Suppose you had the job to interview students for a university. Over the course of an admissions season, you might interview 25 students a week for six months. That would translate to over 500 interviews. Some of those students are probably strong candidates for your school, while others are not. And over the years, you probably get good at separating the good candidates from the bad ones.
Ideally, you would evaluate each new student relative to the entire set you have interviewed in the past. After working for 10 years, that would give you a base of well over 5000 students that you can use to judge each new applicant.
It shouldn’t matter, then, who else you have interviewed that day.
A fascinating study by Uri Simonsohn and Francesca Gino in the February, 2013 issue of Psychological Science suggests that—contrary to this analysis—expert interviewers are heavily influenced by the other interviews they have already done that day.
These researchers examined the ratings given to candidates who interviewed for an MBA program. They had access to the overall rating given to the candidates, scores on subcomponents of the interview like the assessment of the candidate’s willingness to work in a team and interest in the school, as well as the number of interviews already done that day. They also had information about performance (like GMAT scores), qualifications for the program (like the evaluation of the student’s admission essay) and information about the interviewers (like the average score they typically give to interviews).
The researchers did analyses to look at the factors that predict the overall interview score. Even after controlling for lots of other factors (like the candidate’s GMAT score, essay quality, and characteristics of the individual interviewer), there was also a negative correlation between the previous scores given to candidates that day and the score given to the current interviewee.
That is, if all of the previous candidates had gotten high scores, the interviewer gave someone later in the day a lower score.
What is going on here?
It seems that interviewers like to have each day’s ratings balance out. When an interviewer sees 3 or 4 good candidates in a row, they become concerned that they are giving too many high ratings. So, if another good candidate comes walking through the door, they get a lower rating just so that the ratings for the day are not uniformly high. By chance, of course, there should be lots of days in which there are several good candidates in a row. So, interviewers probably should not take the other people interviewed that day into account, but they do.
The researchers tested a number of alternate explanations for the finding. For example, it is possible that being interviewed in the day just magnifies differences among candidates. So, a candidate seen late in the day who is slightly worse than the previous candidates may be rated much worse than that same candidate seen early in the day, because of the contrast with the better candidates. However, an examination of the specific interview characteristics (like willingness to work as a team) did not show the same negative correlation with the previous interview scores. That is, the interviewers were able to give a reasonably objective evaluation of the candidate’s characteristics, but then used the previous interviews for the day to balance out the overall ratings.
If you are being interviewed for a position, this finding suggests that you might want to schedule your interview early in the day to minimize the interference of previous interviews on your evaluation.
If you are doing interviews, then you should be aware that you may try to balance out your evaluations over the course of a single day. Instead, you should do your best to compare each candidate to the overall ideal candidate for the position. Don’t worry that you have also seen other good candidates that day. Remember, that things will balance out in the long-run even if they don’t balance out in the short-term.