Monday, September 19, 2011

More information makes you more confident, if not more accurate.


Confidence matters.  We are much more likely to act in situations when we are confident.  We make purchases based on confidence.  We are also persuaded by others based on their confidence.  A statement made confidently and forcefully is much more likely to sway our opinion than a statement that is hedged. 

Presumably, the power of confidence lies in the belief that when people are more confident in an outcome they are more likely to be correct in their predictions.

There have been many studies over the years that demonstrate that people tend to be overconfident in their judgments about how likely they are to be correct about a prediction or an answer to a question.  But what causes this overconfidence?

A 2008 paper by Claire Tsai, Joshua Klayman, and Reid Hastie in Organizational Behavior and Human Decision Processes suggests one factor that makes people overconfident.  They find that as people get more information about a judgment they are making, it increases their confidence, even if it does not increase the accuracy of their judgment.

In one study in this paper, they found experts in college football and asked them to predict the outcomes of a number of games.  The names of the schools playing the games were not given.  Instead, people were given information about how the teams had performed to that point in the season on a variety of aspects of performance that are useful for predicting the outcome of a game (like the average number of yards that the teams had gained in their games so far that season). 

For each game, the participants were initially given six measures of performance for each team and were asked to predict which team would win the game and the point spread between the teams.  They were also asked to rate their confidence in their judgment.  Based on this initial information, people predicted the outcome of the game correctly about 63% of the time, and their average confidence was about 68%.  That is, they were overconfident, but not very overconfident.

Then, people were given 6 more measures of performance and they repeated the judgment, again predicting the outcome of the game and the point spread, and giving their confidence.  This was repeated 3 more times, so that people ultimately made 5 judgments and saw 30 measures of performance for each game. 

As people got more information, their overall accuracy did not change much.  After seeing all 30 cues, people were only correct in predicting the winners of the game 67% of the time.  However, by the end of the study, they were not 79% confident in their responses.  That is, their confidence went way up as they got more information, even though their accuracy stayed the same.

One thing that seems to happen as people get more information is that they start to go from specific information to more general evaluations of aspects of team performance.  So, based on a few different measures of performance, a judge might now make an evaluation of the effectiveness of a team’s run defense or passing attack. 

Of course, all information is ambiguous.  Not every measure of performance tells the same story about teams.  As you get more information, though, you become more free to pick and choose the information that is consistent with the story you are trying to tell.  In this way, you find information that helps you to confirm what you already believe to be true.  This tendency to seek information that supports a conclusion you have already drawn is called confirmation bias.  This confirmation bias will increase confidence in the judgment you have made.

It is important to realize that confidence and accuracy are not that highly related.  We are often asked to make decisions on the basis of expert opinion.  There is a temptation to rely on the confidence of the expert to decide how much of that expert’s opinion we should trust.  Perhaps we are better off looking at that expert’s past performance to make up our own mind about how confident we should be in the accuracy of their judgments.