Monday, January 9, 2012

The upside and downside of testing.


Multiple-choice tests are everywhere.  Teachers use them, because they are easy to grade and there is an objectively correct answer.  Aptitude tests also use them for similar reasons.  The internet is full of trivia quizzes with odd questions followed by a series of potential answers.

When people are preparing to take a test, they often study by taking practice tests.  How do these practice tests affect performance on later tests?

This question was addressed by a paper in the June, 2010 issue of Memory and Cognition by Lisa Fazio, Pooja Agarwal, Elizabeth Marsh, and Roddy Roediger.  They find that taking practice tests has both positive and negative consequences for your performance on later tests.

In their experiment, they had people read passages about nonfiction topics. Then, people took a practice multiple choice test.  This test was taken either soon after reading the passages or a week later.  People also did two other tests.  One was a recall test where they had to remember the answers to questions rather than getting a set of possible answers.  Finally, everyone took a final multiple choice test at the end.  The tests asked questions both about information that was in the passages that people read as well as information taken from passages that they did not read.  This last set of questions was expected to be difficult, because people would only know this information if they had encountered it in some other place.

What happened?

Taking an initial multiple choice test as practice did improve people’s performance on tests.  They were much more likely to get the correct answer both in a recall test and also in a later multiple choice test if they studied by taking a multiple choice test.  For the cued recall test, this effect was strongest when the practice test was taken after a one-week delay rather than right away.  For the cued recall test, it didn’t matter as much whether the practice test was taken right away or later.

There was also a negative effect of testing.  In both the recall test and the final multiple choice test, there was a tendency for people to answer with one of the incorrect answers from the original multiple choice test. 

Why does that happen? 

There are two ways that people answer questions on tests.  Sometimes they know the correct answer.  For example, if I ask you the address of the White House, you may simply know that it is 1600 Pennsylvania Avenue.  If you see that on an exam, you answer correctly, because you remember the correct answer.

Sometimes, though, a particular answer just feels familiar.  For example, if you are asked for Richard Nixon’s first Vice President, you might not remember his name specifically.  When you see the name Spiro Agnew, though, it might feel right, and you might give that as the answer.  (And you’d be right.)  Using familiarity to help you answer questions is most useful when there has been a long delay between when you first encountered the information and when you are being tested for it.

Prior multiple choice testing increases the familiarity of all of the alternatives that are part of the test.  It becomes harder to use familiarity as a strategy for answering questions when many answers begin to seem familiar.

Overall, the positives of studying by taking tests seem to outweigh the negatives.  The improvement on tests for items that were tested ranged between 5 and 20 points on the test.  You might lose a few points by getting wrong answers from the lures, but overall the effect of the lures is smaller.

Finally, the authors of this paper suggest that when you do study using a multiple choice test, you should try to get feedback on your answers immediately.  That immediate feedback allows you to focus primarily on the correct answer when you are studying and that will reduce the effect of the lures on later tests.