Showing posts with label education. Show all posts
Showing posts with label education. Show all posts

Monday, January 9, 2012

The upside and downside of testing.


Multiple-choice tests are everywhere.  Teachers use them, because they are easy to grade and there is an objectively correct answer.  Aptitude tests also use them for similar reasons.  The internet is full of trivia quizzes with odd questions followed by a series of potential answers.

When people are preparing to take a test, they often study by taking practice tests.  How do these practice tests affect performance on later tests?

This question was addressed by a paper in the June, 2010 issue of Memory and Cognition by Lisa Fazio, Pooja Agarwal, Elizabeth Marsh, and Roddy Roediger.  They find that taking practice tests has both positive and negative consequences for your performance on later tests.

In their experiment, they had people read passages about nonfiction topics. Then, people took a practice multiple choice test.  This test was taken either soon after reading the passages or a week later.  People also did two other tests.  One was a recall test where they had to remember the answers to questions rather than getting a set of possible answers.  Finally, everyone took a final multiple choice test at the end.  The tests asked questions both about information that was in the passages that people read as well as information taken from passages that they did not read.  This last set of questions was expected to be difficult, because people would only know this information if they had encountered it in some other place.

What happened?

Taking an initial multiple choice test as practice did improve people’s performance on tests.  They were much more likely to get the correct answer both in a recall test and also in a later multiple choice test if they studied by taking a multiple choice test.  For the cued recall test, this effect was strongest when the practice test was taken after a one-week delay rather than right away.  For the cued recall test, it didn’t matter as much whether the practice test was taken right away or later.

There was also a negative effect of testing.  In both the recall test and the final multiple choice test, there was a tendency for people to answer with one of the incorrect answers from the original multiple choice test. 

Why does that happen? 

There are two ways that people answer questions on tests.  Sometimes they know the correct answer.  For example, if I ask you the address of the White House, you may simply know that it is 1600 Pennsylvania Avenue.  If you see that on an exam, you answer correctly, because you remember the correct answer.

Sometimes, though, a particular answer just feels familiar.  For example, if you are asked for Richard Nixon’s first Vice President, you might not remember his name specifically.  When you see the name Spiro Agnew, though, it might feel right, and you might give that as the answer.  (And you’d be right.)  Using familiarity to help you answer questions is most useful when there has been a long delay between when you first encountered the information and when you are being tested for it.

Prior multiple choice testing increases the familiarity of all of the alternatives that are part of the test.  It becomes harder to use familiarity as a strategy for answering questions when many answers begin to seem familiar.

Overall, the positives of studying by taking tests seem to outweigh the negatives.  The improvement on tests for items that were tested ranged between 5 and 20 points on the test.  You might lose a few points by getting wrong answers from the lures, but overall the effect of the lures is smaller.

Finally, the authors of this paper suggest that when you do study using a multiple choice test, you should try to get feedback on your answers immediately.  That immediate feedback allows you to focus primarily on the correct answer when you are studying and that will reduce the effect of the lures on later tests.

Saturday, January 7, 2012

Say it loud: I’m creating a distinctive memory.


I’m around a lot of studying.  When I walk through the lobby of the Psychology building at Texas, I often see a few students on benches reading over their notes.  A trip to the library requires navigating a sea of students at tables and carrels deep in study.  Then, when I go home, I have three teens who are usually engaged in some kind of studying.

One thing you can say about studying.  It sure is quiet. 

Now, there are good practical reasons to want to be quiet while studying.  If you’re in a public place, it would be disruptive to the people around you to start talking.  Plus, if you are talking to yourself, that would look more than a little odd.  Even at home, you may begin to think you have lost your marbles if you start muttering to yourself while studying.

A paper by Colin MacLeod, Nigel Gopie, Kathleen Hourihan, Karen Neary, and Jason Ozubko in the May, 2010 issue of the Journal of Experimental Psychology:  Learning, Memory, and Cognition, suggests that every once in a while, it might not be a bad idea if you talked a bit while studying. 

In this paper, these researchers document what they call the production effect.  They looked at people’s memory for items like a list of words.  They found that if people studied the list by reading half of the words silently and the other half by saying the words out loud, that he words spoken aloud were remembered much better than those that were read silently. 

Now, it isn’t that just reading things aloud helps, because people who read all of the items on the list aloud were no better at remembering the items than the people who read them all silently.  And both of these groups remembered the words more poorly than the words that were read aloud by the people who did half the list silently and the other half aloud.

So, what is going on here? 

One of the things that is known to help memory is distinctiveness.  We tend to remember oddballs pretty well.  An experimental example of this idea is the Von Restorff effect.  If you study a list of words where all but one are birds and the remaining item is a sport, you are much more likely to remember the oddball than to remember any particular one of the birds.  More commonly, when you meet up years later with friends from grade school, everyone remembers the names of the kids who were different from the crowd  in some way.  (If you discover that everyone from grade school remembers you, then you were probably the one who was different.)

The production effect works because it makes part of the list of items more distinctive.  The words you speak aloud are now translated into speech and you have knowledge of producing the items as well as a memory of hearing them.  All of this information makes your memory for the spoken items more distinct from the rest of the items that were read silently.

This result suggests that if you are studying material, you might want to identify those bits of information that are most important to remember and to speak those bits aloud while studying.  Even a whisper will help to make those items more memorable.

Friday, December 2, 2011

How do you decide whether you did well on a test?


As a parent, it is sometimes frustrating to talk to your kids about exams.  There are times when they will come home convinced that they did really well on a test only to be disappointed when the results come in.  Other times, they feel like they did quite poorly on the test and are pleasantly surprised that they scored better than they expected.  

How do people judge how well they did on a test after taking it?

This question was explored in a paper by Yana Weinstein and Roddy Roediger in the April, 2010 issue of Memory & Cognition.  They had students take tests of general knowledge questions.  Some of the questions on these tests were easy (What is the name of Tarzan’s girlfriend?).  Some of the questions were more difficult (The general named Hannibal was from what city?)  There were 50 questions on the test.

This study examined a number of different factors that might affect people’s judgments about how easy a test was including whether people could choose how many questions they wanted to answer or were forced to answer all of them, and the way that the question about test accuracy was asked.  Most of these factors had little or no consistent influence on people’s judgments of how well they did on the test.

The one factor that had a consistent effect was the ordering of the questions on the test.  When the test had the easiest questions at the beginning and the harder ones at the end, people felt like they did better on the test than when the questions were randomly ordered.  When the test had the hard questions first and then the easy ones, people actually felt like they did worse than when the questions were randomly ordered. 

Of particular interest in these results is that when the easy questions were first, people actually judged that they got more questions right than they actually did.  So the easy-to-hard ordering made people overconfident in their performance.

Why does this happen?  The authors suggest two reasons for the observed results.  First, the effect appears most strongly when people are looking back on their performance rather than when they make judgments as they are performing the test.  Research on memory finds that items that are seen early in a list are remembered better than those seen late in the list.  So, if you look back on a test, you are most likely to remember items from early in the test.  If those items are easy, then that will give you the impression that you did well overall.

Second, people often enter a test with a rough sense of how well they are going to do.  For example, if you have not prepared well for a test, you may think you are going to do poorly.  If the test starts with a number of easy questions, you may be pleasantly surprised at your performance and adjust your belief upward.  If the test starts with a number of hard questions, you may adjust your belief downward.

So, why does this matter? 

Perhaps the most important issue is one of motivation.  If your performance on an exam is worse than you thought it would be after taking the test, it can be demoralizing.  It is important to remember that your actual performance on a test is only one factor that helps you figure out how well you did.  As this work shows, the ordering of the questions matters as well.  It is no fun getting a poor grade on a test, of course, but don’t compound the disappointment by being upset that you thought you had done better. 

Monday, August 29, 2011

Video games make a lousy gift.


Because it is back-to-school time, it is interesting to think a bit about another drain on kids’ time—video games.  What role video games have on kids’ performance in school? 

There is a lot of correlational evidence suggesting that video games are bad for schoolwork.  That is, kids who play lots of video games tend to do more poorly in school than kids who do not play that often.

Correlational studies are the most obvious studies to do to address this question, because it is straightforward to get information about school performance about a group of children and then find out from their parents how often they play video games at home.  Of course, the problem with correlational studies is that there are many factors that could be causing the results that go beyond what was measured.  Maybe the kids who play the most video games at home have the least parental supervision, and so they don’t do well in school.  Maybe the kids who are struggling most in school are the ones who seek out video games as a way to escape the frustration of homework.

A nice study in the April, 2010 issue of Psychological Science by Robert Weis and Brittany Cerankosky tackles this issue head-on.  They found a sample of boys between the ages of 6 and 9 who did not yet own a video game system, but whose parents were planning to buy them one in the near future. 

They got information about reading, writing, and math performance in school about all the boys as well as information from parents and teachers about behavior at home and at school.  Two groups of boys were created.  The groups were set up so that their current level of school performance was about the same. 

One group of the boys was given a PS2 game system and three age-appropriate games.  The other group did not get a game system.  After four months, the researchers got information about current reading and math performance, behavior, and an assessment from the parents about how the kids were spending their time.

Reading and writing performance of the boys given the PS2 was significantly worse than that of the boys who did not get the game system.  There were no statistically reliable differences in math performance. 

Other statistical analysis found that the difference in reading performance could be explained by the amount of time that the boys spent playing video games each week.

Why was language performance hurt, but not math performance?  Between the ages of 6 and 9, there are many opportunities for kids to get extra practice with language.  Just having a conversation with parents can improve language use.  In addition, kids can sit at the dinner table and read a book.  Parents can spend some time reading to their children.  The video games cut into this unstructured reading time.  There aren’t as many opportunities to practice math outside of doing schoolwork directly, and so math performance was not hurt. 

What about behavior?  There was a tendency for the kids who got the video games to exhibit more behavior problems in school than those who did not get the games.  This difference was small, but it should be of some concern, because the boys in this study had only had the games for about 4 months when the follow-up measures were taken.  These initial differences could get larger over time.

What does this mean?

The problem here is probably not the video games themselves.  The problem is that video games are very enticing.  Kids can sit and play video games for many hours.  In that time, they engage in few real conversations with others, and so they end up with much less practice reading and using language. 

As a parent, I know how hard it is to resist buying a video game system for your kids.  Obviously, if there is no game system in the house, that will limit the amount of time that your kids play games.  If you do decide to buy a game system, then it is important to have strict limits on the amount of time that it gets played to leave enough time for the kinds of interactions that enhance learning.

Thursday, August 25, 2011

Why girls drop math III: Stereotype threat


In the previous two posts, I talked about two factors that lead girls to drop math:  their belief that math is a talent not a skill, and their desire to pursue careers that have lots of social interaction.  I mentioned data from Jacquelynne Eccles that looked at reasons why girls drop math.  Another factor that strongly affects girls is math anxiety.  The more math anxious girls get, the less likely they are to continue taking math.

One important part of math anxiety is a vicious circle.  Girls hear from parents, friends, and sometimes teachers that girls in general don’t do well in math.  So, there is a stereotype out there in the culture that girls are bad at math.  That stereotype then leads to anxiety when girls are confronted with tests that will demonstrate their math ability.  That anxiety causes girls to do worse on math tests than they would if they were not aware of the stereotype.  Their poor performance on tests convinces them that they are bad at math, and the stereotype becomes a self-fulfilling prophecy.

This pattern was first demonstrated by Claude Steele and Joshua Aronson, who called it stereotype threat.  They found that when people are in a situation in which they might confirm a negative stereotype about a group to which they belong (based on gender, race, or ethnicity, for example) they perform worse in that evaluation than they do when they are not aware of the stereotype.  This finding has been obtained in many settings since it was first reported in 1995.

What can be done?

Research from our lab by  Lisa Grimm, Todd Maddox, Grant Baldwin and I (published in a 2009 paper in the Journal of Personality and Social Psychology) had college men and women take a hard math test (using questions from the GRE exam that is used for entrance to graduate school).  In one condition, people got points for each question they answered, but they got more points for correct answers than for incorrect answers.  In this case, we observed the typical stereotype threat effect.  The women got about 10% more of the questions wrong than the men did. 

In a second condition, though, people started off with a number of points, and they lost points for each question they answered.  They lost fewer points for correct answers than for incorrect answers, and so they were trying to minimize the number of points that they lost.  In this case, the women actually performed somewhat better than the men.  In fact, the women who were trying to minimize the number of points they lost did just as well as the men who were trying to maximize the number of points they gained.  That is, the stereotype threat effect disappeared.

Why?

We suggested that having a negative stereotype puts you in a defensive motivational mode.  You are prepared for negative outcomes in the environment.  If the environment actually has positive outcomes in it (like gaining points), then you act cautiously, and your performance in testing situation suffers.  If you are prepared for negative outcomes and the environment actually has negative outcomes in it (like you are losing points), then you are less cautious, because the environment is as you expect it to be.

There is still a long way to go to figure out how to use results like this to demonstrate to girls that they do not need to fear math.  But now, at least, we know that the effects that negative stereotypes on girls can be counteracted.  Hillary Clinton bashed the glass ceiling in politics.  Hopefully the ongoing research in Psychology will bash the glass ceiling in math and science as well.

Monday, August 22, 2011

Why girls drop math II: Who wants to be alone?


We know that girls are less likely to go into math and “hard science” careers than boys.  There are lots of reactions people have to this news.  Some, like Lawrence Summers (who became the former president of Harvard in large part because this remark in 2005) suggested that women may not have the same innate abilities in math as men.  Others suggest that the school system is somehow failing girls in school.  Still others are concerned about social pressure that girls might feel to shy away from looking smart when there are boys around.

These last two ideas have some traction in popular and political culture.  There has been movement to create all-girls’ schools to give girls an environment to thrive.  Indeed, my home town of Austin Texas has a public all-girls school (named after former Texas governor Ann Richards).  I think there is much to like about the idea about an all-girls school for middle-schoolers.  One question we should ask, though, is whether the mass exodus of girls from math (and to some degree hard sciences) is being caused by a failure of the educational system.

Psychologist Jacquelynne Eccles has studied this question extensively.  One of her more surprising findings is that girls tend to attach less value to math than boys.  There are a few sources of the difference in value that boys and girls place on math.

For one, parents may give their children beliefs about how important math ought to be to them.  Eccles’ evidence suggests that girls are less likely to value math and to take advanced math courses when their parents do not believe that math is as important for girls as it is for boys.  So, girls are internalizing the beliefs of their parents.

For another, there is a pervasive stereotype that mathematicians lead a lonely existence.  When we think about mathematicians, we imagine the lone genius hunched over a desk writing incomprehensible formulas that have lots of Greek letters in them.  On occasion, these lonely figures may emerge from their cramped messy dens to shout Eureka and post the solution to a problem that five people in the world care about on a blackboard in a university hallway. 

Girls find this prospect much less appealing than boys.  Girls are much more likely than boys to state that want a career that has lots of social interaction as a part of it.  When girls start making decisions about the classes they are going to take, they are much more likely to gravitate toward sciences like biology and psychology than toward math, chemistry or physics.  Biology usually leads to pre-medical study in college, and the medical professions are seen as very social.  Likewise, psychology, social work, and other helping professions are seen as social. 

This research suggests that one big reason why girls are less likely to stick with math than boys is a difference in life goals. 

The big question, then, is whether the stereotype about math and the hard sciences is true.  In fact, much of modern science is highly collaborative.  Scientists must find ways to work together to solve difficult problems.  The myth of the lone scientist is just that—a myth.  And that means that science has a public relations job to do.  If we want more girls to go into math and science professions, we are going to need to do more work letting girls know what those careers will really involve. 

I do want to be clear about one thing, though.  There are lots of reasons to think that some girls will be more likely to succeed in a school environment without boys.  So, I am not suggesting that we close all-girl schools.  I am suggesting that the reason to open such schools is not because our current schools are failing to give a good math and science education to girls.  However, we might use venues like all-girl schools to start the process of teaching girls more about what careers are possible with a good math and science education.