Assessment as learning: The role of retrieval practice in the classroom

What are tests and questioning good for?

Jonathan Firth

The ‘testing effect’, widely referred to now as ‘retrieval practice’, is a well-known psychological phenomenon whereby people remember things better if they are tested on them. The benefits don’t stem simply from getting feedback on right or wrong answers – although that can help too. It appears that the process of retrieving information from memory actually helps it to be consolidated. In other words, a test can make the memory more secure and less likely to be forgotten. But what does the research actually tell us about retrieval practice, and what are the implications for what teachers do in the classroom?

Key research

A 2006 study by Roediger and Karpicke has become a classic in this area and is highly illuminating to teachers. The study compared how students performed after different combinations of studying and/or being tested on material. They found that studying followed by a test in which students wrote down everything they could remember led to better recall in a final assessment than being given an additional study opportunity, even though no feedback was given. A point to note is that the advantage was only shown after a two-day or one-week delay – if recall was assessed after just a few minutes, the study-only conditions were better. This is important as it suggests that testing helped retention in long-term memory.

A huge amount of research has been done on this area since the Roediger and Karpicke paper, much of it using educationally relevant materials. Benefits that were first established in psychology labs have been shown to apply to real-world learning situations such as a middle school science classroom (e.g. McDaniel et al., 2011).

It is easy to measure the effect of retrieval practice with simple facts or word lists, and the extent to which the benefit extends to complex materials has been questioned (e.g. van Gog and Sweller, 2015; see also Karpicke and Aue, 2015). However, the effect has been shown many times with the learning of texts, and Butler (2010) found that questions on a passage also improved learners’ ability to transferThe processes of applying learning to new situations ideas to other contexts, suggesting that retrieval can help to support a broader understanding.

Overall, the current research seems to suggest that most teachers and learners in most learning situations would benefit from increasing the level of retrieval used, i.e. the amount that learners have to actively recall things from memory, and that this should be preferred to re-reading or re-teaching as a learning strategy.

Implications for teachers

What, then, are the implications of this for the learning process? Setting a class test is just one way of prompting students to retrieve information from memory; other methods could include:

• Direct verbal questioning
• Self-questioning
• Writing notes from memory
• Using flash cards
• Writing essays
• Group discussion.

The key factor in all cases is that information is actively retrieved rather than passively heard or re-read – a principle that can be applied to any subject discipline. A straightforward method for a teacher who uses PowerPoint would be to insert slides with short questions, either at the end or throughout (see Weinstein et al., 2016, for a discussion of the relative efficacy of interspersing question slides as opposed to testing at the end) – but there are many more examples and ways in which retrieval practice can be integrated in the classroom, as in the examples that follow.

Creating effective retrieval practice activities for younger students

Megan Smith

A plethora of research has been conducted on the benefits of practising retrieval to improve meaningful learning, with students of all ages. But do the same retrieval practice activities that work with older students work with primary age students?

Recent research has found that some of the basic retrieval strategies that are particularly good at producing learning in older students (e.g. at university) don’t work well with younger students For example, Karpicke, Blunt, Smith and Karpicke (2014) conducted an experiment with fourth-grade students (aged 9-11) in their classrooms. Karpicke and his colleagues took fourth-grade science textbooks from the schools and modified the materials to make them easier to read. Students first read the modified text, then were given a blank sheet of lined paper and instructed to write down as much as they could remember from the text. Despite being given plenty of time, the fourth-graders still had trouble remembering what they just read. The students were only able to write down 9 per cent of the information on average; typically, university students are able to write down at least 50 per cent of the material. On a learning assessment four days later, the students did not perform any better after the recall activity compared with just reading the modified text. In other words, adding the extra recall task did not improve learning.

The finding that recall did not improve learning, in this case, is not actually surprising. If college students try to practise recall to prepare for an assessment, but they recall only an extremely small proportion of what they read, then they are not likely to benefit from the activity either. They need to work their way up to being able to successfully recall at least a portion of the information. Then, after recall, to maximise benefits, researchers recommend going back and checking class materials to fill in missing information. Recall, review, repeat. This is all well and good, but what can we, as teachers, do to help facilitate successful retrieval? This is particularly important for younger students who likely need more guidance and structure.

Which retrieval-based learning activities work with primary age students?

It seems that in order for retrieval practice to work well with students of any age, we need to ensure that students are successful in the recall activity. ScaffoldingProgressively introducing students to new concepts to support their learning is a great way to help increase retrieval success. Scaffolding could be implemented with any student, but it may be particularly important with students who may struggle to recall on their own from the start.

In another experiment, Karpicke and colleagues (2014) tested ways of scaffolding retrieval with the fourth-graders in their classrooms. To help guide the students to recall information, students were given partially completed concept maps – diagrams that help to represent relationships among ideas about a given topic. Students were first allowed to fill out the concept maps with the text in front of them. Then the researchers took away the texts, and had the students complete these partially completed concept maps by recalling the information from memory. Using this scaffolded retrieval activity, the fourth-grade students were much more successful on a learning assessment later.

Having found that scaffolding with concept maps helps students successfully retrieve information, the researchers completed one more experiment to compare the guided retrieval activity to a study-only control condition (Karpicke et al., 2014). Students completed a question map (similar to a mind map but with a question or prompt for each strand) with the text in front of them, and then completed another question map without the text. This was compared with a control group during which the students just read through the text twice.

On the learning assessment later, students remembered much more of the information when they used the map to practice retrieval compared with just reading. So, while practising recall with a blank sheet of paper did not improve assessment outcomes over simply reading, practising recall with helpful scaffolds in place did.

Retrieval practice works well for students of many ages and abilities but, for some students, writing out everything they know on a blank sheet of paper may be a daunting task that does not lead to much successful retrieval. To increase success, teachers can implement scaffolded retrieval tasks, like the mapping activities presented here. With scaffolding, the students can successfully produce the information and work their way up to recalling the information on their own.

Retrieval practice as a study skill for students

Blake Harvard

One of the largest gaps in my students’ learning that I encounter regularly is a lack of study skills. I see it as part of my unwritten curriculum to equip my students with these skills/strategies: in my opinion they are incredibly important to the development of a self-sustaining student as they hopefully, transition into a lifelong learner. Of the learning strategies I implement with my students, I believe retrieval practice is perhaps the best. By best, I mean I see an increase in grades (as in, for example, Hartwig and Dunlosky, 2012); it is the easiest to teach to my students; and is most likely to be used by my students.

After instruction (sometimes this is at the end of class, but may be up to a day or two later), I put my students in front of questions or situations that force them to think and create answers using the material they have studied. Since I am preparing my students for advanced placement (AP) tests, most of the time this involves AP-style multiple-choice questions. I tell my students to honestly attempt to answer the questions using only their brain, with no notes.

Once they have finished answering, I sometimes instruct the students to share answers with partners to stimulate a conversation about the material. Next, I tell them the answers to the questions. Finally, I ask them to look back at their answers. That reflection is important. I remind them that developing the ability to understand this material, apply it and grow is entirely up to them – they can do this on their own time, using online quizzing and questioning tools or by simply aiming to recall and write down material studied, but they should not simply re-read or highlight their notes.

Since introducing retrieval practice this year, both within class activities and by encouraging students to use it themselves, I have seen the class average increase significantly compared with the previous five years. Yes, there are many confounding variables that could contribute to the higher scores, but I sincerely believe that much of the increase can be attributed to my focus on retrieval practice during lessons. At the end of the day, however, I’ve got students understanding an important strategy that can be applied in all of their classes and believing they can improve their learning ability through effective study habits. That’s a big win in my book.

Retrieval roulette: Implementing retrieval practice in the secondary science classroom

Adam Boxer

Knowing the potential that retrieval practice has for supporting student learning, I was keen to implement a simple, powerful routine for encouraging regular retrieval with my science classes. For each topic, I create a set of ‘flash cards’ of key ideas in question and answer form. The cards are lean and focused to only have the material necessary and nothing extraneous. I print off a set with all the questions and answers and give it to students at the beginning of a unit, and tell them that all my verbal questions will be based on them. They are expected to begin learning them off by heart and referring to them in their verbal and written answers.

Every three lessons, the class receives a mini-quiz using the contents of the flash cards. I use a simple Excel programme that I call ‘retrieval roulette’ to randomly select questions from the full set contained in the spreadsheet, and display the questions on the board. The quiz has five questions from any point in the course so far and then five questions from the current topic. I give students ten minutes to answer in the back of their books (meaning I can look at their responses later if I want to) and they then peer-assess. I then ask them to raise their hands if they got question one right, then question two and so on. Based on their responses, I pick a question that many of them got wrong, and do some boardwork to clear up any misunderstanding.

This approach seems to me to have many strengths. As well as encouraging retrieval practice, the combination of current and previous topics means that areas of study are spaced and interleaved. It is a straightforward process, which students understand well, and as all the questions and answers are available to the students, a high success rate is possible. In time, I intend to make the process available to parents, too, so that they can support their students to revise effectively.

References

Butler, AC (2010) Repeated testing produces superior transfer of learning relative to repeated studying. Journal of Experimental Psychology: Learning, Memory, and Cognition 36(5): 1118-1133.

Hartwig MK and Dunlosky J (2012) Study strategies of college students: Are self-testing and scheduling related to achievement? Psychonomic Bulletin & Review 19(1): 126-134.

Karpicke JD, Blunt JR, Smith MA and Karpicke SS (2014) Retrieval-based learning: the need for guided retrieval in elementary school children. Journal of Applied Research in Memory and Cognition 3: 198-206.

Karpicke JD and Aue WR (2015) The testing effect is alive and well with complex materials. Educational Psychology Review 27(2): 317-326.

McDaniel MA., Agarwal PK, Huelser BJ, McDermott KB and Roediger HL (2011) Test-enhanced learning in a middle school science classroom: the effects of quiz frequency and placement. Journal of Educational Psychology 103: 399-414.

Roediger HL and Karpicke JD (2006) Test-enhanced learning taking memory tests improves long-term retention. Psychological Science 17: 249-255.

van Gog T and Sweller J (2015) Not new, but nearly forgotten: The testing effect decreases or even disappears as the complexity of learning materials increases. Educational Psychology Review 27(2): 247-264.

Weinstein Y, Nunes LD and Karpicke JD (2016) On the placement of practice questions during study. Journal of Experimental Psychology: Applied 22(1): 72.