Listen and Respond: evaluating the use of audio feedback

It has long been recognised that feedback has a very valuable role to play in overall pupil progress and achievement (Hattie and Clark, 2018; Hattie, 1999). However, questions around how best to deliver feedback to students remain. One of the biggest challenges facing teachers is continuous management of workload, and marking has been identified as one of the most time-consuming activities that teachers undertake (DfEDepartment for Education - a ministerial department responsible for children’s services and education in England, 2015). Ensuring that marking and accompanying feedback are delivered in the most time-efficient and effective way can help to control this workload whilst still maintaining a positive impact on pupil outcomes.

We aimed to investigate the effectiveness of audio feedback as an alternative mode of delivering feedback to A-level psychology students.

Research has shown that feedback is most effective when it provides information on correct rather than incorrect responses and when it builds on changes from previous trials (see Hattie and Clark, 2018; Kluger and DeNisi, 1998). When feedback highlights positive aspects of performance on a task, a learner is more likely to listen and respond and thus further emulate what they are doing well in future tasks. Whilst ‘praise’ is not a particularly effective form of feedback in itself, because it is not directly related to learning outcomes, it has been recognised that feedback appears to be most effective when the perceived threats to a learner’s self-esteem are low (Meskauskiene, 2015; Kluger and DeNisi, 1998).

The way in which teachers choose to deliver feedback can therefore have a very powerful impact on the extent to which learners are able to take on board and respond to the feedback given. In their review article, Hattie and Timperley (2007) note that the most effective forms of feedback provide cues or reinforcement to learners; are in the form of video-, audio- or computer-assisted instructional feedback; and/or relate to goals. In a more recent study, Killingback et al. (2019) investigated the value of delivering feedback using different modes. They found that alternative modes of feedback, such as audio or video recordings, enabled students to engage with feedback in a more personal way and achieve a greater level of comprehension of the feedback given.

With the advent of new technologies and the more widespread use of technology to support teaching and learning in the classroom, we wanted to consider how this could be used to enhance the quality of feedback we were delivering to our A-level psychology pupils. We chose to investigate this question using the ‘audio feedback’ function in OneNote Classroom. For one academic year, the psychology department trialled the use of audio feedback to replace traditional written feedback.

Rationale

We considered that audio feedback could provide an opportunity to deliver richer feedback about learners’ performance and that the use of additional cues, such as intonation, could help to deliver feedback in a less threatening way, thus minimising potential threats to self-esteem. Research suggests that self-reflection is an important aspect of effective feedback for students (see Al Bashir et al., 2016) and therefore, alongside the audio feedback, we introduced ‘feedback sheets’ for students to complete after they had listened to their audio feedback. The introduction of feedback sheets provided an opportunity for students to reflect on the more positive feedback they had received whilst also considering areas for improvement. We felt that this capacity to reflect back and build on progress was pivotal to ensuring greater overall effectiveness of the feedback given (Hattie and Timperley, 2007).

Method

Audio feedback was given for all written work completed by our Year 12 and Year 13 A-level psychology students over the course of one academic year (2018–2019).

All work was submitted electronically in students’ OneNote Classroom ‘homework’ areas and the recorded feedback was saved to this same OneNote Classroom area. Because all essays were submitted in each student’s own personal OneNote Classroom area, teachers were able to save the recorded feedback on the same page alongside each piece of work. This enabled a repository of feedback to be built up over time, allowing both pupils and teachers to easily compare the most recent piece of work with previous submissions and so continually build on progress made.

Students were asked to complete a self-reflection feedback form after listening to every piece of audio feedback. This form was also placed in learners’ OneNote areas and included the following questions:

• After listening to your feedback, can you describe what you did well?
• After listening to your feedback, what is the one area that you will try to improve on in future work?
• Paying attention to the feedback, for the one area that needs improvement, can you write that part of the essay answer again?

To capture the impact of introducing audio feedback, we distributed open questionnaires at the end of the academic year to all staff and pupils who had participated in the trial. The qualitative data that was collected through these questionnaires was analysed using thematic analysis.

Alongside this, we also included a quantitative measure of pupil progress during a typical academic year when traditional written feedback was used (2017–2018) and across the year 2018–2019, when audio feedback was introduced.

Results

Qualitative analysis

The responses collected from the staff and pupil questionnaires at the end of the academic year were grouped according to three themes relating to:

1. the level of detail provided in the feedback
2. students’ ability to comprehend the feedback
3. student progress. 

1. Level of detail in the feedback

 Students indicated that in their experiences, this mode of audio feedback generally included much more detailed and specifically targeted comments than typical written feedback.

“[Audio feedback] helps because it gives a more detailed explanation of how you have completed the work.”

“An explanation of essay structure and how I could alter it is much more detailed when listening to feedback than what is usually written.”

Teachers reported that when recording their spoken feedback, they were able to give much richer and more detailed information and there were greater opportunities to give positive commentary alongside targeting areas for improvement.

“More chance to give positive commentary throughout, rather than highlighting things that need to change.”

“Besides the occasional technical issues encountered, the audio feedback on OneNote is quick and easy.”

We also found that extensive verbal commentary could be delivered in a short space of time. This reduced teacher workload whilst still allowing teachers to deliver rich and detailed feedback.

“Detailed feedback given in a few minutes compared to writing comments, which take a lot longer. More opportunity to be constructive and supportive.”

2. Students’ ability to comprehend the feedback

 Students noted that they found that audio feedback was more accessible than written feedback and that it captured their attention more fully.

“I feel like it resonates more.”

“I prefer audio feedback as opposed to written feedback as I am more likely to pay attention and concentrate on what is being said compared to if it was written.”

In response to the question, Did you find audio feedback useful?, one student commented:

“Yes. Helps with my understanding of what needed to be improved or why certain sections of the essay were well written!”

3. Student progress

 Students also reported that coupling the recorded audio feedback with a feedback reflection form helped them to organise the feedback given. It also helped them to take on board recommendations for improvement and implement changes immediately to see progress.

‘The essay feedback form is very useful as it allows me to see where my weaknesses are in certain topics and it gives me a second chance, which is very useful, especially if I’ve gone totally in the wrong direction/need rephrasing. It also makes me revisit the essay, which is actually very useful as I don’t just file it away and never look at it again.’

“… listing areas to improve and rewriting a paragraph because then I know what my targets are for my next essay and also can see immediate improvement when rewriting a paragraph.”

Teachers noted that the ability to save the audio feedback in OneNote Classroom was very beneficial, as it enabled both teachers and students to reflect back on the improvements made. It was also recognised that some of the pupils who showed the greatest levels of improvement across the year had responded very positively to the audio feedback.

“Some of our most improved students have found it very helpful.”

“The feedback doesn’t get lost. You can save it to your own area if you want a copy.”

Quantitative analysis

For Year 12 students, we compared the overall level of progress shown on internal assessments between December and June. The 2017–2018 cohort (without audio feedback) showed an increase in mean performance on end-of-unit assessments of three per cent during this time frame. In addition to showing better baseline performance, the 2018–2019 cohort (with audio feedback) progressed by an average of eight per cent in their overall performance on end-of-unit assessments across a similar time frame.

Analysis of the A-level performance of the Year 13 students indicated that in terms of value-added measurement, the 2017–2018 cohort showed a standardised residual of 0.2, whereas the 2018–2019 cohort showed a higher standardised residual of 0.5. This indicates that the 2018–2019 cohort had a much higher overall value-added score than previous cohorts.

Discussion

Analysis of the qualitative data collected from both students and teachers confirmed the findings of Killingback et al. (2019), indicating that audio feedback is a helpful mode of delivery as it allows teachers to personalise feedback and engage with students in a more meaningful way. This helped to not only engage students’ attention, but to also support their comprehension of the feedback given. Analysis of the students’ performances on internal assessments and public examinations also demonstrates greater student progress in the cohorts involved in the audio feedback trial.

The flexibility afforded through audio feedback enabled teachers to deliver much more detailed, targeted and constructive feedback than had been possible using written feedback. Thus it was made possible to deliver feedback that highlighted positive aspects of students’ work, thereby reducing the level of threat to students’ self-esteem (see Meskauskiene, 2015; Kluger and DeNisi, 1998). The function to save each piece of audio feedback alongside pupils’ work in OneNote Classroom enabled teachers and students to look back on previous submissions and reflect on the progress they had made, thus helping to ensure that the feedback continued to remain an effective indicator of pupil outcomes.

With reference to teacher workload, overwhelmingly it was noted that the amount of feedback that could be given in approximately one minute of audio feedback was much greater than would be possible using written feedback. The capacity to record audio feedback saved a lot of time and therefore reduced teacher workload without adversely affecting the detail and quality of the feedback given.

Although there are many factors that contribute to successful teaching and learning, we did find that when it came to public examinations, this A-level cohort achieved the highest value-added scores we have seen in our department since 2012. This may in part be due to the mode of delivery; it may also be that coupling audio feedback with a student feedback form required a greater level of engagement and response from students, and thus contributed to greater overall efficacy of the feedback. We believe that audio feedback was so successful because students were made to use it actively (Al Bashir et al., 2016). The self-reflection feedback form required a much greater level of active engagement on the students’ behalf than passively reading written comments, and thus may have helped to build greater metacognition in students.

We plan to replicate this trial again with our students this year and continue to explore both ‘audio feedback’ and ‘dictate’ functions in OneNote Classroom as ways to deliver detailed, targeted and effective feedback.

References

Al Bashir M, Kabir R and Rahman  I (2016) The value and effectiveness of feedback in improving students’ learning and professionalizing teaching in higher education. Journal of Education and Practice 7(16): 38–41.

Department for EducationThe ministerial department responsible for children’s services and education in England (DfE) (2015) Government Response to the Workload Challenge. London: Department for Education.

Hattie JA (1999) Influences on student learning. Inaugural professorial address, University of Auckland, New Zealand, 2 August 1999.

Hattie J and Clarke S (2018) Visible Learning: Feedback. Abingdon, Oxon: Routledge.

Hattie J and Timperley H (2007) The power of feedback. Review of Educational Research 77(1): 81–112.

Killingback C, Ahmed O and Williams J (2019) ‘It was all in your voice’ – tertiary student perceptions of alternative feedback modes (audio, video, podcast, and screencast): A qualitative literature review. Nurse Education Today 72: 32–39.

Kluger AN and De Nisi A (1998) Feedback interventions. Towards the understanding of a double-edged sword. Current Directions in Psychological Science 7: 67–72.

Meskauskiene A (2015) Teacher–pupil interaction: Factors strengthening and impairing adolescents’ self-esteem. Social and Behavioral Sciences 197: 845–850.