Awarding qualifications in summer 2021: Urgent action required
This viewpoint on Ofqual’s proposals regarding summer examinations in 2021 is written by Dr Mick Walker, Former Teacher, Researcher, Fellow of the Chartered College, and an expert in educational assessment.
The views presented here do not necessarily represent those of the Chartered College. Views on the best way forward are very varied across the sector. For a different viewpoint, read Shirley Clarke’s thoughts and Michael Chiles’ views, and for a wider perspective on some of the challenges and our members’ views, have a look at Cat Scutt’s summary of members’ concerns for the Chartered College.
On the 15th January, the DfEDepartment for Education - a ministerial department responsible for children’s services and education in England and Ofqual launched two consultations on how qualification grades would be awarded in summer 2021. One deals with GCSE, AS and A-levels, the other vocational qualifications. With a closing date of Friday 29th January, time to consider and reflect is short. Clearly, it would have been ‘wiser’ to have considered and modelled contingencies should exams be cancelled at the end of last summer, and at some point there has to be a very serious look at how the qualification system has been managed during the pandemic.
But to be fair, anyone would have found managing the impact of Covid-19 on our system of qualifications and tests a challenge, and must be said up-front that there are no ideal solutions to the problems we now face in awarding qualifications in 2021. In the end, there will have to be compromise. Running a process that is widely accepted as being fair and equitable is now a near impossible task.
Understandably folk are confused, angry and somewhat bewildered by what has already happened in relation to examinations and what is now proposed. In the context of working day-to-day in a Covid-impacted environment, having to organise and deliver teaching in school and online is challenging enough. But teachers are not merely defined by their role in school: they are mums and dads, brothers and sisters, daughters and sons, friends and neighbours, so they have to deal with all the other fears and uncertainties along with everyone else. So whilst the consultation focuses on the delivery of qualifications that will impact the lives of so many young people not just this summer, but probably for the rest of their lives, any proposals have to be seen in the wider context.
The Consultation proposals
In relation to the GCSE and A-level qualifications, last September, the CIEA, Chartered College of Teaching and Sir John Dunford wrote to the Secretary of State with a proposed ‘plan B’, a contingency should exams be cancelled in 2021 – which even then was a strong possibility. The plan was constructive, non-partisan, and accompanied by an offer to help the implementation. Ministers finally replied in mid-December, but remarkably managed to miss or ignore the actual content of the September letter. In January, I wrote a modified plan cognisant of the fact that almost four months had passed without any signs of a contingency from the DfE or others who really should have been taking a lead. Indeed, I have to register my real disappointment in those who should have been taking the lead – and I’m not just thinking politicians. Sure, there has been lots of noise and posturing, but no ideas, no plan B, no leadership.
So put simply, the adults haven’t stood up.
At least we now have a joint proposal from the DfE and Ofqual (an interesting point in itself – but for another time), but it’s now over-engineered, too complex and unrealistic. To be fair, the consultation proposals lay out a set of robust procedures and if they had been presented earlier, they would have presented quite a way through the challenges we face. Problem is, they weren’t, but they would form a good contribution to a debate on an alternative approach to the administration of general qualifications in the future, just not right now, it’s too much too late. Ant plan now has to be stripped back to the basics, to what can be realistically achieved. So, how do the proposals measure up?
What the grades will mean
The key and really fundamental assessment principle suggested by the proposals is to award grades on attainment – what students know, understand and can do. This is significantly different to what happened in 2020 where centres were required to produce grades formed of predicted performance had examinations taken place. Technically speaking, on pure assessment grounds, the proposed approach for 2021 is the right one. But is it fair?
The suggested attainment-only approach will discriminate against students who have received the least teaching time and provides advantage to those who have had comparatively less interruption to their learning – all of which has been beyond the control of students and centres. Such an approach is open to justifiable criticism on the grounds that some students who have been more affected by the pandemic than others will be penalised. This in itself is likely to fuel an avalanche of appeals (more on that below). The only conceivable way disadvantaged students can now be supported is either to adopt a compensatory award based on predicted performance had exams taken place, or to target key elements of the subject specification in the remaining time and make sure any evidence of student learning and attainment is gathered.
And how will teachers know a grade 6 when they see one? This is considered further below.
When should teachers assess the standard at which students re performing?
The consultation presents mixed messages. On the one hand, the emphasis is on keeping students motivated and extending teaching time as long as possible. This reflects the pre-consultation rhetoric about pushing exams back by three weeks. On the other, the suggestion is that assessments should be made from late May to early June. That’s twelve weeks of school/college time. Why not late June or early July? The reason seems to me to allow as much time as possible for appeals. This speaks volumes! I would prefer longer teaching time, working hard to generate fair grades and reducing the need for appeals.
Teachers should also be allowed to draw on any evidence to support their grades. Teachers are now best placed to judge what evidence of attainment is available to support their professional judgement and for me, it should be drawn from all sources providing it sits within the syllabus requirements and represents the un-aided effort of students. This applies to all evidence, be it in examination conditions that some schools may be able to operate, or in work undertaken throughout the course.
Determining teacher grades
In regard to how teachers should determine the grades they submit to examination boards, the consultation appears to place a heavy emphasis on the potential use of exams and tests and asks for views on their use to supplement other forms of evidence of student performance. Further questions ask whether they should be a compulsory part of this year’s approach.
I am a supporter of using tests and questions/items produced by awarding bodies because they are of a high quality. I proposed in January that a bank of tests should be made available to centres along with mark schemesCriteria used for assessing pieces of work in relation to particular grades, support materials and any available performance data such as item analysis. Such provision would give centres an indication of relative performance. However, and of real importance, the tests should be used at the discretion of centres and certainly not weighted above, or necessarily below, other forms of assessment. Nor should they be used to generate data for any purpose other than informing the judgments made by teachers. In promoting tests, I was at pains to stress that their use and status should be clearly communicated otherwise it would appear as exams through the back door. I’m not sure this point was heeded.
Clearly, if we are to allow some level of flexibility in the use of tests, which is now my preferred option, there is every chance that the content will be compromised. But, by using an item or test bank, this can be minimised, especially if this is explained to students. And whilst I wouldn’t advocate a compulsory regime of tests, I would strongly encourage their use.
Further, if the tests were to be administered as late as possible in the summer term within a test window, there is still a possibility, albeit a slim one, that students’ learning could be focused around tests geared to areas of the syllabus as a means of incentivising learning and generating evidence of attainment, all of which would aid teacher judgments.
We should also remember that tests, even the usual annual examinations, only ever sample the syllabus with full coverage roughly over a five year period. But I rarely hear debates about just how much of a sample is taken each year, so we need to exercise caution in being too exacting in just how much evidence a student has to generate this summer. I’ve already heard figures like 60% of the syllabus, but I for one don’t understand what folk mean by that exactly.
A further and highly significant point is how will teachers award and differentiate between grades? What makes a grade 8 different to a grade 7 or 9? We have no criteria on which such judgments can be made as the current system is geared to comparative outcomes. So rank ordering, though not mentioned in the consultation, might be a tool for schools to undertake under their own steam as an additional piece of information to guide professional judgment. They did this last year.
Hopefully, the proposed guidance and support from awarding bodies will assist in identifying key aspects of grades, but it needs to be clear and well structured. This was again a suggestion made last September to give examiners time to prepare guidance that would illustrate the type of performance they expect around grade boundaries. Examiners are generally good at this, but they need time to prepare materials that clearly articulate their thoughts. If this is accompanied with other performance data from awarding bodies, this has to be of benefit. But time, yet again, is against us, exam boards included. However, wise exam boards may be on this case already!
Supporting teachers: Internal QA
I am pleased to see that standardisation is seen as central to internal quality assurance. What is vital here is that the distinction between standardisation and moderation is really well understood. It is also important that heads of department and the head of centre take on responsibility here. Along with others, I am a keen supporter of schools and colleges having designated Lead Assessors who in effect become the owners of standards set and expected by their institutions – and to oversee quality assurance and quality control procedures – like moderation. Being a Lead Assessor should, in my view, be a step towards becoming a Chartered Educational Assessor (CEA). If every centre had a CEA, robust QA arrangements would be already in place.
But again, Lead Assessors need the support of the head of centre and access to guidance. The Chartered Institute of Educational Assessors (CIEA) has designed such support covering areas like validityIn assessment, the degree to which a particular assessment measures what it is intended to measure, and the extent to which proposed interpretations and uses are justified, controlling for bias, standardisation and moderation; if this was complemented by support from awarding bodies, we could have a powerful set of aides. Taking advantage of such support should also strengthen the position of schools and colleges in the appeals process. My fear here however, is that this puts all the pressure on schools and colleges, which if they were better prepared, would be defensible.
Another excellent idea that would work well if given time would be to call for centres to work together. Some centres are already involved in established networks they can trust, others are not. And then there’s workload (more on this below).
Supporting teachers: External QA
External quality assurance is of course another worthy idea, though the approach advocated in the consultation is one of verification rather than one of moderation. Yet there is a sentence or two stating that if exam boards have concerns over the internal processes, they will look at the evidence used by schools and colleges to support their grading judgements. And where exam boards find that centres have not followed the guidance or that malpractice is suggested or has occurred, they have the right to alter grades.
The consultation also suggests that exam boards should verify the approach of every centre along with more targeted interventions, for example a new school or one where submitted results are unusually high or low. As for supporting public and indeed professional confidence, I see this as defensible and reassuring. But if the decision is taken to visit all centres, it really must be all centres. And if centres are otherwise targeted, they should be told why.
As an alternative, random visits might be more manageable, but care will have to be taken in selecting schools to avoid perceptions of guilt before the trial.
But again, this approach really does enforce the need for clear support and guidance for schools and colleges in assuring their approaches to grading, and the need for demonstrably robust procedures.
Appeals
From the outset, an appeals process has to be welcomed as a matter of principle. But the suggestion to bring results day forward to allow for appeals to begin early raises concern. On the one hand this seems quite reasonable, but as noted earlier, I’d like to think that concentrating on getting a fair process in place would help to reduce the number of appeals. And further, there’s perhaps too much focus on ‘teacher error’ rather than centre error. We then have the suggestion that appeals should be heard by a ‘competent person’ from within the centre. No matter how respected and worthy that person turns out to be, such an approach is unlikely to instil a feeling of impartiality. And even a person from outside the centre but appointed by the centre is unlikely to appease an angry parent or student. So if in the final analysis grades are to be actually ‘awarded’ or certificated by awarding bodies, surely they have to own appeals. My bet is centres are already receiving pressure from parents so, in the most difficult circumstances possible, we are pitting students and their parents against teachers. This has immediate and longer-term consequences for teachers and their institutions, none of which sound welcome.
Workload implications
The final element of the consultation on which I’d like to comment is that of workload. I’m thinking in the main in terms of the demands on teachers, but we should also consider awarding bodies – and importantly, students.
Teachers are already, to put it mildly, under the cosh. Teaching and learning is already under unprecedented strain and now having to implement a regime of assessment geared to the demands and expectations coming out of the consultation no matter how worthy and desirable is a big ask. For me, on balance, I lean to the approach of grading used by centres in 2020: it appears to have been broadly accepted. And if we really do trust teachers more than algorithms, why not use last year’s solution? Where I do have concern is that last year we saw high levels of grade inflation: truly this doesn’t do anyone any favours in the end. So some support in assisting schools to take a level of consistency in their approach is desirable. Support like understanding how to set standards and tackle bias, for example. And for those who think this contradicts my confidence in teachers, you have a point. Teachers in general are not well-versed in the technicalities of running high-stakes assessments. Not because they are not capable, but simply because they have not been shown how to. And setting a standard in your own class is not the same as setting a standard across a school, region or country. This is challenging stuff and despite what some may think, our system of regulated exam boards is really quite good. It’s just not as good during pandemics when last-minute plans are thrown together.
One thing I do want to say, and it may not be popular right now, but although I understand the heightened intensity around workload, I am minded that in my experience now approaching half a century in education, workload has often been raised as an issue around exams. I think this argument has been handled badly in the past. I’m not saying for one second that teachers are not over-worked, but I am advocating a profession that is trained and trusted to assess students.
During the 1960s and up to the early 1990s, teachers were trusted: they ran the Certificate of Secondary Education (CSE) for example, designed and ran Mode 1, 2 and 3 exams (younger readers will need to look this up), and some GCSEs had 100% coursework. I don’t remember too many complaints about workload. Perhaps the introduction of performance tables turned that tide, but that shouldn’t mean teachers should drop out of the high-stakes assessment equation. Workload is a massive issue, but that should be focussed on things that don’t really need to be done – like hours of marking with no proven link to improved learning, crunching daft data and producing flight paths. These things give assessment a bad name. These things are not ‘assessment’.
And let’s remember that exam boards are also under pressure here. Getting the right support in place in such quick time is, to say the least, a challenge. Looking back, we have failed to build robust moderation systems in normal times, so expecting this to happen in a few months is a long shot, but one we must return to at some point if we want to build a better and more valid system.
Then the students. We are in real danger of letting them down. Schools and colleges have bust a gut in developing distance learning whilst simultaneously conducting live sessions, but how has this looked to the pupils? Some students even in the same institutions have had very different experiences never mind across regions. How do they feel right now about their exams? There is real danger in cramming for evidence over the next few months, even in my preferred model. Teachers will no doubt manage this, but teaching to the test is about to take on a whole new dimension. I sincerely hope that before any school or college submits a response to the consultation, time is spent talking to their pupils.
What I think is worth doing right now
This is not meant to be preaching. But there are a few things that schools and colleges can do right now rather than waiting until the report on the outcomes of the consultation.
We are going to rely on teacher-based assessments one way or another, so centres can decide on Lead Assessors who can orchestrate preparations, drawing on last year’s processes, but also looking to how they can be improved or shown to be robust.
Accessing the Lead Assessor support programme offered by the CIEA will help to extend or confirm knowledge and provide confidence in the selected approach. In the longer term, Lead Assessors can go on to be Chartered Assessors: that will raise professional status and confidence.
Students’ work, from all sources but within the syllabus specification, can be collected: 1. to aid teachers in making and evidencing their judgments; 2. to identify gaps in key constructs – the big ideas in the syllabus and 3; to provide the basis for setting the standard to be used by the centre.
Subject expertise and professional challenge on the process of agreeing the standard has to be worthwhile, indeed a must. Moderation then follows; that is, checking that the agreed standard is being applied across the centre. If there isn’t an agreed standard, you can’t moderate. If centres can work together, this can only improve confidence in the process.
Data already available in centres should be collected and shared at subject- or department-level to build a profile over as many years as possible. Sure, exams have changed in recent years, but looking at the profile of results provides a basis to challenge or support decisions made in few months’ time. This is not applying an algorithm, it’s using data to inform, challenge and support professional judgements.
And do think about how exams and tests can be used as points of reference to aid professional judgement. Indeed I have labelled these as reference tests elsewhere to prevent confusion with the term ‘exam’. This may be too subtle, but at least in my mind it redesignates their position and purpose.
And finally, if you disagree with anything I have said, that’s fine – and understandable. But please don’t use it merely as a platform to complain and criticise, well not right now as I am more than happy to be corrected or challenged in due course: I’m still learning. Rather, convert the ideas or produce your own solutions and share them. We need action. We need leadership.
