Context, mechanism, outcome: How one primary school evaluated the efficacy of teaching strategies in a knowledge-rich curriculum

In 2018, we introduced Impact readers to our initial work designing and implementing a knowledge-rich curriculum at Cottenham Primary School (CPS) (Dennis and Kilsby, 2018). At that time, we were focused on striking a balance between teachers’ subject knowledge and the pedagogies that best support retention across primary domains. We recognised that ‘the next phase of our research will necessarily include children’s progress as a means of addressing the “so what?” questions that emerge from a curriculum overhaul’ (p. 23). In this article, we share the process we’ve developed for evaluating the impact of the knowledge-rich curriculum and how we studied whether specific cognitive teaching practices supported children’s learning across domains.

The challenge of curricular efficacy

The challenge of being able to talk with authenticity and authority about the efficacy of our curriculum required us to revisit how we define the curriculum at CPS, and to then enshrine what we do in our ‘First Principles of the Curriculum at CPS’ document. The ‘First Principles’ were designed to provide a clear, succinct and non-negotiable set of guiding principles for us when considering the content, delivery and assessment of the curriculum at CPS.

At the heart of this work is our commitment to having clear definitions of success for children, within a specific domain, and ensuring that, by using a range of indicators, all teachers are well positioned to answer the question: ‘have the children learned what we have taught them?’

These definitions of success (known as ‘intended outcome statements’) were written by the curriculum coordination teams that replaced the old system of subject leadership at the school. These teams are a triad of two teachers, known as curriculum coordinators (CCs), and a senior curriculum lead (SCL), who is also a member of the SLT. The development of the role has been one of the most significant so far in the school’s curriculum journey, placing, as it does, the CCs in the vanguard of design and delivery for their specific curriculum area.

Curriculum coordination

In addition to dramatically clarifying the responsibilities of so-called ‘middle leaders’ (something that we had previously struggled to adequately convey to inspectors), the model has also provided a powerful catalyst in establishing whole-school awareness for all teachers. Prior to the curriculum coordinator role, the level of a teacher’s understanding would tend to be within their teaching year group. With the expectation that CCs are now responsible for the provision in their subjects across the whole school, their understanding now transcends all year groups, with teachers responsible for designing a coherent and logical curriculum pathway, and one that should ensure that children are equipped with the necessary substantive and disciplinary knowledge within a domain that we deem essential by the end of Year 6.

The investment placed in the development of the curriculum coordinators is already showing a healthy return – one that could only have been achieved with the collaboration of a highly committed, capable and increasingly confident cadre of teaching professionals. This has required a detailed and cohesive strategic approach that builds on the school’s curriculum journey and integrates existing and planned initiatives into the arsenal of current systems and structures employed at the school. This includes strategic plan objectives, the programme of in-school monitoring, the appraisal cycle and, most importantly, bespoke professional development.

Indeed, alongside the necessity for the alignment of staff and governors with the school’s stated curriculum aims, the focus on high-quality professional development is the priority for school leaders. Put simply, without this, the phenomenally hard work of teachers and TAs at CPS would be in vain and lacking a coherent focus on how to identify, achieve and secure highly effective teaching.

Nowhere is this more evident than in the development of our chosen mechanism for assessing the impact of our curriculum.

As stated earlier, our target is to ensure that by the end of Year 6, all of our children are at least secure with the intended outcome statements for each domain. There are no other details: no milestones or descriptors. It is all about knowing where we are aiming for and then using our professional judgement to make accurate assessments about where individuals, groups, cohorts and the whole school are on that trajectory, by using a suite of indicators, not least the pupils’ work.

The challenge, however, was to find a methodology that enabled us to take this broad range of qualitative and quantitative information and to shape it into an accurate and reliable narrative, whilst all the time making it a tool that teachers believe in, and that does not set unreasonable expectations in order to prop up a dubious assessment system, which may risk fast-tracking them to burn out.

‘Context Mechanism Outcome’

The solution came from an unexpected source. When discussing the challenge with a group of governors, one of them asked whether James was aware of ‘realist evaluation’ and the work of Pawson and Tilley (1997), which he wasn’t. The resulting research led to the ‘Context Mechanism Outcome’ (CMO) model that we are now developing at CPS.

In this model, our CCs posit a hypothesis about the impact of a specific intervention or initiative on the outcomes for an identified group of pupils within their domain. This is called the ‘programme theory’. They then define the class/year/group of pupils that they are going to investigate against the the programme theory. This is the context. The mechanism is the teaching sequence, technique or resource that has been delivered. They then identify the evidence base (including pupils’ work, their views, teachers’ planning and thoughts, lesson observations, outcomes from any additional diagnostic assessment and statutory reporting stage information) and this will be used to define the outcome.

The final CMO report is a one-side report of the process and the findings. In addition to answering the question posed by the programme theory, it also establishes what was measured, what findings were uncovered and any recommendations or next steps that the CCs have identified.

CMO reports

We selected three CMO reports to highlight in this article: English, French and history, as we believe these to represent a foundational subject (English), a subject that best represents our commitment to the knowledge-rich curriculum (history), and a subject that is often marginalised in the primary curriculum (French). Table 1 outlines the programme theory, context and mechanism for each of these domains.

Table 1: ‘Context Mechanism Outcome’

Our English curriculum is currently centred around Talk for Writing (TfW). The English CCs focused their data collection on whether TfW ‘provided sufficient opportunity to optimally extend the accuracy and sophistication of children’s independent writing’ (English CMO report). Although their hypothesis was that TfW did not offer opportunities for children to write creatively, their findings indicated that children ‘do have a sense of ownership of the direction and outcome of their writing’. This led to the English CCs to recommend that TfW remain as a structure, but with adjustments to allow for more intentional connections between English and other domains.

The French CCs investigated the teaching of grammatical features in Years 5 and 6. They found inconsistencies across the Year 6 team, making it challenging for children to retain and retrieve key grammatical facts. In response, the French CCs revised the curriculum map, beginning in Year 3, using a model of impart knowledge, practise, consolidate and revise, and did so with the goal of increasing sophistication through Year 6.

Looking across Years 2 and 4, the history CCs noted that ‘retrieval practice is incorporated in each lesson (usually in the form of quick quizzes) and good use is made of dual codingIn qualitative research, coding involves breaking down data to support all learners’ (history CMO report). However, in Year 4 there were ‘few deliberately planned opportunities to make links back to previously taught lessons’ (history CMO report). Connecting back to the English findings, history CCs recommended alignment with the writing curriculum to support children in crafting balanced and reasoned arguments.

Next steps

It is important to state that this work is still in its infancy, and we are learning all the time. From the first ‘round’ of CMOs, it is clear that teachers have fully engaged with the logic behind the process and with the potential that it has to drive school improvement. In interviews with curriculum coordinators following the CMO process, CCs described their role as ‘grassroots research’, and they expressed feelings of ownership and responsibility for their subjects. One CC stated, ‘It’s really important that somebody is championing each subject… the role of the coordinator is to put their subject at the forefront of everybody’s mind.’ (Dennis, 2019) One of the main drivers of this engagement was the link that teachers made to their own professional development as a result of the CMO process, which allowed them to feel more confident in supporting colleagues across year groups. Very few limitations of the CMO process were noted by the staff. Initial uncertainty of the process led to some ‘cognitive overload’ as teams began to explore their subject areas. However, as they worked through the process with their SCL, they felt that they had enough time and support to successfully address the task.

The opportunity that the CMO model provides for genuine insight into the impact of individual aspects of agreed practice would appear to be considerable. This can range from specific approaches within an individual subject (e.g. the use of sketchbooks to support the development of artistic techniques) to broader, school-wide initiatives. This year we are progressing with a school-wide programme theory that explores whether systematic and consistent application of retrieval practice(s) will support low prior attaining pupils’ knowledge retention. Each CC team will explore this theory within their domain and we will then look across domains for evidence of practices that support and inhibit knowledge retention. Feedback from CCs was the driving force behind the decision to move to a shared programme theory. Consistently, questions of appropriate pitch of both content and assessment across and within year groups, as well as questions as to whether we were addressing the needs of all pupils, were present in conversations and interviews with CCs. One goal of this shared theory study is to determine the appropriate scope and sequence for teaching across domains, from EYFS to Year 6, within a knowledge-rich curriculum.

References

Dennis D (2019) Research findings: Executive summary 6. Available at: https://powerfulknowledgepowerfulpedagogy.com/2019/09/11/research-findings-executive-summary-6 (accessed 14 October 2019).

Dennis D and Kilsby J (2018) Designing a knowledge-rich curriculum: Where we’ve been, where we are, where we are going. Impact 4: 21–23.

Pawson R and Tilley N (1997) Realistic Evaluation. London: Sage.