The problematic interface between research, policy and practice: The case of attainment grouping

Since 1997, UK governments have advocated evidence-based practice in English education policy (DfEDepartment for Education - a ministerial department responsible for children’s services and education in England, n.d.) ; (also see Ball (Ball, 2013) and Whitty (Whitty, 2016) for a discussion). However, we have yet to see this translation of research into practice realised. In this article, we use the example of attainment groupingAlso known as ability grouping, the practice of grouping students according to measures of attainment as a case with which to explore some of the wider challenges around the interface between research, policy and practice, and the various impediments to a seamless journey between research findings and change on the ground.

Grouping in schools 

While comprehensive education to age 16 is the norm for most young people in England, the majority of secondary schools, and increasing numbers of primary schools, operate in-school grouping by attainment. In secondary schools this is usually in the form of setting, where pupils are grouped by prior attainment for certain subjects, while some schools practise streaming, where pupils stay in the same ‘ability’ groups for most subjects; many operate a mixture of the two (Taylor et al., 2017). In primary schools, grouping by attainment usually takes the form of within-class ‘ability tables’, but research indicates that setting and streaming are also increasingly practised (Hallam and Parsons, 2013a); (Hallam and Parsons, 2013b).  

Where grouping by attainment is used, there is often conflation of the concept of prior attainment with ‘ability’. Young people enter the education system from very different background circumstances, resulting in different starting points, so to see their attainment as an indicator of raw ability is problematic. We adopt the term ‘grouping by attainment’ wherever possible, rejecting conceptions of ‘ability’ as ascribed and fixed; we see ability as malleable, and prior attainment as reflecting a range of societal factors that impact on educational progress and outcomes.  

Despite the prevalence of grouping by attainment, long-standing international evidence shows that this practice does not have a significant impact on outcomes overall; further, it has a significant negative impact on those in ‘low ability’ groups, who make less progress than peers in higher attainment groups, and who perform better in mixed attainment classrooms. The Education Endowment Foundation Toolkit states that: 

On average, studies show that higher attaining learners make between one and two additional months’ progress when set or streamed compared to when taught in mixed ability groups. […] Low attaining learners fall behind by one or two months a year, on average, when compared with the progress of similar students in classes with mixed ability groups. (Higgins et al., 2015)

Slavin’s (Slavin, 1990) systematic review of international research found that the effects of ability grouping on achievement are statistically insignificant, and more recent studies (Steenbergen-Hu et al., 2016) have confirmed this (although conclusions are not without contention). This overall finding could be interpreted as suggesting that ‘ability grouping’ neither helps nor hinders, but as we shall see, there is a negative impact for the smaller proportion of pupils in lower sets, a majority of whom are shown to be from low socio-economic backgrounds.  

The significance of attainment grouping 

Educational attainment in England is highly stratified by social class. There is an especially strong relationship between family wealth and educational outcomes, and between family wealth and post-education occupational outcomes. Obviously, wealth inequality, as well as the impact of socio-economic background on children’s unequal starting points when they begin school, has a strong impact on achievement. However, socio-economic attainment gaps then widen rather than narrow as children progress through school, suggesting that, if anything, schooling exacerbates rather than mitigates inequalities in attainment outcomes. There are exceptions, of course – the evidence that schools can make a positive difference is provided by the many schools that facilitate high achievement for disadvantaged pupils. 

Working class pupils and those from some minority ethnic groups are disproportionately represented in low sets and streams. Although this trend might be predicted given the impact of social inequality on what is often referred to as ‘school readiness’ (meaning that children from low socio-economic status backgrounds tend to start from a lower baseline), practices of allocation may also not necessarily be reflective of ‘ability’ (Muijs and Dunne, 2010).  

Given the research suggests students placed in low attainment groups make less progress than their peers in higher groups, working class pupils in low attainment groups are subject to a double disadvantage; they are disadvantaged on arrival at school, but then subject to practices shown to further limit their chances of progress and attainment. We have conducted a detailed analysis of the research literature and, from this, have distilled seven explanations for poor outcomes of students in lower sets and streams (see Box 1; (Francis et al., 2017)). 

Lack of impact of research on policy and practice 

So how can we account for the fact that this research has had so little purchase? We propose the following explanations: 

Mixed messages 

Our analysis of key policy documents and speeches alluding to pupils’ grouping since 1997 shows that, within the semiotics of policy discourse, setting has somehow become a signifier for ‘academic high standards’. Setting became emblematic of the New Labour ‘standards’ agenda, and was notably advocated within the incoming New Labour government’s first White Paper, ‘Excellence in Schools’, including a statement that: 

[…] unless a school can demonstrate that it is getting better than expected results through a different approach, we do make the presumption that setting should be the norm in secondary schools. (DfES, 1997) 

So, despite the lack of evidence, we have an association and conflation of setting with ‘standards’ in the policy documents we analysed (see (Francis et al., 2017)); the case of attainment grouping illustrates how policy recommendations to schools are not always based on research evidence.  

In schools, even where there is some awareness of what research evidence says, it may not always filter into practice across the whole school. In large secondary schools, different departments often adopt markedly different approaches, and there may be few mechanisms for the sharing of pedagogic practices and experiences between them. Knowledge and/or implementation of evidence-informed practice may therefore become contained in certain departments. 

Challenges with the body of research  

There are some vulnerabilities and challenges within the literature on grouping by attainment that also hinder traction. For example, the literature often conflates different types of attainment grouping, and there is a lack of disaggregation of different factors posited as potentially detrimental. We still need answers to the following questions:  

• Which of these identified explanations have the greatest impact on pupil experiences and outcomes? 
• What is the impact on achievement for students in low sets if detrimental setting practices are mitigated? 
• What actually constitutes good practice in mixed-attainment teaching and grouping? 
• Which of the good practice alternatives is more effective in improving the attainment of low-achieving students? 

The lack of clarity can hinder traction on practice, especially when there are few pointers or little systematic research on effective practice. It is surprising how little has been written on best practice in mixed attainment (‘mixed ability’) practice. Without support and successful exemplars, it is unlikely that many teachers will feel able to embark practice that differs from long-established ways of doing things.  

Structures dictate content 

We have found from extensive interviews with teachers that setting practices are often driven by practical constraints that impede or over-ride the application of evidence-informed practice. For example, the school timetable may be set up so that sets have to be the same for mathematics and science, while student attainment might differ significantly between subjects. Similarly, staffing or finances can constrain setting arrangements, determining class sizes, the availability of subject specialists and the number of subject lessons that can take place simultaneously. Big changes to structures can be seen as too difficult, and so even when schools identify setting as the intention, streaming or partial streaming may be forced (Taylor et al., 2018). 

Knock-on effects of contextual factors 

Interviews with teachers have also revealed that contextual factors for schools have a knock-on effect on setting practices. For example, many teachers told us that they didn’t trust Key Stage 2 results or primary school assessments. This led to the ‘tweaking’ of sets and reliance on internal testing or subjective judgements that, while seen as more trustworthy, allow bias to be introduced. Likewise, the logistical issues mentioned above were often shaped by local contextual approaches. For example, our requirement of fluidity between sets was very difficult in some schools, where timetabling arrangements or the requirement to negotiate movement with colleagues in another department made movement impossible. Finally, movement was actually seen as so undesirable in some schools that teachers construct support structures to reduce movement and keep students in their initially allocated set (Taylor et al., 2018). 

Addressing challenges through research 

Our present research project, funded by the Education Endowment Foundation, seeks to address some of the identified challenges, especially that of the lack of disaggregation of explanatory factors in the literature. The Best Practice in Grouping Students project focuses on mathematics and English in Key Stage 3. The project comprises two studies with randomised controlled trial design: Best Practice in Setting, a fully-powered RCT involving 126 schools, and Best Practice in Mixed Attainment Grouping, a feasibility study involving 13 schools. 

The Best Practice in Setting intervention set out to mitigate the factors associated with the poor performance of lower sets and streams (see Box 1) by establishing and maintaining sets strictly according to attainment and with a maximum of four levels, and by providing professional development to teachers to encourage high expectations of all students and a flexible approach to ‘ability’. These practices are contrasted with a control group, who maintain their usual setting practices. 

The Best Practice in Mixed Attainment intervention provided teachers with support for teaching students with a broad range of prior attainment together in the classroom. We developed this support in collaboration with three schools recognised for their excellent mixed-attainment teaching practice. These practices are again contrasted with a control group who maintain their usual grouping practices. 

The intervention and control groups will be compared for outcomes in terms of both academic attainment and student self-confidence. Our data collection is now complete and we expect to report on our overall outcomes later in 2018, contributing answers to the questions outlined above: the relative impact of explanations for differences in attainment, the possibility of mitigating detrimental setting practices, characterising good practice in mixed-attainment teaching, and the relative impact of setting and mixed-attainment grouping. 

For our part, we hope that the recognition and exploration of some of the issues around attainment grouping in our present study will help us in making a significant contribution to the field, by aiding our reflection on optimal approaches and factors to consider going forward, in order to think with practitioners and facilitate good practice in the important area of grouping pupils. 

The Best Practice in Grouping Students team also includes: Professor Louise Archer, Professor Jeremy Hodgen, Dr Anna Mazenod and Dr Antonina Tereshchenko, UCL Institute of Education; and Professor Paul Connolly, Dr Sarah J Miller and Dr Nicole Craig, Queen’s University Belfast.