Can we help improve wider school outcomes through youth social action?

Stephen Gorard, Beng Huat See and Nadia Siddiqui, Durham University Evidence Centre for Education, UK

This short piece looks at two large, randomised trials attempting to improve attitudinal and behavioural outcomes in school, especially for disadvantaged pupils. It looks at the context, the need for this new evidence, what we did, what we found, and what the results might mean for schools interested in wider outcomes like pupil self-confidence.

Thinking about wider outcomes

In the past 10 years or more, in England, there has been an increased interest among policy-makers and others in schools’ outcomes that are additional to attainment. They have used terms like ‘grit’, ‘character building’ and ‘moral values’. Some of these concepts, such as ‘knightly values’, have been queried for their relevance and inclusiveness. Some are unclear as to whether they are necessarily a good thing in their own right – resilience could be stubbornness, autonomy, an unwillingness to listen to experts and so on. Sometimes these character and attitude outcomes have been seen incorrectly as important – and perhaps cheap – ways to improve attainment for some groups of pupils, by increasing aspirations and expectations perhaps. They generally do not do this (Gorard, 2012). Sometimes they have been more properly regarded as important to consider in their own right – such as pupil wellbeing or the enjoyment of school (Gorard and See, 2011). But can these wider outcomes be taught? If so, how, and what is their relationship, if any, to pupil attainment at school?

One structural prerequisite could be a mixed intake to schools in socioeconomic terms (Gorard, 2018). This appears to establish the most fruitful conditions for improving wider outcomes. It is, in general, harder to teach about religious tolerance in a sectarian school, or to teach about respect for race, gender or disability in a segregated school setting, and so on. Over and above an equitable structure, it would also help for schools to exemplify via their processes the values that they hope to teach. This means obvious things like giving pupils a voice in running the school if you hope to help civic participation in the future, for example. Beyond the obvious, the most important single factor lies in respectful everyday interactions between teachers and pupils (Gorard and Smith, 2010).

However, most of the evidence on the aforementioned matters has not been from a research design appropriate to making clear causal claims. Most research worldwide has been either in-depth, merely asking stakeholders what they thought, or correlational, showing that this or that condition is associated with improved attainment or wider outcomes. Surveys and interviews are not the answer. What has been needed for some time is work that takes the promising approaches from these weaker designs and tests them more robustly. The Institute of Education Sciences, set up in the US in 2002, and subsequently others worldwide, has pushed the case to try to settle some of the most basic education research questions by using more robust designs like randomised controlled trials. Most of their work so far has concerned attainment. Since around 2014, the Education Endowment Foundation in England (set up in 2010) has funded studies to look at wider outcomes as well, and a number of these have involved youth social action.

Two youth social action interventions

Youth social action is where children and young people are involved in activities that may benefit their family, community and society. Clearly, volunteering activities can be undertaken individually, but they can also be encouraged by schools through extra-curricular sessions, perhaps including cadet forces, Scouting, St Johns’s Ambulance Brigade training or similar, and enrichment experiences. Out-of-school activities (breakfast clubs, sports activities, extra music and art lessons, visits and religious services) might be able to improve attainment at school (Chanfreau et al., 2016). Linking them to social action could improve wider outcomes as well, such as greater employability, self-esteem and confidence (Ward et al., 2009; Kirkman et al., 2016).

However, these enriching experiences are perhaps more frequent in low-disadvantage and private schools (Birdwell et al., 2015). Such activities have a negative association for some less-privileged students, and are seen as the preserve of the middle class (Bradbury and Kay, 2005). Disadvantaged pupils may not get the same opportunities, some of which involve a cost for participation, and all of which require either additional staff or extra work from existing staff. There is a poverty gap in access to activities such as after-school clubs, arts and cultural events, and volunteering and community-based projects (Southby and South, 2016).

We evaluated, via randomised controlled trials, the impact of two different approaches to youth social action in schools. One was Youth United, which helps to organise a range of uniformed activities such as Sea Scouts and St John’s Ambulance Brigade in schools. The other was Children’s University, which supports and badges extra-curricular activities, outdoor learning activities, after-school clubs and community social action, taking place via schools and leading to a graduation ceremony. Each was provided free to the participating treatment schools for one year.

There was some prior evidence that both of these kinds of activities might be beneficial in terms of both attainment and outcomes such as satisfaction and attendance at school (e.g. MacBeath, 2012). However, this was generally based on volunteers whose families could afford to pay for the activities, and there was no fair comparator group. Children who are more likely to attend these activities are from families in a higher socioeconomic group, and so are already more likely to have better outcomes at school (Cheung, 2016). There are few evaluations with suitable comparators, and the evaluations that have taken place have often not been rigorous enough to assess the impact convincingly (Booth et al., 2015). We wanted to improve that evidence base.

What we did

There is no space in this brief paper to describe the interventions in full, the prior evidence in the literature, the methods, the in-depth work, or more detail than the headline findings. Further details can be found in See et al. (2017) for the Youth United trial, and in Siddiqui et al. (2019) for Children’s University.

The Youth United trial was conducted with the help of 61 secondary schools in England (38 treatment and 33 control), in which 1,733 pupils in treatment schools and 1,644 pupils in control schools indicated in an initial survey before randomisation that they would like to take part in the kinds of activities provided via Youth United. These ‘volunteers’ from Year 9 provide the main basis for the comparison that follows. Around one per cent of cases are missing final scores. The intention was to use Key Stage 2 scores as a measure of prior attainment, and Key Stage 3 scores as the attainment outcome. However, due to national reforms, the Key Stage 3 data was not available consistently from all schools in that year, and the attainment results are not reported here.

The Children’s University trial took place with the help of 68 primary schools in England (36 treatment and 32 wait-list control). As with Youth United, pupils were asked before randomisation whether they were interested in the kinds of activities associated with Children’s University. There were 670 ‘volunteers’ from Year 5 in the treatment schools, and 588 in the control. These are the main basis of comparison for the results. After two years, two per cent of the original pupils in each group had moved schools and were untraceable for Key Stage 2 maths and reading scores. Prior attainment for both groups came from their Key Stage 1 scores.

Both trials used the same bespoke survey instrument, with all relevant pupils completing it before randomisation and at the end of the trial. The instrument included basic questions about whether respondents had participated in any youth social action activities similar to those offered by Youth United or Children’s University, and how keen they were to undertake such activities. In addition, the instrument contained a set of questions on a range of wider outcomes covering concepts including teamwork, communication, motivation, self-esteem, confidence, resilience, civic mindedness and future intentions. Some items were based on short stories (vignettes) in which the socially desirable responses were not as clear as in scaled tick-box questions. The questionnaire was designed for ease and speed of completion to encourage full response and to prevent drop-out caused by fatigue or frustration.

Summary findings

The range of social action activities involved was wide, including first aid, firefighting, charity fundraising, making packs for donations to food banks, collecting unwanted household items for charity shops and participating in awareness walks. Pupils in one school grew vegetables, which they sold to raise money for donations to a cancer research foundation. Pupils reported:

We help younger children in our school learn new games in the playground. It is real fun and we like helping them to learn exciting games.

I volunteered to help in a local library and it gave me a chance to meet new people. I also learned about books and how to place them in the bookshelves.

I volunteered to help organising a show for St John Hospice. It was a competition for the disabled. It was so great to help people who were helping others.

We want our streets to be clean and no litter around. We want no graffiti on the walls. We have made these posters so that people should see that we care about our area.

We know we can’t see children living in countries where there is war but we can at least collect money for the charity so they can buy food and clothes for children in need.

My school is very nice. We play and do a lot of fun activities. We have raised funds for school buildings in Uganda so that children go to schools like mine.

The traditional ‘effect’ sizes, expressed as fractions of a standard deviation difference between the intervention and the control group, are small but positive for all outcomes from both interventions (Table 1). It is presumably hard to demonstrate the impact of a youth social action programme in a short time (Youth United lasted less than one school year). Taken together, the figures suggest that there is a benefit from both programmes in terms of the previously selected outcomes.

Table 1: ‘Effect’ sizes from two trials of non-cognitive outcomes

In addition to greater reported gains in teamwork and self-confidence in comparison to the control group, Youth United participants also showed greater gains in empathy for others and professional aspirations. Unfortunately, these gains are smaller or non-existent for the disadvantaged FSM-eligible (free school meals) pupils. If true, these results suggest that while uniformed activities would have general benefits, they could actually increase the gap in wider outcomes between disadvantaged pupils and others.

The reverse seems to apply to the Children’s University (CU) results. There are small possible benefits in terms of reported teamwork and social responsibility, but the gains in comparison to the control are greater when considering the disadvantaged sub-group of FSM-eligible pupils. If true, this means that an extra-curricular activity and recognition approach could provide general benefits and actually reduce the poverty gap in wider outcomes. In addition, the CU group also became more empathetic and increased their professional aspirations, in comparison to the control pupils.

Turning to attainment, the situation reverses again. The gains in attainment are more substantial and based on more straightforward data than those for wider outcomes like teamwork. But these gains are negligible for the FSM-eligible sub-group. Put another way, it may be that the more disadvantaged pupils gain more from some youth social action in terms of wider outcomes, while their peers gain more in attainment.

Implications

For schools whose primary goal is to raise attainment, then there are almost certainly more assured and definitely cheaper approaches than the two discussed here. And if schools are interested in reducing the poverty gap in academic attainment, then there are no indications of promise from these programmes. If, however, schools are chiefly interested in the poverty gap in these enriching experiences and the wider outcomes they may support, then any gain in attainment is a bonus.

Describing these two programmes as has been done here is not meant to suggest that they are the only or even the best ways to achieve the same results of improved pupil confidence, etc. They are simply the ones we tested. Most of the activities described could be set up by schools, groups of schools or other agencies.

The programmes appear to produce consistent small gains in children’s motivation, aspiration and attitudes related to education. These will be somewhat muted, since not all pupils in the treatment schools actually took part in the intervention. But the two interventions described here show that giving pupils new experiences could enhance their empathy, confidence and aspirations as good things in their own right, and one programme suggests that such school experiences can reduce the socioeconomic status (SES) stratification of such non-cognitive outcomes.

Therefore, schools concerned about these wider outcomes should take steps to investigate further and try out cost-effective approaches like the ones here. We are happy to discuss these issues further with schools, by email or at invited workshops.

References

Birdwell J, Scott R and Reynolds R (2015) The double benefit of youth social action could help to tackle some of our most social problems. Service Nation 2010. London: DEMOS. Available at: https://dera.ioe.ac.uk/24131/1/ServiceNation2020.pdf (accessed 10 March 2020).

Booth C, Shrimpton H, Candy D et al. (2015) National Citizen Service 2014 Evaluation. Ipsos MORI. Available at: www.ipsos.com/ipsos-mori/en-uk/national-citizen-service-2014-evaluation (accessed 10 March 2020).

Bradbury S and Kay T (2005) Evaluation of the Pupil Centred Stages of Phase One of the Step Into Sport Project. Loughborough: Institute of Youth Sport.

Chanfreau J, Tanner E, Callanan M et al. (2016) Out of school activities during primary school and KS2 attainment. Centre for Longitudinal Studies: Working paper 2016/1. Available at: https://eprint.ncl.ac.uk/file_store/production/218418/42354AD4-8363-48F0-BFA1-1358B69AA7A0.pdf (accessed 10 March 2020).

Cheung P (2016) Children’s after-school physical activity participation in Hong Kong: Does family socioeconomic status matter? Health Education Journal. DOI: 10.1177/0017896916660863.

Gorard S (2012) Querying the causal role of attitudes in educational attainment, ISRN Education 2012: Article ID 501589. DOI: 10.5402/2012/501589.

Gorard S (2018) Education Policy: Evidence of Equity and Effectiveness. Bristol: Policy Press.

Gorard S and See BH (2011) How can we enhance enjoyment of secondary school? The student view. British Educational Research Journal 37(4): 671–690.

Gorard S, See BH and Siddiqui N (2017) The Trials of Evidence-Based Education. London: Routledge.

Gorard S, Siddiqui N and See BH (2017) Can ‘Philosophy for Children’ improve primary school attainment? Journal of Philosophy of Education 51(1): 5–22.

Gorard S and Smith E (2010) Equity in Education: An International Comparison of Pupil Perspectives. London: Palgrave.

Kirkman E, Sanders M, Emanuel N et al. (2016) Evaluating youth social action: Does participating in social action boost the skills young people need to succeed in adult life? London: Behavioural Insights Team. Available at: www.bi.team/wp-content/uploads/2016/01/YSA-Report-Final-Version1.pdf (accessed 10 March 2020).

MacBeath J (2012) Evaluating provision, progress and quality of learning in the Children’s University. Fourth report to the CU Trust. University of Cambridge. Available at: www.childrensuniversity.co.uk/media/1092/cu_evaluation_2012-13_full.pdf (accessed 10 March 2020).

See BH, Gorard S and Siddiqui N (2017) Does participation in uniformed group activities in school improve young people’s non-cognitive outcomes? International Journal of Educational Research 85: 109–120.

Siddiqui N, Gorard S and See BH (2019) Can learning beyond the classroom impact on social responsibility and attainment? An evaluation of the Children’s University youth social action programme. Studies in Educational Evaluation 61: 74–82.

Southby K and South J (2016) Volunteering, inequalities and barriers to volunteering: A rapid evidence review. Leeds Beckett University. Available at: http://eprints.leedsbeckett.ac.uk/3434/1/Barriers%20to%20volunteering%20-%20final%2021st%20November%202016.pdf (accessed 10 March 2020).

Ward F, Thurston M and Alford S (2009) RESPECT: A personal development programme for young people at risk of social exclusion – final report. Chester: University of Chester. Available at: https://chesterrep.openrepository.com/handle/10034/85510 (accessed 10 March 2020).