Harnessing AI for better school reviews: Testing the feasibility of AI to improve quality, efficiency and fairness in school inspections

MATT DAVIS, GLOBAL MANAGING DIRECTOR, ETIO, UK

LEE NORTHERN, PRINCIPAL INSPECTIONS CONSULTANT, ETIO, UK

School inspections, resource challenges and the potential of AI

While the focus on generative AI (GenAI) in education and its potential has typically been on its use for school staff and pupils, its promise of bringing radical productivity benefits to complex tasks (Somers, 2023), also suggests the potential to apply this technology to other areas of work, including the work of school inspections. There may be the potential for it to streamline administration and improve the reliabilityIn assessment, the degree to which the outcome of a particular assessment would be consistent – for example, if it were marked by a different marker or taken again and quality of inspections, with consistency in OfstedThe Office for Standards in Education, Children’s Services and Skills – a non-ministerial department responsible for inspecting and regulating services that care for children and young people, and services providing education and skills inspection remaining a challenge in England, for example (Bokhove et al., 2023). However, there are questions about the extent to which we can trust AI’s outputs (Riley and Bruno, 2024). Ethical concerns, particularly data privacy issues (Madiega, 2019) and the potential misuse of biased or discriminatory insight (Vicente and Matute, 2023), are legitimate. For AI to be feasible in this context, it must add value without affecting quality and reliability.

Exploring the potential of AI through a feasibility study

We have carried out a series of experimental trials using commercially available Generative AI products on a range of activities across the lifecycle of a school evaluation. Our team ran upwards of 150 tests, using simulated inspections evidence under the United Arab Emirates inspections framework. After feedback on each output from experienced school inspectors, we iteratively refined the AI prompts until the outputs were at a level that the team agreed was of sufficient quality to be useful to field-based inspections teams. We then conducted a short field test, using some of the prompts in a non-statutory school review, with the permission of school leaders.

Early results are encouraging and indicate that AI can provide impactful assistance for inspectors in many ways, as evaluated by experienced inspection experts.

Discussion: Three positive predictions about the potential of AI in school review

At the simplest level, it is clear that AI’s ability to process and analyse vast amounts of data quickly, generate reports, plan inspector deployment and carry out time-consuming tasks can support improved efficiency of inspections. What we think is much more interesting, however, is the glimpse of what the technology might offer in terms of improvements to inspection and school review quality and impact. Here are three positive predictions for what could be achieved within a very short period of time if AI is used to enhance the inspection process.

1. AI will increase the trust and confidence that school leaders have in inspection through improvements to reliability and transparency

Where there is a lack of trust in inspections, or the process feels opaque, unfair or monodirectional, there is unlikely to be lasting improvement or buy-in (Leeuw, 2002). Reliability is therefore a crucial factor in earning the trust of schools in the process (Nelson and Ehren, 2014). There is a very real risk that bringing AI into the process will be perceived as heightening these concerns.

However, any increase in the efficiency with which inspections can be delivered provides additional time for inspectors to invest more in understanding context, exploring complex issues and building relationships. This could actually increase trust and credibility and strengthen the relationships between inspectors and those they inspect (Leeuw, 2002).

2. AI will improve the validityIn assessment, the degree to which a particular assessment measures what it is intended to measure, and the extent to which proposed interpretations and uses are justified of inspections by helping more practising school leaders to become part of the school review workforce

Those applying inspection framework standards must be knowledgeable, skilful and – crucially – credible enough to ensure the validity of the use of these measures.

Inspectors’ expertise and credibility increase validity and benefit the schools they work with. Our feasibility study suggests that ‘co-pilot’-type solutions – AI that prompts users during the live task of inspection – might help to upskill new inspectors to ensure that the evaluations and judgements that they make are rigorous, robust and fully consistent with inspection methodology. Equally, AI can also be used to provide an initial assessment of the appropriateness of lesson or curriculum content against national standards and age-related expectations, checking alignment with statutory regulations.

While human quality assurance by expert inspectors will still be needed, AI scaffoldingProgressively introducing students to new concepts to support their learning of this type might provide sufficient support so that individuals with excellent education credentials – but perhaps limited or no inspection experience – can participate in inspection much more quickly and effectively than is currently the case.

3. Insights from AI-enhanced inspection will dramatically amplify the potential for school and system improvement

Repeatedly auditing schools does not make them any better at educating children. AI allows us to see a future where the review process helps to drive improvements to school quality, alongside generating greater and more reliable insight for policymakers and governments. We have seen sufficient evidence in our tests to believe that AI can identify patterns and anomalies that might not be detected by human inspectors, creating deeper insights into school performance and system-level effectiveness.

For example, the volume of evidence gathered in a typical school inspection frequently presents a significant challenge to inspection teams. Therefore, a process of summarisation is essential in making evidence manageable during inspection. This is a time-consuming process if conducted manually, and inspection teams face the risk that important insights and connections may be lost or diluted. AI-supported inspection teams could access these summaries instantly, leaving them time to direct their attention to the judgements and recommendations.

Conclusion

Based on our experimental trials, we are optimistic that AI tools can create substantial productivity benefits and allow different options for how we resource inspections. If we can use AI tools to create more time, space and evidence to inform an improvement-focused dialogue between leaders and reviewers, this is likely to have positive benefits for both parties. 

Etio’s full report, ‘Harnessing AI for better school reviews’, can be accessed at: https://info.etioglobal.org/ai-in-review-whitepaper

The examples of AI use and specific tools in this article are for context only. They do not imply endorsement or recommendation of any particular tool or approach by the Department for EducationThe ministerial department responsible for children’s services and education in England or the Chartered College of Teaching and any views stated are those of the individual. Any use of AI also needs to be carefully planned, and what is appropriate in one setting may not be elsewhere. You should always follow the DfE’s Generative AI In Education policy position and product safety expectations in addition to aligning any AI use with the DfE’s latest Keeping Children Safe in Education guidance. You can also find teacher and leader toolkits on gov.uk .