Fostering maths fluency with the use of EdTech

Conflict of interest statement: This study was carried out by Sumdog, a company providing free and paid versions of their software to schools.

The study design and planned analysis were reviewed by an independent academic and registered with the Open Science Framework prior to any data collection. The registration, along with associated documents, can be found at https://osf.io/3jyhe.

Maths fluency, defined as the ‘ability to recall and apply knowledge rapidly and accurately’ (Department for Education, 2014), is promoted by regular practice of concepts already taught (National Council of Teachers of Mathematics, 2014). Fluency gives students the capability to be confident in their calculations and the cognitive capacity to focus on more complex, problem-solving aspects of the curriculum (Foster, 2019). In recognition of the lack of secure fluency in upper-primary school, and in an attempt to ensure a real focus on improving this, the government in England introduced a compulsory ‘multiplication tables check’ for all students at around eight years old (Standards and Testing Agency, 2018). This means that teachers are required to dedicate additional time to practising and assessing these skills, alongside the competing priorities of the rest of the curriculum. All this is within a climate where excessive workload is recognised as being a major contributing factor to stress and poor retention (Department for Education, 2019). For students who need extra help, time and resources for implementing interventions are also limited.

EdTech offers a way to reduce teacher workload by providing automatic setting and marking of work, and provides intervention requiring little or no additional teacher time. This study found a real and quantifiable impact of time spent using one such online maths resource, Sumdog Maths, a version of which is available free to all schools. Sumdog works by learning the level at which the student is working, then presenting them with multiple choice questions in conjunction with engaging and varied gameplay. When a rolling accuracy of 80 per cent is achieved, this topic is marked as ‘mastered’ and the student moves on to another. Topics are interleaved as revision at regular points to promote long-term retention.

Study design

A quasi-experimental, longitudinal study measured the progress of 221 seven-to-eight-year-old P4 students (equivalent to Year 3 in England and 3rd Grade in the US) in nine primary schools in Glasgow, UK, across the academic year 2018–19. The quasi nature of the study was adopted to ensure that no students missed out on the potential benefits of using the resource by being allocated into a control group. Two pencil-and-paper measures of maths ability were taken:

1. Maths fluency – a test of accuracy and speed in arithmetic. Students were given three minutes to answer as many items as they could, starting with addition and subtraction and moving on to multiplication.
2. Applied maths – an untimed MaLT (Mathematics Assessment for Learning and Teaching) test, requiring students to interpret questions and apply wider mathematical knowledge (hoddereducation.co.uk/malt). Figure 1 shows example items from each task.

Maths measures were taken at the start and end of the study, and amount of time using the resource was recorded throughout. Students were split into self-selecting groups, with ‘regular’ users defined as those who used it for at least 30 minutes a week for at least half of the weeks between the testing points. Remaining students were classed as ‘non-regular’ users. The ‘regular’ users spent, on average, over twice as much time during the study using the resource than the ‘non-regular’ users (1,654 mins (SD=399) compared to 721.5 mins (SD=398)). Progress on the two maths measures was calculated for both groups, and an analysis of the differences between them quantified the impact of the resource.

Statistical analyses and interpretation

As the groups were self-selecting, it was important to consider possible pre-existing confounding variables prior to analysis: age, usage of the resource last year, attitude towards maths and maths ability. The resource had been available across all Glasgow primary schools since 2017, so several students and teachers had been using it previously. Attitude towards maths was measured using a two-item self-report questionnaire, where statements such as ‘I enjoy learning maths at school’ were rated from ‘agree’ to ‘disagree’ on a five-point scale, giving a score between two and 10. Maths ability was taken as the students’ scores on the maths fluency and applied maths tasks.

A check on confounding variables

Table 1 reports the means at both time points and a statistical test of the differences between groups. There were no significant differences across any of the variables.

Table 1: Mean averages for ‘regular’ and ‘non-regular’ users

Maths fluency is represented by the raw number of questions answered correctly in three minutes. Applied maths at the start of the year is a scaled score converted from the raw scores, allowing the comparison between different test papers.

Comparison of progress

Table 2 shows the maths scores for both groups pre- and post-test. Figure 2 and Figure 3 show the interaction plots. Maths fluency progress for students who used the resource regularly was double that of the non-regular users (12.4 compared to 6.0 more questions correct). An ANOVA revealed a statistically significant interaction between usage group and test time (F(1,219)=21.2, p<.001), with . The Education Endowment Foundation (EEF) describe an effect size of this magnitude as ‘high impact’ – equivalent to a six-month advantage (Education Endowment Foundation, 2019). Figure 3 shows that there was no similar interaction between usage group and test time for the applied maths measure.

Table 2: Mean averages and standard deviations for the maths measures

Figure 2: An interaction plot showing the maths fluency scores of the regular and non-regular users (error bars represent standard error)

Figure 3: An interaction plot showing the applied maths scores of the regular and non-regular users (error bars represent standard error)

Making maths progress using EdTech

The study found that use of this EdTech resource had a significant impact on students’ fluency skills. There are some features of EdTech that might be responsible for this:

• Frequency and repetition: Effects of forgetting are reduced by revisiting topics to reinforce learning (Ebbinghaus, 1913). When EdTech facilitates an interleaving of topics and revision, fluency can be strengthened.
• Quantity of practice: When students are provided with individual logins, they have the freedom to practise outside of regular classroom hours, increasing potential practice time. EdTech availability on a variety of devices, such as smartphones, allows practice whenever there’s a few minutes to spare.
• Motivation/engagement: Particularly for students who may struggle to engage in traditionally academic activities, EdTech can provide an alternative. The game format adds an element of competition and constructiveness that allows students to associate the experience with something other than ‘doing maths work’.
• Feedback: In the average-sized UK primary school class of 28 students (OECD, 2019), giving timely and effective feedback to every individual is a challenging task for one teacher. Sumdog provides rewards for correct answers and information about incorrect answers (including an indication of the correct response). This helps to strongly reinforce maths facts and encourages lasting learning (Zerr, 2007).

Although the differences between groups on the confounding variables were statistically non-significant, there is still a theoretical argument that some of these might influence maths progress, and therefore they should continue to be considered in future research. A further study could also consider broader factors that have the potential to affect maths progress, such as the attitudes and effort levels of teachers and parents. This would allow an even clearer conclusion to be made in terms of the impact of the EdTech itself.

In addition to fluency, reasoning and problem-solving skills are also vital in the development of successful and confident mathematicians. This study found that progress in this domain was not impacted using this particular EdTech in the same way as fluency. There are several reasons why this may be the case:

• The format of the resource: The quick-paced, multiple choice, competitive nature lends itself to developing speed and accuracy. More thoughtful problem-solving skills may thrive better in a different format.
• Timing: It is possible that this study did not allow enough time for the effects of using the resource to be seen in the students’ test results. If fluency skills are considered a pre-cursor to reasoning, then this may well be the case. Further research could incorporate delayed post-tests to measure how this changes over time.
• A mis-match between the topics practised by the resource and those assessed in the task: Sumdog is designed to provide students with practice on topics that are in need of consolidation, and therefore are not necessarily the same topics being taught in class or tested in the applied maths task. This is achieved by requiring students to achieve an accuracy of 80 per cent before moving on to the next topic. MaLT tests are standardised against a national sample to provide a maths age. At the start of the study, the average score was equivalent to a maths age of seven years and eight months – six months below national averages. This suggests that there were gaps in knowledge for these students, and the resource will have targeted these. This is most likely to be the reason why a large impact was seen for the precursor fluency skills in particular.

Using EdTech to close the attainment gap

A poignant problem faced by education is the skills gap between children from families living in deprivation and their peers. As a demographic group, these children’s numeracy levels are significantly lower from the very early stages of education, and they don’t get the chance to catch up. This leaves them in a disadvantaged position leaving school, and perpetuates the problem (Joseph Rowntree Foundation, 2014).

Glasgow is identified as being one of the nine local authorities in Scotland with the highest concentrations of deprivation, and Pupil Equity Funding (PEF) (like Pupil Premium in England) allocates additional funding for every child registered for free school meals. Headteachers are responsible for investing this money in programmes ‘grounded in evidence of what is known to be effective at raising attainment for children affected by poverty’ (Scottish Government, 2017). For the academic year 2018–19, Glasgow City Council primary schools were allocated over £16 million for this cause.

Standardised test scores for the students participating in this study suggested their average ‘maths age’ to be around six months behind the national average. This study showed that Sumdog gave those students the opportunity to consolidate their basic numeracy, giving them a chance to ‘catch up’. For schools in which maths fluency is identified as being a concern for pupils, PEF spent on technology that enables access to resources proven to have an impact can pave genuine steps towards narrowing the attainment gap.

References

Department for Education (2014) National curriculum in England: Mathematics programmes of study. Available at: www.gov.uk/government/publications/national-curriculum-in-england-mathematics-programmes-of-study/national-curriculum-in-england-mathematics-programmes-of-study (accessed 10 December 2019).

Department for Education (2019) Reducing school workload. Available at: www.gov.uk/government/publications/reducing-teachers-workload/reducing-teachers-workload (accessed 11 December 2019).

Ebbinghaus H (1913) Memory: A Contribution to Experimental Psychology. New York City, NY: Teachers College, Columbia University.

Education Endowment Foundation (2019) Impact: Additional months’ progress. Available at: https://educationendowmentfoundation.org.uk/evidence-summaries/about-the-toolkits/attainment (accessed 11 December 2019).

Foster C (2019) The fundamental problem with teaching problem solving. Mathematics Teaching 265:8–10. Available at: www.atm.org.uk/write/MediaUploads/Journals/MT265/MT26503.pdf (accessed 11 December 2019).

Joseph Rowntree Foundation (2014) Closing the attainment gap in Scottish education. Joseph Rowntree Foundation May: 1–4.

National Council of Teachers of Mathematics (2014) Procedural fluency in mathematics: A position of the National Council of Teachers of Mathematics. Available at: www.nctm.org/Standards-and-Positions/Position-Statements/Procedural-Fluency-in-Mathematics (accessed 11 December 2019).

OECD (2019) Education at a Glance 2019: OECD Indicators. OECD. Available at: https://doi.org/10.1787/f8d7880d-en (accessed 11 December 2019).

Scottish Government. (2017) Pupil Equity Funding: How it will work. Available at: www.gov.scot/publications/pupil-equity-fund (accessed 11 December  2019).

Standards and Testing Agency (2018) Multiplication tables check assessment framework. Available at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/755745/2018_MTC_assessment_framework_PDFA.pdf (accessed 11 December 2019).

Zerr R (2007) A quantitative and qualitative analysis of the effectiveness of online homework in first-semester calculus. Journal of Computers in Mathematics and Science Teaching 26(1): 55–73.