Researching cognitive development in primary schools: methods and practical considerations

As more and more schools participate in research projects and conduct their own research, it is important for all stakeholders (headteachers, teachers, trainee teachers and researchers) to familiarise themselves with research protocols and understand the nuances and requirements of different research designs and methods. Conducting research with primary school children offers researchers the opportunity to explore the effects of a range of school factors on students’ cognitive development, such as the type of school, teaching style and school environment, as well as to conduct classroom-based studies with enhanced ecological validityIn assessment, the degree to which a particular assessment measures what it is intended to measure, and the extent to which proposed interpretations and uses are justified and direct implications for the teaching practice. This review addresses three main ways in which researchers study primary school children’s cognitive development within the classroom/school context: (1) common measures of children’s cognitive skills, (2) computerised tasks and processing speed measures and (3) children’s explanations of their experiences and ideas. More specifically, the review highlights cognitive tasks that measure skills correlated with intelligence, accuracy and response times metrics, and verbal and written reasoning data-collection methods such as interviews, think-aloud techniques and questionnaires. The strengths and limitations of the use of each method within the classroom/school environment are considered.

Psychology and neuroscience research explores how cognitive skills develop from childhood to adulthood. For children, cognitive development refers to how they think about and understand the world using cognitive skills such as problem-solving, memory, decision-making, attention, information processing, reasoning, cognitive flexibility, etc. Cognitive psychologists use a variety of methods to explore cognition, varying from lab experiments, eye-tracking techniques and neurophysiological measures (e.g. electrophysiological measures and imaging techniques) to standardised testing, non-verbal tasks and computer tasks. For this review, we will focus on the tasks and methods that can be administered in the classroom/school context to many students simultaneously, as these methods are of greater relevance and importance to the educational and teaching community.

Cognitive skills tasks

There is a wide range of cognitive tasks used by researchers studying cognitive development that tap into different cognitive abilities and skills (e.g. intelligence, working memory and problem-solving). Typical measures of intelligence or skills that are correlated with intelligence and are used with school-aged children are the Dimensional Change Card Sort, the Wisconsin Card Sorting Test, the Stanford–Binet Intelligence Scale (5th edition), the Stroop Task, and the Digit Span Task. We will particularly focus on Raven’s Standard Progressive Matrices (Raven, 1938), which is a common measure of general cognitive ability. Raven’s test is used widely in the classroom as it is easy to administer, with each child given one booklet. Moreover, compared to other tasks such as the Wisconsin Card Sorting Test, which require the presence of the researcher or teacher to keep track of the children’s performance while sorting the cards on a one-to-one basis, Raven’s test can be administered to the whole classroom simultaneously, with minimum guidance from the researcher or teacher.

The Raven’s Standard Progressive Matrices (Raven, 1938) is still considered one of the best, if not the best, non-verbal measure of general intelligence (Jensen, 1980; Mackintosh and Bennett, 2005). For the Raven Progressive Matrices tests, children have to complete a series of diagrams or designs with a part missing by identifying the pattern. Students have to choose the right piece out of the options printed beneath. Overall, children fill in 60 patterns of increased difficulty and get one point for each correct item. The benefits of the Raven’s Standard Progressive Matrices include the fact that it can be used with respondents of all ages and requires no vocabulary skills, and the administration does not require much time or effort, making it suitable for classroom-based research.

Computerised tasks and processing speed

Recent developments in cognitive functions assessments include a set of computerised task batteries that capture improvements in various cognitive skills across a wide age range and are sensitive to children as well (Hughes and Graham, 2002). Examples of these tasks include: the Cambridge Automated Neuropsychological Test Battery (CANTAB – Robbins et al., 1997), measuring impairments in executive function associated with ageing; the Test of Everyday Attention for Children (TEA-Ch – Manly et al., 2001), measuring attention; and the Maudsley Attention and Response Suppression task battery (MARS – Rubia et al., 2001), measuring impulsivity control. Computerised tasks enable researchers to easily measure response times for each task.

Verbal and written reports

Primary school children’s age group offers a unique opportunity for researchers studying cognition to study children’s knowledge, ideas, reasoning and how they understand the world through their own words and perspectives (Paes et al., in press). As students start developing their verbal and writing skills, they are able to engage with self-reported measures such as written tests and questionnaires, as well as express their own ideas and thinking processes through interviews and think-aloud tasks, allowing the researchers to capture their reasoning and understanding of the world.

Interviews are a widely used method of qualitative data-collection and enable researchers to understand children’s perspectives, experiences, understandings and ideas on important issues. For instance, Vosniadou and Brewer (1992) conducted one-to-one interviews with primary school children to explore their exposure to information about facts and underlying conceptual models of the Earth. Think-aloud techniques are also used by researchers studying cognitive development to better comprehend children’s cognitive processes, e.g. reasoning and problem-solving. In a typical think-aloud session, the child is asked to verbalise their thoughts, but they are time-consuming and need practice to train the children in the process.

Important practical considerations when interviewing or conducting think-aloud tasks with primary school children are to use age-appropriate language, build effective rapport and take care of the physical setting (Gibson, 2012). Researchers should use simple, everyday language that enables children to understand the questions and feel comfortable. This is especially difficult in cross-age studies, in which researchers work with students of different ages. Researchers have to strike a balance between simple and not simplistic vocabulary that can be comprehended by all children, and concurrently have to be consistent with their questions. Moreover, they should provide time and space for children to talk and follow up their responses and show interest. When working with children, it is important to take into consideration the physical setting. The furniture in the room in which the interview or think-aloud session takes place should minimise the size difference between children and adults (Cameron, 2005), so that the adult and the child sit on an equal level and the child feels comfortable. It is also important to have as few visually and audibly distracting items in the room in general as possible, so that the children can focus on the task more easily and the recording is clearer.

Written reports from school-aged children include questionnaires with closed or open-ended questions, as well as drawings and diaries. Specifically, questionnaires are used to capture children’s ideas and reasoning, and are easy and fast to administer in a whole-classroom setting, thus enabling the researchers to recruit a large number of participants. However, they are more suitable for children in late primary school, as they require a certain level of literacy skills. Moreover, children often give incomplete or irrelevant answers. Although questionnaires are faster and easier to distribute compared to interviews and think-aloud tasks, which require administration on a one-to-one basis, they often lack depth in children’s responses, as the researchers cannot follow up interesting statements.

Ethical considerations

What should not be left out of the conversation, in terms of practical considerations, are the sensitive ethical procedures associated with this age group. Researchers need to obtain children’s consent to participate in the research, as well as parents’ consent, through opt-in or opt-out consent forms (Lambert and Glacken, 2011). For opt-out consent forms, after gaining the headteacher’s approval to conduct the study at the school, researchers send an information letters to the parents explaining the process, along with a form to be returned in the case that they do not want their children to participate in the study. For opt-in consent forms, however, parents need to explicitly give their consent to the researcher by returning signed forms to the school.

Although opt-in forms better safeguard the rights of the participants, especially vulnerable participants, they make recruitment more difficult, as often children forget to give the parents the forms or return them. Researchers should explain the study to the participants and their guardians in detail, and be explicit about what activities and tasks children will participate in, as well as how the anonymity of the participants will be protected. Moreover, children should be informed that they can withdraw from the study at any point (BERA, 2018). In light of the General Data Protection Regulation that aims to protect the personal data of residents in the European Union, all data is anonymised (Ayala-Rivera and Pasquale, 2018). Identities of participants are protected and further analyses are conducted on anonymised data. With regard to confidentiality, only the researcher has access to personalised data, thereby ensuring privacy control. The personal data is only stored until processed and not for longer than that period.

Conclusion

The Raven’s Standard Progressive Matrices is a common measure of children’s cognitive ability. It is a widely used measure, given its ease of administration, and it is suitable for all ages. Computerised tasks are another good measure to assess children’s cognitive skills in schools, as no additional equipment is required. They not only measure the children’s response time but they also provide details about the children’s accuracy, and multiple students can take the assessment simultaneously. Interviews and think-aloud tasks are other effective means of collecting qualitative data about children’s perspectives on a range of important issues and, combined with questionnaires, they can provide both breadth and depth to the question under study. It is imperative that when research is conducted in schools, researchers use age-appropriate language and ensure that the physical setting is suitable, in order to increase the child’s comfort level.

When researching cognitive development in primary schools, researchers need to be flexible. Ellefson et al. (2018) note that when conducting research in schools, it is important to consider the different structures, time frames and semantics. For example, in addition to the typical school day, there are plenty of other activities that children participate in. Consequently, it is critical that researchers build in some flexibility in the time frame of the study’s design to ensure that they are able to recruit the adequate sample size and it does not add pressure to the schools that participate in the study. In addition, communicating effectively with the teachers is critical. Researchers could ask teachers what form of communication they would prefer, and request a weekly schedule of school events so that they can plan their study appropriately around these events. Furthermore, if the study requires the teachers to complete measures, researchers can provide teachers with an example so that they know what the expectations are. Often, teachers would like to know not just the rationale of the study but also the implications, in terms of their teaching practice and its pertinence to their students (Ellefson et al., 2018). Overall, effective cooperation and communication among all stakeholders (researcher, headteacher, teacher and students) and schools’ familiarisation with research protocols and needs can facilitate the process and produce ethical research with benefits for both the research and the educational community.

References

Ayala-Rivera V and Pasquale L (2018) The grace period has ended: An approach to operationalize GDPR requirements. In: Proceedings – 2018 IEEE 26th International Requirements Engineering Conference, RE 2018.

British Educational Research Association (BERA) (2018) Ethical Guidelines for Educational Research. Available at: https://www.bera.ac.uk/publication/ethical-guidelines-for-educational-research-2018 (accessed 26 November 2019).

Cameron H (2005) Asking the tough questions: A guide to ethical practices in interviewing young children. Early Child Development and Care 175(6): 597–610.

Ellefson MR, Baker ST and Gibson JL (2018) Lessons for successful cognitive developmental science in educational settings: The case of executive functions. Journal of Cognition and Development 20(2): 253–277.

Gibson JE (2012) Interviews and focus groups with children: Methods that match children’s developing competencies. Journal of Family Theory & Review 4(2): 148–159. DOI: 10.1111/j.1756-2589.2012.00119.x.

Hughes C and Graham A (2002) Measuring executive functions in childhood: Problems and solutions? Child and Adolescent Mental Health 7(3): 131–142.

Jensen AR (1980) Bias in Mental Testing. New York: Free Press.

Lambert V and Glacken M (2011) Engaging with children in research: Theoretical and practical implications of negotiating informed consent/assent. Nursing Ethics 18(6): 781–801.

Mackintosh NJ and Bennett ES (2005) What do Raven’s Matrices measure? An analysis in terms of sex differences. Intelligence 33(6): 663–674.

Manly T, Nimmo‐Smith I, Watson P et al. (2001) The differential assessment of children’s attention: The test of everyday attention for children (TEA‐Ch), normative sample and ADHD performance. Journal of Child Psychology and Psychiatry 42: 1065–1081.

Paes T, Tsapali M and Ellefson MR (in press) Studying cognitive development in school-aged children. In: Westerman G (ed) The Encyclopaedia of Child and Adolescent Development,  Vol 3: Cognition.

Raven JC (1938) Guide to Using Progressive Matrices (1938): The Design and Use of the Scale-Instructions-Records and Marking-Key-Normal Score Composition-Norms for Children and for Adults Up to the Age of 65 – Bibliography. London: HK Lewis.

Robbins T, James M, Owen A et al. (1997) A neural systems approach to the cognitive psychology of ageing using the CANTAB battery. In: Rabbitt P (ed) Methodology of Frontal and Executive Function. Hove, East Sussex: Psychology Press, pp. 215– 238.

Rubia K, Taylor E, Smith A et al. (2001) Neuropsychological analyses of impulsiveness in childhood hyperactivity. British Journal of Psychiatry 179: 138–143.

Vosniadou S and Brewer WF (1992) Mental models of the earth: A study of conceptual change in childhood. Cognitive Psychology 24: 535–585.