A framework for defining and assessing creative thinking

Claire Scoular and Jonathan Heard, Australian Council for Educational Research, Australia

There is growing consensus internationally that creative thinking is important (e.g., Care et al., 2012; Kereluik et al., 2013). Beyond identifying the importance of creativity, however, there is little consensus on how to develop and teach it (and even whether this is necessary), due in part to a lack of clarity or even agreement about the nature of creativity as a skill (Simonton, 2012), including what constitutes its fundamental building blocks and how it develops and changes in a learner. This is where assessment can help. Specifically, formative assessments that can identify what a learner can do in relation to a skill, and what they need to know or be able to do to progress further, can inform teacher instruction and support students to progress towards mastery (Lucas, 2016).

However, assessing creative thinking is complicated, given that such thinking is largely an internal process. Tasks need to be intentionally designed to elicit certain responses or behaviours from learners, reasonably assumed to be indicative of creative thinking, from which to make valid inferences about their creative ability. At a minimum, this requires teachers to first have a clear and robust definition in their own minds of what – as a skill – creative thinking is: what processes and end-qualities it entails, and therefore what observable behaviours and features would reasonably serve as evidence of it.

Given the complex nature of creative thinking, it also requires a detailed framework, based on relevant academic theory and research, that breaks what is often thought of as a mercurial and elusive construct down into relatively stable components and subcomponents, such that a teacher can isolate aspects of the construct and focus on developing and assessing these individually rather than the entire construct itself. Recent and ongoing research undertaken by the Australian Council for Educational Research (ACER) is providing early validation of a construct framework for creative thinking, assessed using a classroom problem-based learning approach with primary and secondary school students, that shows promise for standardised assessment but also as a resource for teachers.

A creative thinking framework

A seminal framework for considering factors of creativity is that of Rhodes’ ‘4Ps’ model (1961). Rhodes considered creativity a combination of influences from the person, the process, the product and the ‘press’ (the social and cultural environment and context). As personality and environmental factors can be fixed and are therefore not necessarily amenable to change through teaching, ACER’s framework focuses instead on the skills necessary for the process of creative thinking, and the features of the end product that make an idea creative. Both are observable and able to be measured using techniques for standardised assessments that can be easily administered in the classroom.

ACER defines creative thinking as:

the capacity to generate many different kinds of ideas, manipulate ideas in unusual ways and make unconventional connections in order to outline novel possibilities that have the potential to elegantly meet a given purpose.

This definition is further expounded by ACER’s creative thinking framework, which can be viewed in summary in Figure 1. The construct framework breaks the skill down into component strands and subsequent aspects (Ramalingam et al., 2020).

Applying the framework

As the framework demonstrates, creative thinking is multi-faceted and complex, but by breaking it down, the framework is intended to focus teachers and students at the level of individual component skills or qualities of creative thinking. As such, using the framework enables teachers to target instructional interventions to specific aspects and assess evidence of the demonstration of those aspects. The aspects support the development of rubrics, providing criteria upon which to judge student performance and a common terminology to enhance reliabilityIn assessment, the degree to which the outcome of a particular assessment would be consistent – for example, if it were marked by a different marker or taken again when judging the skills as they manifest across different subject domains.

Table 1 outlines an example of a general rubric for assessing brainstorming activities in creative thinking tasks. The rubric identifies generic levels of proficiency in each of the aspects, but it is intended to be adaptable and tailored to the specific details of the brainstorming task at hand. The aim of developing such resources is to support teachers to better understand these skills and to make criterion-based judgements while observing student behaviours, interpreting student assessment data and monitoring progress.

A version of the rubric in Table 1 was trialled in our recent Assessment of General Capabilities project. Task 3 in the assessment required students to generate as many creative ideas as they could to solve a problem of refugee resettlement in a local area.

Figure 2 presents an example of a student response to Task 3. This student scored low on the rubric in Table 1, indicating less proficiency in creative thinking. The student showed limited fluency and flexibility in their thinking, indicated by a limited number of ideas, all of which were variations on a theme of free provision of resources. The ideas showed naivety about the problem of what it would take for refugees to smoothly resettle into a new and foreign community, which limited their effectiveness as solutions. There was a lack of detail as to how the ideas would work, and the solutions suggested that only the perspectives of the refugees receiving assistance were considered.

Figure 3 presents a second example of a student response to Task 3. By contrast, this student scored higher on the rubric in Table 1, indicating more proficiency in creative thinking. This student showed a greater fluency (number) of ideas, of which several were fit for the purpose of assisting refugees to resettle smoothly. The ideas were reasonably distinct from each other, showing a degree of flexible thinking (range). While relatively diverse, they mostly reflected expected solutions: making refugees feel welcome, providing housing solutions, education services, personal assistance, etc. However, novel ideas emerged about creating host families in the local community and an awareness-raising event. Ideas such as these, which discussed bringing the refugees into the wider community rather than simply providing them with goods and services, also indicated shifting perspectives on the nature of the problem. The student could elaborate their reasoning about the outcome of one idea and therefore scored higher on this specific criterion. This response also demonstrates better understanding of the topic, highlighting that level of understanding of the topic can impact on ability to think creatively.

Monitoring growth

As specific behaviours demonstrated by students can indicate the processes or cognitive ‘moves’ by which they complete creative tasks and solve problems, identification of these behaviours allows for capturing evidence of the processes involved in thinking creatively. Using item response theory (IRT), assessments such as these help us to understand what growth in creative thinking might look like. Based on the assumption that less-frequently demonstrated skills or responses in an assessment are, by definition, more difficult, student responses can be plotted along a scale of difficulty. Scales such as these, derived from multiple assessments of a construct such as creative thinking, help to inform the development of learning progressions. As such, the emphasis in assessment should be on developing an understanding of how learning progresses, rather than on the generation of summative scores and grades.

Table 2 presents an excerpt from ACER’s levels of skill development for creative thinking that focuses on the aspect of ‘shifting perspective’. To ensure an evidence-based approach, these levels have been, and continue to be, validated and corroborated with assessment data. Levels of skill development can support understanding of these aspects and how they develop. They can also support teachers to appropriately target teaching interventions at a student’s level of attainment. For example, students who demonstrate being less proficient at shifting their perspective on a problem to generate novel ideas can be taught techniques and strategies for reconsidering problems and solutions from a different ‘angle’: from the perspectives of different stakeholders, to see other causes to a problem or to explore less obvious uses, properties and qualities of objects or solutions.

Table 2: Excerpt from the levels of skill development for creative thinking, Aspect 2.1: Shifting perspective

Through interpretation of the ability–difficulty scales derived from assessments, it is possible to make broader inferences about which behaviours can be associated with lower or higher levels of proficiency. For example, in the creative thinking tasks of the Assessment of General Capabilities project that assessed Aspect 2.1: Shifting perspective, most students took a fairly singular perspective on what it means to help refugees resettle. Very few students tested out multiple pathways or responded in ways that would indicate that they renegotiated the boundaries of the task. Those students who did tended to perform the best overall in other creative thinking tasks; therefore, this behaviour was interpreted as a higher level of proficiency of creative thinking (full results can be found in Scoular et al., 2020). The analysis of this data allowed a shift from a hypothetical progression to one that, through relevant adjustments, is empirically derived. Of course, inferences should not rely too heavily on a single assessment alone; rather, a number of scales derived from different assessments are being reviewed. Where the evidence is consistent across assessments, broader conclusions about how creative thinking develops and can be enhanced can be made.

Conclusion

The findings from our research supported the creative thinking framework as a robust foundation on which to base assessment. The data provided validation and iteratively informed the definitions in the skill development framework and the levels of skill development. On the basis of such project work, ACER is contributing to the international research in this area by developing frameworks for creative thinking and other constructs, such as critical thinking and collaboration, useful for classroom assessment and learning. A number of recent projects, and subsequently countries, have now adopted these frameworks, and a process of validation through comparison to assessment data is ongoing. Future work includes considerations not just of the framework’s application in assessment but also of how the aspects can be utilised for the purposes of teaching strategies and linking to learning outcomes in the curriculum.

References

Care E, Griffin P and McGaw B (2012) Assessment and Teaching of 21st Century Skills. New York, USA: Springer.

Kereluik K, Mishra P, Fahnoe C et al. (2013) What knowledge is of most worth: Teacher knowledge for 21st century learning. Journal of Digital Learning in Teacher Education 29(4): 127–140.

Lucas B (2016) A five-dimensional model of creativity and its assessment in schools. Applied Measurement in Education 29(4): 278–290.

Ramalingam D, Anderson P, Scoular C et al. (2020) Creative thinking: Skill development framework. Australian Council for Educational Research. Available at: https://research.acer.edu.au/cgi/viewcontent.cgi?article=1041&context=ar_misc (accessed 17 March 2021).

Rhodes M (1961) An analysis of creativity. The Phi Delta Kappan 42(7): 305–310.

Scoular C, Ramalingam D, Heard J et al. (2020) Assessing general capabilities: Skills for the 21st century learner. Full research report. Australian Council for Educational Research. Available at: https://research.acer.edu.au/ar_misc/47 (accessed 17 March 2021).

Simonton DK (2012) Teaching creativity: Current findings, trends, and controversies in the psychology of creativity. Teaching of Psychology 39(3): 217–222.