Evaluating effective Teacher Professional Development
VICTORIA COOK, EDUCATION AND RESEARCH PROJECT SPECIALIST, CHARTERED COLLEGE OF TEACHING, UK
For those responsible for improving teacher professional development (PD), it is important to understand which characteristics of PD make it effective. Some claim that the field has begun to reach a consensus on the characteristics of effective PD (Darling-Hammond et al., 2017). However, this is contested by others who argue that studies have failed to distinguish the active ingredients of effective PD (Sims and Fletcher-Wood, 2021).
A new theory of effective PD
Instead of focusing on common characteristics of PD interventions, Sims and colleagues analysed PD interventions through the lens of behaviour change strategies and principles, developing a new theory of effective PD called the I/M/T/P theory (Sims et al., 2021). This theory is based on combinations of causally active components (referred to as ‘mechanisms’) targeted at developing teachers’ insights, goals, techniques, and practice (Table 1).
Table 1: The four (IGTP) purposes of PD and their associated mechanisms (Sims et al., 2021)
| Purpose | Mechanism |
| Instil insight (I) | 1. Manage cognitive load
2. Revisit prior learning |
| Motivate goals (G) | 3. Goal setting
4. Credible source 5. Praise/reinforce |
| Teach techniques (T) | 6. Instruction
7. Practical social support 8. Modelling 9. Feedback 10. Rehearsal |
| Embed practice (P) | 11. Prompts/cues
12. Action planning 13. Self-monitoring 14. Context-specific repetition |
This table is licensed under a Creative Commons CC BY 4.0 License
The theory provides a framework for those designing or commissioning PD to use when thinking about how to combine different components of PD to develop teacher insights, motivate change, develop techniques, and embed these in practice.
Sims et al. (2023) have found qualified empirical support for the theory, with the number of mechanisms incorporated in a PD program associated with greater impact on pupil test scores. They also found that balanced PD programs have a larger impact on pupil test scores than imbalanced PD programs in general, although the difference is not statistically significant at conventional levels.
In developing their list of potential mechanisms, Sims and colleagues used a common practice called ‘analogical abduction’, drawing on research outside of the teacher PD literature (specifically, from cognitive scienceThe study of the human mind, such as the processes of thought, memory, attention and perception, behavioural science and medical education). Whilst they recongise that this may be viewed as a weakness, they justify their approach given the lack of strong empirical evidence from within education (Sims et al., 2023). Furthermore, their analysis focuses primarily on the design of the PD rather than the content, and they note that there is a need for future studies investigating the interactions between content and mechanisms of PD, because content may be an important influence on the way PD should be designed.
The Teacher Professional Development (TPD) Monitor
Richter and Richter (2024) have developed a practical tool to evaluate PD quality. The Teacher Professional Development (TPD) Monitor (Richter and Richter, 2024) covers four dimensions: clarity and structure, practical relevance, cognitive activation and collaboration (Table 2). The tool has been rigorously tested using a sample of 173 PD courses with a total of 2314 participants to ensure validityIn assessment, the degree to which a particular assessment measures what it is intended to measure, and the extent to which proposed interpretations and uses are justified and reliabilityIn assessment, the degree to which the outcome of a particular assessment would be consistent – for example, if it were marked by a different marker or taken again.
Table 2: The Teacher Professional Development (TPD) Monitor (adapted from Richter and Richter, 2024)
| Dimension | Item | Strongly disagree | Disagree | Agree | Strongly agree |
| Clarity and Structure | 1. The goals of the course were clearly stated. | ||||
| 2. The course content was explained in a comprehensible manner. | |||||
| 3. In the event, the time was used efficiently. | |||||
| 4. The agenda of the course was clarified at the beginning. | |||||
| 5. The individual components of the course built on each other in a meaningful way. | |||||
| Practical Relevance | 6. The discussion of the contents was based on real examples from school practice. | ||||
| 7. The aspects covered in the course were related to my current professional practice. | |||||
| 8. The course addressed problems that I encounter in my everyday professional life. | |||||
| 9. What I learned in the course I can apply in my professional practice. | |||||
| Cognitive Activation | 10. My prior knowledge was incorporated into the course. | ||||
| 11. In the course, I was asked questions or given tasks that made me think. | |||||
| 12. I was able to reflect on my professional tasks in the course. | |||||
| 13. The feedback I received during the course provided me with concrete advice for my professional development. | |||||
| 14. The course offered the opportunity to question previous routines in my work. | |||||
| 15. In the course, I was able to think my way into the unknown. | |||||
| Collaboration | 16. During the course, I had the opportunity to discuss the course content intensively with the other participants. | ||||
| 17. The course offered the opportunity to work on tasks with other participants. | |||||
| 18. The course allowed for work in small groups. |
This table is licensed under a Creative Commons CC-BY-NC License
This tool could be used to compare different PD courses or obtain feedback on the perceived quality of a PD course. However, whatever your chosen method for evaluating PD, it is also important to bear in mind the funnel effect.
The “funnel” effect
Not all the professional development activities intended by designers are deployed by teacher educators, with even fewer deployed by teachers. Called the ‘funnel’ effect of teacher PD (McChesney and Aldridge, 2019), research suggests that teacher educators change the intended design of PD interventions by up to 25%, whilst the proportion of the intended PD that is used by teachers ranges from 25% to 75% (Warner et al., 2022). For example, research undertaken with two experienced science teacher educators revealed that they omitted instruction (direct advice on how to implement a teaching method) when delivering science teacher PD despite it being included in the PD design. The findings suggest that changes to the intended mechanisms identified by Sims and colleagues may be made due to a lack of practical teaching resources combined with a lack of teacher preparedness or enthusiasm for a particular approach (Greitāns and Namsone, 2023).
To help overcome this, it is important to establish a shared purpose for PD. Engaging in professional dialogue around the question ‘why are we doing this?’ will help to align collective values prior to implementation (Barton, 2021).
Reflection questions
- How could these frameworks or tools be used when designing and/or commissioning PD in your setting?
- How significant is the funnel effect in your setting? How do you know?
References
Barton, A. (2021). Implementing education reform: Is there a “secret sauce”?. 10.13140/RG.2.2.26997.76000.
Darling-Hammond, L., Hyler, M. E., & Gardener, M. (2017). Effective teacher professional development. Learning Policy Institute. https://doi.org/10.54300/122.311
Greitāns, K. and Namsone, D. (2023). Analyzing an effective in-service science teacher professional development model through the lens of professional development principles and mechanisms: developer and teacher educator perspectives. The 15th Conference of the European Science Education Research Association (ESERA 2023).
McChesney, K., & Aldridge, J. M. (2019). What gets in the way? A new conceptual model for the trajectory from teacher professional development to impact. Professional Development in Education, 47(5), 834–852. https://doi.org/10.1080 /19415257.2019.1667412
Richter, E., Richter, D. (2024) Measuring the quality of teacher professional development: A large-scale validation study of an 18-item instrument for daily use, Studies in Educational Evaluation, Volume 81, 101357, ISSN 0191-491X, https://doi.org/10.1016/j.stueduc.2024.101357.
Sims, S., & Fletcher-Wood, H. (2021). Identifying the characteristics of effective teacher professional development: A critical review. School Effectiveness and School Improvement, 32(1), 47–63.
Sims, S., Fletcher-Wood, H., O’Mara-Evers, A., Cottingham, S., Stansfield, C., Van Herwegen, J., & Anders, J. (2021). What are the characteristics of effective teacher professional development? A systematic review and meta-analysisA quantitative study design used to systematically assess the results of multiple studies in order to draw conclusions about that body of research. Education Endowment Foundation. https://d2tic4wvo1iusb.cloudfront.net/documents/pages/ Teacher-professional-development.pdf
Sims, S., Fletcher-Wood, H., O’Mara-Eves, A., Cottingham, S., Stansfield, C., Goodrich, J., Van Herwegen, J., & Anders, J. (2023). Effective Teacher Professional Development: New Theory and a Meta-Analytic Test. Review of Educational Research, 0(0). https://doi.org/10.3102/00346543231217480
Warner, J. R., Plowman, D. L., Osman, D. J., & Schallert, D. L. (2022). The influence of collective and individual sensemaking on how facilitators and teachers implement a professional development program. Teacher Development, 26(5), 706–726. https://doi.org/10.1080/13664530.2022.2123849
