The current system often claims to assess an apprentice as occupationally competent before they have gained experience of doing the job in the real world.
In this post, I argue that college-based knowledge and competence tests cannot support valid inferences about workplace competence and knowledge transferThe processes of applying learning to new situations. Further, I suggest that such competency assessments are a dangerous way of testing apprentices’ performance in a safety-specific occupation such as plumbing, which often involves electrical and gas work.
My research
Plumbing is a cornerstone of civilisation in contributing to human health, so it is in the interest of the public that competency qualifications are fit for purpose. However, plumbing is not licensed in the UK, unlike in other countries, such as Germany, where you must be a Meister (master) with 5 years of work experience to establish a business. It became apparent to me – as a plumbing employer, Further Education plumbing teacher and student – that the English system of training and assessment was poor by international standards.
I adopted an ethnographic approach for my 2014 doctoral study into the assessment of occupational competence in college contexts. Ethnography enables the researcher to ‘get closer to social reality than other methods’ (Hammersley, 1992: 44), and, as a researcher, plumber and teacher, I was able to explore, first hand, the terrain and complexities of the plumbing curriculum from the perspectives of both tutors and students. Alongside informal conversations with tutors and students, I conducted 29 semi-structured formal interviews.
Participant responses
Although many tutors liked online multiple-choice tests as a way to assess subject knowledge (codified knowledge), others regarded them simply as memory tests. Many tutors doubted that multiple-choice tests were suitable for testing the problem-solving skills and analytical abilities demanded of plumbers in the field, and were despondent at the process of allowing students to continually resit online assessments until they achieved the pass mark:
I think the online testing and keep on going back until you pass the exam is not a very satisfactory way of doing it.’ – Tutor Den College 2, cited in Reddy, 2014: 227.
The situation we find ourselves in with students, which is unforgivable really, is that we are not in a position to fail a student, which I find most difficult. It devalues the qualification for someone who is competent and because of the numbers, retention, achievement, the pressures we are all under, success has to be success and there are questions asked at why it is not 100%. I think the culture is extremely detrimental to learning. – Tutor Gordon College 3, cited in Reddy, 2014: 228.
There were structural pressures on tutors to meet externally imposed targets and, judging from the majority of tutors’ responses, the credibility of the assessment process was highly questionable. Indeed, teachers across the three college sites were equally sceptical about the quality of practical plumbing assessments:
…there are a lot of tasks that are nothing to do with plumbing. Making a square of plastic pipe means nothing…there are a lot of board exercises, a lot of measuring, which needs to be done, but in those exercises there is nothing real about them. I don’t think you walk away from it thinking that you know a lot about plumbing. – Tutor Matt College 2, cited in Reddy, 2014: 155.
Tutors in the study were unanimous in their judgements about college-based assessments failing to adequately represent the reality, problems and experiences of plumbers operating in the workplace:
College exercises in the workshop are not real-life exercises; they are simulated. Everything is nice and level and flat. There are no problems, it‘s all there. It‘s like I said, when you get into the real world and you get into someone‘s house, it’s completely different. – Tutor Darrel College 3, cited in Reddy, 2014: 156.
Many tutors were keen to report on City & Guilds plumbing qualifications and how they failed to adequately prepare full-time students for the workplace. According to some tutors and on-site assessors this was leading to substandard workmanship in the trade:
…they haven’t got a clue, not a clue…you wouldn’t have them in your house and you know they have got no experience…it does degrade the qualification…he had a nice sign-written van saying ‘Plumber, City & Guilds’…but looking at the job he did, it was shocking really. – Tutor Darrel, College 3, cited in Reddy, 2014: 236.
Summarising the findings
While many of my findings in relation to the inadequacy of competency assessments supports the existing literature on vocational education and training, my research brings new insights in relation to health and safety. My study reinforces growing concern from the Gas Industry Safety Group (GISG) and the Institution of Gas Engineers and Managers (IGEM) (2016), who reported an increase in unsafe gas work by recently qualified engineers, from 1% to 5% (Gas Safe Register, in GISG/IGEM, 2016: 1).
They highlighted an over-emphasis on classroom theory and a dependency on online multiple-choice tests with novices sitting gas safety exams who ‘were allowed to keep re-sitting the tests until they passed’, GISG/IGEM (2016: 20). These findings build on the earlier work of the Unite Union (Unite 2012: 10), which stated that college-based courses can create ‘under-qualified individuals, who have the misconception that they are then able to undertake safety-critical work’.
Being a plumber involves dealing with the unexpected and problem-solving, which requires reflection and deliberation, drawing on previous meaningful events to inform present circumstances. However, problem-solving capability is unlikely to develop if the apprentices’ history is furnished with ‘low-fidelity’ training experiences, which bear little resemblance to the reality of practice (Rush, Acton and Tolley, 2010: 469), supplemented by snippets of abstract knowledge which are often quickly forgotten. Wolf (2011: 33) emphasised the effectiveness of apprenticeship learning in a ‘genuine workplace’ over ‘any education-based simulation’, and the Gas Competence Review (Health & Safety Executive, 2012: 6) stated that ‘experience was a central part of competence’.
Despite this, the plumbing Trailblazer separates technical assessments from work. This means that apprenticeship assessments show signs of poor quality, putting public safety at risk and diminishing the professional reputation of plumbers.
This post calls for more consultations on Trailblazer standards and a review of current methods for assessing occupational competence in safety-specific occupations such as plumbing, gas and electrical engineering.