In preparing for headship, I had recurrent Ofsted nightmares. These frequently involved me taking an inspector along a corridor, not really knowing the way, and arriving at a class that was out of control.
Which I was meant to be teaching.
During this time, from 2014 to 2016, I studied for an MSc in Research for Public Policy and Practice with the EPPI Centre and UCL/IoE. I quickly learnt that we need to beware well-intentioned interventions that do more harm than good, for example terrible “expert” advice to put babies to sleep on their fronts that is estimated to have killed thousands British babies in the late 20th Century. Systematic Review of research literature can provide more empirical evidence to help us make better decisions. As Ofsted’s practice of grading lessons was unravelling I became obsessed with school inspection as an intervention that might be doing more harm than good, particularly in terms of classroom practice, and developed the dissertation question: how might school inspection improve teaching?
My dissertation took a systematic approach to synthesise the findings of twenty-seven studies that examined school inspection in OECD developed countries. Having used these studies to develop a context – mechanism – outcome rough theory of change, I tested this rough theory on four practitioners who had worked to improve teaching at whole school level in good or better English schools since 2010. Using in-depth interviews in Summer 2016, I got their feedback on the extent to which the theory matched their experience; how useful it could be in their future practice; and areas where more research is needed.
The findings were that despite a lack of evidence linking inspector feedback with subsequent actions taken by teachers in their classrooms; research shows that the inspectorate framework heavily influences school self-evaluation in England. The four practitioners described how the 2015 Ofsted Framework informs rather than defines school self-evaluation, but one expressed the view that in the past the quality of teaching has been limited to the quality of the framework. The September 2015 Ofsted framework gives freedom to teachers to teach in any way they want, which was seen by the four practitioners as a better opportunity to improve teaching, however there is no research available on this yet. There was a caveat raised by one practitioner who had recent experience of inspection under the September 2015 framework; that the inspectors themselves may be mis-interpreting the freedom as an opportunity to judge teaching and feedback based on their own personal preferences and expectations.
Feedback to teachers as part of school self-evaluation is thought by school leaders to improve teaching, as the theory of change model suggests it should, but there is a lack of research evidence on the extent to which feedback from internal observations as part of school self-evaluation changes classroom practice for the better. The practitioners testing the model welcomed that observations now look at progress over time and raised issues with previous practices of the inspectorate grading lessons. Practitioners were positive that lesson grading had now stopped. Some recognised ongoing issues raised in the research base of staged performance lessons that don’t relate to everyday practice, though others felt that the latest framework and removal of lesson grades had addressed this.
Therefore it is not so much inspection that changes teachers’ behaviour, but the anticipation of inspection. Over the years expectations set by the English inspectorate, such as students having to make “progress every twenty minutes” became part of schools’ self-evaluation, and this and other aspects of inspectorate frameworks over the years have led to unintended consequences (tunnel vision; suboptimisation; myopia (short term targets); ossification and measure fixation). There was insufficient evidence in the literature to develop a theory on the sub-question, “What do school leaders and teachers do to minimise the effect of unintended consequences?” However the practitioners spoke at length about this in their contexts and identified areas where further research could provide more insights. Minimising unintended consequences would remove some of the main barriers to school improvement in a system that involves high stakes inspection; and would also be beneficial for staff morale and longer term student outcomes.
Possible implications for policy makers
Policy makers need to be aware that school self-evaluation and feedback and resultant training given to teachers is very sensitive to the inspectorate framework. Initial feedback suggest that the September 2015 framework is popular with school leaders in England working to improve teaching because it allows freedom for teachers to teach according to their own style.
Possible implications for the inspectorate
Inspectors need to be trained on what to look for and on how to give feedback and need to do this consistently for all inspections. There is no evidence yet to show that inspections since September 2015 offer more consistent feedback on teaching. One practitioner identified the risk that new freedoms in the framework leave more scope for inspectors to feedback based on their personal preferences of teaching. Training on the inspectorate framework is needed for inspectors and school leaders to develop a shared interpretation of what the framework looks like in practice and how teaching can be judged against the framework.
Possible implications for schools
School self-evaluation based on the inspection framework is a key mechanism for school improvement. Training is needed for school leaders to achieve this. When observed by inspectors, teachers will be more likely to act on their feedback to improve if they go to see the inspectors to hear it directly from them.
More research is needed on the headteacher’s role in school improvement. For example research to identify the features of effective school self-evaluation and the extent to which this relies on the Ofsted framework.
The full dissertation is here if you want to read more or see details of the underlying research base.
Rough theory of change: how might school inspection change teaching?
|1.Leaders are equipped to understand the expectations
Inspectors are trained and qualified to judge schools against the framework
|Expectations of quality teaching set out in an inspection framework||School uses inspection framework to complete self-evaluation of teaching quality
|2.School uses inspection framework to complete accurate School Self-Evaluation(SSE) of teaching quality and set improvement targets
Leaders have access to the relevant data and evidence and are able to analyse that accurately to complete SSE
Leaders have access to guidance and support for SSE
Leaders manage the demands of internal and external accountability in pursuit of school-defined improvement
SSE is seen as collaborative and non-threatening
|School Self-Evaluation in line with an inspectorate framework is used to identify actions to improve teching
SSE is honest and is written for the school rather than for the inspectorate
|Improvements are made to teaching so that SSE meets inspectorate expectations more closely
Inspectors make judgements to confirm or contest SSE, thereby endorsing leadership or recommending that it needs to change
|3.Inspectors are of high quality and make consistent judgements on teaching
Inspectors make recommendations that will help teachers improve their practice
Inspection process allocates time for inspectors to see teaching
|Independent external inspectors visit the school and gather evidence on teaching quality to make judgements against the framework||Inspectors provide school with knowledge it lacked to make improvements
Inspection findings contribute to school improvement: dip in reults in inspection year, but increase in two years after
|4.Inspectors take time to observe teaching
Inspectors are trained to give feedback
Specific and clear recommendations
Teachers and leaders accept the feedback
|Feedback from inspectors on how to improve teaching
Feedback from school leaders on how to improve teaching as part of school self evaluation
|Teachers act on feedback
Leaders follow up on the extent to which teachers have acted on feedback, and monitor whether this has a positive impact
Possibility that feedback may have negative consequences is very real, lack of evidence on this
|5.Leaders and teachers take ownership of inspection findings
Action plans are formulated and implemented as
result of findings from the inspection
Leaders manage the demands of internal and external accountability in pursuit of school-defined improvement
|Leadership makes use of inspection process to improve teaching
Teachers make use of inspection and SSE process to improve their teaching
|Leaders implement appropriate actions to improve teaching, avoiding unintended consequences
Student progress and outcomes improve
6.Schools use SSE effectively and have access to relevant monitoring data
Leaders are motivated/compelled to improve teaching to reap rewards and to avoid sanctions (for example in England achieving “Outstanding” for teaching means that a school can apply to lead a Teaching School Alliance; achieving “Inadequate” can mean a school is forced to convert to academy status)
Leaders are motivated/compelled to address inadequate teaching
Carrots and sticks: rewards and sanctions
In England, for “good” schools there is an incentive to improve further to “outstanding”.
Schools that perform very poorly will either improve, with the extensive support provided, or be closed down/put under new leadership – action taken quickly to turn around schools judged to be failing
Rewards and sanctions may have unintended consequences: tunnel vision-narrow focus on certain areas to meet inspectorate demands; sub optimisation – actions for inspection, rather than for best interests of pupils; myopia (short term targets); ossification-sticking to known teaching methods, risk avoidance; measure fixation-focus on data rather than bigger picture (Nelson & Ehren, 2014)
|7.Teachers are motivated/compelled to improve their classroom practice so that they are not judged adversely by leaders for school self-evaluation or by external inspectors
|Teachers are held to account in line with high stakes accountability expectations, via appraisal and capability procedures||System is likely to be less useful for school improvement and professional development
Teachers fear the system and may “resist, subvert or sullenly acquiesce”
The volume of work associated with the accountability agenda was so intense that it severely reduced the time and energy available for enhancing teaching and learning in more traditional ways, such as by offering extra-curricular activities or spending longer planning and resourcing lessons.
|8.If (Head of dept) saw an ‘outstanding’ teacher teach a lesson that did not accord with the Ofsted requirement for manifest student progress within half an hour, he graded it as ‘satisfactory’, in the hope that this would jolt the teacher into a more sophisticated understanding of the game they were engaged in.||Gaming: activities to get positive inspection feedback, irrespective of whether these are in the best interests of pupils or represent everyday school life.||Teachers become reflexively aware that they are involved in what many described as kind of ‘staged performance’ for the inspectorate during the event itself.
Teachers are more resistant to recommendations because a)they are based on a staged performance, b)condition of trust in inspectors is not in place
|9.Inspection reports published so that stakeholders are aware of teaching quality in the school
|Accountability to stakeholders, including parents, means that schools work hard to meet inspectorate expectations to get a positive report
|Inspectorate recommendations are taken seriously because they are in the public domain and schools are held accountable if they do not act upon them to improve|