Last year the world of educational leadership research was rocked by a study, summarised in two Harvard Business Review articles (here and here), that introduced to the world the idea of ‘Surgeon’ and ‘Architect’ headteachers, among other types.

The findings, if more generally true, would radically re-shape the advice we should give governing bodies about how to recruit school leaders because they showed subject background of a school leader is strongly related to their leadership style.

“We decided to see if there was a pattern between leadership types and their subject. And it’s not just a pattern, it’s completely clear. Maybe the subject you chose to study is a measure of other factors – how you grew up and what you believe. With teachers there’s usually no external influence from other sectors. So you fall back on the subject. If you’re a PE teacher, for example, it’s about winners and losers.”

Co-author Alex Hill quoted in Schools Week

Given the scale of recommendations arising from the study, it is important that they are interrogated by other researchers. This isn’t straightforward because the articles themselves have no published methodology, as pointed out here.

This presents certain difficulties for us in trying to replicate the study’s findings. For example, the most recent HBR article is framed around the turnaround of ‘failing’ schools, although it is not clear whether this relates specifically to those in special measures. Yet the article then goes on to advocate for the appointment of more ‘Architect’ heads across the entire schools system, whether at schools in need of turnaround or not.

As such, our analysis doesn’t seek to replicate the HBR work – but instead asks whether these broader lessons are ones we should really be taking. It does, though, also look at things when schools rated good or outstanding are stripped out, to see whether we’d come to different conclusions if we looked at schools that were considered underperforming.

We can use the School Workforce Census (SWC) to see whether we can find a general association between school performance and subject background of leadership. We can do this for all secondary schools or for any sub-sample we choose. We identify the school leaders in the 2010 SWC and split the school’s subsequent performance into during and post-tenure periods, calculating the annualised change in GCSE results for each[1].

It is a ‘quick and dirty’ analysis. If we were really interested in answering these research questions we would spend more time carefully crafting the datasets and thinking through each analytical decision we make. We don’t do this. The corresponding advantage is that we are not data mining – trying out every possible specification until we find one we like!

On school performance during tenure:

They say: ‘Surgeons’ (mostly PE, RS and biology subject background) improve examination results dramatically in the one or two years they are at a school.

We say: subject background of senior leaders is not an important factor in explaining variation in school performance.

We take the annualised change in GCSE results for each senior leader between 2010/11 and the year when they leave the school, or 2014/15 if they do not leave their post.

We perform an analysis of variance (ANOVA) test to show whether the subject background explains any of the variation in GCSE improvements at the school.

The below table shows the average annualised percentage point change in the number of children achieving five A*-C GCSEs or equivalents, both for heads and for all members of senior leadership teams (SLT), with results also given for all schools and just for schools not rated good or outstanding by Ofsted (Ofsted 1&2)

There is certainly no evidence that leaders with a PE, RS or biology background have good GCSE improvements.

In fact, none of the relationships with subject background is statistically significant at the 95% level.

On school performance post-tenure:

They say: ‘Architects’ (mostly history, economics, music, physics subject background) have the most positive long-term impact on exam results – on average, 15 to 23% higher than other leaders. The schools of ‘Philosophers’ (mostly English, languages, geography, with no experience outside education) either coast or decline.

We say: subject background of senior leaders is not an important factor in explaining variation in school performance.

This time we take the annualised change in GCSE results for each senior leader from the time they leave the school until 2014/15 and perform an ANOVA test to show whether the subject background explains any of the variation in GCSE improvements at the school.

The below table shows the average annualised percentage point change in the number of children achieving five A*-C GCSEs or equivalents – which shows that subject background does not explain the variation in school performance after a head leaves.

When all members of senior leadership teams are considered, English shows a statistically significant deviation – but we’d expect to see one or deviations like this even in random data.

On leadership pay:

They say: ‘Surgeons’ (PE, RS and biology) were paid an average of £150,000 a year. ‘Architects’ (history, economics, music, physics) were paid about £86,000 a year.

We say: There are some differences in pay by subject, particularly among senior leadership teams rather than heads specifically. But the pay levels do not match the findings of the HBR study.

Looking at the mean pay of heads with different subject backgrounds, as the chart below shows, differences are not large.

We also can’t reproduce specific claims about pay. Very few secondary heads earn over £150,000 – in 2015, only around 45.

We do find salary differences by subject background of SLT. They do not correspond with the findings of the HBR articles. Interestingly, they are persistent when we include demographic background characteristics of the teacher – we will write this up shortly in a separate post.

On careers outside education:

They say: Some 86% of ‘Architects’ (history, economics, music, physics) had experiences outside education before teaching (normally working there 5+ years).

We say: Some subject groups have spent more of their adult life outside teaching before qualifying, but not the group mentioned above and not in similar magnitudes.

In the SWC we can use each teachers’ qualification date and age to calculate the number of adult years prior to qualification[2].

Most teachers who make it to SLT were qualified by their mid-20s (three-quarters had achieved QTS by their early or mid-twenties). The heads with an economics background did qualify slightly later than others on average, but only by about a year. The same is true of heads with a languages background.

And in any case, there is no evidence that spending time outside education before teaching improves leadership quality. The correlation between years before qualifying and GCSE improvement (either during or after tenure) is essentially zero.

Want to stay up-to-date with the latest research from Education Datalab? Follow Education Datalab on Twitter to get all of our research as it comes out.

Notes
[1]A couple of notes on our methodology here. From the published HBR pieces it isn’t clear over what timeframe results have been considered. We work with copies of the School Workforce Census from 2010-2015, as the years for which data is available. This therefore isn’t likely to have perfect overlap with that used in the HBR work, but which shouldn’t lead to strikingly different results.
From the School Workforce Census it is possible to identify a subject background for around two-thirds of headteachers. Full details of how this has been done can be found here.
Full details of the sample considered can also be found here.
[2] We do make the assumption here that teachers do not gain QTS and then work outside of teaching for several years before returning to the profession in large numbers.