Provisional KS4 data 2016: Superstars and under the bars

By

Today will be a day of celebration for some schools, and concern for others, as the new Progress 8 performance measure re-sorts schools into piles.

Some who did very well on the old 5 A*-C measure will find they are struggling under a progress measure that does not reward them simply for recruiting a high attaining intake.

As the chart below shows, most schools bunch around a Progress 8 value of zero, which is exactly as it should be. Well-functioning schooling systems tend to have low variation in school quality; everyone gets an equal chance to do well.

(You will notice that schools with higher attaining intakes tend to do better on Progress 8. Want to know why? Read our numerous posts on this topic here, here and here.)

2016 Progress 8 scores versus KS2 intake, by school

floorsstars

For those of us who are interested in system improvement, though, the outliers might provide clues as to what a school at its best can achieve, and the places to focus energy on improving outcomes for those falling furthest behind.

It is worth saying here that today’s data is provisional – final results won’t be available until January, when school-level results may differ somewhat – but there are some interesting points we can draw from the data.

Learning from the superstars

There are seven schools with a provisional Progress 8 score above +1.0.

This equates to every child at these schools being on average a grade better than children with the same prior attainment – an amazing achievement.

These seven schools are all very different from each other, but I hope in each case we will take the time to understand what it is that they are doing so well.

Some of them – the Steiner Academy in Hereford, the two Tauheedul Islam schools in Blackburn and St Andrew’s Catholic school in Surrey – have unusual intakes. This means that we cannot really be certain as to how much of the amazing progress students make from ages 11 to 16 is a reflection of pupil characteristics and home learning environments.

The other three schools – City Academy Hackney, Ark King Solomon Academy and Harris Academy Battersea –  are London schools, but all have intakes at or below the national average and operate admissions policies without socially selective criteria.

Is it possible that we can replicate their success, perhaps even outside London?

To answer that we must understand the resources they have at their disposal.

It seems very likely there are policies, procedures and teaching approaches that could work across a variety of settings.

However, if their outstanding results relies on recruiting outstanding teachers who are willing to work exceptionally long hours, then their success isn’t something we can replicate across the system because such teachers are a scarce resource.

Falling below the floor

Schools will only be officially judged to be below the floor standard once revised data is published in January. From this year, the floor standard is set at a (statistically significant) Progress 8 score of below -0.5.

A total of 294 schools appear to be below the floor standard in these provisional results, though (20 of which appear to have closed since this summer’s KS4 exams). If things are the same in January, these schools will come under scrutiny from inspection and, if they are not an academy already, may be subject to forced conversion to academy status.

That is the theory. What happens in practice depends, in part, on the capacity of their region to find new sponsors willing to take them on. And it is noteworthy that well over a third of these schools are already sponsored academies so, if we believe the Progress 8 is a fair and valid method of judging school performance then the sponsorship model hasn’t worked for them.

There are also a surprising number of the government’s flagship free schools that, based on this provisional data, appear to be below the floor standard.

Those with the extremely low scores are all studio schools and university technical colleges (UTCs), which we wrote about last month.

Of greater concern are the eight mainstream free schools (of a total of 26 which have KS4 results) that appear to be below the floor standard.

Is it just ‘teething problems’ at these schools? In which case, can they demonstrate that these have been resolved? Or is there something more systemic that must call into question either the process of setting up free schools, or the ability to hold all schools to account using Progress 8?

Want to stay up-to-date with the latest research from Education Datalab? Follow Education Datalab on Twitter to get all of Datalab’s research as it comes out.

By | 2017-10-23T12:44:03+00:00 October 13th, 2016|Exams and assessment, School accountability|

About the Author:

Rebecca Allen is Director of Education Datalab and an expert in the analysis of large scale administrative and survey datasets, including the National Pupil Database and School Workforce Census. Her research explores the impact of government reforms on school behaviour, with a particular focus on accountability and teacher labour markets. She is currently on leave from her academic post as Reader in Economics of Education at UCL Institute of Education.

4 Comments

  1. ciran October 13, 2016 at 9:04 pm - Reply

    School intake score – is that the difference between the average fine point average score at KS2 national average and your school (i.e. the one that is one the Raise on line table 3.1.1)?

    • Mike Smith October 19, 2016 at 6:28 pm - Reply

      I thought a Z score is a measure of how many standard deviations a result is away from the mean – so to really get an idea we would need to know the standard deviation of school level ks2 scores – DO YOU HAVE THIS FIGURE IT WOULD BE VERY USEFUL?

      Looking at the data, something doesn’t look quite right – if it is a normalish distribution, you would expect 68 percent with + or – 1 on the Z score, but it looks like only about 100 schools (out of 3000?) that are outside a z score of 1?

      It is a long time since I did this sort of stats though so quite possible that I am horribly wrong on all of it!

      • Dave Thomson October 19, 2016 at 6:56 pm - Reply

        You’re not wrong Mike. However, the standardisation is first done at pupil-level then aggregated to school-level. So a score of 0.5, for example, means that the average pupil at the school was 0.5 standard deviations above the average pupil nationally. Hence the rather strange pattern. Selective schools are those on the right hand side of the graph.

  2. Alison Davies October 20, 2016 at 9:41 pm - Reply

    I would very much like to know my school’s z score to see where we are on the graph. Also the equation of the line to see whether we are above or below it. Is it possible to get this data?

Leave A Comment