In recent days there has been some discussion about the way grades in current GCSEs will be scored alongside reformed GCSEs when calculating Progress 8 in 2017 and 2018. In 2016, when Progress 8 will be fully introduced into Performance Tables, GCSE grades A*-G will be converted into points on the 1-8 scale. By 2019, all GCSEs will have been reformed and will be scored on the 1-9 scale. This leaves a mixed economy in the intervening two years in which some GCSEs will be graded 1-9 and others A*-G.

To bridge the divide, DfE recently announced a set of revised points to be used for current GCSEs in 2017 and 2018. These are shown in the ‘2017 Points’ column in the table below.

Points mapping for A*-G in 2016 and 2017

Selective schools are the biggest winners from grade re-scaling

So what difference might this make? To investigate, we calculate Progress 8 based on 2014 data for each state-funded mainstream school in England. We do this using a) 2016 points and then b) 2017 points. In the chart below, we look at the difference between a) and b) having allocated each school to one of nine bands based on the prior (Key Stage 2) attainment of their cohort. It seems reasonably clear that the small proportion of schools with particularly high ability intakes tend to benefit most from the 2017 points scale at the expense of schools with lower ability intakes.

Impact of re-scaling by intake composition of school

Moreover, the 2017 points scale, if applied to all qualifications and based on 2014 data, would drop a further 60 schools below the floor target of 0.5 points below expectation.

Whilst the differences in the chart seem large we need to remember that schools with a mean KS2 intake level 5 or above represent less than 7% of all secondary maintained schools. They are almost always schools with a selective intake. If we look at differences by school type then, going from 2016 points to 2017 points:

Progress 8 score by school type

Changes to Progress 8 are not substantial for the typical school

Whilst a gain of 0.15 for selective schools seems (and is) rather large we need to remember two things. Firstly, that Progress 8 is a value-added (VA) measure. Schools with high-attaining intakes tend, on average, to achieve better VA scores than schools with low-attaining intakes. As we noted at the time that Progress 8 was announced, pupils with high levels of prior attainment tend to make more progress when taught alongside pupils of similar ability. Secondly, we might expect some of the gap to close by 2016 (and beyond) as a result of non-selective schools increasing their entry rates in qualifications counted in Progress 8.

Even though the proposed change has little impact on the overall Progress 8 scores for comprehensive schools there is variation within this. Whilst the impact for most schools will be a change of less than 0.1 points some might see an increase or decrease of as much as 0.3 points.

How can we judge fairness in assigning points to grades?

The proposed change has led some to question the apparent unfairness of ‘2017 Points’. Moving a pupil from grade C to grade B, for example, would be valued more than moving a pupil from grade G to grade F. Might this result in a perverse incentive for schools to focus their efforts on pupils working at grades C or above?

Implicit in this question is an assumption that the ‘2016 Points’ are fair and that intervals between grades are linear, i.e. the level of difficulty of moving pupils from grade G to grade F is, on average, the same as the level of difficulty of moving pupils from grade C to grade B. As we have written previously, we do not necessarily think this is the case. Might the 2016 scale encourage schools to focus on moving lower ability pupils through the lower GCSE grades?

It may be that the proposed 2017 points more accurately reflect the degree of difficulty between grades at GCSE. But what is perhaps unfair is the lack of consistency between 2016 and 2017 and the additional volatility that the change may introduce, particularly for schools with lower ability intakes. As a result of awarding lower points to grades B and C, the national average Attainment 8 score is very likely to fall between 2016 and 2017.

As was suggested here  there is a danger that the introduction of Progress 8, generally regarded as a good thing, could be derailed by the complications of introducing reformed GCSEs on the 1-9 scale. Maybe the best thing to do would be to use the new points scale from 2016 onwards. That would, at least, reduce the level of change from 2016 to 2017.