Getting from B to A (or G to F)

By

In recent days there has been some discussion about the way grades in current GCSEs will be scored alongside reformed GCSEs when calculating Progress 8 in 2017 and 2018. In 2016, when Progress 8 will be fully introduced into Performance Tables, GCSE grades A*-G will be converted into points on the 1-8 scale. By 2019, all GCSEs will have been reformed and will be scored on the 1-9 scale. This leaves a mixed economy in the intervening two years in which some GCSEs will be graded 1-9 and others A*-G.

To bridge the divide, DfE recently announced a set of revised points to be used for current GCSEs in 2017 and 2018. These are shown in the ‘2017 Points’ column in the table below.

Points mapping for A*-G in 2016 and 2017
20150311-Table1

Selective schools are the biggest winners from grade re-scaling

So what difference might this make? To investigate, we calculate Progress 8 based on 2014 data for each state-funded mainstream school in England. We do this using a) 2016 points and then b) 2017 points. In the chart below, we look at the difference between a) and b) having allocated each school to one of nine bands based on the prior (Key Stage 2) attainment of their cohort. It seems reasonably clear that the small proportion of schools with particularly high ability intakes tend to benefit most from the 2017 points scale at the expense of schools with lower ability intakes.

Impact of re-scaling by intake composition of school

Moreover, the 2017 points scale, if applied to all qualifications and based on 2014 data, would drop a further 60 schools below the floor target of 0.5 points below expectation.

Whilst the differences in the chart seem large we need to remember that schools with a mean KS2 intake level 5 or above represent less than 7% of all secondary maintained schools. They are almost always schools with a selective intake. If we look at differences by school type then, going from 2016 points to 2017 points:

Progress 8 score by school type
20150311-Table2

Changes to Progress 8 are not substantial for the typical school

Whilst a gain of 0.15 for selective schools seems (and is) rather large we need to remember two things. Firstly, that Progress 8 is a value-added (VA) measure. Schools with high-attaining intakes tend, on average, to achieve better VA scores than schools with low-attaining intakes. As we noted at the time that Progress 8 was announced, pupils with high levels of prior attainment tend to make more progress when taught alongside pupils of similar ability. Secondly, we might expect some of the gap to close by 2016 (and beyond) as a result of non-selective schools increasing their entry rates in qualifications counted in Progress 8.

Even though the proposed change has little impact on the overall Progress 8 scores for comprehensive schools there is variation within this. Whilst the impact for most schools will be a change of less than 0.1 points some might see an increase or decrease of as much as 0.3 points.

How can we judge fairness in assigning points to grades?

The proposed change has led some to question the apparent unfairness of ‘2017 Points’. Moving a pupil from grade C to grade B, for example, would be valued more than moving a pupil from grade G to grade F. Might this result in a perverse incentive for schools to focus their efforts on pupils working at grades C or above?

Implicit in this question is an assumption that the ‘2016 Points’ are fair and that intervals between grades are linear, i.e. the level of difficulty of moving pupils from grade G to grade F is, on average, the same as the level of difficulty of moving pupils from grade C to grade B. As we have written previously, we do not necessarily think this is the case. Might the 2016 scale encourage schools to focus on moving lower ability pupils through the lower GCSE grades?

It may be that the proposed 2017 points more accurately reflect the degree of difficulty between grades at GCSE. But what is perhaps unfair is the lack of consistency between 2016 and 2017 and the additional volatility that the change may introduce, particularly for schools with lower ability intakes. As a result of awarding lower points to grades B and C, the national average Attainment 8 score is very likely to fall between 2016 and 2017.

As was suggested here  there is a danger that the introduction of Progress 8, generally regarded as a good thing, could be derailed by the complications of introducing reformed GCSEs on the 1-9 scale. Maybe the best thing to do would be to use the new points scale from 2016 onwards. That would, at least, reduce the level of change from 2016 to 2017.

By | 2016-12-07T12:55:38+00:00 March 11th, 2015|Exams and assessment, School accountability|

About the Author:

Mike Treadaway is Associate Research Fellow at Education Datalab, and was previously FFT’s Director of Research. A former teacher, lecturer and LA adviser, Mike has over 20 years of experience in working with education data in the context of school improvement. Since founding the FFT Data Analysis Project he has developed models for analysing pupil progress, has led the processing, matching and data analysis for the National Pupil Database, and has provided advice to the Department for Education to support the development of new school accountability indicators and analysis of national trends in school performance.

2 Comments

  1. […] our notional ability scale for a subject of average difficulty. This suggests, as we have discussed previously, that intervals between grades are not equal. In other words, the difference between G and F is not […]

  2. […] a previous blog, we noted that Grammar schools tended to achieve above average Progress 8 scores based on 2014 […]

Leave A Comment