Will more training in moderation for Key Stage 2 writing, announced by Justine Greening yesterday, help reduce the inequities that were highlighted in a recent blogpost on Key Stage 2 writing? Sadly, the evidence suggests otherwise.
If you look through the literature on examination standards over the last 20 years, you will find references through the 1970s, persisting through to the early 1990s, to examiners being able to carry standards in their head. In the same way cricket umpires used to be able to say you’re out because I say you’re out, an examiner could say, it’s a grade C because I say it’s a grade C. These days, umpires and examiners often have to defer to sources other than their intuition. After decades of grade inflation in the 1970s and 1980s no-one would seriously argue for a return to examiner judgement in the maintenance of standards within the examination system.
However, while the idea of examiners holding a privileged view of standards has lapsed, we still find the idea alive and well in moderation. One person’s mark, the moderator’s, is privileged over another, presumably because, through training or experience, they can see something that others can’t. In the literature on marking you will sometimes see the senior examiner’s mark defined as the true mark, against which all other marks should be consistent. This definition is outdated. Modern measurement theory defines the true mark as the one that would be given by an infinite number of markers.
The great failure of the privileged or hierarchical true mark definition was exposed in the 2012 GCSE English judicial review. Moderators had been unable to prevent inequities in marking of controlled assessment, which led to schools receiving inequitable grades. No one could question the integrity of the moderators, or the training they had received; they were simply being asked to do a task they cannot do. Humans cannot perceive absolute standards. Humans cannot make absolute judgements on marks.
So how do we get an infinite number of markers, so a pupil receives their true mark for their Key Stage 2 writing? Well, we can’t get an infinite number, but there are around 200,000 teachers working in primary education. If teachers start working together, by making relative judgements on writing, and sharing those judgments with each other, we can make a great start. Comparative judgements of pieces of work is something that humans, in particular teachers, are very good at. If we can replace the idea of the privileged knowledge of a moderator, with multiple expert views gathered nationally, then we can avoid future inequities in moderation.
Our research has now shown that Comparative Judgment, which is designed to collect multiple judgments on every piece, presents an efficient, reliable alternative to moderation. Last year No More Marking worked with 5 primary schools to devise an alternative to their Key Stage 2 moderation procedures. This year we want to work with 250 schools. Once we have 2,500 schools we will be able to present the Department for Education with a solution that works.
If you are a primary school you can sign up to join us here (there is a cost) : https://www.sharingstandards.com/
Anyone else can get started with Comparative Judgment here (free for all research and educational use):