Josh and the Dragons: Measuring Creative Writing

By

For the last three weeks myself and twelve post-graduates who are studying Creative Writing around the UK have been immersing ourselves in dragons, bears, vampires and ghosts. No, we haven’t been overdosing on Netflix, we have been judging the creative writing of 11 year olds in what we think is the largest experiment ever of its kind.

In order to measure how the writing of children develops in Year 7 we asked 12,815 pupils at 84 schools to read a short text and then continue the story in the same style. We will ask them to repeat the exercise in June, and then measure whether their writing has improved.

So how do you measure creative writing? Surely creativity cannot be measured?

We do not ask our judges to place absolute measures or marks on the pupils’ writing, but rather to make comparisons between pieces, and to state, in each case, which is the more creative response. By taking this Comparative Judgement approach we do not need to define in advance what good creative writing looks like; rather we can rely on the tacit knowledge of judges. By making a series of comparisons we slowly build up a scale of creativity that pools the knowledge of all our judges and delivers a reliable measure of creativity.

So on what basis are the judges making their decisions?

Here is some of their feedback:

thinking about the format or point of view of the story is important

 

make sure your tenses all match up and your grammar leaves space for the reader to breathe

 

I noticed those writers who used an unusual metaphor or simile. I liked the writer who was not afraid to stay on a character, like a scene in a movie, drawing out the moment, rather than lose the impact by too much action or activity. 

 

‘and then my best friend Josh came and we played football and forgot all about the big scary monster’ (I love those ones – good old Josh!)

 

I wonder if some good examples from the latest pieces of writing could be made available for the pupils to analyse to find more creative ways of ending their stories?

If we trust in the expertise of our judges we don’t need to construct complex rubrics or schemes for them to follow.

In order to measure whether pupils’ writing improves, we will repeat the exercise in June. When judges come to judge, we will include pieces from September, but the judges will be unable to tell which were written in September, and which were written in June. By directly comparing the writing from the two sessions we will learn whether pupils’ writing has improved. Participating schools will learn whether the writing of their pupils has improved or deteriorated relative to other pupils in the project.

So if you are ready to ditch your mark schemes and make a genuine attempt to measure progress, get in touch and find out more about the proof of progress tests we are developing in English and Maths. You never know, next year you might be able to persuade your pupils to ditch Josh and the dragons and start developing a writing style that will serve them well through to their GCSEs.

More information on the project is here:

To get in touch contact: chris@nomoremarking.com

By | 2017-03-03T09:48:28+00:00 November 17th, 2015|Exams and assessment|

About the Author:

Chris Wheadon is a Research Associate at Education Datalab and is one of the UK’s leading experts in assessment. He is the Founder of No More Marking Ltd, a company that uses comparative judgement to assess work more accurately than traditional marking techniques. He has designed the Proof of Progress tests from FFT.

Leave A Comment