On blaming the test

Standardised testing does not cause poor writing. Poor instruction causes poor writing.

Photo by Johannes Plenio on Unsplash

For the moment, leave aside your observations that this piece reads like an advertorial for Steiner schools. TL;DR - the article, based on a recent study, states that the likes of Hemingway and McCarthy would not have passed NAPLAN. In one sense, it’s a case of same argument, different day. There’s a long history of teachers, leaders, systems, and pundits blaming the test for student literacy.

The difference here is that the researchers deliberately set out to promote their approach to teaching writing by setting it up in opposition to the standardised test. Sounds clickbaity, but the argument itself is a kind of canny marketing strategy. The total cynic in me was intrigued and thought it was worth a deeper look.

The basic argument presented is that NAPLAN (our big, scary standardised numeracy and literacy test in Australia) kills creativity. Strangely, in the SMH article, the authors try to argue that there is little lexical complexity in the below passage by Cormac McCarthy and thus he would not pass NAPLAN.

The road was empty. Below in the little valley the still grey serpentine of a river. Motionless and precise. Along the shore a burden of dead reeds. Are you okay? He said. The boy nodded. They set out along the blacktop in the gunmetal light, shuffling through the ash, each the other’s world entire.

I don’t know about you, but I find this argument a bit of a conceit. I challenge you to find a Year 9 student who would have the kind of post-modern self-reflexivity and knowledge needed to break these rules. The claim that a savant-like child would produce this and be penalised is a very weak analogy indeed.

The purpose of NAPLAN in the article and the research paper is willfully ignored. This nationally standardised test is not designed to pick up on creativity (but certainly background knowledge doesn’t hurt in creating an engaging response with scope for flair). The test is designed to measure literacy. It is also designed to be nationally standardised and therefore useful for measuring progress and making comparisons, something the Steiner program didn’t achieve¹.

I also suspect that the claim that teaching to the test is an issue in secondary schools is false. I can only speak from experience, but I have never taught NAPLAN, other than to revise text types and features to familiarise students with the format of the test. We know coaching has a small but limited benefit when it comes to navigating examinations.

The researcher/instructors seem to be of the workshop persuasion, with one of the authors offering sessions where, “I want to share my love of words with the next generation of writers. You don’t have to be a good speller. You don’t have to be a good grammarian. You just have to hop online with a pen, some paper and an open mind.” The paper itself argues for the teaching of creativity over instruction and with an essence of balanced literacy approaches (italics mine):

Seemingly unmeasurable, “creativity” is not taught; only the various forms of writing, such as narrative writing, essay writing, personal writing, persuasive writing, or speech writing are explicitly taught.

Without this unquantified capacity, writing tasks are mechanistic. It is every young writer’s job to make their words work well and to reflect as best as they can an imaginative realm of ideas, a story, or landscape. To do that, young writers need the space to imagine things not prescribed and secondly to be taught craft-based skills.

The program seems to focus on macro textual features rather than the mechanics of writing and encourages students to “perform creative pedagogical strategies in the classroom in order to model (and assess) student efforts at being creative.” This seems to have a strong affiliation with the kind of ‘thinking like a mathematician/scientist/entrepreneur’ that we have seen in recent syllabus iterations. And the focus on achieving a minimalist style (such as in the Hemingway example) means that poor scores when measured against NAPLAN criteria are contrived as a foregone conclusion.

Unsurprisingly, “students showed improvement in the areas covered in the creative writing project that aligned with the creative writing rubric; it is evident that the NAPLAN rubric does not assess imaginative capacity.” So the researchers develop criteria, teach to criteria, assess according to criteria, and then imply that comparison to NAPLAN makes this an ‘x vs not-x’ study design. High fives all ‘round.

I’m going to leave you with a passage written by someone who, in all likelihood, didn’t get the benefit of progressive creative-writing-immersion. He probably had to sit in rows. I’m sure you will recognise it as a failure of the system.

“He remembered the time he had hooked one of a pair of marlin. The male fish always let the female fish feed first and the hooked fish, the female, made a wild, panic-stricken, despairing fight that soon exhausted her, and all the time the male had stayed with her, crossing the line and circling with her on the surface.

He had stayed so close that the old man was afraid he would cut the line with his tail which was sharp as a scythe and almost of that size and shape. When the old man had gaffed her and clubbed her, holding the rapier bill with its sandpaper edge and clubbing her across the top of her head until her colour turned to a colour almost like the backing of mirrors, and then, with the boy’s aid, hoisted her aboard, the male fish had stayed by the side of the boat.

Then, while the old man was clearing the lines and preparing the harpoon, the male fish jumped high into the air beside the boat to see where the female was and then went down deep, his lavender wings, that were his pectoral fins, spread wide and all his wide lavender stripes showing. He was beautiful, the old man remembered, and he had stayed.”

1. “The analysis of inter-rater reliability showed that the raters applying the creative writing rubric criteria did not reach the high level of inter-rater reliability required by a standardized national test.”