On the ethics of measurement
Just because you can, doesn’t mean you should
Each year in the media, we see a new version of this story: schools should measure more than just academics; students are not just a number; the stakes are too high for final year examinations. In many ways the topic should be moot. Record numbers of students now receive early offers, including unconditional ones. Schools already measure effort, organisation, communication and many other traits and dispositions. Where ATAR matters is mostly, as far as I can see, for the professions where one might be required to stick a scalpel into a person or build a sturdy bridge.
The movement is gaining traction it seems due to the policy agitation of a group of schools working with Melbourne University on New Metrics, a project to develop measurement tools for academic-adjacent skills and dispositions. Policy influence is a stated mandate of the group. I can certainly see a use-case for this, having worked in schools for special purposes for a number of years. I know New Metrics is working in this space, where so many assessment practices just don’t meaningfully fit with the specific challenges these teachers and students face. But this isn’t where New Metrics seems to predominantly be pitched.
Their blurb plays to the ‘industrial-model/jobs-of-the-future’ paradigm that we know so well. It denounces the ‘grammar of schooling’ positioning adopters as ‘innovative’ and ‘likeminded.’ I could mention that the 21st Century has hardly heralded the glory days of education. The university claims their measures have been validated and this interests me. I’m curious to know how these biologically primary skills are taught, and how we could benchmark the same. If academic assessment suffers from issues of human subjectivity then I do wonder what assessor training goes into this difficult to measure area.
The work of the ‘very different future’ tends to be preceded by several years of university, with its attendant broadening of horizons, challenges, expanding social circles and higher demands. Workplaces further enculturate students. The knowledge economy is said to be T-shaped: the requisite knowledge needs to be a couple of kilometres wide, a couple of centimetres deep and - to be really successful - a couple of kilometres deep in one area. This takes time. While noble and worth striving for in their own right, without the breadth of knowledge that a rounded schooling provides, traits like ‘kindness’ and ‘navigating diversity’ are not exactly bankable.
But the most obvious flaw to the argument for their measurement is that it raises the stakes on things like being a good person. When it comes to opportunity and reporting, these traits are ironically just as subject to manipulation as any other form of advantage. Privilege is one that has been acknowledged, where students with enough wealth can afford to volunteer to support the disadvantaged, while other students are the disadvantaged. It’s also ironic that the loudest voices in opposition to students being presented as simply a number want to quantify the traits that should be pursued as ends in themselves.
The equity argument is well-intentioned. We love to see every child acknowledged for their unique gifts, especially the high-effort, mid-level achievers who bring their A game to every lesson. But just like the heritability of intelligence, there is a school of thought that posits that working memory and executive function are largely biological. So metrics on ‘being systematic’ or ‘being diligent’ will favour the school-ready, while potentially disadvantaging boys and the neurodiverse1. Just as Freddie De Boer talks about the fetishisation of intelligence, we can now have the fetishisation of other traits. So while there’s an ideological shift away from meritocracy that is admirable here, it’s possible we will substitute one set of lucky circumstances for another.
In the depths of my soul, I’m worried that the problem is even deeper than this. Schools certainly do offer more than academics. Hartley2 talks about the contemporary model of school as filling a premodern, atavistic need for community. Rather than once upon a time giving through our churches or helping a neighbour, our children engage in community service through school. School communities have stepped in where extended families - and often even families - once were. Something about quantifying the outcome of good-person-ness makes me bristle for reasons that are deeply felt but hard to explain.
School choice is increasingly an arms-race. Is the goal to out-gun on every metric? As the expression goes, when you’ve got a hammer, everything looks like a nail. Just because we can assess anything that’s not nailed down, doesn’t mean we should.
No, this is not an argument for eugenics or an argument for not bothering to teach anyone anything at all. Read this if you’re a good faith actor who wants to know more.
Hartley., D. (2007). The emergence of distributed leadership in education: why now? British Journal of Educational Studies, 55(2), 202–214.
You have articulated concerns I have had on this topic so succinctly. Thank you.
Thank you very much for this Rebecca. Most insightful and thoughtful, as always. I couldn’t agree more. Students need to be able to do things at school and in life, that are not measured. Recorded, maybe; but measured and graded, definitely not. School is essentially a series of academic exercises that do need measuring, for reasons you state. I am appalled at the subjectivity that champions of the alternative movement wish to apply to these new measurements. One said it should be like job promotion. Well, there’s an area I have seen more corruption, misjudgement and consequential harm than in many other areas.
Melbourne University has been responsible for the worst assessment imposed on student-teachers, the QTPA test. It was a ridiculous attempt to measure Hattie’s ideas of impact. If you want a good example of unethical, immoral assessment, look no further.
Thank you once again.