On intangibles and accountability

Why schools should be able to support their claims about non-academic outcomes

Photo by Andrea Tummons on Unsplash

When it comes to questions about merit or performance pay, there’s an interesting paradox in the claims of teachers and leaders that, ‘You can’t measure what we do.’ In opposition to this, the marketplace of education suggests that schools are claiming to offer a lot. Can we have it both ways?

I looked at a small selection of private and government offerings and found that there wasn’t a substantial difference in their graduate outcomes. This school aims to develop your child into a man who is “creative, confident and accomplished” and “equipped to succeed in workplaces and industries yet to be created, and jobs and roles yet to be imagined.” This school aims “to develop young men and women of character who will stand up against injustice and serve the communities in which they live.” Similarly, this school supports “the development of self–aware, empathetic and resilient lifelong learners.”

These offerings seem kind of abstract and immeasurable. If what schools are offering at the end of 6-13 years of schooling can be distilled as ‘nice, capable people,’ then why state it at all? What really makes one school so different to another? And can this even be substantiated?

I spoke to Scott Eacott at UNSW about his work on alternative measures of schooling and why they’re needed. Here’s what he had to say:

Schools, educators and communities have long claimed that they deliver more than academic outcomes. There is an interesting article by James Ladwig that engages with this very topic. However, while the claim is common there are often two things that follow. First is the declaration that those things cannot be measured, and following that, an inability to hold anyone to account on whether they actually deliver on the claim because after all they cannot be measured.

Rather than simply seeking to hold schools to account, the stimulus for the Building Alternative Indicators for Schooling project is about working with schools to evidence what it is they claim to do – a focus on impact. Building on Ladwig’s argument, if you can claim to deliver something (e.g., 21st century critical thinking) then you need to have some idea what that looks like. If you articulate what it looks like you can backward map that into action. You can translate that into a model of instruction (or curriculum, assessment etc) and this becomes what you would expect to see in a classroom. The very claims of what the school says it delivers become the core focus of teaching and learning, and instructional leadership in the school. By developing this degree of organisational coherence, a school should be able to evidence whether it is delivering on its stated claims and justify decisions on what is done (or not done) based on its alignment.

In the absence of such a project, communities (including not just prospective families, but systems and government) have to just accept at face value claims of delivering something even in the absence of any data or evidence to confirm it.

What do we need to be cautious about when designing these measures? 

The key thing is to not simply add more measures on to school – and more importantly the lives of children and youth. The last thing educators and schools need is more data generation for the stake of data generation. Similarly, we do not want to new measures for the point of comparison. There are already sufficient measures to make comparisons (not saying we should) within and across schools.

Any new indicators should not come at the expense of existing performance. That is, we cannot sacrifice current performance (e.g., teaching reading, writing and mathematics) to achieve these new indicators. If anything, if schools are claiming these distinctions already, they should be able to evidence them without too much work, as it is more likely to be about curating existing data or recording something more systematically.

What do you envision as the end product and application of BAIS? 

My idealistic end product is that schools are able to shift the narrative from MySchool to Our School. That is, schools are able to make empirically defensible claims about what they deliver. These claims do not need to be the same for all schools, but the idea of being able to actually support your claims in the face of critique or questioning is important. The types of schools this would be most of value for are small schools, and those who claim an identity build on being different. Schools are relatively standardised places, if we want to make claims to being different we should be able to show what it looks like in graduates, in classrooms, through curriculum and in the design of assessment tasks.  

Most importantly, as a project, the Building Alternative Indicators for Schooling work is about working with not on schools. It is about empowering educators and schools to develop the type of evidence they need to support their work and build the types of supports they need to sustain that over time. In this way, schools build their practice on continuously improving their impact on students. It is not easy work, but then again things that are worth it rarely are.

There’s an interesting example presented in the article that has shaped Scott’s work. The author uses the example of the Cronulla Riots as evidence of the failure of character education to live up to its mandate. Ostensibly, everyone involved went to a school that aimed to develop a more enlightened citizenry. Similarly, it remains to be seen whether the cross-curricular Sustainability priority will have any impact on the ways that future generations conduct themselves, consume, and even vote.

A greater rationalisation of what schools are willing to invest time and resources in will potentially have two positive impacts: schools will be able to refocus their non-academic offerings on ‘what works’ in a similar way to the uptake of evidence-based classroom practice; and communities can have greater assurance that their choice of school might result in their child developing into a critical/empathetic/resilient individual.