Critiquing culture sector KPIs via school socks

Yesterday I joined in the quote-tweet trend:

I’ve been asked to say a little more, but as a swerve, I’ll begin with school uniforms.

My kids’ school is a little obsessive about school uniforms — their view is that it promotes pride, reduces bullying, and a sense of common purpose. The lives experience of the students, however, is expressed by their meme:

Reframing this. The school’s goal is ‘pride, cohesion, safety’. The Performance indicator they are currently using for this is ‘how many kids are in correct school uniform today’? This tells us something, but tells us nothing about whether the cohort feels pride, cohesion, or that they are bully-free.

There is nothing wrong with the KPI in itself. But it is an indicator, not an outcome. The purpose of the uniform is not to have pupils in the correct coloured socks — but that is the result of applying the measure.

Taking one museum as an example, Museums Victoria (MV) has an amazing clause in its founding Act:

23 Functions
The functions of the Board are —
… (aa) to control, manage, operate, promote, develop and maintain the Exhibition land as a place…
…c) to exhibit material from those collections for the purposes of education and entertainment;

Check that out: ‘entertainment’! Excellent. I love it. Leaving aside whether MV is or isn’t doing that, let’s look at the KPI that’s in place to monitor that. In the Victorian Government’s Budget Papers, there is this:

Ignoring the numbers (2015–16 was good though!), look at what’s being measured. ‘Satisfied with visit’. Satisfied! I’m unsatisfied with that. The Act says ‘Educate and entertain’ — but the KPI is ‘Satisfied’. This sure is necessary to achieve the Act’s goals, and it’s perfectly possible to measure education and entertainment — but the KPI does not demand such measures. This could be for a range of reasons; perhaps commonality of benchmarks across the sector or through time, or perhaps ease.

It’s a complex system. There’s a board, an Executive, a staff, with professional skills. there are marketing and visitor services staff, audience surveying, and many more things all working to make the visitor experience great. But my point is that the mechanism of applying KPIs is not driving this outcome. The various forces working to deliver it (or not) are doing so despite its absence from the KPIs.

Just like with the school socks, KPIs are Key Performance Indicators — they aren’t supposed to be the goal. If i want to buy my friend a birthday present a KPI might be ‘i have bought and wrapped a present’. But the actual goal is ‘my friend is delighted with her present’ (that ‘my friend is still my friend…!’). If i can achieve the be deemed successful by buying some tat and leaving it on her table after dinner — this meets the KPI 100% — then maybe i’d do just that. In reality, i need both the KPI — i got her a present — and measurement of the outcome — she was delighted by her gift.

To bring it back to museums: if all we have to do is leave visitors ‘Satisfied’, we could cut some corners on educational efficacy while looking after the baselines. We have created a system where the only guarantee a museum is educational is from the desires and skill of its leaders and staff. That’s not good enough.

The attention of Boards, CEOs, CFOs and staff will always be influenced by what is formally measured. Measuring only satisfaction seems such a small target. Where is the demand for education, for entertainment?

I lament the missed opportunity. Museums should do more, should work harder. Should earn their social license constantly — they should be an active instrument of building a better society. So we should have outcomes for ‘healthy society’ or ‘socially cohesive town’ — for which KPIs could include ‘loved by locals’, or ‘prescribed by doctors for wellbeing’.

So what happens when the KPIs known to be pointless lip service? Well, then they are ignored and the Board and CEO have freedom, for good or ill: if nothing is measured tightly, then you can’t be held accountable for the wrong thing. This can lead to innovation — no KPI initiated Industry co-working spaces- but it can also lead to rogue activities that aren’t within the remit of the organisation and aren’t serving a good and valuable purpose. You can all think of examples, I am sure. (perhaps the errors are worth accepting for the innovation benefits? I don’t know.)

There is certainly a cost to collecting KPI information — is it worth it? Would it be better to use this time and money to do, say, one case study per year as a deep dive into an area of impact and value?

I focused on one example above, but you can pick any number of other issues in contemporary museums and the KPIs are misaligned:

• ‘Collections kept in industry-standard conditions’ — bleurgh. Creates a rush to super-serve collection conditions, arguably to the detriment of digitisation etc. efforts, which vary enormously according to the type of collection.
• Wouldn’t it be nice to have a ‘collection relevance’, for which deaccessioning might be a KPI?
• ‘education visits’ — bleurgh. Value to school pupils is the actual goal here. This KPI creates a bias to serve a 1.5hr drivetime and neglects to measure teacher CPD or online resources, which are arguably of better value to formal education sector.

I am sure there are lots of contrary views to this, and I am far from expert in policy formulation at a government level. I’d love to hear contrary views! In the meantime, I think we could all pull our (matching) socks up and focus on outcomes, not the indicators that we’re on the right track.