The Impact of Cultural Heritage: creating a common language
If you work in a purpose driven, non-profit organisation and more specifically in the digital Cultural Heritage sector, we share a problem: we should be feeling fine and dandy because we work for a great cause, but we have very few instruments at our disposal to assess the results of our actions and be happy about it. This leaves us in a very unsatisfying and vulnerable position: the work we do comes at a significant cost to society but we can’t systematically assess that it was worth it. Did it improve the way our children are educated? Did it result in a stronger, more cohesive society? Did it enable artists to create groundbreaking new creative works? What positive change in society have we contributed to? In one word: did we have impact?
The digitisation of cultural heritage collections has been going on for several decades now, promising unprecedented potential for libraries, museums and archives to fulfill their public mission of opening up knowledge and culture to the participation and enjoyment of all citizens. Over the years we digitised tons of books, paintings and (audio-visual) archives to preserve them for future generations, develop new insights and allow others to create new works using all the advantages of new technologies. And so we did. We have seen some impressive examples of how it changed the way people engage with their heritage. Think Rijksstudio. Think Statens Museum for Kunst ‘Mix it up.’ Think Europeana 1914–1918. Think Retronaut.
Let’s assume for a moment that these initiatives have what we call ‘impact’. Any one of the examples above have attracted above average visitors online or offline. And I am confident that we can find lots of pictures of people experiencing our services with big smiles on their faces. The big question to answer is: how do we assess that positive change in a more structured way? What’s so good about it and can we do more of that? And once we understand the anatomies of success, can we demonstrate that we are consistently achieving our ambition to have that kind of impact? Because if we do, we could design for real improvements on people’s lives rather than blindly make content available and count the visits on our websites as the main sign of success.
Can we become more systematic in our approach, so that we can iterate faster, develop better services and provide real material evidence that we are reaching our objectives that goes beyond the anecdotal and the circumstantial?
Impkt.tools
With the beta launch of Impkt.tools we aim to bring focus to this discussion. Supported by the digital heritage networks of Europe, America, Brazil, New Zealand and Australia this initiative wants to develop the common language necessary to understand what we mean by ‘impact’ in the Cultural Heritage field. Impkt.tools aims to measure and assess impact, help us tell our stories in a way that is univocally convincing thus enabling us to design projects that create more impact. So join us, you are invited.
Impact, so what?
But let’s take a step back. Is impact perhaps the new ‘business model’ (which in common parlance is still too often interpreted as new ways to generate revenue rather than value creation- sorry Alexander Osterwalder)? Is it the latest fad taken over from the start-up lingo that will make your project proposals look snazzy but that you never got round to fully internalise in your organisation? Not if we are smart about it, and really integrate a sound methodology into our operations.
There may also be a sense of urgency to consider that we haven’t had to deal with before. In the old days nobody really questioned the need to have a publicly financed library, museum or archive. We have seen budget cuts yes, recently as severe as 40% or more in some countries in Europe. But in essence the need to invest in memory institutions and subsequently in digitisation was never seriously debated, so far. With the upsurge of data driven decision making we may have to hold our ground against other unquestionably meaningful purposes for public money. As Rob Stein in his excellent article Museums, so what? noted:
“The problem lies with the cultural sector’s inability to mount a compelling case of evidence to convince these “effective altruists” that tangible and meaningful benefit does indeed result from investing in the arts and culture.”
Never heard of effective altruists? That’s Bill and Melinda Gates wanting to see the highest possible return on investment from their charities, some call that your social return on investment... They’ll want you to report on that too to see how you contribute to the Sustainable Development Goals launched by the United Nations in 2015. And why not? Is it unreasonable to ask us to report on our contribution to ‘quality education’, ‘sustainable cities and communities’ or even ‘industry, innovation and infrastructure’ if everyone else including big for profit organisations do?
Impact, oh yeah!
But that’s just the defensive side of the story, a place where I don’t usually like to linger. Much more important is this: being clearer about the impact that we (can) have will make us more effective and relevant.
Consider the whole debate on Open GlAMs. For years the merits, advantages and possibilities of opening up digital collections for free re-use have been cast against a background of rather corporate thinking about the risks of losing money, control and copyright limitations. Not against the possibilities to achieve a much higher impact. Recent studies show that the institutions that do make the leap really benefit from it. Read Effie Kapsalis’ study The Impact of OpenAccess on Galleries, Libraries, Museums, & Archives.
‘It is the organisations that prioritise openness and sharing that are reaping the most benefits’.
Nothing new to advocates of the first hour. Prominent organisations like the Rijksmuseum and recently the Met have jumped on the open bandwagon. And while some debates relapse in the old ‘open’ vs ‘profitable’ arguments (read the interesting exchange between James Shulman and Merete Sanderhoff in Open Access can never be bad news) many organisations now see that the benefits of opening up their collections outweigh the costs of not doing so.
That’s great news, but I see one serious defect. The decisive arguments that get listed are still things like cost savings in the licensing department and improved brand recognition (which in turn may lead to better sponsorship deals). Now I am not against a bit of commercial common sense and pragmatism, far from it. But it seems to me that we are still attacking the problems and opportunities from the wrong paradigm. Our raison d’etre is not to break even. It is to fulfil our mission, whatever that may be!
Impact assessment can help us reflect what we are really about and get us over the tipping point where not only a couple, but most organisations will adopt re-usable and preferably open licenses as their default rights statement where possible. Not only because it makes economic sense, but because they see a direct relation between re-usable content and the impact they can have. Because impact is always about people, not about things, this should bring the debate to a whole new level. By making success dependent on the effect we have on people’s behaviour, knowledge, skills, status, wealth, well-being, or effectiveness we will start to look at open content as a mere pre-requisite to a higher ambition. This will effectively lift us from the stage of access to the stage of experience.
Under development: a new language
So how do we propose to do this, develop this common language? Not a trife question. It’s long known that we can gain interesting insights from people visiting our websites but that our value cannot be captured solely by Google analytics. There is more, much more out there that we contribute to. But that value is so much harder to capture and transmit in ways that really make sense to us.
Any language development starts with babbling, experimenting with uttering articulate sounds until we find some recognisable words. This should be followed by semantic relations so that when we combine words like ‘impact’ and ‘statement’, we still have a common understanding about what this means. It is upheld by a structure, a grammar if you like, which allows us to communicate about things like impact assessment in logical formats that we can compare.
We have developed a basic structure, based on Professor Simon Tanner’s seminal work, the Balanced Value Impact Model (Tanner, Kings College 2012) with a number of components that- while still far from poetry- we believe have the potential to become cornerstones of our framework.
Case study
We have taken a test run with that structure on the Europeana 1914–1918 project. This initiative, started in 2011 by Oxford University, invites people to share their personal family histories about the First World War and to have to have their memorabilia (often diaries, letters, notebooks) digitised. If you are interested in numbers: over the course of the last 5 years over 7.000 people in 24 European countries contributed their stories and in total close to 200.000 items were scanned, catalogued and stored. But what does that tell you about the impact this had on people’s lives? We‘ve been on location, and witnessed first hand the sometimes gripping and very emotional stories about suffering, longing for loved ones, the occasional happy moments, which have been part of people’s family histories for decades. But how do we transmit this to you who weren’t there?
We have interviewed and filmed a couple of dozen of people on location (users and non-users) and analysed survey results from 1506 respondents. The results were promising. We established that close to 70% of the respondents agreed that the service contributed significantly (8+ on a scale of 0–10) to their sense of common European identity. Apparently sharing your story and seeing that thousands of other families across Europe have similar experiences creates a bond. This meant a lot to us as this is the main impact the project strives for. Other results were less favourable. It became clear that people where disappointed in their expectations to learn from our project (69% hoped to learn a lot, 39% indicated they did).
The insight was that the potential to learn from all these stories is severely hampered because of the language barrier: believe it or not, very few Europeans master 16 different languages. You could argue that this could hardly have been a revelation to us, which thank god is true. But seeing this assumption reflected in numbers based on evidence makes a strong case against organisational lethargy. It brought us back to the drawing table and led to an add-on to the project which allows people to transcribe handwritten letters to machine readable (and therefore translatable) texts. A pretty good outcome in my books.
Some results just showed how immature our language is at this stage. We felt we needed to stay close to the original framework and therefore wanted to also assess the monetary value that people attach to the service. At the end of the day, judging your impact is only interesting when you can relate it to the investment you are making, right? As I have a commercial background, I am completely attuned to the idea that we should be looking not only for impact but for impact maximisation (small investments, high returns). But in this case it lead to the rather quirky conclusion that an investment of 1 euro in this project would lead to 1.93 in return (calculation: average amount that European citizens valued the service at divided by the costs of running the service. (See full calculation in the case study and dataset: Workers Underground). It’s not that this would not be a good result. It just feels like we are expressing our (social) value in the wrong currency.
At its core, the structure helps us to understand the causal relationship between the things we do, the direct results from that activity, the change that this leads to and ultimately it’s impact. By applying this systematically to our projects, we are able to formulate better impact statements to describe our desired impact (social, economic, innovation) and the output and outcome measures that will serve as key performance indicators of success. The instruments that we currently have at our disposal are crude, and will stay so for a while until we invest in creative new methods.
Rob Stein: “What if we focused instead on keeping a catalog and evidence of the creative imprint our audiences are exposed to and the impact they make on the world.”
For this we need cultural heritage thought leaders, cultural avant-gardists, conceptual designers, data crunchers & visualisers and survey experts. We invite you to work with us to develop a language that really sings. Join us on impkt.tools (beta).
This initiative is supported by Europeana, the Digital Public Library of America, Digital NZ, Trove, the Brazilian Institute of Museums and Whalebone & Greenstone.
This post was originally published on www.impkt.tools
