Impact Assessment. A collaborative strategy for continuous improvement
Take a moment to think about an educational project you have developed. You have designed the resources, put in practice some tools and finally, you got a result, but how did you measure during all the process how people felt with your activities or how your project had a great effect on the participants and all people involved in?
Maybe the answers you can tell are: I talk with my participants in focus groups or after the activity; I collect some data using surveys about their satisfaction to know what they think; I reflect on how the sessions have been going for doing better next time and engaging more with my audience.
All of these strategies are part of an Impact Assessment methodology, but what do we mean by impact?
Impact Assessment
To have a social impact on someone is defined as having a strong effect. In culture, cultural heritage, and also in education, the social effect that produces an action has a powerful repercussion on how people perceive their daily life (Bollo et alii 2013: 22; NEMO 2020).
So, designing an effective impact assessment strategy is something really important for all. It implies thinking about how to measure this effect, narrate it, and learn from it using tools and actions taken from previous case studies to know how your actions affect your surroundings.
In this impact assessment, it is important to think about how the stakeholders are involved in. They should be part of all discussions but it does not happen in all cases (Pereira et alli 2013).
On the one hand, as all stakeholders are part of the project from the beginning, any action they take part in the project needs to be evaluated. In this way, whether or not your stakeholders are undertaking the role of actor or the role of the audience is needed to be taken into account and to identify their roles and their actions.
On the other hand, the effect of the project on the stakeholders should be internally and externally measured. If stakeholders are not engaged with the project, they are not going to be involved and the project is not going to go far away. It implies the outputs of the project and the development of it in the same way.
In the design phase, for example, any indicator created should answer a question and should help to measure quantitatively. To get it, as any project the SMART rule should be followed: to be Specific, Measurable, Achievable, Relevant, and Time-bound Objectives.
In this process not only stakeholders must be under control but also resources used, activities made, and outputs in short term and outcomes in long term. The way all the quantitative analysis is taken and transposed to an understandable mode is also important. For this reason, it is recommendable to use the most effective digital storytelling tools to involve all stakeholders in the post-project actions to improve what they do. An example of this is the tool Muse, developed by the École Polytechnique Fédérale de Lausanne ‐ EPFL to monitor the visits in Swiss museums implying the audience through multi-sensory digital storytelling to evaluate their satisfaction.
Now, let’s have a look at the Europeana Impact Playbook, a toolkit for digital heritage that can be very useful for doing an impact assessment of any digital heritage education resource and project whatever the audience is.
Europeana Impact Playbook
The Europeana Impact Playbook is a toolkit developed in 2017 by the Europeana Foundation in collaboration with the Impact community of the Europeana Network Association.
It is based on four phases related to the main tasks of an impact assessment: Design, Assess, Narrate and Evaluate. For now, the foundation has published on their website Phase 1 and Phase 2.
It includes suggestions, recommendations, and tools like the Empathy Map, the Change Pathway, the Value Lens, and the Strategic perspectives. These tools help to evaluate how the project is going, to evaluate how your audience feels about your actions, to find the correct indicators to measure and plannify their assessment among other purposes.
Bibliography
- Bollo, A.; Nicholls, Ann; Pereira M; Sani, M (2013): Report 3 — Measuring Museum Impacts. Emilia Romagna: LEM — The Learning Museum Network Project.
- NEMO (2020): EU Presidency Trio Conference. Museums and Social Responsibility. Values Revisited. Berlin: NEMO.
- Pereira Roders, A. R., Bond, A., & Teller, J. (2013). Determining effectiveness in heritage impact assessments. In Impact Assessment: The Next Generation: Proceedings of the 33rd Annual Conference of the International Association for Impact Assessment (IAIA13), 13–16 May 2013, Calgary, Canada (pp. 1–6)
