The Agent of Change Challenge: Measuring Results to Amplify Impact
By Agnese Cigliano, ASEF Higher Education Young Reporters
Ultimately, the objective of the Capacity Building Workshop on Equitable Access and Success in Higher Education is to create more equal, accessible, and sustainable opportunities for everyone everywhere. This target is a very practical one and, to be effective, requires the proactive engagement of all participants operating as agents of change, championing equality and success in education. Ancillary requirement, yet not to be overlooked, is the ability to capture that change, being persuasive in describing its footprint, and fundamentally confirming the role of equality as a decisive priority in the field of higher education. Referring to the monitoring and evaluation aspect of equitable access and success in education, Matthew Pavelski, Evaluation and Impact Officer at Lancaster University (UK), and participant of the CBW, commented:
“It’s a really tricky problem. Having ambitions in developing projects that we hope and we think will have a real positive impact on students from the view of widening participation backgrounds is one thing; devising an effective method of evaluating that and embedding it within processes, encouraging practitioners, researchers, and others involved in this area to think with an evaluation hat at all times, and trying to make their activities and interventions demonstrable in terms of impact, is an ongoing, but separate, challenge”.
Then, the question is how to address this multi-layered issue to make impact both informed and effective?
Budgetary Allocations: Problem of and Solution to Equality, Diversity, and Inclusion
In an ideal world, there should be no justification required to promote equitable access and success in higher education. In practice, this principle requires the implementation of policies, processes, and investments that have to be “won” over other priorities within higher education institutions.
One of the expert facilitators of the CWB, Prof. Greame Atherton, provided a clear picture of how complex the issue can be from an operational perspective:
“It’s not all about quantitative data, it’s also about a qualitative approach and a balance between quantitative and qualitative approach. Particularly, it’s a balance of devoting resources. One of the reasons why many countries don’t have the data collection systems adopted by countries like Australia and UK — that probably lead the world in terms of depth, nature and range of collected information — is resources. In a world of limited resources, if you have X million to spend, are you going to spend that in collecting better data, or are you going to invest that in supporting students and building universities at national level? We have got these trade-offs in the real world, so that particular context may imply that there is less data available”.
Expert facilitator Mary Tupan-Wenno, Executive Director of the Centre for Diversity Policy in The Netherlands, reiterated how resources distribution and budgetary issues impact the conversation on equality, diversity, and inclusion: “Diversity and inclusion should be complementary, they should be part of the design, they should be a thread. In fact too often, diversity and inclusion as a policy area is seen as a separate silo track within policy and practice, while it should be a facet of every policy track within institutions. Everybody agrees that it’s important, but the moment you have to put money to achieve it, that’s when it becomes difficult. That’s definitely one of the challenges for the future. That’s why I suggest including diversity and inclusion into designing processes and making sure that the voices of the communities you would like to change are taken into account. It’s important that institutions co-create with students from scratch”.
This problem, and true challenge, is almost a reality check for higher education professionals, who have to make a strong case for equality, diversity, and inclusion, making the trail towards equal access and success in education more uneven. And yet, it is in the most challenging situations that we usually find the driving energy and inspiration to come up with transformative solutions.
Quantitative and Qualitative Data: The Human Experience Behind the Numbers
“Numbers don’t lie”, clarified Dr. Edizon Fermin, Vice-President for Academic Affairs at the National Teachers College of The Philippines, during his session on monitoring impact and innovation. How numbers are generated is a complex corollary question, but it is apparent that we cannot bypass quantitative data to measure impact.
Equally, we cannot omit that numbers have stories to tell.
Referring to the impact of the Covid-19 pandemic on measuring and evaluating the outcomes of equity programs, Dr. Edizon Fermin highlighted: “When we deal with outcomes and evaluation of our programs we are not just dealing with numbers, but more importantly with the stories that accompany them”. This is the most meaningful part of monitoring and evaluation, in the words of Fermin.
Based on these considerations, higher education professionals are called to balance numbers and stories if they intend to effectively portray social innovation and the merit of their programs in the field of equality, diversity, and inclusion. This is where thick data comes in place. Thick data is to qualitative data as big data is to quantitative data. The inner qualitative nature of thick data makes it such that it doesn’t focus on volume, as bid data does, but on the variety and complexity of the qualitative data acquired. Here is where the different stories — in the form of case studies, focus groups, interviews, and any other resource involving beneficiaries and offering more insights on the impact achieved — are collected and can define the way we look at the success of programs.
The full depth of this set of information, collecting numbers on indicators in a quantitative fashion as well as the impact experienced by the beneficiaries from a qualitative point of view, it also seems appropriate to zoom out and look at the contexts in which the impact to be achieved will flourish. As pointed out by Pusa Nastase, Lead of the Global Professional Development Programme at the Central European University, “There are all sorts of indexes on corruption, on good governance in countries, and these numbers correlate strongly with a more inclusive higher education system in many parts of the world. There are outliers, but overall, good governance is the prerequisite for a strong university system which is embedded in a society promoting the values of transparency and equality”. This is an important reminder to always measure social impact against pre-existing conditions and looking at the multi-layered variable of change.
Finding the “Why”: A Path of Empathy
Numbers and thick data are the tools to measure impact and, from a practical point of view, to showcase the meaningfulness of programs provoking budgetary allocations and inviting funding. However, the key solution to the challenge we looked at lies in empathy.
Through empathy, and by connecting to the real stories hidden behind data — the stories of students who could complete their studies only thanks to focused attention on equality, diversity, and inclusion, and tailored solutions to respond to specific complex needs — we are able to identify the “why” of social impact, and the purpose of programs aiming at achieving it. The needs of the beneficiaries, the indicators we want to focus on, the outcomes that matter the most, are all connected by the fil rouge of purpose, and they would only be partial and incomplete if they did not look at the human experience through the lenses of empathy. That “why” and that understanding for a common human experience are what will lead to social change on a small scale, such as the implementation of equality policies in one single university in one country, to a global scale, including the achievement of the Global Goals.
