MEL Tech @ Mercy Corps — Two Years On
By Hanna Camp, Senior Advisor — Monitoring, Evaluation, and Learning Technology
In late 2020, Mercy Corps started an effort-intensive process of improving its use of Monitoring, Evaluation, and Learning (MEL) Technology across our global programming. MEL Tech (adapted from and inspired by the MERL Tech community) at Mercy Corps means any software platform used frequently by Monitoring, Evaluation, and Learning teams to collect, store, analyze, and visualize data about program activities and outcomes achieved. This data is a critical source of information about how well our programs serve communities and where we could improve. But too often, without a unified organization-wide MEL Tech approach, the data was also difficult to use well.
You can read more about the MEL Tech challenges Mercy Corps faced and faces, and the steps the MEL team took to address them here. This work was greatly supported by the Cisco Foundation, which generously funded a multi-year Data-Driven Decision-Making initiative focusing on improving Mercy Corps’ MEL Tech processes and also funded the creation of a MEL Tech training program for Mercy Corps. This training program consists of both self-paced and facilitated learning modules covering all of the priority MEL Technologies, and was rolled out to students for the first time in the summer of 2022. This article details the results achieved in that first training and the adaptations made along the way.
What have we learned?
After two years working to improve MEL Tech, utilization of technology for MEL at a global level has improved, as has collaboration with key stakeholders such as IT. The MEL team has also made many other observations, some expected and some unexpected, especially in the wake of the first MEL Tech training.
One observation that was very much expected was that technology champions working directly in programs have played a critical role in uptake of priority MEL technologies. The training program, and the personal connections developed through it, opened up opportunities for those tech champions to try something new to address stubborn problems, such as disorganized data, underutilized data, and inefficient data collection and storage systems. Through a series of small awards — called “jumpstarts” — the MEL team was able to strategically fund projects pitched by MEL Tech Training graduates that allowed them to implement priority MEL technologies in their programs.
Data collection and dashboarding technologies saw the quickest uptake among both jumpstart project awardees as well as trainees in general. This outcome was also expected due to pre-existing knowledge of the unmet needs in programs (observed in multiple global surveys of MEL Tech use) and because the global MEL and IT teams had done significant work to make procurement and payment for those technologies significantly easier.
Finally among the “expected outcomes”, the team found that opportunities to build skills in MEL Tech are valued by program staff. The feedback from the training was very positive, and Mercy Corps’ MEL team has committed to making regular MEL Tech trainings a part of its work for years to come.
There were also some unexpected outcomes observed. In addition to providing training on foundational data collection and visualization technologies, the training also sought to build skills in less commonly used technologies, such as platforms for qualitative analysis, GIS analysis, and statistical programming. After the training, we observed much faster uptake of qualitative analysis softwares than anticipated, even when considering the demands expressed in the global surveys. Trainees and program teams repeatedly said that they had had a great deal of qualitative data that was underutilized due to the difficulty of analyzing it in Excel. The skills developed in the MEL Tech training enabled them to think through new ideas for use and put those ideas into action. Multiple trainees even conducted cascade trainings on qualitative analysis technologies within their offices to demonstrate the value.
Another unexpected — but welcome! — observation was that across our interactions on MEL Tech, whether during trainings, in jumpstart projects, or in regular design and support conversations, there has been significant interest in more standardized MEL Tech systems and tools. Standardized tools include things like data collection forms, database structures, dashboard templates, and even automated data pipelines that connect all these components together. For years, there was debate about whether standardized systems would even work, let alone be accepted by program teams which each operate in a unique context. As the Mercy Corps MEL team has continued to refine its MEL Tech approach, we’ve found that there are workable levels of standardization that can be implemented in almost all data technologies. These approaches each strike a delicate balance between giving teams a needed level of flexibility, while also maintaining consistency in data structures, naming conventions, and other key parameters.
What next?
These standardized tools, aggregating up into fully automated data pipelines, are increasingly where MEL Tech is headed within Mercy Corps. While there is a significant amount of work still to be done, automated data pipelines have dramatically reduced the effort and time required to produce MEL information in every program where they have been implemented. However, there will need to be more internal learning on how best to adapt and manage them in each context, and a more intensive focus on developing internal cultures of data interpretation and team discussion, in order to ensure that information is used to maximum effect.
On top of this, we’re seeing an increased need for context analysis. More and more programs are operating in a remote or reduced access capacity, and many more simply in a highly dynamic, fast-paced context. Almost all face increasing risks and volatility from climate change. As a result, monitoring of the physical, social, and environmental context is more important than ever. For all the progress made, there is still a tremendous amount of data and insight that we have yet to make use of, and which could significantly improve our ability to serve and empower the communities we work in.
