If no one uses the data, is it useful?

Nine thousand mothers reached, 50% reduction in disease burden, 100% satisfaction rating on program implementation. These numbers are music to donor and development practitioners’ ears alike. It seems that everyone is now in agreement that accountability, transparency, and data-driven strategy are all critical factors for responsible development practitioners. However, in the wake of a paradigm shift within the social sector towards measuring impact and program performance, some key components around developing a deliberate but malleable system have been lost.

Collecting reams of data is commonplace in the business sector, but are now just becoming de rigueur for non-profits. Rather than straightforward measurement of overall profit or product demand, the social sector deals with assessing complex objectives that tackle social, political, and environmental factors. We cannot afford to take the same approach if our goal is to fully understand the issues we hope to address.

Unfortunately, a majority of development M&E systems are built around short-term (2–3 year) projects and within static frameworks that assume a linear process. But the reality is that most of us are trying to create long-term change around complex and non-linear processes. Many areas of development are still complete black boxes. If a known playbook laid out how to solve our social problems there would not be a need to monitor or evaluate any of our programs. However, that simply is not the case.

Developing the Splash M&E program:

Monitoring and Evaluation will not serve as a silver bullet to the challenges we face implementing our program. However, we believe developing our M&E program in a deliberate, child focused, and participatory way will ensure a more sustainable and effective M&E program in the long run.

Over the last few months, countless hours have been spent connecting with other organizations in the health, water, and development sectors, understanding the processes used to create M&E systems. Each organization has a different story about their development strategy, but each focused on the importance of encouraging a participatory and iterative process to develop M&E objectives, goals, and overall programming.

Ideas from headquarters do not always translate exactly as planned during on the ground implementation. Our funding model has allowed us the opportunity to take an iterative approach in developing our M&E strategy, and with that, truly understand and identify the necessary qualitative and quantitative data for informing our programs.

It is not an easy task to change deeply imbedded cultural beliefs and behaviors. Therefore, our programmatic efforts are focused on tracking the enabling environment for behavior change, such as clean latrines, soap availability, as well as understanding social norms around hand washing behavior. To be successful we will need to react and change in a continually shifting landscape.

Lessons Learned Along the Way:

· You can’t measure everything, no matter how interesting it is. Keep it simple.

· Prioritize — What is necessary right now, and what is our capacity as an organization?

· Remember why you are collecting the information – If you cannot take action with the data, don’t collect it.

· Involve staff at every level – Staff should feel invested and place value on the data we collect.

Moving forward:

Splash has always had the foresight to incorporate M&E into our work to increase transparency and accountability. However, there has never been the opportunity for dedicated staff to undertake a full assessment of the organizational data needs and respond to them. We are now in a place to develop a system that addresses programmatic, as well as external reporting needs of Splash. Moving forward, our system will be based on clear Key Performance Indicators (KPIs), which will provide a clear snapshot on progress and failures of our program as we move steadily forward towards our goals. The KPIs will be based on a set of sub-indicators that will allows to have flexibility in our process to refine, reestablish, and truly get the best information possible.

Water System Metrics

1 Water is clean — maintains WHO water quality standards

2 Water is flowing — less than 24-hour downtime

3 Water is used for drinking and hand washing

Hygiene Program Metrics

4 Soap is present at hand washing stations

5 Knowledge of proper hygiene practices

6 Hand washing with soap

There is still an incredible amount of work left in developing and implementing our M&E program, but we are moving with deliberate steps in the right direction.