Scaling up is a process, not a solution
As The Water Project reaches more people with reliable water points each year, we are working to better define terms of scale in all aspects of our work. This piece outlines a major trap associated with scale and how avoiding it will result in safe, reliable water for more people.
If you were to play a game of nonprofit BINGO, the term ‘scale’ is as good as a center free square.
Terms like “achieving scale” and “scaling up” are used to indicate that an organization is growing, has ideas that are worth growing, and/or is signaling to supporters that their investment is in some part contributing to the organization’s expansion.
Having that word in your pocket can make you feel like unlimited growth and global impact is possible. And it helps communicate that an organization is succeeding and it can reach even more people with its programs.
But it also comes with some heavy baggage.
When scale is assumed, or unrealistically projected, the end goal can easily become disconnected from the people involved in the process. There is a subtle-yet-important difference between a program that grows as a result of uncovered solutions over time and one that sets scale as the solution.
The problem of setting scale as the solution is akin to the rise of data dredging (also known as p-hacking) in statistics. Data dredging is the act of manipulating and analyzing data in order to show a significant effect of an intervention when there is no real effect.
It is illustrated best in this XKCD comic:
Scientists tested whether jelly beans cause acne and find no link. So they check every possible color to see if there is a link and eventually green jelly beans produce a link. The finding is considered to be statistically significant, but it is such a small portion of the total study that it is virtually meaningless.
It required data dredging to produce the positive result.
So, what happens if you build a new program, with new ideas, and make scale the ultimate measure of success? The data that comes from the project is forced to serve the scale set from the beginning. A recent example illustrates what can go wrong.
A report from the Special Inspector General for Afghan Reconstruction revealed the spectacular failure of an aid project that intended to empower more than 75,000 women in Afghanistan. The scale of the impact was the measure of success when the U.S. Agency for International Development announced the program in 2013.
After three years, there is no way to really tell whether the $87.9 million spent did anything. That is because dozens of changes were made to the performance indicators developed at the start of the program.
“From the 78 original performance indicators, USAID/Afghanistan modified 32, deleted 23, and added 13 new indicators. Of the indicators USAID/Afghanistan modified, it changed the definitions for 12, changed the targets for 11, and changed both for 9. Specifically, USAID/Afghanistan lowered the targets for 12 indicators,” says the report.
When you are having trouble meeting the standards, you change the standards. That is because it is easier to change the rules for determining success than change the program itself. As a result, changes are made to keep achieving the narrow goal set at the start — “the scale.”
When the answer to a problem is predetermined, the ends justify the means. In the case of the comic, the scientists manipulated the data to show what they wanted it to say — not what it was actually saying.
When that answer is scale, then data can be changed and program compromises made to achieve that end. The ability to change what counts with such ease is only possible when the power is held by one person or group. In the case of Afghanistan, it is the USAID program that holds the power over the women it aimed to empower.
Ironic, right?
Achieving scale that is sustainable and impactful is an emerging, interpersonal process, not a solution.
Success is only possible when the power to choose, to shape the systems and processes of the program, and to hold others accountable begins with those closest to the problem. The solutions lie with them.
In the case of The Water Project, we grow, we learn and update our prior assumptions based on lessons from each water point. We get a better understanding of what is working and why in each community. That helps us work towards our mission to unlock human potential by providing reliable water projects in sub-Saharan Africa.
It is a lot easier to do one thing, like drill borehole wells, over and over again. It is also easier to scale when doing one thing really well. But a one-size-fits-all approach to water provision is a surefire path to failure.
There is a wonderful opportunity to determine scale in an entirely different way. Growth is not simply more water points reaching more people, it is one of learning.
Defining scale in a way where the process serves the relationships with all involved for the benefit of those closest to the issues draws everyone into deeper and more complex conversations about the problem at hand. Long-term discernment presents opportunities for solidarity. An inappropriate water point is doomed to fail, no matter what is done to support it. The appropriate solutions are implemented because that is what local leaders and communities demand.
The appropriate solution in Kenya depends on the region of the country. Southeastern Kenya is arid and the rainy seasons that provided enough water for farmers to grow some crops are less reliable due to climate change. So we build dams in riverbeds that collect sand during the rains which retain the water for many months. We construct wells near the dams to provide access to the water throughout the year.
In Western Kenya, reliable rains shape the contours of the verdant and variegated land. But access to safe water is difficult due to contamination. So we are protecting the natural springs that communities already use to meet their water needs, transforming them into safe sources. In schools, we install rainwater tanks that collect water from the roofs of buildings.
Scale in Kenya is multi-pronged. Lessons from both regions inform all of our work but are also specific to each context. Just because something works in Western does not mean it will work in Southeastern. Organizational achievements of scale depend on contextual possibilities.
We learn from our work and from the broader water sector. Sometimes, indicators and metrics need to change. Scale in context requires us to consistently improve our monitoring work. Real impact happens when changes are informed by lessons learned and from assessments of each and every goal set within ongoing relationships.
Doing the hard and contextual work does not prevent growth — it personalizes growth.
We are scaling up. But how we are doing it matters most.
