A recipe for success: humility and resilience in humanitarian innovation failures
Deep within the depths of the UN Refugee Agency’s headquarters in Geneva, Rebeca Moreno Jimenez, UNHCR Innovation Service’s resident Data Scientist, rolled back her sleeves and tied back her hair as she, with one hand, pulled out a sack of flour and, with another, glinting in the dim light. In one swift stroke, she slashed open the sack of flour, letting the powder flow out into an enormous vat, filling the room with a dusty white haze.
At least, that was how she saw herself as she stared intensely at her computer screen, the empty black screen on the page taunting her, challenging her to find a complete algorithm for Project Jetson, a machine learning algorithm designed to predict displacement in Somalia.
In Rebeca’s mind, the machine learning algorithm was a cake (a predictive analytics product), and she and her colleagues were tasked with finding the best recipe (the algorithms/models). Just as a cake is constituted of certain ingredients, which are then joined together using a specific recipe and processed to make the rich product, so too is Jetson constructed from a number of different input data streams that are joined together and processed according to a specific mathematical algorithm.
Instead of one cup of milk and two times a cup of sugar, it’s a certain numerical multiplier times a certain data stream, such as market prices or weather conditions. The multiplier represents how closely related a certain set of data may be to the decision to migrate — for example, weather may be not very direct (and therefore less important to the overall decision to migrate), while market prices can clearly indicate the flow of assets, and with them the flow of people.
A real success within misteps: building Project Jetson’s team
But, just like baking a cake, it was easier said than done. Sofia Kyriazi, the project’s Artificial Intelligence Engineer, was critical to architecting the machine learning side of Jetson. She and Rebeca had been working for months to achieve a working algorithm, with no success. Every algorithm that they produced would overfit or underfit the actual numbers of arrivals, to their great frustration. In their search for the perfect cake, they only were able to achieve cake that was so bland it tasted like cardboard, or cake that was so cloyingly sweet it tasted like cardiovascular disease. But, they could not find, as they put it, the Goldilocks algorithm.
The hunt continued, day in and day out, as they modified various parts of the algorithm to generate different results. They had several knobs to turn: the number of months in advance that the algorithm would be able to predict arrivals, which data streams they used, and how much the data streams were processed to account for delays. But, with every variation of these parameters, every iteration of Jetson, nothing worked. They were desperate.
When they felt they had finally hit rock bottom, and weren’t even sure whether Jetson could be made using the data that they had, Hans Park, the Innovation team’s Strategic Design Manager, made the suggestion that they flatten their approach and adjust their ambitions. Instead of trying to predict several months in advance with data that was several months delayed, why didn’t they predict one month in advance with data that was all up-to-date? They hadn’t tried this before, as they expected they could do more. But, at such a low point, why not give it a shot?
Resilience and innovation walk hand-in-hand
So, they tried it, almost more as a joke than anything else. They didn’t expect it to work, as simple as it was. They came up with the algorithm, fed it the data, and waited, expectations numbed from past failures.
This time, though, it worked. Sofia excitedly shared news of the success with a disbelieving Rebeca. Neither of them could believe it. After months of work, it turned out that their ambitions had been too grand at the outset.
“We started with scattered data,” says Sofia, “which meant that we thought we could go large and use all of the sources that would give us data. So, for example, we had historic data on cases of donations, river discharge, and other things that we couldn’t use because we were not getting continuous data feeds every month. And, we also thought we could predict a few months in advance. The failures were probably in our ambition. Later on, we removed a few points and we focused on one region. It’s about limiting the scope — so the failure was on the scope. We now know to start small and then go big.”
Don’t build a virtual paperweight
Once it began to work, Jetson would have to be made available to users without technical backgrounds. That’s where Babusi Nyoni came in. As the UI/UX (User Interface/User Experience) Designer on the Jetson team, Babusi was charged with turning Jetson from a jumble of algorithms into an appealing experience for various audiences. In a sense, Babusi had the challenging task, as, no matter how good Jetson was as a machine learning program, if nobody knew how to use it, it was as good as a virtual paperweight.
Babusi began working on a web portal through which people could access Jetson. He, along with input from the team, created a site with a view to the future: it was user-friendly and well-designed, and successfully addressed previous challenges. The team was very excited about the website.
One day, though, when Babusi was testing the site with Rebeca outside of work, they discovered a problem that they hadn’t encountered before: the site wouldn’t load. When Babusi brought up the issue, Rebeca explained that the Internet connection where they were testing it was spotty. That struck Babusi like a bolt of lightning. “The ‘weight’ of the site [in terms of its size as a set of files] was a little too heavy for people in low bandwidth areas,” he says. “And, when I say the weight of the site, I mean specifically the map visualizations and predictive engine, which we were actually hosting on the website, and it would load as the site loaded as well, so you had no option to opt into loading an experience or taking on extra bandwidth.” In other words, the site was built as one package, even though some people would only visit the site to use certain parts of it, they weren’t even able to visit other parts.
They set out to change the site to make it accessible to all of their target audiences, with a view to equity. Babusi explains that, “if the site took longer than X amount of time to load, you would have this little pop-up [offer] to open a lite version of the website that was stripped away of all the gimmicks, all the animations, all the cumbersome things that would take upload time. And, that was okay, but it still was a broken experience in the sense that it’s not good to stagger the same experience for two kinds of people [low-bandwidth and high-bandwidth users].” He added that the site’s experience should be something that is all-inclusive.
To ensure this, the team explored what it meant to have the lightweight version of the site and the heavier version of the site on one platform and the result was akin to reading a newspaper. It was very pragmatic and did a bad job at selling visitors the idea of the project. He added, “… after that, we decided to add a little more character to the site. So, now, we have a really good compromise. We have a generally okay interface — one that all the senior managers can look at and understand while also allowing a separate crowd to derive some sense of excitement on the nature of the project — and also one that really cut down the cost of loading the page by having people opt into certain experiences.”
Turning failures into value
Perhaps the chief factor contributing to the success of the Jetson team is their recognition of failure as integral on the path to success. Learning to start out simple, and restrict well-placed enthusiasm, is perhaps one of their biggest lessons. Maybe more importantly, though, through their successive failures, they had the opportunity to re-learn one of the key aspects of innovation: iteration and experimentation.
“We dealt with a constantly changing environment and a constantly changing product, and to have to change a sufficient number of times to be agile in this constantly changing space is a quality that I can take forward going even beyond this project, because when it initially starts, you always have your set-in-stone expectations, but as you start to fail more often, you start to see the value in picking up a conversation again and the value in not being too tied to an idea, and looking at things in terms of cycles versus a departure point and an arrival point,” Babusi explained.
What Babusi says bears extra weight in light of Jetson being a project dedicated to helping refugees. The refugee situation is often conceptualized as a journey from the point of destination to the point of arrival. But in reality, the struggles and joys of life for a refugee come in cycles. The end of one cycle may be the beginning of another. In that sense, Babusi’s point is doubly relevant: true not only at face-value, but also in its consistency with the principles of equity and accessibility he learned in making Jetson available to users in low-bandwidth areas. Humanitarian innovation must mirror the experiences of refugees for it to be truly effective. Without consideration of the population in need, the developments made in innovation are worthless.
And, maybe in the future, with greater consideration for the needs of all potential audiences, innovations don’t need to be restricted to one audience or another — the high-bandwidth or low-bandwidth audience, in this case. Perhaps a better, more equitable future will be one in which we can have our cake and eat it too.
We’re always looking for great stories, ideas, and opinions on innovations that are led by or create impact for refugees. If you have one to share with us send us an email at innovation@unhcr.org.
If you’d like to repost this article on your website, please see our reposting policy.
