Tl;dr Reminders from a volunteer effort to build an app for the World Health Organization (WHO) about the basics of working with large organizations
There’s a volunteer effort helping the World Health Organization deliver an app for people, beyond the governments and health and policy professionals that WHO technology typically serves. I joined that effort early on, and recently scaled my participation back. This volunteer collective and its collaboration with WHO started with excitement, sincerity, and the best of intentions on all sides and… stalled in a variety of interesting ways.
I shared some of my notes on this project with friends, both tech and non-tech. Based on those conversations, I’m sharing here some of what happened in the hope that the arc might be useful for others. The initial theme of the conversations with friends was how easily product/engineering people and non-tech industry people can “lose track of the plot.” Over time, the topic shifted — to bureaucracies and the management/leadership issues that accompany them, and to the power of volunteer and not for profit efforts, and the additional emphasis these kinds of efforts place on trust.
What happened? Did rude Silicon Valley volunteers waste the time and get in the way of noble public health professionals? Did maneuvering political intergovernmental agency bureaucrats obstruct technical professionals who showed up to build something helpful? Was this effort structurally challenged, or could different decisions along the way have made a material difference in outcome?
Is it just a reminder of how easy falling short can be despite wanting to work together and actually help?
I’ll offer one incident, unfairly and free of context, to illustrate some of the complexity and “fog of war” around the effort. The team understood it was close to fulfilling the requirements for the app’s first release, the “MVP” to establish a baseline to build up from publicly. While ticking down the final list of tasks, the team got “shipstopper” feedback from WHO IT: change the shade of blue, change the animation used for handwashing, and change between a left-right pan and an up-down pan in one place.
Clearly, that is so absurd that it’s best taken as an indicator of underlying problems. And no participant has a monopoly or dominant share of the problems.
I don’t know who is qualified to evaluate or assess the WHO, just that I’m certainly not. The people who are deployed in harm’s way, working on the ground, are amazingly impressive. I am not used to hearing people on a call say, from their own experience, “In Yemen, the bombing hasn’t stopped for COVID” or “Boko Haram is not intimidated by the virus.”
Someone super smart (and neutral) already wrote about this. The gist was “few organizations are staffed with people as qualified, sincere, and dedicated to the work at hand than WHO” and “at [the] same time [WHO] was under pressure from all directions no matter what they did… [that it’s either] too much/little.” (Text from this Twitter thread, particularly 13 and 14.) Criticisms are easy to find, whether specific to what’s going on now (like this tweet or this other one) or criticisms in general (to the effect of “great, more Davos people, flying around the world in First Class and going to meetings, and enjoying Geneva when they’re not”).
And at the same time: no one else is doing the public health job for the world. And while all these apps and hackathons are happening… WHO doesn’t have an app “for people” (as compared to epidemiologists, policy folks et al.).
So, my response when a friend asked me about joining and helping was affirmative.
Helping them can’t be bad, right?
Around the same time, what a different friend working on another effort wrote to me captured, I think, much of the current feeling: “It’s a lot of activity, but I have no idea what kind of dent it is. But it’s a way to try to help so worth it for that reason alone.”
I ran across this thread and agree with its guidance wholeheartedly.
I think it was in this spirit that a bunch of people showed up to help WHO deliver an app.
Daniel Kraft, MD, drafted an initial plan, reached out to friends in tech and many other fields, got the right meetings set up and… the effort got a green light: go forward! Here’s an early call for volunteers:
A Mindset and Point of View: Listen, Ship, Learn, Repeat
Super experienced industry veterans — folks who had been in senior positions at Google, Square, Microsoft, and more — showed up. They were interested (referring to my notes from one call) in “what actually helps.” Self-organizing followed quickly via GitHub, Slack, Zoom, and WhatsApp.
People who were used to building architecturally sound systems that could scale up to huge user bases started making assumptions and figuring out what we could figure out in the absence of clear guidance.
Delivering an app today at all still involves many gnarly details, much less delivering an app quickly and ready to scale.
The particulars involve accounts with the app stores, navigating the approval process, figuring out what language, frameworks, and libraries to use in building the app, the back-end “cloud” side, i18n expertise so that people around the world will be able to use the app, and much more. “Team stuff,” like communication channels for a decentralized and distributed team and how to approach documents and issue tracking, is also necessary work. Teams have their own dynamics. “Forming, storming, norming, and performing” is just a start.
I go into the weeds here because the speed with which all this stuff happened reflects a somewhat shared culture and point of view of “builders:” here are the things we know we’ll need no matter what we build… now let’s figure out what really, really needs to be in that first release… because getting that first release to the public is almost always harder than expected and is necessary in order to listen to people to understand what to do next: what’s valuable, what’s too hard today, what people expect, and more.
Another Point of View: “Where were you for ebola?”
Now, no one said those words that bluntly, because at some level, WHO is an international, intergovernmental organization that operates with a level of diplomacy and civility that are not typically found in the tech industry. Genteel phrasing and an abundance of thanking people profusely for showing up and doing something they said they’d do were novel for me.
A sequela of all that subtlety and politeness is an abject lack of clarity.
The team could not identify the lack of clarity until we’d gone in circles long enough to damage relationships and trust.
When visiting a new country, preparation to understand how things get done and what phrases mean is a good use of time. For example, “quite” means very different things in the US and the UK.
Similarly, to the outside, there are “Americans.” From the inside, there’s a lot more subtlety.
The team did not understand the different internal “tribes” within WHO… how they worked together (or didn’t), which individuals had authority for what, which teams needed to be included on what issue and how, and the like.
Worse, we did not understand that we didn’t understand. We lacked a guide or interpreter for WHO culture as well as the awareness that we needed one.
The professionals we engaged with had been at WHO for previous crises, they were present for this crisis, and they planned to be at WHO for the next crisis.
We were not part of any tribe. We had no social capital.
And They’re Off…
The app team didn’t wait for guidance after we received our initial approval. There was enough experience to anticipate that we needed to figure out all the non-medical, non-public health stuff. The list above continued to grow as we worked through plans for security, privacy, legal and compliance, and more. The collective effort was focused on the fastest path to a software product that helped, not anyone’s self-actualization.
The team was almost entirely new to each other, and figuring out in real time how to work together. And everyone was working from home; kids and dogs showed up to (well, interrupted) Zoom meetings. This wasn’t disaster relief for some distant natural catastrophe. Everyone was somewhat in the middle of their own local experience of the same catastrophe, dealing with conflicting local guidance, news and concerns of friends and family, and the anxieties that everyone else in the world had.
Here’s an excerpt from a draft document from this early phase:
The WHO app will be primarily focused around:
• General Public
• Eventually it will become a long term platform for WHO’s wider mission
v1.0 is a native look-and-feel, dynamically-updatable “port” of the WhatsApp bot content on iOS/Android, with information able to be served specific to your country and language (with at least 2 languages actually implemented), and the ability to push notifications in the future.
We were off to the races — and we didn’t know exactly where we were racing to. There was a tremendous sense of urgency. “Now is better than perfect. Speed and focus!”
No one stopped to think through a product plan or roadmap. This was clearly a mistake.
As the days, and then weeks, passed, we learned. Illustrative examples, by no means complete:
- How much work we can get done sustainably. That first week, engineers pulled all nighters to get the basic app that we thought we had clearance to release ready to go. We didn’t release, and we had (justifiably!) tired and cranky people.
- Particularly sticky legal implications. People on the team were experienced with financial services sensitivities, enterprise customer expectations, GDPR and the like; information security risks in this app had the potential to enable human rights violations. Of course we wanted to respect strict European privacy regulations; at the same time, from looking at my notes, one comment from a medical professional on this issue stands out for me:
“The patient died, but their privacy was intact!”
- Changing, and different, guidance. Should people wear masks or not? In what situations? Answers were a moving target. Local guidance varied widely. One discussion involved what guidance the app should offer in an African country with a single-digit number of ventilators.
For WHO, this was an entry into a new business line. While not entirely new (their WhatsApp bot was already live), it appeared to be well outside historical comfort zones. Health ministries appreciate clear information; the broader goal here was different: to provide people not just information but (from my notes) “self-management tools to influence their behavior.”
It was around this time that we entered a loop, rapidly cycling between
- Getting v1 out the door, so we can build further on it
- Asking if this app was sufficiently interesting, so users had good reason to come back to it
- Trying to plan when we would figure out the really good post v1 version
We cycled here because at almost any moment in time we thought we were just a few days and one more meeting from getting approval to release.
Just a few days and one more meeting… to release
Around this time frustration began to build. We thought we were ready to go and (to the example mentioned earlier) got feedback to change the shade of blue etc. The subsequent feedback, referring to my notes, was “it’s not complicated enough… it needs more animations and graphics!” (But it should still work great on older, slower devices with poor connectivity.)
We were going in circles. There was always “just one more change, and then you can release.” And then, silence: at the last minute, our anointed contact canceled the meeting at which we expected to get the go-ahead or find out what we needed to do to release. This happened three times. They said they were still awaiting feedback from elsewhere in the organization.
While this is going on, we were seeing a lot of neat work release to the public: lots of self-reporting tools like https://covidnearyou.org/ and https://covidnearme.org, lots of risk assessment and diagnostic tools, Apple and Google announcing their joint work, and more.
And the app remained unreleased.
“Bring Me a Rock”
We were in the middle of a well-known game that goes by a variety of names: get me a rock, bring me a rock, go fetch me a rock. Here’s a description of how to play:
“Go fetch me a rock. No, that rock isn’t the right kind. Go fetch a bigger rock. No, that rock does not have the right shape. Go fetch a sharper rock.” …and thus you rock fetch.
Forever. It’s even better when new guidance contradicts previous feedback. For example, without a symptom checker you can’t release, followed by why are you doing a symptom checker; work with older phones, poor connectivity, and even offline, and also have more and fancier animations.
The simplest explanation was that the anointed person “in the middle” who we were looking to to clear the release didn’t have (to be charitable) the authority, or the will, or the capacity to figure out and convey what needed to happen in order to release. They were canceling meetings because they could neither approve nor reject the work.
We also realized that in working with the IT side of the house, we’d been isolated from the groups with people on the ground. The groups developing recommendations and engaging in direct action had the best (or at least most-informed) sense of what would be of benefit. When you don’t know the organizational silos, you can’t work across the silos.
This is when, I think, trust began to erode, perhaps in all directions. And a friend, after hearing how things were going, asked me something I’d never thought about before: “Have you ever met a brave bureaucrat?”
Sponsor. Champion. Bureaucracy Whisperer.
When people start talking about decision making, and “deciders,” it’s just a bad sign about so many things.
When the finish line keeps moving further away no matter hard the team is running toward it, frustration, confidence, and trust all go in bad directions.
It was (in a very, very scaled down way) Windows Vista-esque.
We found a whisperer just in time: a senior, connected, respected person within WHO, someone who genuinely cared about delivering a good product as soon as possible. The details of how we found this whisperer, like many other details, aren’t worth going into here. What was important is that we had someone to navigate the complexity of the large organization.
Initially, meetings with The Whisperer inspired confidence. Some of their comments, from my notes: “There are 50 countries looking forward to this, in 27 languages” and “You are approved — WHO wants this app, there’s room for it, here’s who we will work with and bring together…”
They were on board, before we said anything, with focusing on the absolute must have features up front (the ‘minimally viable product’), and understanding that there’s room to learn and update over time after that.
Another of their comments, from my notes:
“What would be the way to save the most lives?”
because “that’s how we figure out MVP.”
Progress! Now we were concerned we might not be moving fast enough for what was needed. They asked about bringing our dates in. We had conversations with folks in other parts of WHO where we saw movement, hints of nods that this effort could release and be useful.
There was an odd meeting in which Daniel was asked to draft the flow of questions for symptom assessment that the app would use. Sure, he’s an MD, we thought, but is there a shortage of qualified people at WHO? Weird.
In the spirit of kremlinology, we weren’t sure if this was a brilliant tactical move (whether “Make it easier for internal groups to converge by giving them an outsider to gang up against!” or “Provide a neutral external party so different teams can agree without any loss of face!”) or… random. Daniel ran with it.
Whatever it was, time kept passing, the news around the world kept getting worse, people on the team got discouraged, and more rounds of bring me a rock continued.
Were we clueless boors who didn’t know how we were messing up subtle interactions within WHO? Are the problems, cooperating with governments and local authorities, just that hard? Is this actually 10x better than the alternatives that have been tried to date?
I don’t know. We did feel like we had momentum, that the end of the delays was in sight even if we still had a few more delays before the first release.
An amazing PM on the team, who opted out of this effort before I did, when discussing their decision, framed the issue perfectly: there are many, many ways to help. Based on the experience on this app so far, does this one make sense to continue, or could they find a better use of time and energy? They scaled their time down appropriately.
Back on the project, we had another meeting with WHO. And another. More days piled up into a week, and then more. Will this next meeting be The One?
And then, we went backwards.
We were informed that whatever previous agreements and expectation setting had happened, we had a new set of rules going forward. We were to work with a diffrent team within WHO. The Whisperer remained optimistic. I admit I wasn’t sure why.
Earlier, we’d heard that the only reason the effort hadn’t been canceled outright was that the head of WHO had tweeted about it (a separate story of miscommunication). Going into this latest meeting, we heard that a negative reaction from some other part of WHO to a more recent article about the app under development had resulted in this reset.
After this latest twist, I was in the same place as that amazing PM: basta. Time to scale back.
And you know — it’ll all work out. I have confidence that I am not the difference between WHO and the team doing a great job a week sooner or a week later.
I continue to respect the complexity that so many people sign up for, working in large organizations in order to achieve scale. We’re just at the beginning of this COVID thing; there is still lots of upside, and lots of ways to help.
I have confidence that the team will do good and deliver a solid app and the best infrastructure that WHO could have, in a way that will help long after this particular crisis is over.
Friends have asked me if my opinion or point of view on WHO has moved (one way or another) after this experience, especially given all the political commentary (some of which I link to toward the start of this post). No. I maintain that I’m insufficiently informed and not qualified to offer anything intelligent or helpful on the topics that are getting so much attention now.
I put my learning from this experience in three categories: about bureaucracies (in specific, and organizations in general); about volunteer efforts (especially technical ones) and trust; and personal.
Management in bureaucracies rarely rewards leadership. The formulation “you manage complexity; you lead change” still resonates with me. We have bureaucracies in order to manage complexity. People in bureaucracies don’t want to look stupid. I respect this is hard for them.
“you manage complexity; you lead change”
When anyone can say no, and there is comfort and power in saying no, and there’s risk in saying yes, and no one is sure if they even can say yes — the outcome is clear before the start. I heard the expression “small ‘p’ politics” in a call on this project, and it makes more sense to me now.
When anyone can say no… and there’s risk in saying yes…
The experience was a strong reminder that optimizing an organization for one set of activities typically makes novel efforts outside those activities even harder.
I strongly recommend working on a volunteer effort like this one because it is such an amazing exercise in connecting with others, understanding their motivations, and finding common cause to work together. There is no positional authority to be abused here, or threats to be made.
I think builders (particularly in tech) often think they’re having a feature conversation when they’re actually having a trust conversation, or they focus on a technical problem, missing what might be the more important but non-technical problem beside it. As an example of the latter: however comfortable the devs are with CompanyA’s cloud services, the organization is committed (as only an enterprise can be) to CompanyB’s cloud services. This conversation might look technical but it’s really not. Wow there are a lot of these.
think they’re having a feature conversation [but] they’re actually having a trust conversation
I’d forgotten just how corrosive doubt and distrust are, and how much work goes into getting ahead of them. This is particularly acute on a volunteer effort like this one.
Finally, I feel bad at representing things (ex “if we do X, we can release”) that were not true. It doesn’t matter that I was about as surprised as anyone when they turned out to be false. Volunteers invested their valuable time and energy to build something to make a bad situation better. Similarly, I’d forgotten how important clarity on rules of engagement are.
There’s a lot of pain that I think I should have been able to anticipate and help the team avoid. I wish I’d have done a better job at that. Personally, the balance between leaving something too early and leaving something too late, especially when I believe in the larger mission, is still hard for me to assess.
Did we ever meet with the right people at WHO? I don’t know. I will continue to root for this effort for sure.
Many thanks to SCS, RM, RH, RAH, PO, KW, JEM, HS, DK, and AM for the conversations and feedback that made this piece shorter and better.