Livestreaming’s early exits: taking stock of things that did not go to plan

“ In setting up the Frontier Tech Hub and Livestreaming, the intention was always to learn from failure — to ‘fail fast and pivot.’ Working in Sprints has enabled this to happen quickly in practice. The Frontier Technologies (FT) Hub, with 45 pilots under it’s belt, have taken a step back to look across the portfolio to talk honestly about where pilots have not worked and the elements needed to adapt and fund success.” — Magdalena Banasiak, Senior Innovation Adviser, FCDO

2021 marks four years since the Frontier Technologies (FT) Hub started supporting frontier technology pilots under our Livestreaming programme. It has been quite the journey for our various pilots.

As we expected at the outset, not all of them have gone according to plan.

While FT Livestreaming has had numerous successes, like any innovation portfolio, our programme has also seen its fair share of pilots that we and our partners had hoped would turn out differently. While we always knew that an experimental approach designed to explore the most emergent tech comes with its obvious risks, the pitfalls themselves have been novel and would have been impossible to predict at the start. Along the way, we have learned a great deal, tweaked and iterated our approach, and in doing so changed how we think about and deliver our support.

FTL pilots most often integrate learning in real-time to keep moving forward, but sometimes there just is no viable way forward.

Experiences that have not proceeded to plan may not be the easiest or most glamorous in the moment, but they can be the most valuable for learning and improvement. Just as we expect our pilot teams to learn and pivot when things do not go as intended, we also make it a point to “eat our own dog food” and also learn from the unexpected.

This article will recount some of the stories and insights from that which did not quite go to plan for us and our pilots and, more importantly, what we learned from these experiences to make our programme stronger.

What are some of the things that have gone wrong?

Our methodology provides a great deal of flexibility to pilots for teams to pivot and adjust their approach when things don’t quite go to plan, provided that there is a viable and sensible way forward. This liberty to change course has allowed many of our pilots to keep working, experimenting, and striving towards their end goal.

However, having supported a portfolio of more than 45 pilots, there have been instances where this flexibility has not been enough and we have had to exit pilots. This has occurred for several reasons:

• Pilot that never got off the ground because nobody was available locally to drive the initiative forward: Every pilot needs an engine to drive progress. This is normally some combination of an incumbent technology or implementation partner, local expert, or in-country FCDO team. Persistent recruitment challenges and competing obligations unfortunately held us back from making any real progress.
• Instance where key stakeholders lost faith in our local partner and a crucial opportunity was missed: Conversely, another pilot had all the right team ingredients in place, but a lack of transparency and communication from a local partner ultimately led collaborators losing faith in our pilot’s ambitions.
• Initiative that was just too politically sensitive: Sometimes, when a challenging undertaking could only really move forward with numerous official partners in full lockstep, not moving ahead at all is the right choice. This was the case for one of our first pilots.
• Local operating context that was just too challenging: Working in fragile and conflict-afflicted state contexts can be fraught with many dangers and pitfalls. With one pilot, we simply could not find a way to move forward strategically and safely.
• A pilot where delays were so persistent that we ran out of time: Procuring highly specialised, unique technology can require inordinate amounts of time. Combine this with humanitarian teams that often need to drop everything to respond to the next crisis, and it all adds up to a recipe for enormous delays.
• Mismatch between the vision of the FCDO Pioneer and the structure of FTL: Miscommunication from the very get-go doomed one of our pilots from the very start, and we could never truly get off the starting blocks.

Our programme has a healthy UAV portfolio, but not all pilots we approved with this tech landed smoothly, or even took off.

So, how have we iterated our approach?

Our methodology and Livestreaming programme are optimised for learning. It thus naturally follows that, just like the pilots we support, we as a programme also evolve and pivot in order to improve. It all starts at the beginning with…

A more informed, rigorous selection process

Getting FTL pilots right starts at the very beginning of their time or, to be more precise, even before they start! Yes, over time, we have gleaned a number of lessons relating to how we pick from the litter:

• More risk = more upfront rigour: Our programme will never shy away from challenging environments or ambitious goals, but it is a fact of life that — all else being equal — working in tricky environments (for example: FCAS countries) will be more difficult. Having seen multiple pilots come to an early end due to challenges in the proposed operating context, we have learned to ask more questions and harder questions of applicants right at the start.
• Tell it like it is: We pride ourselves on FTL being a programme where FCDO staff can come to us with their ideas and ambitions. Even when there is not an exact match between their aspirations and what we can offer, we will work collaboratively to see if we can find a way to support an initiative that which holds great promise within the bounds of our programme. However, we do also have hard lines for certain parameters. Now, if there is a mismatch we are not able to get around, we let applicants know right away.
• Clarity on the team: With multiple pilots slowing or stopping due to lack of time and resources, we now want to know up front more details around how a pilot could move forward if approved.

Our playbook is a great resource for pilot teams, and we follow suit in learning what works best for us

Evolving how we work with pilots once underway

Selecting the right pilots is an important part of the equation, but it is not the only element that is crucial to get right. We have also evolved our method once things are moving on the ground:

• Be frank if things are not working, and bring things to a close more quickly if they aren’t: Previously, we might have let pilots continue for too long and given too much leeway to see progress. This led us to sink more resources than would have been ideal. Now, we are more up front with teams if progress is not as expected, exiting pilots earlier than we might have in the past.
• Push and nudge in strategic ways to move things along: We are always thinking about ways to bring more pace into our work and that of our portfolio. We have introduced various tweaks to facilitate faster progress, such as more regular check-ins, time-boxed Sprints, and a pilot leaderboard (read more below).
• Working smarter and drawing on more local resources: Early experiences proved that our team’s involvement was crucially important to how pilots would turn out. When and how frequently we engaged was an important determinant of how things might evolve. Over time, we have learned to target our involvement differently and draw on in-country expertise as our network has grown to support our pilots in the best way possible.

Developing new tools and approaches

Over time, we have also created and iterated new tools to enable our pilots to thrive and for us to help them do so. Here are just a few:

• Shift away from technical experts to marketplaces of ideas: For our earlier cohorts, we would often lean on independent technical experts to help us design pilots and drive progress on the ground. Often, we could not find the right profiles and, when we did, the process of engaging them could be slow. So, we decided to experiment with the marketplace of ideas. This enabled us to not only design pilots more quickly, but it also brought more experts around the table, allowing them to build off of one another and expand the network for us and our pilot teams.
• Creation of a leader board: We had two goals in mind when we created a leaderboard across our pilot portfolio: reward teams that progressed quickly and shared learnings out loud, and nudge those who were lagging into doing so. Shortly after its introduction, multiple pilot teams that had been moving slowly suggested an exit and teams became more open to sharing learning.
• And much more… which you can read about on our methodology page

Introducing a leaderboard was helpful in nudging the behaviour we hoped for

Beyond these more significant shifts to our approach, we have also considered how to spot these and other signs that might indicate that a pilot exit might be on the horizon. We have integrated these signals, as well as possible mitigation strategies into our internal playbook and the learning we share with our sector.

Our experience with our exits has also led us to reflect on our risk appetite as a programme and formally track when and how our pilot partners pivot. In doing so, we are evaluating our hypothesis for what impacts our “failure rate,” while also acknowledging that some degree of this is to be expected. If every pilot was successful and we did not make any early exits, it would mean that we were not pushing the technology frontier after all.

That is just not what we are about.