Programme as portfolio of investments
Frontier Technology Livestreaming* (FTL) launched an eventful 18 months ago, with our very first pilots selected in December 2016.
A lot has happened in that time, with many studies, workshops, sprints and livestreams since. We’re now in 9 countries across Africa and South Asia.
As the programme matures, we’re looking at better ways to recognise and reward progress — and ways that we can treat our programme of pilots as a portfolio of frontier tech investments.
Like any investor with a portfolio, we have introduced mechanisms for assessing the pilots to help us know where we’re enjoying impact and where we might be better off redirecting our efforts and investment elsewhere.
Four new developments on FTL
At the beginning of 2018, we introduced four new developments onto the programme: 1) the way we measure progress; 2) a leaderboard; 3) monthly decisions to exit/double down on some pilots; and 4) investment committees.
1. The way we measure progress
When we first designed FTL, we had in mind two broad phases for our work with pilot teams: scoping & design, followed by a series of implementation sprints.
When we first designed the programme, it seemed really elegant to turn sprint cycles into the overall rhythm of a pilot’s work, contracting periods, and to serve as a way to measure progress.
To simplify, we thought: no. of sprints completed = progress made
The incentive in this model, however, was for teams to put as much effort into sprints to minimise contracting. This creates a skewed perception when progress is measured simply by number of sprints completed, because it is the effort and diversity of activities within each Sprint that matters.
Because each pilot is unique, what makes sense to include in a single sprint in terms of activities could be very different depending on the realities the pilots deal with on the ground, as well as the specific assumption(s) being tested.
What unites all the pilots is that the activities must be geared towards learnings. Therefore, we have shifted away from counting sprints, and towards counting the learning within each sprint.
We realised: no. of learnings completed = progress made.
This is what we will measure and display on our Leaderboard (more on that later). In their Sprint reports, this will be counted as each vertical column of work within a Sprint. This demonstrates that teams have gone through the process of: 1) having a belief; 2) doing something to test it; 3) observing whether there was a difference between reality and that belief; and 4) pivoting or doubling own.
2. A leaderboard
A really valuable lesson learned using frontier tech is all the more valuable if that lesson is shared with our readers. Ultimately, it’s this sharing of what works — and what doesn’t — that will ensure our programme is greater than the sum of its own pilots. And we really wanted to do more to incentivise sharing or ‘livestreams’, what we call the stories our teams write from the field here on our Medium publication.
As such, you’ll see our leaderboard = Number of posts on Medium vs. Number of learnings generated. Below is a mock up of a leaderboard for illustrative purposes only:
You’ll see that the leaderboard is made up of three zones including ‘potential additional funding’ and ‘potential exit’. The poorest performing pilots — those not learning or sharing their learning — on the bottom left are at risk of being exited. And the strongest performing stand the chance of further investment.
3. Monthly decisions to exit or provide additional funding on some pilots
This is where it gets interesting. This programme is really a portfolio of investments in frontier technologies and as such, we have started managing our portfolio as an investor would.
That means exiting pilots that have made little progress or have failed to demonstrate the potential to achieving impact at scale, so that we can use the money where there is greater promise. On the other end of the spectrum, this would give us flex to provide additional funding on pilots that are showing great promise and progress.
The decision will be made both on the basis of the leaderboard, contextual factors and a compelling case that further a sprint could provide a significant batch of learning or significant leap towards scale.
Ultimately, we hope that focusing on our highest performing pilots means that we achieve greater impact for our investment.
4. Investment Committee
The decision on whether to discontinue or provide additional funding will be made via an Investment Committee, consisting of the FTL Hub and our DFID sponsors.
Results of our first investment committee meeting
The FTL Hub has now gone through our first cycle of new measurement and hosted our inaugural investment committee meeting.
After discussions with the Committee and the Pioneers/pilot owners, we have agreed to discontinue our work on two pilots: Smart Meters Challenge in India and Humanitarian Response in Ethiopia.
We would like to take this opportunity to thank the Pioneers** on those pilots for applying to the programme, working with us over these past months and for prioritising the greater good of the programme by agreeing to exit their pilots.
“Failure is an option here, if you are not failing, you are not innovating”
— Elon Musk
Failure is an inevitable part of the innovator’s journey. We are proud to embrace this reality and learn from our failures and we believe sharing these insights will improve all of our work. Certainly, the lessons we have learned through these two discontinued pilots have already informed the programme’s ways of working.
What we’ve learned:
- Independent Technical Expert (ITE): Typically we appoint an independent technical expert to assess the proposed technology, or in the case of pilots on the bottom left of our typology (pic below), to help us scope the pilot. In both cases the FTL Hub struggled to find an ITE which made progress very slow.
- Pilot type: In both cases, our Pioneers were starting with just an idea and needed to work with the Hub to consider how best to turn that idea into a reality. This is not unusual for FTL, as you can see from the pilot typology below, 8/18 pilots fall into this category. What have learned is that there is an unavoidable, far greater amount of time and close collaboration required from the Pioneers for pilots of this type.
How this will inform what we do next:
- Skip-the-ITE experiment: We are running two experiments this quarter to assess whether we are better off skipping the ITE in some cases and going straight to the market instead. Our hypothesis is that a well-framed problem (even without deep understanding of the tech), instead of a well thought through answer, will significantly increase our velocity without eroding impact.
- Pilot type: Now that we understand the real level of commitment required from Pioneers of this type, we can and must be very clear about the requirements during the application process and through Dragon’s Den***.
The funds that had been provisionally allocated to these pilots will go into other pilots which are finding more fertile ground. We have invited our front-running pilots to apply for this additional funding, should they be interested.
We’ll keep you posted on how it all plays out and what we learn and do next.
We are always keen to learn from others and discover new and better ways of working, so do share your reflections, insights and thoughts with us.
Here are some quick links to more on our methodology, experiments in electric motorbikes in Rwanda, plans for long range drones, alternate connectivity for market traders in Mozambique, contraceptive ending machines and machine learning.
* FTL is a programme for DFID advisors with ideas about how frontier technologies might apply to their programming. The programme, fund and Hub are set up to make space for their experimentation and collaboration with innovators, entrepreneurs and partners.
** Once a DFID advisor has submitted their idea and been selected by the team to be part of our work, they are referred to as Frontier Technology Pioneers, to recognise their ingenuity and cutting edge thinking.
*** Dragon’s Den is the second stage of the application process. First, DFID advisors submit their ideas and then the shortlisted ideas are invited to pitch to a panel of Dragons to be selected for participation in the programme
