Inside FTL’s kick-off meetings: 5 ways we focus on and accelerate learning
We have now welcomed 18 pilots onto Frontier Technology Livestreaming. The whole process is hugely engaging for us, but the part I’m beginning to enjoy most of all is the Kick-off Meeting with each of the successful teams.
During this 2-hour session together, after submissions, pitches and welcoming pilot teams into the fold, we can finally get stuck into the work and think about how best to move things forward.
FTL offers a small grant fund of up to £100k for each of the pilots — we use the word ‘pilot’ with great intention, that is to say that each of the projects we accept into the programme must pilot something real, in the field. I believe the size of the grant funding and the imperative to pilot something real combine into a useful constraint that focuses our minds to learn, fast.
Our kick-off meetings are where we set the tone and precedent for this mindset and sensibility. These are the 5 key things we aim to convey and put into place for our pilots during this kick-off session:
1. Data as information, not evaluation
One of the most profound mindset shifts we have to make with our teams is how we use data on our programme. We work in Sprints, taking small, incremental and iterative steps forward based on what we learn. At the end of each Sprint we reflect on what was supposed to happen, what did happen, why there was a difference and how that will inform our next Sprint.
Our disbursements work in line with this rhythm and focus on the insight that has resulted from the Sprint of work. It’s a nuance that deserves underlining: we’re not necessarily looking for outputs or ‘hitting targets’, we’re far more interested in the insights gained by taking the action. By framing success in terms of insight, we give our teams the freedom and flexibility to course correct and use evidence to progress based on what works. This shift to data as information is a key philosophical tenet of how the programme runs.
2. Increase velocity of learning by converting unknowns into assumptions
During our Kick-off Meetings we spend a good chunk of our time together extracting assumptions from the original idea. Together, we list out all the reasons we believe the project idea holds promise.
We think this through broadly as it relates to feasibility (will the tech work?), desirability (do people want this?) and viability (how will it endure beyond FTL?). The intention here it to take a skim of the riskiest or most critical assumptions and test them first.
To increase the velocity of our learning further, we work hard to think about the unknowns that might be most critical and convert them into testable assumptions.
For example, our overarching ambition might be to pilot a frontier technology in humanitarian response. While the feasibility of the technology might appear to be the most critical assumption, after further conversation we could realise that the most critical factor of all is an unknown: whether any of the humanitarian actors will be interested in our findings. To work quickly, efficiently and to make sure we’re focussing on the most important thing first, we convert this unknown into a testable assumption. To do that, we might assume a certain level of interest and then have a discussion with the relevant humanitarian stakeholders and audiences to validate or invalidate that assumption.
This is why it’s more useful to think of Lean as a mindset than a process per se. Often there is a misconception that Lean can only apply to building, testing and learning… the inference here is that we have to build a piece of kit before Lean iterations can come into play. In truth though, on FTL we focus more on using Lean methods to close our knowledge gap, tackling whatever is most critical first and typically, that’s not the technology.
For those readers who work in development, the idea is to work through something similar to the assumptions within a Theory of Change, but for us, these assumptions are live, dynamic, assessed, measured and challenged over the course of a project.
3. Measure proxy signals that tell us if we’re on track
One of the questions we hear most on FTL, or any programme that seeks to use Lean and agile methods is how we measure successful Sprints of work when the overall impact we set out to make will take years to come to fruition.
For us, understanding the relationship between longer term goals and short term proxy signals is key to working in Sprints. While it is unquestionably true that many pilots will only achieve longer term goals in longer terms, it simply isn’t true that we only learn whether something is on the right track right at the very end. In truth, no matter how long a programme of work is, one begins to get signals as to whether it is evolving into a successful endeavour long before the end.
On FTL we work to get really precise about what these proxy and interim measures of success are for each Sprint of work, measure them and make decisions based on whether we achieve them or not. The intention is for the small tests to fail so that the overall pilot doesn’t. For instance, we might cite technological take up as a longer-term measure of our impact, but along the way, engagement could serve as a useful proxy or signal that the ultimate take up is likely, or not.
In the kick off meeting we think about the kinds of proxy measures that will tell us we’re on track, or not, quickly. Here we’re asking ‘what can we do tomorrow to find out whether this assumption is correct?’.
4. Include multivariate testing to learn which strategy is best
A key part of a Lean mindset, is holding the overarching goal quite tightly, while being relatively loose with our potential strategies for achieving it. If our goal is to prove the usefulness of a specific technology, the way that we prove that can and should flex in line with what we learn.
This gives us great freedom to think critically about different strategies we might pursue in order to achieve our goal. In fact, we believe going wide on ideas or potential routes to achieving goals is often the best way to manage risk in emergent contexts.
Some of our pilots work through a range of ideas in the first couple of Sprints, shutting down ideas that don’t hold promise and doubling down on ones that do.
5. Creating time and space for discussion and reflection
Having space on a programme to think and discuss options as a group is critical for the success of our pilots. The Kick-off Meeting is an important cultural moment for us because it sets the tone for how we’d like to work together. We work hard to make those sessions fun, fast-paced and action-orientated with our teams, the first part of a collaborative relationship.
Beyond the kick-off we reflect with the teams at the end of a Sprint to think about what we’ve learned and how best to invest that fresh insight into the Sprint that comes next.
__
We’re always looking for new ways to work and improve our processes so if you have any suggestions on how we might accelerate our learning further, we’d really like to hear from you.
If you’d like to learn more about our method and the key decisions that shaped our ways of working, please read our Methodology Series.
