Validation bingo: testing big ideas bit by bit

When I joined the British Red Cross Innovation Team two months ago, I picked up the proof of concept for Leaps & Grounds. It’s a new social enterprise coffee cart designed to support refugee women into work by:

• Training and employing refugee women to work on our coffee cart to help build skills, confidence, social networks and English language skills.
• Providing regular 1–1 mentoring with a professional in their dream industry to help them take the next steps on their individual career paths.

The plan is that the sales from the cart will fund the mentoring program, and over time generate a profit that can be reinvested in other services.

The backstory:

The project started in late 2019, when Jon Brown won the ideas challenge, with the idea to set up a service that could support refugees and asylum seekers into employment. Throughout 2020 Alice, Lucas and Sarah conducted a pre COVID and post COVID discovery, ran ideation sessions and identified the Leaps & Grounds concept as the strongest opportunity to address barriers to work whilst being a self sustaining or profit making service. They had it ready to test in the wild in October 2020, when Boris announced Lockdown 2.0.

The challenge:

Leaps & Grounds was a big piece of Swiss cheese. It was a delicious, exciting prospect but there were also loads of holes in it. These holes were our assumptions, things we believed but hadn’t proved yet. With COVID restrictions still in force, we knew we couldn’t run a full proof of concept for a couple of months. So instead, we started to play validation bingo — which assumptions could we prove with a series of small tests that would help us refine and tweak the model?

In some projects — particularly software development — you can confirm your assumptions by releasing new features regularly. You think you’ve built the right thing, so you put it in customers’ hands as quickly as possible and then they tell you if it’s true.

With ‘real life’ projects, this can be more difficult. We can’t know if the coffee we make is good enough until we can get a cappuccino in someone’s hand. But there’s lots we can test ahead of time. We used a great model from the book ‘Testing Business Ideas’ to help us get started.

1st assumptions map.

Map the gaps:

First, we worked out where the holes in our idea were by mapping all of our assumptions across a Venn diagram of feasibility, impact, desirability and viability.

• What had we assumed we could do?
• What had we assumed people wanted?
• What had we assumed we could afford or earn?

Assumptions prioritized by the importance and existing evidence

Fill in what you know:

Next, we plotted each of those assumptions based on how important they were and how much evidence we already had for them. We had built up a lot of data over the course of refining the winning idea into Leaps & Grounds.

From here, we could work out what needed to be in our validation bingo card. We could set aside less important assumptions, or ‘snooze’ ones we had more evidence for and focus on testing out the crucial, unproven assumptions.

For example:

We assumed that work and training on the coffee cart would be a desirable opportunity for refugee women seeking employment.
We already knew:
•That the role addressed the key areas highlighted as a barriers to work or missing opportunities in our discovery of needs.
• That the idea was co-created and socialised with members of the target audience in 2020.
• That women were underserved by existing employability services and this service filled those gaps identified in our industry discovery.

Because we already knew all these things about our potential users, it was less important to start testing this assumption straight away. It was definitely still a hole, but we could hold off and test when we were ready to pilot a coffee cart.

Whereas some of our other assumptions were far less known, but still critical to the projects success. What if our estimations on the price of wholesale coffee were wrong? Well then, maybe our profit margins wouldn’t be as high. What if that meant we wouldn’t be self-sustaining? Then, instead of saving or making money, we would have committed ourselves to spending more.

Testing assumptions is like bingo. You define the assumption, define the test and see how the numbers come up. Each test fills out another number on your scorecard, and in time you build up a feasible, viable and desirable model which you can then test in real life. But also, you have to accept that you might not win. Your assumptions might just be wrong, and that’s okay.

So far, most of our Leaps & Grounds assumptions have been validated, giving us more momentum. They say creativity loves constraints and coming up with accurate, cheap tests without being able to meet in person has been a great constraint.

Testing our assumptions on volunteers for Leaps & Grounds is a good example of this. Our financial model assumed that volunteers would be interested in working on the coffee cart. Our ideal model assumed that those volunteers would be skilled baristas. But we actually had no idea if this was true, so we tested it first:

Test 1:

Assumption: We believe expert baristas will want to volunteer with Leaps & Grounds.

To test this we will:
•Run an unbranded ad for the volunteering position on the Leaps & Grounds cart for one month and measure how many people apply
•Cost: £18

We are right if:
• Anyone applies for the role

Evidence Strength:
Very strong — 10 volunteer baristas apply
Strong— 5 volunteer baristas apply
Weak— 3 volunteer baristas apply
Not true — 0 volunteer baristas apply

We put up a basic post for one month on Coffee Jobs Board. We shared details about the project, but didn’t mention the British Red Cross. The idea was to see if the role on its own was desirable enough to generate any genuine interest from baristas.

We also set up some confidence intervals. One person would confirm that the this could be an attractive opportunity, but we needed five applicants to feel confident that our assumption was valid. It’s not a lot, but without any branding or budget it’s enough to validate the assumption.

We got six applications from great candidates over the four weeks — three of those in the first weekend. When people applied, we emailed them and explained that we weren’t up and running yet but would be in touch soon.

Now that we have strong evidence that expert baristas will want to volunteer, we can look at testing other assumptions about their journey.

For example:

We assume that expert barista volunteers will:
• Be more likely to apply when they know its run by the British Red Cross
• Be willing to complete any required training ahead of their shifts on the cart

Again, these are tests we can run quickly and ahead of time. We can advertise the volunteer barista opportunity ‘officially’ and try to recruit volunteers for our pilot. This time, we’ll leverage our brand and see if others in the coffee industry can help to promote it.

We can also set up a lightweight volunteer journey, so that if we do get the applicants we expect, we can see if they complete their training. Where we have successful tests, we can keep moving. Where we have failures or weak evidence, we’ll pause, speak to potential volunteers and try to understand where we went wrong.

So far our bingo numbers have come up, but the key to validation bingo is being open to change. If none of the barista volunteers are willing to complete any training, then we’ll pivot. We might have to test recruiting from our existing volunteer pool and training them as baristas instead. We might change how we onboard volunteers. The point is that running these smaller tests, quickly and cheaply helps us know sooner if where the holes are.

What’s next?

At the moment, we’re continuing to test loads of assumptions with small tests. We’re looking at the right collaborations within the coffee industry, seeing if we can recruit Leapers from our existing service networks and trying to work out just how lightweight and easy we can make a volunteer journey.

At the same time, we’re gearing up for a real life pilot in August — hopefully the cappuccino is good.