This webinar was recorded live on April 28th, 2020, and sponsored by the Columbia Venture Community (CVC) — a global network of nearly 6,000 Columbia University alumni, students, and staff who are interested in all aspects of entrepreneurship and innovation.
In this webinar, we explored the subject of entrepreneurial risk — how to measure and understand it through several different lenses. We also touched on the relationship between startup risk and experimentation and how our tolerance for risk might change in the wake of COVID-19.
- Identifying and testing risky business assumptions as quickly as possible.
- Why rapid experimentation and learning often provide a clearer picture of early-stage risk than other more traditional measures.
- Why the book Testing Business Ideas should be required reading for all virtual accelerator directors and participants.
- How our understanding of entrepreneurial risk might change or need to adapt in the post-coronavirus era.
David Bland: Co-Author of Testing Business Ideas with Alex Osterwalder. David is an advisor on product-market fit using lean startup, design thinking, and business model innovation.
He pioneered GE FastWorks with Eric Ries, advised emerging product teams at Adobe, and mentored Toyota on lean startup practices. Before his transition into advising, Bland spent over 10 years of his career at technology startups.
He remains connected to the startup scene working at Singularity University, Draper University, Backstage Capital, and several other Silicon Valley accelerators.
Beth Mckeon: Founder & CEO of Fluent Studio, an entrepreneurial R&D lab.
Fluent Studio has designed and facilitated over a dozen virtual accelerators across five states and developed a revolutionary new tool, The Fluency Score, to measure product-market fit in early-stage innovation. It works like a FICO Score for startups.
Now, accelerators, investors, corporate innovators, and government agencies are using The Fluency Score to select startups for programs, diagnose stage and risk factors, and measure business growth and traction over time.
Beth and her team built both products using the kind of rapid experimentation described in Testing Business Ideas and consistently advise founders to bring laser focus to their business model design and strategy.
Ryan MacCarrigan: Welcome to the testing business ideas webinar, featuring guests David Bland and Beth McKeon, and sponsored by the Columbia Venture Community. I am your host today and moderator Ryan MacCarrigan. We are extremely excited to welcome both of these fantastic speakers. Both of them will have an opportunity to introduce themselves momentarily, but first a few words about the Columbia Venture Community
Founded in 2006, the Columbia Venture Community, or CVC, is a private network of nearly 6,000 Columbia University alumni, students and staff who are interested in all aspects of entrepreneurship and innovation, with chapters in New York City, San Francisco, London, Washington DC, Seattle, Los Angeles, San Diego, Denver, and Seoul, South Korea, with several new chapters launching soon in Asia and in Europe.
Our mission at CVC is a simple one. For over a decade, we have aimed to foster entrepreneurship among Columbia alumni, students, and staff by facilitating communication, providing access to needed resources, and creating a vibrant and supportive community. To this end, we promote events in major cities around the globe, including happy hours, startup pitch events, incubation seminars, speaker panels, and more.
Given the worldwide impacts of coronavirus, however, we have pivoted to running all-virtual events for the foreseeable future. Some upcoming events: our next big online event will be on May 15 with Hubble co-founder and co-CEO, Jesse Horwitz, about marketing a startup. If you’re interested in learning more about how to market an early-stage startup, we’ll talk about keeping to a budget, achieving product-market fit, aligning customer feedback with product development, and much more. That will be hosted by CVC’s Global VP, Courtland Thomas.
If you’re interested in learning more about CVC in general, just type in columbia.vc in your browser and that’ll bring you to our homepage. For all of these links and more information about CVC, check out the email that we’re going to send you later this week that will also include a recording and written transcript of today’s webinar.
To get started, what are some of the takeaways that you will learn about in today’s talk? We’re going to cover the importance of identifying and testing risky business assumptions as quickly as possible; why rapid experimentation and learning often provide a clearer picture of early-stage risk than other more traditional measures; why the book Testing Business Ideas should be required reading for all virtual accelerator directors and participants; and finally, how our understanding of entrepreneurial risk might change or need to adapt in the post-coronavirus era.
We have some fantastic guests today. I’m excited first to introduce David Bland, who is a longtime colleague of mine in the Lean Startup community. David advises companies on how to find product-market fit using Lean Startup, Design Thinking, and Business Model Innovation. And of course, he is the co-author of Testing Business Ideas with Alex Osterwalder. David, why don’t you tell us a little bit more about your work?
David Bland: Thanks for having me, Ryan.
I grew up in startups. I went to school for design, and I joined my first startup out of college and realized I had to learn to code and everything else at a startup, as you do. An interesting point in my career ‒ something that really influenced me ‒ we thought we were a B2C company, but we ended up being a B2B company before we were acquired for $16 million. I think that shaped my whole career.
Fast forward to today: for the last 10 years or so, I’ve been in the Bay Area helping entrepreneurs at accelerators, and helping corporations around the world, to basically test their way through anything new. The way you shape ideas and the way you test them ‒ they don’t always come out looking how you imagined them in the beginning.
What I have in the book is basically what I do in my work, just written down. So that’s what I do. I help a lot of pre-product-market-fit companies and teams find out if there’s a “there” there.
Ryan MacCarrigan: Thank you, David. We’re excited to have you here today and eager to hear about how that experience informed the design of your new book.
Our next guest is Beth McKeon. Beth is the founder and CEO of Fluent Studio, an entrepreneurial R&D lab that has designed and facilitated over a dozen virtual accelerator programs across five states, and developed a revolutionary new tool, the Fluency Score, to measure product-market fit in early-stage innovation. Welcome, Beth.
Beth McKeon: Thanks so much, Ryan. It’s a pleasure to be here with David as well during this webinar with all of you.
To share a little bit about myself, I am an entrepreneur first and foremost. I built a company, went through the process of going through an accelerator, trying to find product-market fit and growing, and ended up selling that company in 2017, while also getting recruited into accelerator management.
I ran an accelerator for two years ‒ an in-person, traditional 13-week accelerator program ‒ and I was interested at that point in how we could expand our understanding of how we support founders in programs like accelerators. So I launched Fluent in 2018, really as an experiment to see what would happen if we tried to expand the idea of an accelerator ‒ not just to see which companies should get investment, but how we could predictably and scalably support startups everywhere.
My interest has always been in how we can bring this knowledge and this expertise to founders that aren’t just in New York City or the Bay Area, or in specialized locations, and how we can ensure that startups can be successful no matter where they want to live. We’re based out of Denver, but we work in Alabama, Massachusetts, Iowa, Nebraska, Colorado ‒ non-traditional startup communities all over the country.
Ryan MacCarrigan: Thank you, Beth. We’re eager to see how your experience and your work intersect with David’s work around risk, which is the subject for today.
As you said, you’re coming into us live from Denver. David and I are based in the Bay Area. Most of our listeners are in the New York metropolitan area, obviously because of their affiliation with Columbia, but we also have folks coming in from East Asia and from Europe that I see. So we’re going to get started now.
David’s going to talk about what led him to write the book Testing Business Ideas, and he’s going to give a bit of an overview of what the book talks about. David, I’ll let you take over from here.
David Bland: Thanks. We came to this conclusion by working with teams all around the world, and I had worked with some accelerators out here in the Valley. We realized that people are still jumping to build way too quickly.
Alex has had two successful books already (he just released a new one this month called Invincible Company), about business model generation and value prop design, so he’d already created this canon about how you structure using a Business Model Canvas. How do you test those ideas? When we had this conversation about possibly writing a book together, we talked about how we needed almost like a job story ‒ if you think of jobs to be done. What we came up with was this: when I have a business idea, I want to be able to rapidly test it so I don’t build something nobody wants. I feel like time and time again, we’re still jumping this scale way too quickly. There’s a big gap between people sketching out their Canvas, and then running experiments.
Time and time again, when I’m in an accelerator, I’d say, “Raise your hand if you’ve ever used a Business Model Canvas,” and the whole cohort would usually raise their hands. They were exposed to it either online, or through some kind of entrepreneurship program at a university. But they were having a hard time making it actionable, and that’s really what the whole point of the tool was: storytelling and making it actionable. It wasn’t the fault of the tool; it was more that people needed a link between taking what we have there, and actually going on to run the experiments.
What we ended up doing was saying, in my work, what I do with that Canvas is I say: write down your assumptions. That whole thing is a bunch of question marks. Whether you’re using a Value Prop Canvas or a Business Model Canvas, write down those assumptions, and then go map them and see where to focus.
With regards to the book, it’s a simple facilitated exercise, but I think where people were stuck was they weren’t sure how to write down their assumptions and focus on their risk.
And then the 2x2 we created ‒ I first picked it up from Jeff Gothelf and Josh Sidon, when we were working with them out of Neo, and I just kept iterating on it. I was pulling in design-thinking themes, and now it’s been pulled into the whole Design Sprint Canon. If you go to Google’s Design Sprint Kit site, they have this mapping exercise there listed as part of their resources. Basically it went ‒ I don’t want to say viral, in this time we’re in ‒ but it did spread.
We took that and said that we can create a library of experiments, to help people match their risk to the experiment they’re running.
There are two big blocks. One is Discovery Experiments. There are 44 in the book. We built on Steve Blank’s work from Four Steps to the Epiphany, which was hugely influential to my career and also influenced Eric Ries who I’ve also worked with in the past. Why I love Four Steps is because it was very customer-focused. It was very much about customer discovery, customer development and customer validation.
What we decided is we want to go broader. What Alex and I did is we took all these experiments that we’d created and helped manage, and we categorized them. We looked at which ones are desirability, which ones are viability, which ones are feasible. It was a really interesting exercise.
Basically, we have categorized discovery and validation. In that, you can choose where you’re at in your journey, and what kind of risks ‒ what’s your next best test run? I meet all these startup founders who have talked to, maybe five customers, and then they go build their app. They’re going from a small-scale experiment ‒ maybe even just friends and family ‒ to a large-scale, $600,000 experiment, potentially. We can do things in between to pay down our risk.
The book is basically an actionable textbook for founders, for corporate innovators, for people who have side hustles, to help them match the experiment they’re trying to do to the risk they have. So many people I meet, they do interviews, they do a landing page and maybe a survey, and that’s it. There’s so much more available. We tried to create a library to help people choose what experiments to run to pay down their risk.
So, why now? And I’m really interested, Ryan and Beth, in your take on this as well. We are in a time of great uncertainty, to say the least. If you look at this chart, when you start on the left, you have an idea, everything’s really uncertain and it feels messy. Can we apply a process to that mess ‒ an almost scientific method, pulling from the Lean Startup approach to apply it to building a business.
I feel like we’re getting dragged back to the left here. A lot of companies that are becoming digital are being forced to be digital, whether they like it or not. We almost have to go back into search and testing mode again, and I feel like we’re getting pulled back into this area of great uncertainty. Especially for the next 12 to 18 months, specifically here in the US, how do we test our way through that again, and can we use methods remotely instead of things that we would do in person? So Ryan and Beth, I’m curious what your thoughts are on this too.
Beth McKeon: I have so many thoughts.
This may be one of my favorite pages in the whole book. The book is phenomenal, and since discovering it, I’ve recommended it to everyone that I work with. This page, in particular, struck me because it’s part of the story that I’ve been telling founders for years, which is that in the beginning stages, it does look like this garbled mess of backward and forwards, and a lot of exploration, but in the end, it is going to feel like a straight line.
So often, early-stage founders, when they’re in this process of sorting through the uncertainty, they compare themselves against companies that have already navigated all of that uncertainty, and there’s a straight line to their discovery process. They’re hearing all these stories that make it seem like there was a really straightforward approach that got these companies to success, but it wasn’t like that at all.
I feel like this diagram gives permission to founders that they don’t have to know the answer too soon. It allows them to really sit with the discovery that has to happen and the experimentation that they need to do, rather than ‒ like you said, David ‒ just doing three tiny experiments that maybe point in one direction and then considering that as the proof that they need to go build the big thing. It encourages them to own the messiness of that innovation process for as long as they need to, to really know that they have it, and to do that de-risking in a much more disciplined way than I think is fairly common.
Ryan MacCarrigan: I think we can all agree that discovery and validation work, and experimentation in general, are really important. Whether startups actually have been doing this hard work is sometimes questionable. We’ve been teaching for a long time that these are important principles, but how does our new environment, with all of this added uncertainty and risk due to coronavirus, and the fact that a lot of investors are going to be more hesitant about throwing money at startups ‒ how is that going to impact how we approach experimentation and risk moving forward?
Beth McKeon: I’d love to take a stab at that. I think that what it’s doing is it’s laying bare that we can’t just build without having some of these things figured out. Having solid business fundamentals ‒ actually de-risking those assumptions ‒ is more critical than ever.
For startups that were punting on having to figure those things out, and figuring that they can maybe just throw enough marketing dollars at it down the road when they can hire a team to do that, it’s urgent to do it now. Ultimately, this is an amazing opportunity for scrappy, creative founders to solve legitimate problems that we’re facing in society. There’s no margin for the gut-reaction build. We need to do it in a systematic way.
The good news is there is an actual process for this that can ensure that founders reach that place where they’re creating value, and innovating in a successful way. It’s actually not as risky as it might seem if you use a process like this.
Ryan MacCarrigan: How much of this do you guys think really can be reduced to process? How much of it is luck?
David Bland: It’s not necessarily a spreadsheet you follow and then you have a billion-dollar business. It’s about whether you can create your own luck, but behind luck is a lot of hard work. If you’re systematically breaking it down to: what’s my risk and how do I address that risk, and let’s do that over and over again, that does feel like a process to me.
When I talk to founders, it’s really interesting that some of them say they would never use a Canvas. I say, “Explain to me how you’re working through this.” And sure enough, they say, “This is my value proposition to my customer and how I’m getting to them and how I make money.” So they’re checking the box. They’re explaining their way through this very much like they’re using a tool.
Sometimes founders have challenges communicating what’s going on inside their head, and that’s not just externally ‒ that’s also with co-founders and within their team. That’s something we need to work on, because you do need to have a shared vision and a shared explanation of how you’re working, with a common language.
I feel like it is a process and you are trying to make your own luck, but there are so many founders that I talk to who tell me they can never be constrained by a tool, but when you have them explain how they work, it’s very similar to how you’d visualize it in a tool. That’s really interesting to me.
If we’re going to compare this to science, it’s a lot like a social science. We’re 10 years in with the Lean Startup movement, which is still pretty early. I don’t think we’ve even scratched the surface of what we can do, but I feel like it’s a process where you’re trying to manufacture your own look.
Ryan MacCarrigan: Let’s switch gears a little bit. If this is a process ‒ not a dogmatic one, but a process that facilitates the art and science of some of this decision-making around risk ‒ what are some ways that we can begin to apply these principles?
All three of us have had a lot of experience teaching Lean Startup, teaching experiment-driven design, having a mindset around experimentation, and a lot of the models that are in the Testing Business Ideas book have been tossed around for the better part of a decade.
Now we have more of a practical roadmap that we can use from the book to put into the hands of startups and accelerators, and see what they do with it. What are some of these practical applications? How do you envision the book and these methods being used moving forward?
David Bland: I can talk about a real story with regards to a team I was working with. This looks very conceptual here in the book, but is based on a real story.
I do a lot of advising with teams who are trying to find out who their customers are. If you take something as simple as an online ad and treat that as an experiment, you should probably have a hypothesis before you go in and start running ads. What are you trying to learn when you’re running your ads, and how do you take that learning and inform what you’re doing next?
I think where we’ve tripped up over the years has been that we see a lot of diagrams in books, but we don’t know how to make that practical. We don’t know how to make that influence our day-to-day lives, especially in the craziness of the startup world.
If you go back to Four Steps, where you say, our customers need to have the problem, be aware of it and actively seek a solution to that problem, that’s a pretty standard way of thinking. But how would that inform your practical day-to-day work? It depends on your startup idea and where you’re at, right?
If you’re going after people that just have a problem and you’re trying to build awareness, the way you run your ads ‒ the way you treat them as experiments ‒ would be a lot of awareness building: trying to help people understand that this is a real problem that we, as experts, recognize. It’s quite different when people are actively seeking a solution to that problem. Then it’s more about where they go to seek the solution to that problem. Do they go online? Do they go offline?
I was working with a team that had a mouth pain product. We thought we would use ads to push people to our landing page ‒ a very polished web page ‒ and then we’d have a call to action on a mouth pain product that we were testing. It was going through compliance hoops ‒ it was moving forward.
Almost nobody converted. We had a 1% to 2% conversion on our landing page. Before we gave up, we thought: if I had mouth pain, what would I do? I would go to my dentist, I would go to my doctor, I might even go online and Google and just search. So we decided to check search volume on that problem. Are people actively seeking a solution to that problem online? Sure enough, there was a pretty decent-sized amount of search volume going on there.
So we took the same ad and basically the same page, but we did Google search-only results where you type into the search bar and your result comes back as an ad. We had a 40% conversion. It’s ridiculous. We had 1% to 2% conversion trying to push ads to people, but 40% catching people while they’re seeking a solution to the problem.
It seems really trivial. I know, Ryan, you were even in a session I did once about explaining this concept when I was testing content for the book ‒ I remember that, in San Francisco, in that open space we were in. But people don’t connect those dots. We look at diagrams and stuff in books and we think we don’t know how to use that in our day-to-day work.
I’m trying to help people that are feeling very uncertain about their ideas. This team almost gave up. They almost gave up too early because they were trying to just push ads. Nobody was converting and they were really demoralized. We suggested trying a different approach, trying to find people who are actively seeking and pull them over. Granted, it’s a smaller set of people, but 40% conversion is huge. That’s a ridiculous amount of people converting.
I feel like this is a glimpse into that window of what it’s like working with teams and founders day-to-day. We get demoralized and we need to be able to suggest trying another thing and learn more. This is a perfect example to share because it’s taking these conceptual things and applying them to what it’s like in a day-to-day startup situation.
Ryan MacCarrigan: Drawing from my own personal experience, I’ve encountered this issue often as well. I think one of the main emergent benefits of a book like this, is that you have a larger toolbox. You don’t always quickly know what is the right approach to do an experiment, but then you can reference it, see that there’s this big toolbox of options to draw from, work with your team to figure out which approach you want to try, and create a plan for how you’ll conduct those experiments. Then you’re off and running a lot faster than you would be otherwise. Would you agree?
David Bland: I do. Beth, I’m curious, in the accelerator world, do you find this opportunity to make abstract things practical to people? Have you seen things like this?
Beth McKeon: 100%. Every interaction that the accelerators we’ve run have focused on, is this core question of, how can we focus on that riskiest assumption? How can we identify what it is, first of all, because working on the right assumption in the right order is half of the battle.
Once you’ve identified that assumption, crafting an experiment that actually helps you resolve that question is crucial. So much of what this process does is it designs for easy-to-understand human behavior. Without a model like this, without a systematic framework for running these experiments, as humans, we tend to either do the things we’re good at or that we like. Sometimes those are the same thing. If you’re a founder who maybe has a lot of expertise in online advertising, it’s easy to gravitate to the thing that you’re good at, or that you understand really well and that you like, rather than doing the thing that’s actually going to get you to success.
That’s what I’m interested in: how do we predictably help founders get to success? This framework, this model for identifying the assumption, developing a hypothesis that can be tested and that there’s a success metric attached to, and then running that as fast as possible to find out whether we’re going down the right path or we need to adjust things in some way ‒ I find that when founders really own that methodology, they bring a discipline to their decision-making process that gets them to the answer faster.
David, to your point, there’s still luck involved. There are so many things to figure out in any kind of innovation, but the path to figuring it out is so much faster, and it ends up being less risky. Investors think in terms of risk, but founders think in terms of opportunity. Founders are very optimistic. We have to be optimistic in order to tackle the kinds of challenges that we’re working on, but the more that founders embrace risk as a core part of figuring out the opportunity, I’ve found that they get to the answer so much faster.
Ryan MacCarrigan: Let’s talk about risk then. We’ve talked about risk through the lens of a company’s business model ‒ risky business assumptions, mapping, prioritization, using Testing Business Ideas with the Business Model Canvas, designing experiments that flow from those connections. Risk is a much bigger space though, right?
Entrepreneurial risk, startup risk ‒ that’s one form of risk. There are whole industries that have grown up around modeling of risk. That’s an interesting segue into what you work on, Beth. How can we improve the way that we model startup risk, and so you’re going to talk a little bit about Fluency Score, and then maybe we can begin to connect the dots between Testing Business Ideas, Fluency Score, and how we should understand risk moving forward in our new environment.
Beth McKeon: Using the process that David has described in his book and has been sharing with us here, my team and I went out into the market to test some assumptions around ways we could help startups succeed. We discovered that what was most needed to help everybody understand startups, to better evaluate and allocate resources to them, was better data.
No startup is going to be successful if they’re not measuring what they’re doing, and then optimizing for what works and getting rid of the things that don’t, but there weren’t really any tools for doing that at the startup support level. Accelerators are saying that they help entrepreneurs make two years of progress in 90 days. My question as an accelerator director was, “What does two years of progress look like if we don’t have any data-driven metrics to describe that progress?”
The Fluency Score is addressing that in a system-wide, industry-level way, by bringing data to the conceptualization of, specifically, product-market fit and business model risk. You can see in this diagram that it does two interesting things. One, we’re able to put a number on companies. When we measure companies, they’re being measured out of 7,000 points, and it gives us a way to rank and compare similar companies. Secondly, the wheel ‒ that visualization around it ‒ is demonstrating where the risks are in both the business model itself, and in the approach to de-risking it.
The blue side is looking at where the risks are in the assumptions the founders have, and where they have actually de-risked those assumptions, which is super helpful for knowing where to pay attention. If you know where the risks are, you can start to quantify that. That’s very useful to both accelerators and investors, but also to the founders themselves in bringing an objective, standardized approach to understanding those risks, and using them to diagnose where to pay attention.
The green side is looking at how fast and how focused the team is at de-risking. It’s more around process and approach. We know that a huge amount of the risk in a startup is that they’re just going to run out of time and resources before they’ve had a chance to solve for the blue side ‒ the business model side. So it gives us a picture that allows us to measure companies from day one ‒ literally we could measure a company on the first day they exist ‒ all the way up to Series A, and use it as a tool for starting to investigate what’s happening in these companies, both at an individual company level, but also as a massive data set of companies across the world, all at different stages, in different industries, demographics and geographies, and start to see the patterns and trends on what’s happening in these companies, what makes a successful company move faster ‒ that sort of thing. There are some really interesting applications as our database grows.
Ryan MacCarrigan: Beth, as I mentioned a minute ago, risk modeling exists in many different industries. It’s nothing new to those industries. Why hasn’t this type of risk modeling ‒ a bit like a FICO-score modeling ‒ not existed in the startup industry up until now? Is it rooted in data as a problem, like you mentioned, or has it been something else?
Beth McKeon: I think that’s exactly the right question. As we were talking about earlier, the processes for innovation are still so new. As David mentioned, Lean Startup has been around for 10 years. Relatively speaking, that’s not very long. I think there’s been a long period of time that these methods of managing innovation ‒ whether it’s Design Thinking or Lean Startup or Agile ‒ have all been emerging and developing in parallel tracks, across lots of companies.
While they were still in the early development phase, a lot of people were doing this kind of work and testing these processes, but no one had codified it. It wasn’t a repeatable thing yet, and so a lot of the things that were working felt like luck. It felt like lightning in a bottle. We figured something out, and we unlocked this mystery, and it’s like this magical thing. Let’s put $5 million on it and see if we can grow it in an insane way.
Now, we have these really systematic approaches. Not everyone’s using them so it still feels like magic most of the time to most people, but there is a process for doing this work that can be measured, and it can be optimized, and we can use that as a tool to get better at this work and not just rely on luck.
I think we’re just starting to reach a place in this industry where there’s enough maturity of the systems, and there are ways to measure this now that probably there weren’t even three or five years ago. It’s about the maturity of the industry, moving toward a more process-driven, data-driven approach to innovation, which to me is a really exciting opportunity, because founders are already putting everything they can into being successful, and if there are ways to increase the likelihood of success, I think that really changes the industry.
Ryan MacCarrigan: Let’s switch gears and talk about the topic that’s on everyone’s minds, which is how coronavirus is going to impact the startup industry. Even corporate innovation is being impacted across the board. What do you guys think? How are we going to continue to innovate and validate and test and design experiments in this new environment? Is it going to be easier, harder or just different? What do we think?
David Bland: I think it’s definitely going to take on a different form. Even 5 or 10 years ago, we’d run exercises with companies where we’d say, “What would happen if you can no longer sell your product? What would happen if all your channels went away? (Especially with retail ‒ this is only going to expedite some of the extinction of retail, unless it can take on a different form) And how would you create another company that would take you out?”
What we didn’t have in that conversation is what would happen if we had a pandemic. We talked about climate change in a lot of my Design Thinking workshops. In terms of access to customers, there were already some people that could do it remotely, but now they’re going to be forced to.
In the book, I have a case study of a company that did a bunch of pop-up stores in San Francisco and some other cities. They tested their product by grabbing people off the street, interviewing them, walking them into the store, doing pre-sales, finding all these great quotes from them about whether they had the need that they were trying to solve.
We’re going to have to be a little more creative now, because some of those physical experiences ‒ I’ve worked with consumer packaged goods companies where we would literally intercept people walking through a store and we would do an interview there, or we would do some kind of in-store experiment. Specifically for the physical products, we’re going to have to be a little more creative in terms of what we can do remotely.
With software, a lot of it was happening remotely already. You could go and find customers online, you could reach out, you could interview them via video calls, you could have them virtually rank things online using surveys or using an online whiteboard. There was a lot already happening with software companies that was remote. A lot of this in-person stuff, where we were doing physical intercepts with folks ‒ how do we recreate those remotely? I’m curious, Beth and Ryan, about your take on this.
Ryan MacCarrigan: I do a bit of teaching in addition to consulting, as both of you know, and one of the things that is so important to me is teaching my students to be grounded in proper user-research methods. Traditionally, we’ve always strongly advocated for in-person discovery interviews or usability testing. We prefer to do this in person so that you can see someone’s body language and have a deeper human connection with the subject. All of that is going to change, obviously.
I’ve had to pivot the way that I teach some of these research methods, because students and clients are having these interactions over video conferencing instead. You still see them, you’re still interacting with them, it’s human, but it feels different. It is different in a way, and it’s something that we’re going to have to adapt to, moving forward. Beth, what do you think?
Beth McKeon: I have so many thoughts because I really wanted to see what it looks like to do virtual entrepreneurial training and development long before we reached this moment.
What this kind of pivotal moment in our industry is doing is it’s clarifying our priorities. I think that’s great, because it allows us to reorder the priorities and focus on the things that make the biggest impact. I can speak to that on two levels ‒ both on the founder level, and also at a programmatic founder-support level.
At a founder level, there’s a common activity where founders know they need to do the user research and the customer discovery and they think, if I need to talk to customers, let me just go to a place where there are a bunch of these people and just interview as many as possible. I have been guilty of doing this myself in the past!
That immediate ability to find people in physical spaces didn’t require the same discipline that we ask of the founders that work in our programs, which is to realize that not everyone is your customer. You have to figure out those key characteristics of your early adopters that really set them apart from 99.9% of the people that will buy your product in the future, and identify where those people are in the world.
What’s interesting about the founders that we’ve worked with, because we tend to work in these non-traditional startup communities, is that a founder working in Iowa has the same ability to build a world-class competitive business, because you can build a company anywhere, but your early adopters might not be in rural Iowa. They might be in Chicago or New York or San Francisco. For a long time, we’ve been helping founders brainstorm how to find these really clear early adopters in the little pockets of the internet, or of the world.
In many ways, using the internet and virtual tools allows us to get really clear about who these people are and where they exist, which ends up being a huge boost later on when you’re trying to market to a lot more people like them. So it gives you a leg-up on that future, more scalable marketing approach.
That’s on the founder side, but I would say the same thing is true on programs for founders around this clarifying and reprioritizing. A lot of entrepreneurial programs have been built around optimizing for collisions and optimizing for luck, which is the old model, right? It’s the thing we needed to do before we really understood how this work got done in a systematic, repeatable way.
This is an opportunity for programs to reimagine the services that they offer to startups, and to think about what the most important things are for founders right now. If I had that conversation with any accelerator director at any time, my answer will always be, it’s to help them figure out their riskiest assumptions, help them run those experiments and figure out what to do with the data they collect. If you can do that well, a program can support founders, and that can be done remotely. It also scales really nicely.
Ryan MacCarrigan: Thank you, Beth. Looking forward, one of the things that we wanted to think about was the reality that a lot of these programs ‒ and granted, you’ve been running virtual programs for a long time, Beth ‒ but a lot of programs that have not had a virtual component are going to have to change. They will have to adapt to this new environment, using all the tools that both of you have been talking about that have been accessible for quite a long time, but maybe it was not so readily apparent how they could use them effectively.
How is this going to impact the numbers game that we all think about when we think of that funnel of companies coming into accelerator programs or the funnel that venture investors think about ‒ the numbers game that underpins how all of this works? How is that going to adapt as we move forward?
David Bland: We’re still learning. On this slide, from the book, what I was trying to do was to visualize what I’ve seen happening as far as changing investment behavior goes, and I think it’s just going to be accelerated.
For example, how do we fund smaller things ‒ basic-level funding? I know there’s this “Go big or go home” funding strategy as well, but I think especially now, it’s really risky. I think VCs are going to become a little more risk-averse and do more basic-level funding ‒ seed level and then launch level. We’ve been talking about this for years ‒ can we get portfolios of companies to behave more like, “We’re showing evidence that we have traction, we’re building on that evidence, give us a little more funding to get further down the road.”
I was just reading a tweet today from a founder, in which he said that he’s never going into a VC’s office again, because he’s raised his series A on Zoom. I do think this is going to have ripple effects in the community with regards to how we look at investments and what we expect.
The power dynamic is different over Zoom. You’re not driving down Sand Hill Road all nervous and sweaty in your rental car, right? You’re sitting at home. A video chat has a different power dynamic. I think it’s going to be super interesting to see how that plays out. From an investment point of view, I feel like it’s going to be a lot more like, “Let’s give you enough funding to get started and show evidence, and then let’s build on that and do more incremental investing.”
Ryan MacCarrigan: I think that there’s a sense among especially venture investors that the current model will have to shift quite a bit. I listened to a podcast recently that was with Mike Maples from Floodgate ‒ I think it was on the Knowledge Project Podcast. He was talking about how grueling it can be to listen to all of these pitches in person all the time ‒ the traditional Sand Hill Road startup pitch ‒ and I think some of them are relieved that now a lot of this will be virtual.
You still want to capture the key relationship-building component. You’re investing in a person and a team ‒ it’s not just about the metrics ‒ but there’s definitely this palpable sense that things are going to change a lot, maybe more so than they did after the dot-com bust and after the global financial crisis, which were both major contractions in the industry.
Let’s pivot into some questions that have been submitted. A question that David and Beth and I liked was from Eduardo who asked, “Is this de-risking and assumption-testing strategy the same for new businesses in markets that already exist?” Good question. David and Beth, what do you think?
David Bland: I’ll start and then I’m curious what Beth thinks too.
Yes, in the sense that you’re going to have to position yourself as having a unique value proposition. Nowadays your solution has to be almost 10 times better than what people are already doing. Otherwise, the behavior change isn’t going to happen and they’re never going to adopt your thing, even if it’s slightly better.
It requires a lot of the same techniques, but it’s a little easier if you’re doing it in an existing market because there is a known quantity. If you’re going in and creating your own market, it’s a little tougher because there’s no evidence to pull from. You have to manufacture all your own evidence. In an existing market, you still need to have a unique value proposition. Your business has to be defensible, and you have to compete on more than just price. You have to think about how you actually get the job to be done behind the customers where they’re struggling with existing solutions, and how do you make your thing unique.
HotelTonight is an amazing example of that. They decided to do one thing and to do it really well. You’ve got on the plane, you’ve landed, but you forgot to book your hotel room so you need a hotel tonight. Click; see where the available hotels are tonight; book. They were able to go into an industry that was saturated ‒ super expensive to compete in ‒ and they did quite well. They used these techniques. So I think there are great examples you can pull from about how to do this in existing markets.
Ryan MacCarrigan: I would add on top of what you just said, David, there’s a key concept that people should learn about, which is the idea of category creation, or becoming the king of your category. It’s an extension of the idea of solidifying and understanding your value proposition ‒ the idea that Salesforce, for example, owns the category of customer relationship management. If you’re a new business in an existing market, it’s still possible to differentiate yourself by carving out that niche for yourself and naming the category and owning it. That is another good practice to think about and to learn more about as you are validating that value proposition.
Beth McKeon: Absolutely. I would add one other idea to this. For anyone who’s still trying to wrap their heads around where to apply this methodology, the answer is, wherever there are unknowns. The job of the entrepreneur ‒ whether they’re running R&D in an established company and trying to open up a new product line, or it’s a founder working in their home trying to build a startup anywhere in the world ‒ in any place where there’s innovation happening, where there’s the opportunity for value creation, it is the job of the person doing that innovation work to think about what they believe to be true. So this is the pitch. Their pitch is what they believe to be true about their business. Then they need to identify the places where they have data to back up those assertions, and the data comes from those experiments. If you don’t have data yet, it’s still an assumption and it’s still worth testing.
In the innovation game, there are always areas where you have more evidence that you’re on the right track, and then there are places where you don’t. It’s about identifying those and then ranking them so that you can focus on the biggest risk at the right time.
Ryan MacCarrigan: Absolutely. Here’s a good question that I think we can address quickly: How much time should a founder actually spend on the searching and testing phase?
Beth McKeon: Can I take that one, please?
You should take as long as it takes, and at the same time move as fast as possible.
Ryan MacCarrigan: I agree, definitely.
Beth McKeon: This is an interesting tension for founders to manage because our goal is to move super fast, but at varying points in this de-risking process, there are questions that have to be answered. There are assumptions that have to be tested. Sometimes you can’t rush the discovery and the learning process.
This is maybe one of the biggest tensions that I see founders struggle with, because there’s an expectation that you can show up on a certain day at a certain time and stand on stage and give a pitch and say, “My business looks like this, it’s going to work like this, it’s going to grow like this.” Often that moment comes before a lot of these things have actually been validated through experimentation.
Founders need to give themselves the space and the time to get the real answers, optimizing constantly for running very fast experiments in the short term, to get closer to that answer so that you’re constantly directionally moving towards the right answer.
Ryan MacCarrigan: I would agree and add to that, that in my experience, it’s difficult to have your research and discovery process be part of an Agile Sprint process. For those teams that are working in, say a typical two-week Sprint cycle, I often recommend decoupling the discovery and research process and the experiment process from that two-week Sprint cycle so that the people who are running those experiments don’t feel like they’re just being measured on their velocity or their output.
We have a lot of discussion in our community right now about outcomes over outputs, and why that’s important and how to think about it. As you said, Beth, it’s really important to give those teams space to breathe and do the hard work they need to do so that the outcomes or the discoveries they make will inform the development process and not feel like there’s a fire lit under them to try to adhere to that Sprint cycle. David, what are your thoughts on that?
David Bland: The quick answer is it never stops. Discovery never stops. At the first startup, I joined, which was a fintech startup that pivoted from B2C to B2B, we didn’t make that change until a couple of years into the journey. I was there for eight years. It’s a long marathon.
That discovery for you, depending on your company, needs to happen. If we had given up, we would have shut it down a couple of years in, but we realized that financial advisors were the ones that really loved our product, even though they weren’t our customer. If we hadn’t been aware enough of that, we would have shuttered the startup.
So I think it never stops, to an extent. You have to give it time. This idea that with a startup, success happens quickly, that hasn’t been my experience.
Ryan MacCarrigan: The final question comes from Ameen:
“With COVID we noticed how desperately we need to upgrade our public sector. How would you approach helping public-sector companies with the same processes? Are the methods even applicable to large public institutions?”
David Bland: I can touch on that quickly. From my experience ‒ I’ve done work with Singularity University, I’ve advised Code for America and their accelerator program in the past, and I’ve done some work with the IMF in Washington DC ‒ the principles are the same, but the tools end up being a little different.
I’ll use something like a Mission Model Canvas, which is an adaptation of the Business Model Canvas. It goes more into impact and getting buy-in, but it’s a similar flow. We sketch out our Mission Model Canvas, we theme it desirable, viable, feasible. We identify our risk and then we design experiments to address that risk.
So yes, it does apply, but the tools are a little different. Either way, you have to reduce risk. If, for example, you’re trying to solve for a Hepatitis C outbreak in the south, you still have to measure whether you’re putting something in place that actually is working, and you still have to take that rigor and apply it. So for the projects I’ve worked on, the methods are applicable, it’s just the tools look slightly different.
Ryan MacCarrigan: That brings us right up to the top of the hour. Thank you, David and Beth. We really appreciate your time and your insights today. Check out David’s new book Testing Business Ideas. It’s definitely part of the canon of startup books moving forward. It’s a valuable resource.
Also, make sure to check out Beth’s work at Fluent Studio and the Fluency Score, I think that we will all be hearing much more about the Fluency Score and risk modeling for startups moving forward.
Thank you again to our speakers. The webinar recording and links will be shared by email later this week. That will include a transcript of the talk. If you’d like to follow up with us, feel free to email us at firstname.lastname@example.org.
Thanks, and have a great day.