What I have learned from the MilValChal
Before you read my article, I want you to understand WHY I wrote this. One of the goals of MilValChal is to learn, share what you learned, and mentor others. I hope that this will help others better understand the challenge, and find their own success through this experiment.
As with any story, it’s best to start at the beginning. Five months ago, I started the process of trying to turn my passion into a business. I have always been a problem solver and a dreamer, and my biggest dream is to contribute to the electric vehicle revolution. This is something I have been passionate about, very literally, since I was 5 years old, but I will save that story for another time.
5 months ago, having quite a few very good books, and having spent months of research online, I thought I knew what it meant to be a startup. I developed my idea into a solution, one I am still very proud of. I compiled a financial model of my business, authored a business plan, executive summary, and a pitch deck. Then I set out to pitch my idea and look for investors to turn this dream into reality. I found an open event, and pitched my huge, world changing idea to a group of 30 investors. I learned more in that one night than I had in the months preceding, and haven’t stopped learning since. The one thing I learned that night was that I had no idea what “traction” was. So I set out to understand this new, missing element.
One avenue of learning was an open startup coffee shared on Meetup and hosted by John Sechrest. I think back to my first time attending John’s open coffee, and my naivety must have been so obvious to him. I kept trying to explain that I needed to put the cart in front of the horse, and John kept pounding into my head that I didn’t even have a horse yet. What I mean by that, is I didn’t have customer #1. And more importantly, I had no idea what the cart even looked like. I had this grand idea of saving the wold, but the world is made up of people, and before I can save all of them, I must save one. JUST ONE. Then and only then (as John likes to explain) I turn one person into 2, 2 into 5, 5 into 10, 10 into 100, 100 into 1000 and so on. In each step of this process you learn, and plan for the next scaling leap. What it takes to make each step is different than the last. And very importantly, the customer is different.
Now, I need to take a step back here, I’m already getting ahead of myself and talking about scaling customers, when again, I haven’t talked about what the cart looks like. This is where lesson #2 comes in. Customer Development. At this stage I still had no idea how to understand the cart problem. I could design the best cart in the world, but if no customer is interested in my cart, I have no business. So it was time to learn how to learn from customers. I had to go out and find people I wanted to be my customer, and learn from them. This is when John linked me to MilValChal and I started to learn what a testable hypothesis is.
What MilValChal taught me
My very first lesson; I didn’t know what hypothesis meant in the context of a startup. I thought I did, but I was wrong. Regardless I submitted a card into the experiment: My First Experiment
I believe that $100 will prove that 10 of 50 Nissan Leaf owners who took my survey (with whom I have completed and submitted 4 interviews) will agree to phone interviews about their range anxiety within 3 weeks.
My hypothesis was formatted correctly, and my goal of performing more interviews was good, but I wasn’t testing something specific to my startup. I wasn’t learning anything about the cart. I did get my interviews, and I did learn from my customers, but nothing about that was in this experiment. More importantly, I met the stated goal, but by meeting that goal, I didn’t learn anything through this experiment about my business.
I tried again, My Second Experiment
I believe that $500 will prove that 3 of 50 Nissan Leaf owners who I interview (with whom I have completed and submitted 4 interviews) will agree to place a deposit towards my future range related product(s) within 3 weeks.
My first lesson from adamberk was that my hypothesis was not specific enough. How much is the deposit he asked.
So I tried again, My Second-and-a-half Experiment
I believe that $500 will prove that 3 of 50 Nissan Leaf owners who I interview (with whom I have completed and submitted 4 interviews) will agree to place a $250 deposit towards my future range related product(s) within 3 weeks
Ok, now I’m getting somewhere. I have a specific hypothesis that I can test and validate with a specific call to action from my potential customers. Great! So how are the interviews going? After this card I completed another 6–8 interviews, I think totaling around 9 at the time. And I was starting to learn from my customers. I felt like I was on a roll, making progress, and starting to learn. I was beginning to see what a cart looked like!
During this time, I saw an article on LinkedIn that Adam wrote: Mapping Your Business Model Canvas to a Landing Page Experiment. And that really resonated with me. THIS was an experiment I could very quickly perform and learn from. So I spent a Friday evening re-working my landing page, and published this about the results: Turning My Web Presence into a Lean Experiment. I consider that to be my third experiment. I got immediate feedback from my customers, and some kudos from Adam, I think I’m finally getting this!
But I jumped the gun, My Fourth Experiment.
I believe that $50 will prove that 10 of 60+ EV owners attending the National Drive Electric Week event this Sunday in Issaquah, WA will agree to problem interviews related to their range needs by the end of the car show. Bonus: I believe that I can prove 10 attendees of an unknown number who do NOT have a car in the show, but drive a compact, economy, or hybrid vehicle will also agree to a problem interview related to their interest in a future EV purchase before the end of the show. Of both of these hypothesis, I believe 5 will agree to the interview on the spot AT the show.
I attended the show, and I got a great response from people and collected another 11 interviews that day. I thought this was success, and I made a post saying so. But I was still wrong about my hypotheses. My hypotheses were still not directly testing things related to my cart. My cart was not taking shape BECAUSE of my hypotheses, it was taking shape DESPITE my hypotheses.
At this point, I’ve completed about 25 interviews formally, another 25–30 informally (and not recorded on MilValChal, since I don’t consider them deep enough conversations for real learning.) But I have learned, so lets look into what all of this has taught me.
What I really learned from MilValChal experiments
I learned in four different categories:
I learned what a TESTABLE HYPOTHESIS is.
I learned how to run a LEAN EXPERIMENT.
I learned how to perform a PROBLEM INTERVIEW.
I learned about MY BUSINESS.
On to the results:
TESTABLE HYPOTHESIS:
This is really all about distilling down your startups core problem statement to it’s true essence. Remember, you’re probably starting with assumptions. Identify what those assumptions are, find a way to isolate and validate them.
I made the assumption that ONLY electric vehicle owners that have cars with short range would be interested in my products. I identified that my assumption was that the owner’s range was what I needed to test, and performed interviews with people on BOTH sides of that line. I interviewed people with older Leafs, and I interviewed people with new Teslas. I learned my assumption was wrong, there were customers on both sides of that line.
Hypothesis: Vehicle range will determine if an EV owner is my customer. Test: Talk to EV owners of a variety of vehicles with a variety of range. Results: Range ALONE does not identify my customer.
Then I move on to the next assumption, perform a test, and learn again. Repeat until I understand what DOES identify my customer.
LEAN EXPERIMENT:
This is all about how you test your hypotheses. It can take many forms, in my example above, I expanded my interviews to include people I thought would not be interested. In my landing page experiment listed I tested if removing my product description impacted my web traffic behavior. Specifically I distilled my landing page down to a problem statement, and a value proposition. It can also be as simple as sharing your web address in a new forum. But the experiment should be specific to the hypothesis you are trying to prove. It’s results should give you an answer to the assumption you are trying to test. And these results should be MEASURABLE.
PROBLEM INTERVIEW:
There are many resources online about how to perform a Problem Interview, one of the better ones is Lean Customer Development by cindyalvarez. Get it, read it. Worth every penny. But most importantly DO IT. Get out there, and LISTEN to people. The interview is not about talking, it’s about listening, and learning. Ask the Five Whys, don’t talk about your solution, get them to talk, take notes and LEARN.
Remember, WHAT you are learning from them will NOT be immediately obvious. Keep listening, do more interviews, watch for the patterns. In my first few interviews I listened to the problems, in my first 10 I started seeing who had the problems, in my first 20 I started seeing correlations! As an example, out of 20 interviews of electric vehicle owners, 4 of them that wanted to solve a range problem did so for recreation. 2 wanted to drive farther to go camp, 2 wanted to drive farther for weekend destinations.
Don’t stop learning. I have only completed 25 formal interviews and have had around 30 more informal conversations, and I’m still learning from each completed interview. I have more scheduled next week, and I’m going to keep going, 50, 100, and more.
MY BUSINESS:
This is really why we’re all here. It’s not for the easy paypal funds for a properly written hypothesis. It’s to get better at building our startups. I’m going to share a few lessons that I have learned from my hypotheses, from my interviews, and more importantly, from the dozen or so experiments I have performed outside of this challenge.
The results:
My customers are divided into 3 categories (Remember this is for my core belief that people will pay to solve Range Anxiety in an electric vehicle): Peace of Mind, Charging Flexibility, and Living Life. Peace of Mind just wants a little more reserve range, just in case they don’t make it to a charger. Charging Flexibility wants to get rid of the annoyance of dealing with public chargers. Living Life just wants to go farther, to not be limited by outlets and chargers.
EV owners are starting to Cross the Chasm from early adopter to mainstream customers. And both customers are very different for my business. The Early Adopter is eager, a DIY, and is willing to try unconventional things to solve their needs. They are also informed, can articulate their car’s ROI and TCO, and more importantly; skeptical of companies like mine as several have tried and failed. The mainstream customer just joined the club, they like to share their EV experience with other EV owners, but are still testing the waters, and likely are only leasing their vehicle, for now. BOTH customers are skeptical of the opinions of people who do NOT own an EV, and can even be unwilling to share their POOR EV experiences with someone who does not also own one. They ALL want to be positive evangelists for EV, and try not to talk about bad experiences.
Product cost is not important, neither is deposit cost. EV owners are willing, and often capable of, buying high dollar accessory items that show real value. But they want to see that value articulated in real, tangible data, or proof.
And finally, there are some EV owners who are the hobbyist. They built their own conversions, did their own research, and have very strong opinions about a variety of products, technologies, and solutions. This group too is split. The YOURSELF category takes the Y in DIY very seriously, and will solve their own problems, at any cost. The PROBLEM category are the ones that are actively trying to solve their range problems, and are looking for new ideas.
All of these customers know very well that once you become an EV owner, the world changes for you. They are (almost) all convinced it’s for the better, but they are all also skeptical of unproven solutions. Without proof that my solution works, they have very little interest in making a deposit.
To get their attention, I need to join the club and own my own EV, and I need to prove that my solution works.
As I continue to work on my interviews, customer development, and various experiments, there is one specific hypothesis that I must validate soon to prove my market…
My current experimental hypothesis:
I’m formatting this in the style required for the MilValChal to help people better understand how to apply what we’ve all learned in the context of our own startups.
My name is John Bysinger and I believe that $6,850.00* will prove that owning an EV, and having a functional demonstration prototype will change the conversation, and that 10% of 100 EV owners that I provide a demonstration to will commit to a $250 deposit for my product within 3 weeks of acquiring the car and prototype materials. This is my 5th Experiment.
- $6850 breaks down as follows: $5000 for one SPECIFIC car I already have lined up, $850 in tax, license, etc, and $1000 for prototype materials consisting of COTS (commercial-off-the-shelf) components requiring NO development costs hardware OR software.
Conclusion:
Through all of this, I must keep learning, experimenting, and refining my core business hypothesis. I refine the cost it takes to prove the hypothesis, the target customer, the scope of the market, the expected results, and compile all of this into a viable business model. I am currently working on these refinements using Ash Maurya’s Lean Canvas model. But more on that in another article later…
