Lessons learned from a $100 Lean Experiment

I have always being a big fan in Lean Models but this time I had a unique experience Thanks to adamberk #leanstartup Milvalchal model to help entrepreneurs run their experiments the lean way by funding these small experiments.

I was part of an experimental incubator that gives startups and entrepreneurs funding in exchange for testing and sharing leanstartup hypotheses. I started my first experiment with a hypothesis assuming that by spending $100 on targeted online advertising I would be able to get 50 users interested to be early adopters of Wnna (AKA $2 is the user acquisition cost) who usually eat at restaurants with a minimal coming soon landing page and basic information.

I assumed by only getting 1,000 visitors to the website I will be able to get these 50 early adopters (5%), this assumption was extremely wrong (less by more than 5 times) however there was a lot of interesting findings.

What I did:

• Run a $20 sponsored StumbleUpon campaign to target people interested in restaurants
• Run $30 advertising campaign on Facebook targeting people living in big urban cities like New York, London, Paris etc.. and using apps to find and discover restaurants like Yelp, TripAdvisor and Foursquare
• Run $50 advertising campaign on Facebook targeting people traveling in big urban cities like Newyork, London, Paris etc.. and using apps to find and discover restaurants like Yelp, TripAdvisor and Foursquare
• A/B Testing with multiple e-mail opt-in forms in landing page (Embedded, Popup and Exit Intent)
• Interviewed 25 users who frequently eat at restaurants

Findings:

• The assumption of 5% of 1,000 visitors that cost $100 of advertising will be converted into signups was wrong, first with $100 I was able to to get more visitors (around 3600) and the conversion rate was 0.85% instead of 5%.
• StumbleUpon Campaign didn’t result in any signups
• Facebook campaign with people traveling in big cities had a better CPC (0.02) and a better conversion rate
• With $80 spent on Facebook campaigns I was able to acquire 30 users AKA Cost per acquisition is $2.6 per acquisition vs the assumption of $2
• Popup Opt-in forms had better results followed by Exit Intent and the least results came from the embedded form
• By Interviewing 25 users I realized that %60 of the users have this problem and require a better solution and %48 is willing to download an app that solves this problem (full results and report here)

Whats Next?

Next experiment will be a Hypothesis on Acquiring Alpha users from the same target audience discovered (customer discovery) to actually download the Alpha / Early Beta App using a better landing page with more information and visuals about the product.

So my hypothesis will be: I believe that $500 will prove that 2.5% out of 18,000 people (450 people) who use apps to find restaurants and use an Android Phone (with whom I have completed 25 surveys) will download the alpha version of http://wnna.co app. This is my 2nd experiment.