Craft Tapp — 100,000 Hypotheses

Testing AdWords and landing page while conducting user interviews using primary and secondary level data.

Below are the findings from one of the 100,000 Hypotheses tested and posted on a data-driven decision making group on Trello. I was referred to this team by adamberk through the 150Startups slack channel.

Hypotheses

I believe that if I spend $50 on AdWords, 15 craft beer enthusiast in Calgary, who are educated males between the ages of 30–45, will sign-up with emails and phone numbers through a landing page featuring a Craft Beer Club by November 16, 2017.

Results:

• Only a single phone number, email and name was reached out of 15
• Five postal codes were obtained
• No video made, no leads
• High converting AdWord and Landing Page was found at the end of the test
• $35 was spent in total
• Three customer and brewerie interviews were conducted

Important Facts Learned

• Users want rare craft beers they have never experienced before
• Exclusivity/VP brews are a big selling point
• Users go to multiple breweries at a time. These breweries and drinks are reffered to by the app Untappd and the website RateBeer
• Enthusiast in Calgary will rate beers on Untappd and RateBeers is used to find experiences
• Enthusiasts will take their friends to local bars like Craft to try particularly exciting craft beer experiences

Surprising Facts Learned

• Breweries may have a bigger and better problem to solve

Next Actions

• 70% of leads will answer a phone call to improve our AdWords and Landing Page.
• Using the data received from google analytics, three in-person customer interviews will be filmed that best model our leads.

Data and Results Journal

November 12th, 2017 — Development Day

Adword and landing page posted, the value prop is off. Edits will be made later on tomorow.

1. AdWord

First part of the user journey

2. Landing Page

Main Call-2-Action, key partners

How it works

Value Prop One

Value Prop Two

Value Prop Three

The Team

November 13th, 2017 — First Test Day

Cleaned up the search terms and phrases from the ads. It seems like my assumption of generating 10 sign-ups in a week was too aggresive. A better time frame would have been two weeks. Google AdWords is problem solving focused, the advertisement I used and landing page focused too much on creating a trend. Below are the changes I made. Irshad and I are also organising some customer interviews to be done concurrently with the test to better improve the outcome and understanding of the customer.

Spent $15 on random search phrases.

It actually has a call to action now!

November 14th, 2017 — Day Two

After making the changes to the ad and the search phrases, we had our first lead convert at $4 spent. Unless the ad and landing page convert better, the test will fail. No minor changes will be made unless a customer interview warrants it.

Phone Interviews:

1. If the customer agrees to a craft beer taste test, book a time, end the call.
2. Otherwise, ask “What is the number one problem faced drinking and having craft beer delivered?”
3. Ask “How does this make you feel?”
4. Repeat two more times ideally

To better outline everything that needs validation for a better converting landing page a Mural was developed here. Preview may be seen below:

Taken November 14th, 2017

November 15th, 2017 — Day Three

6% Conversion, Need To Do Customer Interviews

Two Customer Interviews Done At Craft

We learned that potential users follow a journey that resembles:

1. Browse a brewery on RateBeer.com
2. Drive to brewery
3. Browse a beer on Untappd to try
4. Purchase a bundle of beers to try

November 16th, 2017 — Day Four

I have made some changes to the “How It Works” section and only asked for Postal Code. This will allow us to get a demographic on who our users are through Google Analytics, ensuring we do user interviews with the right crowd.

Asking only for postal using a landing page that converted well in the past

Users mentioned they used Untappd and RateBeer, Untappd gave us the clear to use their API

November 17th, 2017 — Day Five

Visited Cold Garden Brewerie, the founder mentioned the top problem he is facing as an early stage brewerie are found below:

• Finding the balance between making sales and brew
• Social media engagement
• He mentioned sales so far have arrived organicly
• He REALLY wants to get into the liquor store and is struggling with creating a beer box

I am starting to realise that we do not understand the users problems when it comes to craft beer. However we do understand the breweries problem very well.

November 20th, 2017 —Day Eight

The new page is converting postal codes better, as the test goes on we will need to test what kind of contact information we can get from our leads. This shows that users are willing to risk their location information to see what breweries we offer.