A Growth Marketer’s guide to “Smoke Tests”
A secret weapon to validate your next great idea
Sitting on a game-changing idea? The next must-have product or service with unicorn potential? If only there was a way to validate your startup idea before jumping in head first … welcome to the world of Smoke Testing.
What is Smoke Testing?
Smoke testing is a term most commonly found in the realm of computer programming and software testing; referring to the initial testing process conducted to check whether software is ready / stable enough for further testing. The name originates from hardware testing, where a device passed the test if it did not catch fire (or smoke) the first time it was turned on.
In the world of growth marketing, smoke testing is one of the most cost-effective and efficient ways to test newly formed ideas — resulting in tangible learnings and actionable insights. Smoke tests can even preempt the Minimum Viable Product (“MVP”) concept. Helping you answer the fundamental question “Will people actually pay for my product?”
Smoke tests often comprise little more than one well-put together landing page centered around a strong call to action (“CTA”). It is a method of providing users with an insight into your concept alongside an option to purchase or sign-up. Behind the scenes the product may be in its infancy or may not even exist!
The webpage can then be used to easily track and quantify early demand for your product in actual dollar-terms, which in turn can validate (or disprove) your hypothesis and determine whether you have a feasible business case (or not). Any information collected from these first few highly motivated users can also be subsequently used to target them as your first customers. As these early adopters have already essentially “pre-bought” your product.
All this achievable on a tight budget AND in under a few weeks? “Not possible” you say!
Read on for more insights in to this game-changing tool …
Why Smoke Test?
Whilst conducting customer interviews can be insightful and should form part of any early-stage market sounding, inherent difficulties with surveys and response biases such as acquiescence bias (a.k.a “yea-saying”) is where smoke testing sets itself apart. As nothing quite signifies concrete buyer intent like a potential customer clicking “Buy Now” vs. just saying “I’d probably buy that” when asked.
As entrepreneur and author of the Lean Startup, Eric Ries, puts it,
“The grim reality is that most startups fail.” and
“The problem with entrepreneurship is we are often working really hard producing high quality products that no-one wants”.
Long gone are the days of “Build it and they will come.” Which is where smoke tests come in. They can provide crucial validation and proof of a viable concept with minimal resources, ahead of investing (further) precious time, effort and money into building your next great idea.
So prior to quitting your job and jumping in head first, what is the best way to smoke test and find out if people are actually willing to pay for your product?
5 Step guide to Smoke Testing
Step 1: What to test?
Armed with your hypothesis or idea, firstly it is crucial to define what you wish to test and what would success look like? In most cases, the most important question to answer is “are people actually willing to pay for my product or service?”, which in turn translates to “do I have a feasible business idea?”
Put down your early markers / targets for success. For example:
- If more than X% of visitors to the landing page across X weeks click “Buy” or “Join waitlist”, we may have a product worth building
- If we receive X orders in X weeks, we may have a product worth building
- If we can pre-sell $X in X weeks, we may have a product worth building
TIP #1: Whilst useful to set yourself ballpark figures to strive for and benchmark against, these yardsticks for success will also inevitably evolve as the data you collect comes in. Don’t be afraid to reassess and reevaluate to ensure that these goals are a) realistic and b) appropriately reflect the degree of success necessary for validation
Step 2: How to test?
Build your landing page!
Underpinning smoke tests are the fundamentals of speed, efficiency and cost — we want to efficiently validate and de-risk our idea as quickly as possible, as cheaply as possible.
Using simple drag-and-drop page builders such as Unbounce or Instapage allow you to quickly construct professional landing pages without the need to tie up precious engineering or developer resources.
Your landing page should include an explanation of what you do, the benefits you provide, and any products and pricing. Most importantly make sure your call to action is clearly marked front and centre. The best CTAs are short and compelling. Strong verbs demanding action are particularly impactful (e.g. Buy now, Sign-up, Subscribe!). Driving traffic and conversions of this CTA will determine the success of your smoke test (more on this later).
Smoke tests also provide a valuable opportunity for you to craft and test your Unique Value Proposition (“UVP”). This is the statement that describes the benefit of your offer, how you solve your customer’s needs and what distinguishes you from the competition. It’s the primary reason why a prospect should buy from you and needs to be clearly and concisely noted on your landing page.
Get your language and messaging right in the first instance, and only then test aesthetics such as color, font, size, positioning etc.
The below is an example smoke test landing page I pulled together over a weekend (using Instapage) to test a concept.
TIP #2: Ensure your branding, messaging and visual identity are consistent across all platforms — from your social channels to any advertisements. This is crucial for adding legitimacy to your offer and call to action and can improve conversions and brand building
Step 3: What to track?
Relating to the most important feature on your page, make sure your call to action button is trackable!
Helpfully, Unbounce and Instapage both automatically provide basic conversion tracking, however based on your level of technical ability it can make sense to also track custom user events or links — particularly if you have one or more CTAs on the page.
The point of your landing page is to see if anyone exhibits enough intent to complete your desired action. So ensure your site is setup to capture all valuable insights around site activity, conversion rates, potential buyer intent and customer information (such as email signups).
KPIs you will want to pay particular attention to include overall conversion % of customers from landing page to completed action, the click-through-rate from any ads you set up based on different keywords, and the bounce rate of your landing page for different keywords
TIP #3: Don’t be afraid to test with a “Buy Now” button — even if your product isn’t ready! One simple solution is to lead any potential customers who click on the button to a waitlist / email sign-up modal where they are prompted to enter their email information for restock alerts and / or an exclusive discount at launch.
Not only have you secured a firm indication of buyer intent, but you now have an email for a potential future customer. And if you’re worried about turning off potential customers, Eric Ries explains here why not to worry.
I employed this tactic successfully in the same smoke test noted earlier, resulting in a healthy list of initial email leads! (See below)
Step 4: Who to test on?
Next is all about getting people to your site.
Whilst getting a lot of eyeballs on your landing page is valuable, what matters more is getting the right eyeballs. In other words, the right traffic that will potentially be interested in your offering. For this, it helps to know your target customer and their personas.
Utilizing paid channels such as Google Ads and Facebook ads and running small campaigns lets you target specific audiences based on high intent keywords or lookalike audiences, pulling from their wide user bases to drive relevant traffic to your landing page.
Daily budget caps can also be set to ensure your ad spend remains affordable and is no more than necessary at this early testing stage.
As with every stage of smoke testing, the ads themselves present yet another opportunity to test and learn. Running multiple variations can allow you to refine your ad copy and challenge your existing assumptions about your target audience, whilst also determining viability of untested marketing channels.
In the below example, A/B testing of ad copy as part of a Smoke Test resulted in a c.100% increase in click-through-rate!
TIP #4: Choose a maximum daily budget for ads that is affordable for you to run a campaign for a few weeks. The key determinant of length of any smoke test will be when sufficient data has been gathered to provide conclusive insights
Step 5: Analyze results and optimize
As the data and the clicks flow in, be sure to continually track and optimize your campaign to maximize insights and learning. This may include tweaking of ad copy and / or elements of your landing page.
For example, if you spend $100 on Google Ads and received 100 clicks on your ads, however receive 0 clicks on your “Buy Now” CTA, you may have to question either your messaging, your pricing or your fundamental idea. Altering your landing page to test these variables can help you land on an answer.
On the other hand, if you receive 100 clicks on your ads resulting in 10 “Buy Now” clicks for $50 a piece — your $100 in ad spend would have generated $500 in potential revenue. In which case — Congratulations! — you may have unearthed a promising business.
TIP #5: A/B testing or multivariate testing can be employed as part of a smoke test, allowing you to test one or multiple elements across different page variations — including value proposition, messaging and pricing. Unbounce and Instapage both have built-in functionality to help you easily test page variations and track comparable performance. As part of your smoke test be bold; A/B testing can be a great way to test your riskiest assumptions!
Conclusion
Next time you’re struck with a wave of inspiration or a killer concept, run a smoke test to validate it! A smoke test can be one of the most powerful ways to help you figure out if you’re sitting on a Facebook or a Friendster — just follow the 5 simple steps in this guide:
- Define what success would look like
- Design your landing page experiment
- Define your key metrics and track them
- Drive the right traffic to your experiment
- Analyze the results and iterate / optimize!
I’m always interested in meeting new people and hearing about the latest things people are working on! Feel free to reach out and let’s grab a coffee!
Follow me on twitter to hear more about my adventures in Growth Marketing!
