The Hypest Validation Method of Them All🔥: Apple Testflight (#TT03)

Why Apple’s beta environment is the best for product validation.

This post is part of an ongoing series about Startup Validation called Testing Tuesdays.

Gain the edge with the hacks, tools and smarts to build start ups the right way early by subscribing on venturism.substack.com.

Testflights are by far the hypest validation method right now. They combine user-feedback with pre-hype in a way that is hard to recreate through other channels.

Testflight, Apple’s iOS Beta Environment, is nothing new, but the recent hype around it is, in what has become a weird example of accidental product-zeitgeist-fit.

It hits all the right notes of startup culture: early access, exclusivity, ‘discovering the next it’, and a feedback-channel that actively solicits your ‘hot takes’ on products.

Source: Twitter

If you’re looking for proof of concept, just look at the buzz @JordanSinger was able to create with his ‘App Store for Testflight Apps’ (making them easier browse and discover) called Airport.

Since nearly all of them have waiting lists, it is a FOMO nightmare. But I am addicted in a way that reminds me of when I first got my hands on the original App Store.

Almost as if by design, Testflight’s second youth comes with a brand new logo. Perhaps it was meant to be. In this post we will explore what makes Testflight work so well as a validation tool for your app.

So, Why do Testflights work so well?

Early adopters and VCs naturally gravitate towards becoming test-pilots.

The people who spend their free time looking for the next big app, are the same people who will be interested in testing those apps early. If someone enjoys sites like Product Hunt and Indie Hackers, chances are they will enjoy test-piloting. Beta-testers often become customers or even investors.

Source: Twitter

Scarcity is built in.

The limit on any Testflight app is 10k users. As a bootstrapping founder, that might sound like a lot, but for a massive audience of product-hungry neomaniacs the FOMO is real.

To make matters worse (or better), developers can choose to make their beta ‘invite-only’, giving them full control over the supply-side and guest list. As with the line outside the club: The more people want to get in, the more people want to get in.

Access is the highest status-signal when you’re a 40-something-year-old ex-founder on Tech Twitter.

Pre-Hype is built in.

People who have access to your app will talk about it, because it gives them social proof, and they are excited about trying something new (early adopters, remember?).

People who do not have access to your app will want to be part of the conversation, and do whatever they can to move up the waiting list and gain access at launch.

Either way, people become genuinely excited about having their hands on your product. So when you do officially launch demand may already be there.

Feedback is built in.

Lastly, Testflight’s purpose is to help you test your app. Apple has added in features to make it easier for users to give feedback and send bug reports, f.i. by taking a screenshot.

People talking about your app on social media can be easily approached for a more in depth conversation on usability and their favourite features.

Gathering feedback is an important part of building a product, and Testflight is a great feedback tool.

How to use Testflight for Validation.

If you are considering using Testflight to test your app, I’m assuming you already validated both problem and solution using other methods beforehand. If not, check out this Database of 52+ validation experiments and do so first.

If you have, let’s dive in.

First and foremost, always provide clear instructions to your users in the ‘Test Information’ section of your submission, outlining the app, the purpose of the Testflight, and the type of feedback you are looking for. Based on your idea’s level of validation, your approach might change.

Source: venturism.substack.com

To Test Solution Fit.

• Launch to a beta-group you control (n ≤100). Make sure they seem willing to provide feedback.
• For technical feedback (feasibility), Testflight’s built-in feedback system is your go-to spot.
• For desirability feedback, monitor social media for mentions of your app. See if you can convince some of your test-pilots to do a 1-on-1 interview.

As with any Solution Fit experiment, your goal is to find out to what extent your solution solves the user’s problem. Ask them to rank features. Ask what they couldn’t do without. Ask what the alternative would be if they didn’t have your solution.

Source: tenor.com

To Test Commercial Fit.

To Validate Commercial Fit, you have two options.

1. Launch a public beta (limited to 10k users). A public beta is good for:

• Testing paid go-to-market strategies
• Platform building
• One-to-many Interactions
• Accessibility

2. Launch an invite-only beta. An invite-only beta is good for:

• Testing organic go-to-market strategies
• Community building
• Close interactions
• Exclusivity (Luxury software, memberships, …)

Always incentivise people to move up the waiting list. The amount of hurdles a user will take to gain access is a good measure of the value your product has.

To Test Product-Market Fit.

Even though I would not advise a Testflight to test product-market-fit, sometimes it just happens. The hype behind an app like Clubhouse is so substantial there is hardly any question how the market will respond after they officially go live on the App Store.

That being said: by allowing invitees only, you can control cohorts, and get small scale data on retention, MAU / DAU, etc, which could give you an early indication of P-M Fit. However, a successful Testflight is by no means a sign you can lean back and relax. You’re not making any revenue yet.

Source: Tenor.com

What if my app doesn’t create a buzz?

If your app isn’t the talk of the town yet, don’t freak out. In this series we focus on validating your ideas — Hype isn’t a criterion.

Don’t mistake a distribution problem for a product problem. Even though organic growth is a welcome sign of desirability, the goal is to talk to the users you do have, and find out what they like and dislike about the product.

Once your product is so good people start asking when you will going live, you can worry about distribution.

If you want to learn more about launching on Testflight or becoming a test-pilot, check out these links:

How to become a Test-Pilot

How to submit your App to Testflight

Thanks for reading! 🙏🏻

Want to get the tools, hacks & smarts to build startups the right way?Subscribe to my Substack or Follow me on Twitter.

If you feel the need to reciprocate, feel free to share, leave a like, or buy me a coffee ❤️.