Launching & Learning — A Beta Release
A Beta phase or release is an early attempt to test that you’re solving the basic customer needs which you have identified, your solution works as intended and with this you can scale.
When you first decide to release a Beta program, it’s important to recognise you still have a lot of learning to do. A Beta release is not an excuse to get away with a sub-standard products or a shortcut on quality, it’s a controlled experiment to maximize learning before scaling. Consider this activity as an opportunity to learn and maximize value, whilst reducing the potential risk of discovering damaging issues after launch. Beta programs executed well can reduce the chance of failure for your business and customers, effort, company and/or brand damage.
Running a Beta programs usually involves a cross-functional, cross-departmental engagements and projects where you will identify and coordinate the necessary activities, dependencies and learning together as you capture feedback. A diverse, open and committed group with clear roles and responsibilities for each would be recommended.
What kind of things will you be expected to do in Beta ?
- Releasing regular updates and version changes of your product using feedback and learnings obtained from the customers using your Beta product.
- Establishing close relationships, to support the customer being able to use the product and to maximise learning on the products effectiveness.
- Measuring and testing your hypotheses against customer cohorts using experiments.
- Setting up specialized release programs and marketing campaigns for early adopters to participate. Some examples of some well known Beta programs and platforms include:
- Apple Beta Software Program
- Google Play Services Public Beta Program
- Android Developer Preview
- Windows Insider
5. Setting up clear and transparent data policies and feedback channels with the customers consent. (Attention to local data protection laws required)
6. Generating some basic on boarding and support materials.
7. Regularly and sharing feedback and knowledge with your teams.
The Beta Learning Process
You might know a great deal, but your on a journey to learn as much as possible as fast as possible and discover the unknowns. Remember it’s impossible to be aware of the Unknown,Unknowns you are yet to encounter until you have encountered them. As these emerge, you will need to understand and solve them. Exploring the many possibilities to solve a problem is a beneficial way to maximise growth, customer value and release potential in your business model whilst also learning how to change the solution as needed to be more effective.
Hopefully with the early Beta launch of your product, you are taking small, rapid learning steps with an observational state, supported by controlled time based experiments. A tried an tested practice to support learning is the Lean based Plan-Do-Check-Act (PDCA) introduce by William Edwards Deming, long before the now popularized Lean Startup Build-Measure-Learn methodology. Maybe somewhat pedantic, I tend to favor defining the PDCA process as it explicitly calls out the ‘Plan’ phase first. Although Build-Measure-Learn advocates this through Hypotheses, it helps remove any assumptions and re-enforces the need for success criteria. To maximize learning in Beta, the scientific method should always start with a structured hypotheses first with clear success criteria.
Make sure you explicitly set the criteria before you run the experiment, and make a record of it. It’s too easy to fudge the numbers later and rob yourself of any valuable insight. –
https://www.teaguehopkins.com/2014/03/set-minimum-success-criteria/
5 Things to look out for when launching a Beta program :
- You need to grow and learn as fast as possible, but at a pace that your business operations and culture can sustain. Consider soft launching and managing customer cohorts to maximise learning as you apply changes. Ideally you would structure a specific Beta program such as those demonstrated in the example above.
- Early adopter expectations of quality in terms of your product can be managed better if Beta is stated on your product. Generally this sends a signal to expect some minor bugs and issues, which can result in a little more tolerance from customers. Customers being explicitly aware of this are likely to provide bit of slack as you explore the unknown goes a long way and typically will be more engaged with providing feedback.
- The effort to maintain and transfer knowledge across your organisation will increase as your team sizes grow and new skills and resources will emerge. Make all information as visible and transparent as possible. Be mindful to observe the impact on services as you engage customers. This will inform your business model.
- It takes time to diagnose issues and learn. I’d highly recommend you have good enough diagnosis tools in place. Don’t over do it in the early days though as it will increase cost, time and effort to manage tools. There are plenty of great tools out there to kick off such as Kibana or Crashanalytics.
- Consider keeping a low key initially and avoid the big bang marketing campaigns. Introduce this when your confidence grows.
If you don’t have the might of Apple and Google behind you to build your own Beta platform, there are a number of great tools available that can help you structured and manage a Beta program. These tools can also be useful for giving you more structure as well should you be new to the Beta process. Here are a few examples :
5 Beta Test Platforms & Tools
Please use the comments below if you would like to add useful links, experiences or insights to share with readers.
