Failing Fast and Second Chances
I’ve long had a theory that there is a downside to the “Fail Fast” mentality that many software companies claim to subscribe to.
The spirit of Fail Fast is a positive one. If you can fail fast enough you can arrive at a desirable result faster than simply focusing or trying to perfect the solution. At its core it is de-stigmatizing failure and looking for opportunities to learn and grow. Instead of taking months to plan and build a project, it’s rooted in creating minimum viable products that can prove if there is enough interest or potential to continue to pursue the idea. It’s all about minimizing the time and resources devoted to finding out if you should stick with something.
It’s not enough to fail and move on to the next idea, it’s about measuring and analyzing why things fail and making necessary adjustments in future efforts. As long as you can identify what you want to learn and measure, because you can’t control what you can’t measure.
A Confession
This is by no means a take down piece on the merits of Fail Fast. I for one subscribe the the Fail Fast mentality, but in the back of my mind I’ve wondered what is the cost of failing on your first attempt at someone’s attention? Meaning if I fail the first time I get someone’s attention how likely are they to give me a second chance and how hard will I have to work to wash off the stench of failure.
Personally some of my thoughts around this have been influenced around customer feedback I’ve received in the past. A couple of years back I was receiving feedback on a new feature from one of our customers. Early in the conversation the customer cited that one of our existing product features was failing in his eyes and he didn’t see why he should trust a new feature if an existing feature wasn’t working. Hearing that as a Product Manager was a bit of a gut punch. That loss of faith is hard to regain. Our product had some short comings and this individual wasn’t ready to trust new features with our previous failure still in plain sight.
So the question is: can you wash off the stench of failure that comes with failing fast? If so at what cost? How do you get customers to give you another chance?
1N = 5P: An Experiment
In 1970s Dr. John Gottman studied a series of unhappy couples. He tasked the couples with solving a conflict in their relationship in 15 minutes. After observing the couples he could predict which couples would divorce with 90% accuracy. What he discovered was a ratio. For every negative interaction a stable marriage needs 5 or more positive interactions.
Now let’s assume something similar applies to software. Your customer has to have 5 positive experiences to outweigh a negative one. So ask yourself when you are attempting to fail fast with your product or features how many positive experiences have your users had with your product?
Do you have enough good will built up with your customers? Are you being selective about who you fail fast with? In today’s day and age we have tools like NPS surveys and/or usage and engagement metrics to try and derive who is having the most positive experiences with our products.
Engagement Metrics: One of the first places I will start looking for customers to test with is customers who have high engagement in areas related to what we are doing. In most cases their engagement is high because they are deriving some sort of value.
NPS: In a recent example the new product we were working with didn’t necessarily correlate to usage of any of our existing features so instead of starting with highly engaged users we opted to reach out to a random sampling of customers considered NPS “promoters”. These customers have communicated they are having a positive experience with out software and thusly have some of the “goodwill” we are looking for when failing fast.
Users having positive experiences with your product are more likely to stick around and see what other solutions you throw their way while those having negative experiences might be pushed over the edge.
Expectations
Setting proper expectations is one of the most important aspects of successfully failing fast…(That makes sense right?)
We come from a culture that has a low tolerance for failure. You may need to set the stage properly for your users. Do as much as you can to help your user see your vision for the product. Users need to understand the context of the product or feature to have a positive experience.
This can ideally be learned quickly by prototype testing or even live beta testing with users. Give them tasks and sit back and see if they can accomplish them. Look for the right amount of context needed and try to build that into any training or walkthrough material when future users would first use the product.
Gratitude
Do all you can to make your attempts to fail fast positive. Be gracious to users taking time to test your work. Recently I received a “break up” email from a customer saying they were no longer interested in beta testing one of our products. While this was initially a letdown, and I could have just left things at that, I responded with the following message:
Thank you for trying things out and for the feedback. I completely understand if it isn’t currently meeting your needs. Admittedly our offering is still a work in progress. We’ll keep working on improving things moving forward and we would love for you to give things another look further down the line.
Treat customers like they are doing you a favor, even if they choose not use your products (try to create a positive experience). Be cognizant that what you are asking of them might be outside of their normal work. Be responsive if they have questions or suggestions. Engage them regularly and show them when their feedback helps shape future iterations.
Fail Safe
Limit impact where you can and know the risks.
For example if you are testing a new login method for your app and the potential impact is users will be locked out the impact to the user is high. If your test fails your user loses all value from your product. In this case try and keep your test group small to mitigate your high risk.
In contrast if you are testing the click through rate of a redesigned button there is likely no loss in functionality with your new approach. The risk here is instead minor inconvenience instead of loss of access or functionality. In this case feel free to test with a larger group as you can get more data points with minimal impact to value.
When working with my team we try to ask the following questions before we release something to customers.
- How easy is it to roll back?
- What is the potential impact it will have on users?
In a most cases with our team asking this question leads us to consider adding further control, in the form of a feature flags, on our ability to enable and disable a feature for our beta users. Adding flags allow us to play defense if needed and reduce our risk when releasing features.
As a general rule of thumb consider this decision matrix before you fail. In some cases you can swap out Downside with something like “ease of rollback or reversal” and Upside for “potential impact on customer”. Either way thinking through this matrix will help you shape or potentially rethink your release strategy.
Parting Thoughts
With all I’ve said here the one thing I don’t want is anyone living in fear to release new products or features. These are merely things to consider, some words of advice and caution. What is most important is to ship valuable things early and often, it’s the only way to fail fast. If you aren’t embarrassed by the first version of your product when you look back, you waited too long to ship.
