How I Turned Our #1 User Complaint Into a Growth Engine
As a product manager, I always keep my eyes and ears open to collect users’ feedback.
This is probably the most important thing I can do for understanding what works or doesn’t work in the products I’m building, for coming up with new features and for prioritizing the backlog.
Assuming you do the same, you know how frustrating it can be when you come across a complaint about something you “can’t control”. As long as it’s something minor you may be able to live with it. But what do you do when it’s the #1 user complaint on your site?
Meet My Lemons
I’m working on the E-Commerce site of a top US retailer. Since the company sells both online and offline, one of our biggest challenges is to keep the inventory count up to date.
More than a few times this resulted in showing an item as in-stock when looking at the product list, only to find out that it’s out of stock when looking at the product page (I won’t get into the technical details, but just explain that the product page is the place we get the real time availability of the item without compromising the site’s performance).
I guess you can imagine how annoying this experience can be for a user. In fact, it was so annoying that the #1 complaint on our site was “Why are you showing me out of stock items???”. When diving into the numbers to check the frequency of this sub-optimal experience, I saw that it happens way too many times, especially in times like the holiday season when sales peak.
As I wrote, we couldn’t get the items’ real time availability on the product list, and this happened to many times to accept, so I had to find a solution to something I “can’t control”.
Making Lemonade #1
It didn’t take long until I figured out how to turn this into an opportunity —
Whenever a user watched an out of stock item, we replaced the disabled “add to cart” button, with a “notify me when in-stock” button.
This solution provided value both for the users and for the company —
- For the users. Instead of leaving them with a frustrating “dead-end” experience, we provided them with a way to take action. Even though they still saw the “out of stock” message on the product page, they appreciated the fact they can request to get notified when the item is back in stock. Moreover, getting notified when the item is back in stock creates a great personalized experience.
- For the company (1). In the competitive landscape of E-Commerce, retention is a key factor for success. Since our users are being bombarded with promotional emails and notifications from multiple sellers, it’s crucial to design our communications in a personalized way that will stand out and drive users back to our site. The “back in stock” notification does exactly that as it’s an on-demand notification rather than a “send and pray” one.
- For the company (2). When a user clicks the “notify me when in-stock” button we can be pretty sure that she has a strong purchase intent. This can be leveraged in several ways, ranging from interests collection to suggesting similar items or category-specific promotions.
Making Lemonade #2
As I mentioned above, retention is a key factor for success. One of the ways to increase retention is to have your web / mobile-web users download your App. Of course this is not enough, but it opens up an entire new channel for your users to use your product and for you to reach them.
In my opinion, one of the best methods to drive non-paid App downloads is to promote it in the right context while emphasizing the relevant value it will provide to the user.
So this is just what I did —
When a user clicked the “Notify me when in-stock” button on our mobile web site — we would show a “download app” teaser. The teaser’s message said “Download our app to get notified in real time”. Clicking the ‘download app’ button would take the user to our App’s page on the App Store or Google Play based on her device type.
This in-context ‘download app’ teaser showed an amazing 8% CTR
Bonus: How We Validated This Feature Pre-MVP Using “Fake Door” Testing
As you know, the purpose of an MVP is to validate your hypothesis and learn as much as you can while investing a minimal effort.
It just happened to be that at the time I was writing the Spec for this feature, I’ve read this wonderful article about using “fake door” testing to validate initial assumptions.
In a nutshell, the concept of “fake door” testing is about running a quick test that doesn’t provide value to your users, yet still allows you to know you’re going to build something your users would actually use.
So before investing the effort of building a system that would queue these back in-stock requests per user, match them with our inventory update job, prioritize the items, send the notifications and remove the requests from the queue — I’ve asked our developers to place the “notify me” button without actually making it do anything and track the number of impressions and clicks.
By doing this, I was able to know (almost for free) if our users would use this feature if we actually built it.
Since this was a test, I only exposed it to 10% of our users, which was enough for validating my assumption that users would click this button.
After I saw the numbers, I was confident that it’s worth investing in building this feature, and I was also able to evaluate the expected impact.
Recap
As Product Managers we must continuously listen to our users.
When facing with a complaint about something “we can’t control” (i.e. “The Lemons”), we should try looking for ways not only to overcome the pain, but to even leverage it and turn it into an opportunity (i.e. “Make Lemonade”).
Before running to build a feature, we should try to validate our assumptions in the most cost-effective way. One way of doing it is using “fake door” testing.
I really appreciate you took the time to read my post, and I hope you’ve enjoyed it. Just so you know, when you “clap” — it makes me smile :)
👋🏾 Get to know the people and ideas shaping the products we use every day. Subscribe to Noteworthy — the product & design newsletter written by the Journal team.
