Amazon’s Add to Cart dropped by 14% | PM Interview: Root cause analysis.
You are a PM at Amazon and you noticed a 14% drop in Add to Cart. using the PM senses evaluate the root cause of the issue.
Root cause analysis is a common PM interview question that is asked of PM candidate to gauge their PM sense. Root cause analysis is also known in general as RCA-type questions.
In simple terms, RCA deals with finding the cause of some sort of problem within the product.
This type of question may sound simple, as most of the RCA question does. But RCA can be tricky sometimes. Especially if you have not practiced a structured approach.
The important aspect of solving an RCA problem in a PM interview lies in the structuring of the solution and analysis. This provides a clear picture of the candidate’s thought process and also helps the candidate to think clearly.
The basics of RCA-type questions lie in asking too many clarification questions.
This will help the PM candidate to get more information about the problem at hand, but will also be able to think clearly with more data points.
Below is one of the RCA questions asked in a PM interview.
The Amazon’s Add to cart has decreased by 14%. You are the PM for the Add to Cart flow, analyse why the drop happened and what was the reason behind the drop. Also, suggest how would solve the issue.
How will I go about this Analysis?
Before delving into this issue, I propose a structured approach to address the problem as an Amazon PM.
- Firstly, I will provide a concise overview of the product and service in question, establishing a shared understanding.
- Following this, I intend to pose clarifying questions, ensuring we’re aligned and that I am comprehending the issue accurately.
- Subsequently, I’ll dissect the problem, examining it through four distinct factors:
a. Basic Systems,
b. External Factors,
c. Internal Factors, and
d. Factors Beyond Our Control.
4. Then based on the data collected I will summarise the findings and try to pinpoint the factor that caused this issue.
5. Lastly, I will offer potential solutions to address the issue, along with insights on how similar situations might be preempted in the future.”
I have created a Template explaining the structure that I use during the PM interview. If you wish to get the template for free. Share your email id in the comments below and write: “RCA template”.
Brief about Amazon.
Amazon, established in 1994 by Jeff Bezos, has evolved from a pioneering online bookstore into a colossal e-commerce and technology enterprise. Its diverse range of offerings now includes everything from consumer goods and electronics to innovative services like cloud computing through Amazon Web Services (AWS) and digital streaming.
This diversification, underpinned by a relentless focus on technology and customer experience, has positioned Amazon as a dominant force in both the retail and tech industries.
Central to Amazon’s success is its cutting-edge logistics network and advanced data analytics, which have redefined retail efficiency and customer convenience.
The company’s commitment to innovation is further evidenced by its development of consumer technology products like Kindle, Fire tablets, and the Echo smart speaker with Alexa.
Now that we have had a look at the History of Amazon. Let’s dive into the problem that the PM at Amazon is currently facing.
The Amazon’s Add to cart has decreased by 14%.
Me: To clarify, the Amazon add-to-cart decreased by 14%. And by Add to Cart, we imply that it is the product that people add to the cart on the site.
Interviewer: yes
Me: This decrease of 14% was observed over what period? And was it observed in any specific product or product category?
Interviewer: It was observed in the last 7 days as compared to WoW. This drop was observed in all product categories.
Me: This decrease since we are measuring WoW, what is the average fluctuation seen in this metric?
Interviewer: We mostly see an average drop or increase of 4–5% WoW during the normal period.
Me: So the 4–5% fluctuation is during normal days, that’s when we don’t have any sales or offers.
Interviewer: yes
Me: This add-to-cart functionality we are talking about here is this for Amazon.com or other Amazon entities? Like Amazon Fresh, audible, etc.
Interviewer: This is for amazon.com the parent website.
Me: Was this drop just for the website or the App as well? ie Android and iOS?
Interviewer: This was absorbed across all three platforms.
Me: Have me checked the metric logger(analytics tool), which reports the count of the Add to Cart event. If there was any issue with the event counts at their end? If the Analytics system is working properly and there are any issues in that product?
Interviewer: Yes we have checked, and no issues are present with the Analytics tool.
Me: Is this a seasonal effect or a repeating trend that is seen during this time of the year?
Interviewer: No not a seasonal trend.
Me: Was this drop seen over any particular geography?
Interviewer: yes, this drop was observed only in India.
Me: Over any particular demographic/segment of users in India? Were we able to identify any specific type of user for whom this drop was noticed? Like females, Youths, etc?
Interviewer: No, we have seen this constant across all types of users.
Me: Did we see any action on the marketing side from our competitors? Or any backlash from our marketing efforts?
Interviewer: No significant viral content from our competitors. Just the usual stuff. Nor did we have any backlash recently over any of our products or services.
Me: So I would like to talk more about the competitor landscape. Amazon some of its biggest rivals in India currently include Flipkart, Meesho, and Myntra. So did we have any sales or offers run on these sites that we know of?
Interviewer: No, we have not observed any offer or sale happening on these sites. It’s business as usual.
Me: Do we know if these sites have also experienced any drop in their Add to Cart?
Interviewer: We can’t verify that.
Me: Have we recently pushed any changes, or has there been any design changes on the website?
Interviewer: Changes are pushed weekly so nothing new.
Me: In the last 7 days what changes have we pushed?
Interviewer: There were some button design changes.
Me: Was the button designed changes for all three platforms? That is Web, iOS, and Android?
Interviewer: Yes for all three.
Me: Apart from that any other changes been made to the website in the last 7 days?
Interviewer: Yes, minor changes on how we show data?
Me: Show data? Like description and product data or Pricing data?
Interviewer: Pricing data.
Me: So we have changed the way we show the price? What type of change?
Interviewer: We used to show the total cost now we show the cost of the product + delivery cost.
Me: I have seen the Price + delivery cost shown on the product page earlier as well. Then what is new? What has changed here?
Interviewer: We changed this in the catalog page as well. Earlier it used to show just the cost.
Me: OK so this is when, for example, you search for a table fan and you get a grid view of all the table fans. This is where you have made those changes right?
Interviewer: Yes.
Me: Was the Button UI change and pricing change implemented only in India?
Interviewer: Yes
Me: OK, so in the catalogue page you now show product price + delivery? But delivery is free for Prime users, right?
Interviewer: Yes and Yes.
Me: So now I want to know was the drop seen for all users ie: Prime + non-prime or just the Non-prime users?
Interviewer: Now that I look at this data,I see the majority of these are Non-prime users. But some amount of data for prime users as well. The drop is a 70–30 split between non-prime and Prime.
Me: Ok apart from these. Have the internal team found any bugs or issues?
Interviewer: The team routinely tests the site and logs bugs and issues, but nothing out of the blue.
Me: Any issues or bug reported to Add to Cart?
Interviewer: None that i know of.
Me: Has any user/QA reported any bug or issue related to add-to-cart experience?
Interviewer: No
Me: Since the drop was observed in India, was there any natural disaster in the region, which might have occurred in the last 7 -10 days?
Interviewer: Nothing that I know of.
Ok, I believe I have enough information to go ahead and create a hypothesis, about what caused the drop in Add to Cart on Amazon in the last 7 days.
What caused the Issue?
Based on the thorough analysis and investigation I was able to narrow down the issue to 2 major possible causes, because of which the drop might have occurred.
UI Factor: The change in the button design for add to cart might have confused the users about, whether this is where he/she needs to click to add the product to the cart.
UX factor: Changing the Pricing style might have had a psychological impact on the user and have motivated them to visit other sites and buy the product.
What will be my Next steps?
Now that I have a hypothesis I need to validate both of them.
My first step is to validate if Add to Cart is the only one impacted. For this, I will look into the funnel of the following events.
Session started > Product Search > Product View> Add to cart.
Based on this data I will be able to confirm that It’s just the Add to Cart button that has dropped.
Now the drop can be due to both issues. So we will A/B test with the following variations of design and experience:
- Old Add to Cart Button and Old pricing style
- Old Add to Cart Button and New pricing style
- New Add to Cart button and Old pricing style
- New Add to Cart button and new pricing style.
The A/B test will have these 4 variants and based on these 4 variants we will compare this with 2 weeks old data. If the drop continues in any of the 4 variants then we will be able to pinpoint what the issue is, the new Button UI or the new Pricing UX.
Button UI: If the issue lies with the Button UI, then will either roll back to the older style or will A/B test with a different button UI to identify the best-performing option.
Gather feedback based on user behaviour and actions and try to optimise the UI accordingly
Pricing UX: If new pricing experience is the issue, then firstly will revert to the old pricing style and will analyze more deeply what better approach to pricing could have been taken.
Will follow up with user interview to gather more feedback and based on it will take further decisions on what needs to be done.
Final Thoughts:
RCA type questions never has one right answer. There might be multiple reasons for an issue to happen.
So don’t get fixated on the solution, always try to get hints from the data the interviewer is providing.
Sometimes what you might think is a problem might just be the part of the issue. So always asks for more insights and let the interviewer direct the interview.
The most important of all is to stay calm. Be clear on your head and think from multiple angles. Because thats what PMs do.
I hope this mock interview style content was helpful, I am always open to feedback. You can share your thoughts in the comment section below.
