Analytical Approach to Solve the Seemingly Unsolvable
Tackling large-scale problems by breaking them into small chunks.
This May was a big month for Indonesians, and also for Tokopedia. For a country with the highest number of moslem populations in the world, Ramadan is the month where many Indonesians will get all out and spend a lot to prepare for the Eid celebration. To accommodate the needs of our many users and Indonesian people,Tokopedia held Semarak Ramadan Ekstra, our annual biggest online-shopping tradition which took place on 17 May 2019.
In order to support that, Nakama are doing their best in providing the best user journey. There are various aspects to be improved: from browsing the product inspiration, to finding and securely purchasing the items. My team and I were responsible for one of the first funnels of the user journey, which is product inspiration created by Tokopedia users themselves through Tokopedia By Me.
Tokopedia By Me allows users to curate any product that they like in Tokopedia, create content about the product, share it to their friends out there, and earn a commission from every transaction generated. We believe that this feature can help other users in getting a product recommendation and reviews easily. As this feature was relatively new, we wanted to make it perfect before Semarak Ramadan Ekstra.
Expect The Unexpected
“You won’t know whether your product actually works or not until you ship it to the real market” — somebody said.
It was March 2019. One week prior to the backend-freeze release (a term to indicate that the Product and Tech team can’t release any feature for a certain period of time to secure stabilization) and App Store/Play Store submission, we did a big “bug-bash” for the end-to-end Tokopedia By Me journey in the app. The bug-bash caught 80-ish bugs. Yep, once again, that was a week prior to the deadline. Bang! Slap on your (or my) face.
Breaking A Big Issue Into Small Chunks
One time, my lead said to me that a big issue is nothing if you break it into smaller pieces. So, what are you going to do with all those 80-ish bugs? Solve it all in a week including re-testing the whole funnel? That would only be possible if I had all engineers in Jakarta working on it. So here comes the prioritization.
But wait, those are bugs, meaning those 80-ish are a priority! Then what to do? Submit your resignation letter as a Product Manager. LOL, kidding. I didn’t.
My Test Engineer advised us to make a Bug Triage file. It’s a quick hack sheet to categorize the bugs so that we know what the priorities are in these 80-ish bugs (priorityception? LOL). We divided them into Severity 0 to Severity 3. The following is the definition for each:
Severity 0: Critical bug. It totally BLOCKS the flow. Users can no longer continue or even use the app, e.g. App crashed, can’t add to cart, failed to post content, etc.
Severity 1: Major bug. It doesn’t block the flow, but it’s very noticeable by users and causing a bad UX, e.g. post content with only 1 image but took 10 minutes to load, clicking “back” but not going to the previous page, etc.
Severity 2: Minor bug. Not a really noticeable bug, gives minor impact on UX, e.g. typo in copywriting, wrong color code used, etc.
Severity 3: Nice to have. This is more of a suggestion for a better UX e.g. to add a message in a search box so users can understand better, a micro-interaction, a small change of position, etc.
This idea is just to give us a clear visibility/framework on what things to work on first within a tight deadline. Since everything is a priority, what are the top priorities? That scheme above may help to answer that..
Prioritization Beyond Data
I do support the prioritization based on data, as this might put backlogs back in order based on what to solve first. Yet, take a deeper look into the smaller part right before releasing your feature (usually a funnel-type feature), when no data is there to support it yet. Doing sanity check based on severity level can give you a clear path to know what to deal with next.
So, to conclude my writing, here are some suggestions and learnings that I want to share:
- For the funnel-type feature, partial testing is good but not enough. Do an end-to-end testing as soon as possible, and do it iteratively. It’s actually a mitigation plan to reduce the upcoming “big bang” and unpredictable development cost.
- Getting many bugs isn’t the end of the world. Put everything together and consciously prioritize based on a clear severity matrix.
- Get a commitment from every stakeholder to support you in solving the issues in the same view of the timeline.
- Even in a tight deadline, do daily sync up with all stakeholders to know exactly the current status, so everyone is on the same level of urgency and can help each other ;)
Ciao~
