Designing a UX Health Dashboard
In any SaaS or consumer based startup which is tech-led, it is very essential for the product teams to always be on top of data. And by data, I mean loads and loads of it. Usage data, User behaviour data and data about anomalies in the product. With huge data, comes huge responsibilities. If capturing the right set of data points is itself a huge task, making sense of it and streamlining it to derive insights for future product decisions is another arduous task. But this ensures that the health of the product is always in check and makes data-backed decisions easier across the company.
In order to examine the health of any product, one must take into consideration all the things that are going wrong with the product. These can be anomalies, which disrupt the User Experience and Usability of the platform. Once the anomalies/issues are identified, UX enhancements are worked upon. Categorizing these issues into broader product areas and giving a score to each of them would help one determine the weighted score of each product area, which can later be distributed into health bands based on the weighted score. The aim of the Product Health dashboard would be to help us to become ‘Green’ (healthy) in all the broader product areas being classified after discussion.
Data Sources
Data Sources are the various avenues, from where the anomalies in the product are tracked:
1. Support Requests from users (tracked on Asana/Intercomm)
2. Mixpanel Analytics and FullStory Sessions Replay
3. All bugs which get raised from crowd testing or feature testing (raised on Github or Asana)
Issues and Issue Score
The anomalies recognized from all the above Data Sources are called Issues. Issues are associated with 2 properties — Issue Type and Issue Score. Each Issue can fall into one or more of the following Issues Types:
1. Information Request (originate from Freshdesk/Intercom)
2. User System Problem (originate from Freshdesk/Intercom)
3. Service Request (originate from Freshdesk/Intercom)
4. Backend Issue (categorized on Github)
5. Messaging (categorized on Github/Asana)
6. Workflow (categorized on Github/Asana)
7. Error (originate from Mixpanel)
8. Bug (categorized on Github/Asana) — Any anomaly in the product behavior which is different to the expected behavior according to the product design
9. Usability (categorized on Github/Asana) — Any usability issue with the platform, which create unease to the experience of the user
10. Enhancement (categorized on Github/Asana) — A small feature which is non-existent on the platform currently
In a general setup, all the issues collected through various Data Sources come to the product and engineering teams with priorities P0s, P1s and P2s. This system doesn’t work well during prioritization of the issues. Since issue owners are usually biased more towards their own issue getting resolved, they would pass on the Issue as a P0 and it is extremely difficult for the Product team to decide which P0 issues are to be prioritized first.
There is a need to quantify the importance of each of the issues. Issue Score is a parameter which will help the Product team in prioritization of issues. Each issue is given a score based on the Issue Type and the Product Area it falls under. This will also be a function of the Importance of the Product Area is, which is determined by the no. of users getting affected by it. This data comes from Platform Usage Statistics from Mixpanel.
Issue Score = f (Issue Type, Priority, Customer Size, Product Area)
Here, the Issue Score is a function of Issue Type, Priority, Customer Size and the Product Area. The reason why Customer Account Size is also taken into account, is because in SaaS startups, the account size also matters to keep the customer churn low. Below is a sample list of weightage given to various Issue Types, Priority and Customer Size:
In case of multiple customers raising the same issue, the scores of the issues are also be multiplied by the combined weightage of all customers who raised the issue request.
Data Collection
The cadence for data collection from each of the above Data Sources varies. While the Support Requests, Intercom Requests and Mixpanel Error logs get triggered everyday based on the user activity on the platform, the Asana requirement collection has a cadence of once in 2 weeks, where the Product Enhancement Stream owners talk to Customer Success, Support, Sales/IO and sometimes Customers directly, to understand the problems being faced by them.
Product Score
Once the data collection is done, each of these issues from various Data Sources is categorized into Issue Types, under the Product Areas and given an Issue Score, as described earlier. Then a Product Score is calculated for each of the Product Areas, which would be the weighted sum of all the issues under the product area. Since we have already given weightage to the Issues based on the Importance of Product Area, the Product score would be a simple sum of all the Issue Scores under that Product Area.
Product Score = Σ (Issue Score) = Σ ( f (Issue Type, Priority, Customer, Product Area))
UX Dashboard Metrics
Platform Usage Statistics from Mixpanel should be rolled out on a monthly basis, which should include the usage of various product features by users on the platform. These monthly statistics determine the importance of the key Product Areas defined above and help various teams including Product, Engineering, Customer Success and Support to be aligned to the Product enhancements roadmap. This UX Health report should include the following graphs showing the health of the product and the tracking of issues being solved to ensure the product is in an utmost usable state:
● Health Bands for Product Areas
Each product area would always lie in one of the Health Bands, which are the range of scores determining the health of that area. Health bands will be of 4 categories — Green, Yellow, Orange and Red and the range of scores can vary depending upon the Product Area.
● Issue Type Analytics
No. of Issues categorized by Issue Type under each Product Area.
UX/Product Enhancements Prioritization
Based on the product health and scores discussed above, the product team would work on several UX issues identified, in each of the sprints, to improve the overall health of the product. The Prioritization of UX Enhancements would depend upon the Health Bands, under which each of the Product Areas fall. The Issues of a red band Product Area would be given priority over a Yellow Band Product Area.
This methodology helps product teams stay on top of the issues in the product and ensure that it is always products the best user experience to its users.
Call to Action
If you are a Product guy, I hope this article has inspired you to further your passion. Please consider subscribing to our publication and sharing it in your circles.
At Parallax+, we design, develop, and ship world-class software applications that help startups and enterprises accomplish the impossible. Please do shoot a message in case you need any help!
Website: http://www.parallax.plus/
Personal Portfolio: http://msnkarthik.com/
Instagram: https://www.instagram.com/thecarthick/
