Why Data Should Inform Product Design — Elijah Kingson
Elijah cares deeply about building products that help people and businesses succeed. He has spent the last few years building products across multiple industries; from health, human resources, and now fin-tech. He is currently designing products at Revolut.
What’s the relevance of data as it relates to product design?
Looking at data helps us understand users better and it’s super relevant for making different types of decisions about your product.
What should we build?
What parts of our app don’t work the way that users expect?
Why do users prefer our competition?
Why aren’t users using a particular feature?
Finally, as people in product, we can use data to get buy-in from stakeholders when shaping the vision of our products or deciding what features to prioritize.
How can we get data and how do we use it to aid decision-making?
Broadly speaking, you can get data by:
- Talking to users (surveys, interviews) and/or
- Observing their behaviour (contextual inquiry, A/B testing, analytics).
Depending on the industry you can also get data from reports published by organizations in the space.
As for the how-to-use, let’s take a practical example of an e-commerce store. You can use analytics to get data about conversion/drop-off i.e what % of people are completing purchases for a particular set of products on the store.
With user interviews, you can speak with specific users to get a better sense of WHY they’re not completing purchases. This should give you a better sense of what the issues are and if they’re UX-related (i.e can’t find the checkout button) you have a better idea of what needs fixing.
What are the values attached to using data before and after designing a product or feature?
Before designing or building, it helps to clearly document your assumptions about the problem and what the proposed solution could look like. Using data helps determine if those assumptions hold true. For example, a survey could show if a significant number of users are experiencing something that you think is a problem.
After or during the lifecycle of that product, engaging users or studying their behaviour can help in identifying ways that a particular feature can be improved and what to prioritize when developing a solution to a certain problem. When you’re speaking with stakeholders, you can use data to support product/design decisions as well.
It’s a good way of showing that you have a handle on the business and user needs as well. Doing x product change should result in an improvement in y business metric, and we can make this assumption because of z data points.
How do we find the right balance between data, empathy, and gut feeling?
This question reminds me of a popular quote: “Not everything that can be counted, counts. Not everything that counts can be counted.”When the outcomes are strongly tied to user behaviour, data should inform your decision.
Also, when deciding between multiple options for a particular interface or feature, then data (possibly A/B testing) should take the cake. When it comes to things that aren’t easily measurable or quantifiable, it helps to trust your gut feeling.
In all these scenarios, empathy for your user should never be compromised.
What are some of the tools of the trade?
- Surveys
- User Interviews
- User Testing
- Analytics
After the discussion session, we had questions from the members of the community, here are some of them.
They say research is important throughout the product dev cycle. At the beginning of the product, how can we balance choosing the right participant(s) for our research with being broad enough to not skew or taint our data?
I think the answer to this would depend largely on the problem space and what you’re trying to get out of the research. At the beginning, if you have a clear idea of who the right participant is, then I’d prioritize looking for deeper insights rather than covering a wider spectrum.
Just to get a full picture of customer surveys, can you give an example of how you have done surveys? Do you send mails requesting specific customers to fill the forms or just post the form publicly?
I do emails to customers based on specific criteria or NPS surveys within the product. I think you can get away with posting publicly if you define criteria before sharing the survey.
Is it the role of the product designer or the PM to do this research?
I’d say both, but depending on how the team is set up, the responsibility can sit with a dedicated research team made up of Data Scientists and User Researchers.
There’s a lot that goes into research; crafting questions, setting up analytics dashboards, and so on. At the end of the day, research is a collaborative effort but the owner of the role depends on the team or company in question.
What questions should one ask to get users interested and not come off as bothersome in airing their views, opinions, and likely recommendations?
What questions should one not ask as well?
Start by writing down your assumptions (or observations about what you’re doing research about). Then ask questions that help you determine if your assumptions make sense.
Also, ask questions that are strongly aligned with the metrics that the business cares about.
Questions not to ask would be “leading questions” that subtly suggest what a preferred answer to the question would be. Ensure your questions are clear and to the point.
To get users interested, you could sweet talk them: let them know you value their feedback and that it’s useful for making their experience better.
Furthermore, if you have the resources, you could reward them for their time. Rewards could take the form of cash, coupons, discounts, and so on.
