Are your designs making an impact?
Last week I spoke at User Research London 2018. It was a very a special day to be amongst 300+ user researchers from across the globe in one place.
To prepare for the talk, I reflected on how researchers and designers could show the impacts of their designs after their work is implemented. And when looking back to the various teams I’ve worked with, a common method and theme that kept coming up was the use of quantitative data as a mean of decision making and to capture impact outcomes.
Answering the impact question
To think about if your design is making an impact or not, we must first define the outcomes we want to measure. So this is when using quantitative data becomes really useful.
As user researchers, we have to start getting comfortable and familiar with different sources of data you can use (and beyond the typical use of analytics).
The problem with Quant vs Qual debate
To user researchers, we’re very familiar with the qualitative side — observed behaviours, research insights, interviews and conversations. And then there’s the quantitative side (which we may be less familiar with) — revenue, cost savings, call volume and analytics, for example.
The problem with this thinking about quantitative and qualitative data is that we tend to think about them as two separate entities and not interacting with each other. But, thinking about the data around our products and services separately is very limiting and doesn’t help user researchers to see the bigger picture.
What if we start thinking about quantitative data as a vehicle in telling us what is happening and the qualitative data as why is this happening? When we start thinking quantitative and qualitative as one you will get deeper insights into your products and services.
Ways to start using data
Create a ‘success’ culture
The organisation or team culture is very important in enabling us to think about data and incorporating them into your research.
Create a culture that is ok to start thinking about outcomes and metrics to use. And continuously review and question your metrics as your products or services move through the different stages of a lifecycle.
Ask these 5 questions before you start work:
Why are we doing this work?
What problems are we solving?
Who is this work for?
What outcomes are you expecting?
What are the key metrics?
(You can read more about the reasons behind these 5 questions on Ben Holliday’s blog).
Outcomes may change over time, so document them!
A nice way to keep track of design decisions, changes and outcomes is using hypothesis-driven design. It roughly follows this format:
We believe that [behaviour]
So if we [do this action]
We will see [this outcome]
The outcome part of the hypothesis is important as that may change over time — from observable behaviours to measurable outcomes. This change happens as you become more confident about your customers’ behaviour and your products. Think about what outcomes you might see as acceptable in alpha vs beta vs live in the development cycle and keep track of your hypotheses as you progress. Your hypotheses help you keep track of designs you’ve tried, decisions made and expected outcomes.
Track your design impacts
How do you start showing or communicating the impacts you’ve made? A good way to do this is to keep a simple ‘dashboard’ on the wall with your key metrics — it allows you to update your findings weekly with post-it notes. The ‘dashboard’ is a simple and effective way to show what it’s happening with your products or services. Plus you can include insights and quotes you’ve heard in the field!
Talk ‘business’ with people you need to communicate your research with
Consider the different stakeholders or teams you work with in your organisation who might have a stake in the work you do. What do they care or worry about? For instance, CEO or CTO will probably focus on whether the product or service is going to generate more revenue (or save costs), product owners might focus on if their product is ready for launch, and designers or researchers might focus more on the overall usability and interactions. Knowing what each key players focus on will help you identify the right detail of data to include in your research communications.
My challenge to all: share your stories
To get better at using data, we need to start talking about it more in our research practice. So, I would like to hear from you about how you’re using data in your organisation, how it is working and your war stories. And if you aren’t using data yet, I’d like to hear if this is something of interest to you and how you might start.
My slides to the talk can be viewed here. Measuring impacts is something I care about, so please get in touch if you’d like to have a chat!
Talk feedback
I founded Melon Experience Design to help clients gain better research insights from their users and make a social impact, get in touch if you would like to work with us.
