In Defense of Data-Driven Design
A breakdown of how the UX design team approaches data, measures success, and stays true to its instincts.
At Liberty, we meet about once a month to discuss a UX-related article — we call it the UX Reading Group. These meetings are generally unstructured so we can openly talk about our reactions and thoughts about an article and let the discussion progress from there.
Recently, we read “Data-Driven Design is Killing Our Instincts; valuing data over design instinct puts metrics over users” by Benek Lisefski and discussed our takeaways. We came to the conclusion that we actually disagree with his main point. We believe data-driven design is not killing our instincts.
It got us talking about how data plays an important role in our UX process and helps us improve user experiences, strengthen our brand identity, and promote a collaborative culture here at Liberty.
What does data-driven design mean to us at Liberty?
Data-driven design is the end result of a design processes informed by both qualitative and quantitative user data to create user-friendly and intuitive digital products.
There’s still room for a “designer’s instinct”
While we rely on data to inform our design process, our designers’ influence on product design taps into their expertise and instinct and goes beyond just “pushing pixels” and filtering out unnecessary test variations and design inconsistencies.
Expand your list of KPIs
Lisefski says “Data is good at measuring things that are easy to measure. Some goals are less tangible, but that doesn’t make them less important.” while illustrating that only optimizing for clicks and conversions can negatively impact some of your more qualitative goals, such as brand trustworthiness.
The solution here is simple, and Lisefski even alludes to it; optimize for a broader set of KPIs that accounts for a more holistic view of the customer journey.
Measurable data goes beyond clicks and conversions — you just need to identify what data points to measure. Consider user surveys such as net promoter scores, reputation scores, or PrEmo results. Be sure you have the tools you need to collect data for heat maps, time on page, call deflection, task completion time, average order value, drop off rate, bounce rate, and so on.
Map out the entire user journey with a content map and make sure you’re accounting for how changes you make at one point of the journey might affect KPIs at another stage. For example, if you’re testing the copy of a CTA button to optimize clicks, are you increasing cart abandonment? If you’re testing product scarcity messaging, are you decreasing the average order value? If you’re testing a cross-sell, is your NPS affected?
What’s another way to avoid sacrificing brand trust and “fugly” data-driven design? Put a sophisticated design system and clear brand guidelines in place and share them out with stakeholders. We have a design system that promotes consistency throughout user journeys and is accessible to anyone within the organization. It also provides designers a concrete artifact they can point to if anyone suggests they add flames shooting out of a button to increase clicks :)
Always be testing
Lisefski argues that “When our Agile processes are geared toward efficiency, it’s too costly to prototype and test innovative solutions. So we blindly churn out the same tried and true solutions over and over again.”
We have an unofficial, Agile-inspired motto here of “fail fast and learn.” Creating prototypes and running user tests doesn’t have to be costly — or terribly time consuming. We regularly organize hallway tests and invite teams to test ideas in front of participants we reward with snacks, beverages, or SWAG. We’ve also enabled all UX team members to run low-cost user tests with Validately software (there are tons of similar software available).
User tests don’t have to be incredibly sophisticated or high-cost to quickly validate design decisions and provide statistically significant data. Just be sure to follow best practices when creating test scenarios, user questions, and analyzing results.
Data, design, and delivering delightful UX
Always consider a complete view of your users’ journeys when optimizing or innovating product features to achieve specific KPIs.
Quantitative and qualitative data can and should inspire you to design better experiences for your users. Product designers, content strategists, and other UX professionals should help identify relevant KPIs and hypothesize about how they might be impacted by suggested changes. And of course, make sure reliable data reporting methods are in place so that KPIs can be monitored.
