Get Started: Data-driven Design
Over the past two weeks we’ve talked about the benefits of data-driven design. We’ve written about how it can support “designer’s intuition” when making UX decisions and we’ve discussed how to get your stakeholders on board with the idea using data to help make those decisions. But we haven’t talked about how to get started in a data-driven design process.
How do you introduce data-driven design into your framework? What are the essential components for you and your colleagues to be able to effectively execute on a data-based approach?
Good News, Other News
The good news is that you’re probably already using a somewhat data-driven approach. If you’re performing user testing (and good gravy we hope you are), then you’re collecting data on how your design is being used, understood, and interacted with. Taking that information and using it to help inform or guide your decisions is a data-driven design approach, albeit a bit of a “light” version of it.
That will make pursuing a more data-centric mindset easier to achieve — you’ll be able to build on an already existing framework for incorporation of it with colleagues and stakeholders, since your user data-driven approach will serve as a foundation to bring more data-centric ideas to your process. But amidst all the good news, there’s some other news…
The basics of user testing, while hugely important, don’t give us everything we need to know when it comes to things like incorporating big data, working with analytics, choosing which metrics matter, and breaking down data silos, all of which are important aspects of data-driven design.
“Highly data-driven organizations are three times more likely than others to report significant improvement in decision-making. Yet, 62% of executives still rely more on experience than data to make their decisions.“ Karen Budell, Content Marketing Manager, Google Analytics
When establishing a data-driven design process, there are things we can pattern our process after that are familiar, but there is a lot more that will be new. We’ll need to grapple with information in new ways and develop new intuition about data in the way we did with design.
Get the Data
Every organization will be different, but getting the data may be the single hardest part of your journey on the path to data-driven design. In larger organizations, especially, departments often have very distinct functions and cross-over can be nearly impossible to come by.
To truly have a successful data-driven process, you’ll need to have access to the data. This means you may need to put up a bit of a fight — in a business appropriate, respectful way, of course — to break down some of those barriers to information access.
“I get these excuses, and they are no longer acceptable… ‘Well, we don’t have control over the analytics. That’s a different group, in a different building, under a different vice president’,” says UX & Design expert Jared Spool. “That’s not an acceptable thing anymore.”
As designers, we have a bit of an advantage over other groups seeking out the information, because we’re typically not departmental adversaries. While we may know that when one group gets better, we all get better, we know not everyone recognizes this, so being in a non-competing group is helpful.
Secondly, we can actually make a solid case for supporting the group that holds the data. If we can use the data to improve the product, for example, marketers will have a new feature they can use to help market the product.
Finally, we’re not looking for all of the data. We only want the data we can make use of — specifically, information about user behaviour. “This idea that we own the experience and they own the quantitative data is the wrong answer,” says Spool. “We’ve got to own the quantitative data, at least that data around behavior.” Other stats collected can, for now at least, be left to the side as we build the relationships that allow us to get access to data.
And while ideally you won’t have to lean on stakeholders to gain other departments’ cooperation, if you’re encountering roadblocks you may find it necessary to ask for their assistance too. Just be sure that you’re being respectful of everyone involved — it’ll help your collaborative efforts in the long run.
Tip: Make supporting your needs something that pays benefits for the current data holders and they’ll be more inclined to help you out.
Get Good At Data
Designers don’t tend to have statistics and information analytics backgrounds. But incorporating data into our process means we’re going to have to brave the idea of numbers and learn to understand how they operate.
“As designers, we need to accept and embrace the world of metrics and use their amazing powers to change the way we’re doing things.” Jared Spool
First and foremost, ensure you have collected information over an extended period of time — or at least at multiple intervals. You can get a stat that says you had 1,000 users on a day, but unless you know how many users you usually get, you won’t have any idea whether you improved your numbers, diminished your numbers or had a completely normal day. Try and compare across the industry you’re working in as well, to get a sense of how you compare.
Once you have a decent span of data you can compare across, you’ll need to learn how to interpret and understand the information. Some of it is straightforward enough that a little intuition and a brief explanation from an experienced colleague will give you all you need to be able to start to work with it. A lot of it, however, takes practice and continued effort.
“Data science is now an essential skill for every UX team,” says Spool. “If you don’t have people who understand how to do data science, you cannot create great designs.”
Whether it’s an online analytics course, a hands-on course offered by a digital learning hub, or a chunk of time set aside regularly with your in-office data expert, the most important thing is that you take the time to dig in, learn the various ways data can be used and manipulated (watch out for red herrings!) so that you can begin to interpret the information and understand what it’s telling you in terms of design direction — and where it might be leading you astray.
And when it comes to big data, understanding will need to bridge across teams. IBM, one of the leaders in data insight, states it clearly:
“Insights from big data can enable all employees to make better decisions — deepening customer engagement, optimizing operations, preventing threats and fraud, and capitalizing on new sources of revenue. But escalating demand for insights requires a fundamentally new approach to architecture, tools and practices.”
Be ready to rethink and re-evaluate your processes at every stage. If you’re not faced with challenges and upheaval, you’re probably not doing it right.
Tip: Practice. Get familiar with the resources and ask for help from people more familiar with data than you while you’re learning.
Use Data To Find What Matters
Figuring out what the key areas of improvement are for your digital product or experience can be a hard thing to do. And a glut of data can feel like it’s just adding to the overwhelm, but it can also be the key to figuring out how to attack it.
When you examine the data, you’ll often see areas where specific patterns emerge: people abandoning the site at one page in particular, extremely high or low bounce rates or time on page indicators, a usage drop off after roughly 3 weeks on your app. Whatever it is you’re seeing, that’s a good place to dig deeper into the data you’ve been gathering.
And while figuring out surface level patterns isn’t always indicative of an issue — maybe people abandon the site at one page because they’ve completed their task — but we won’t know until we look into the data at places where the surface stats look off.
That’s where you gut instinct comes into play, balancing the quantitative with the qualitative. This is a prime spot for you to be able to use the two to support and reinforce one another. If users say “X” is a problem, but the stats don’t show it, maybe you need to ask different questions that will surface the real issue, instead of the symptom of the real issue.
Conversely, if you’re seeing a spike in cart abandonment ⅔ of the way through a checkout process and your user testing shows people getting confused at step 4, you know that’s the place you can focus your design energy and, presumably, start to see real improvement on your ROI.
Pamela Pavliscak notes that, “[A]t its core, data about Web sites and applications is data about the people who use our products.” What you’re really looking to do in any data gathering and interpretation work, is understand as best you possibly can, what your users need and want.
Tip: Look for anomalies to begin with — anything that seems “off” in the data set that aligns with your gut instincts on what might need improvement is a good place to dig deeper.
If getting the data isn’t the hardest part of implementing a data-driven design process, having patience with it probably will be. It is perhaps the most important of all the steps to getting started with data-driven design, but being patient is absolutely crucial to the success of your work.
When we get any information we have a tendency to want to react immediately. “The data shows this didn’t work! People are abandoning the app! They’re jumping ship like crazy!” Our brains leap to conclusions, trying to categorize and understand everything right away.
It’s important to monitor the effects of your changes. It’s important to critically evaluate the responses you’re seeing to your designs. But keep in mind that no matter what you’ve done, there will be a period of adjustment.
“With any new design you should consider the time and indeed effort it takes for users to adjust. People don’t like change. Change is scary. Change is unsettling.” — Neil Turner
Whether you’re seeing a spike in user satisfaction or a dip, the early feedback isn’t likely to be telling you the whole story. Give users a chance to adjust and settle into a new pattern before jumping to any conclusions.
Think about Facebook: people hated the “Like” button. They hated Newsfeed. They have hated just about everything they’ve ever implemented… at first. But users eventually settled into the new changes and, it turns out, found the site more usable, more engaging, and harder to turn away from than ever before. And Facebook had the courage to wait out the reactions because they had data supporting the choice before they went into it.
Take your time with implementing changes based on data, and be patient in waiting for the success after you’ve made the changes.
Tip: Don’t just look for immediate changes in metrics after a design change — let users get used to a new feature before making judgments about success or failure.
There you have it. Four (relatively) easy steps to get you going on your path to a more data-driven design process. The best news is that once you get a good handle on it, you’ll wonder how you ever designed without data.