The hard truth about designing with data
As a part of SF Design Week, Segment hosted a group of designers and researchers from top Bay Area companies for coffee and discussion. The topic? All things related to data’s impact on design. Why did we care? Many of us build products that help companies get better data (like Amplitude Analytics, Datadog HQ, Mode, etc) and many of us use data to inform our design decisions (think customer insights, product analytics, usability metrics).
Our coffees got cold as we got sucked into conversation — we asked each other tough questions ranging from “how do you measure the success of your design” to “what are the ethics using data to power product experiences”. Pretty soon, one tension became quite clear: we all see good data as an increasingly critical tool in our design process, but know it can often be a double-edged sword — as much as data can inform and delight, it can also obscure and leak.
So how do we navigate this tension? I certainly don’t have all the answers. Instead I’ll share the top questions every team should ask themselves as they design with and for data in their day to day.
1. How much is this data actually telling us? Having the data does not mean you have the answers. A common misconception is that because something is data-backed, it’s right. But the truth is the data we often get is merely information, not intelligence. When you hear a data point, be sure to interrogate it — capture what questions still remain, and try to triangulate it with other context or knowledge that might give you true signal.
2. What are you not showing me? Data often feels objective (it’s ~data~ after all!) but I think we’ve all seen instances where data is bent to support a certain point of view. Knowing this subjectivity exists, it’s important to always ask “what are you not showing me” when you are handed insights or analysis. It’s not (entirely) about fact-checking your teammates — it’s great way to better understand the process they went through to get to their conclusion and reveal any potential blindspots or jumps in logic.
3. What is the best way to consume this data? This question is really a friendlier way to get at how data literate are we, really? A great quote from the session: “machines read tables, humans do not.” There are so many different ways to cut and visualize data, and it’s absolutely okay that most of us aren’t experts. But without shared understanding of insights from the data it’s really hard to get to shared conviction in next steps. Communicating the data is a huge responsibility that requires design principles of objectivity as well as accessibility.
4. Can we be more transparent about the data we are using? We often use data to do good by our customers — to power more relevant personalized emails, to remove friction in on-boarding, or to give timely notifications and alerts to name a few. But how can we be more transparent about the data that is powering that? Transparency is of course the right approach because it gives your customers the visibility they deserve. But don’t discount that it can also be a moment where you demonstrate the power of your product — if you’re designing a smart experience, highlight the work you are taking off your customer’s hands!
5. What is the minimum viable data we need to solve this problem? As data privacy becomes increasingly top of mind for companies, it begs a question that almost goes against all instincts: how can we as designers and businesses actually use less data. What is the minimum viable data you need from your customer to create the ideal personalized experience, or the minimum data you need to successfully understand a users behavior? How can we be respectful and efficient, making the data we collect go even further.
Have your own big question or hot takes on designing with data? I’d love to hear them (comment below!). In the meantime, stay tuned for another event from Segment Design at SFDW 2020.
Interested in roles at Segment? Apply here.
