Designing with data, not for data

How I leverage analytics to guide the NerdWallet app experience

Data-driven design — sounds good in theory, right?

Well it is good, but I’ve found that indexing too heavily on quantitative data can lead to design decisions that miss the nuances of how real people use apps and websites. So, rather than letting data dictate my designs, my approach is to stay data-aware and leverage the numbers as one of the many tools — and perhaps my favorite tool—in my toolkit. In fact, the Analytics team is one of the most important cross-functional partners I work with while designing the personal finance management app at NerdWallet.

It took trial and error for me to evolve from a self-proclaimed data-driven designer to one who is data-informed. Let me walk you through some examples of how that plays out in my daily work.

Using data to inform requirements and uncover user pain points

Data can be a great way to find out what’s happening with your experience and help guide a design direction that aligns with users’ mental models. Recently, my team was tasked with redesigning our product’s onboarding flow. There were a lot of possible design directions we could’ve taken, each with its own benefits and drawbacks. Do we allow users to choose their own journey, or do we craft a linear flow to reduce cognitive load? Do we require users to activate certain features during onboarding, or do we let them activate later, once they’re in the product? In order to understand the right level of “pushiness” to meet business goals without alienating users, we started by conducting a series of A/B tests on different approaches. The results gave us a preview of how each would impact completion rate, activation rate, and retention.

When it came to quantitative data, we didn’t stop there. It was also important to uncover what wasn’t working with the existing experience, so I partnered with Analytics to examine drop-off by screen. We were able to identify that a significant portion of drop-off was occurring in four specific steps.

However, those numbers alone couldn’t guide thoughtful design choices. There are so many things that could lead to drop-off. So, instead of making assumptions based on the quantitative data, we worked with our User Experience Research team to find out why it was happening. By being data-informed and combining the “what” with “why,” the team was able to get a well-rounded understanding of drop-off and craft a flow that best solved for users’ concerns. I can’t talk about specific numbers in this post, but let’s just say this data-informed approach paid off.

Using data to gauge user interest

Working in a highly collaborative pod structure, I get to partner with my product manager to influence the roadmap and prioritize projects. Naturally, there are going to be times when we’re not aligned on whether a particular initiative is worth the time and/or resource investment. Data is one of the most efficient ways to take bias out of this process and let the features speak for themselves.

One example of this happened when my team was considering incorporating calculators into our logged-in experience. These tools — think: retirement calculators and home affordability calculators — are popular on our website, so we thought they could be a potential value-add to our app.

However, it’d be a pretty time-consuming lift to redesign and rebuild them on our native platform without a proven return on investment. For that reason, the team was ready to move this feature into the backlog and move on to other projects.

Instead of shifting gears, however, we arrived at a win-win: we created a lightweight solution for Design and Engineering by linking to the mobile web experience in the app, rather than rebuilding the features natively. That way, we could monitor the metrics to see how many users interacted with the feature before investing further.

However, it’s not quite that simple. Those metrics will help us directionally, but they won’t tell the full story because the MVP version that we shipped is less robust than the final feature would be. When analyzing results, we’ll be sure to keep in mind that user engagement is driven by a lot of things — including the concept (do users want to engage with calculators in our app?) and the execution (is this the right way to display calculators in our app?). If we rely on the data alone, we could run the risk of scrapping a potentially useful feature for the wrong reasons. But by being data-informed, we’ll be able to account for some nuance and develop new hypotheses to explore based on the numbers.

Using data to iterate and improve with confidence

In addition to helping inform what to build, data can also help guide how to build something. Before starting design explorations, I always make sure the team is collecting the appropriate metrics to help us design confidently.

This was particularly relevant during a recent project that involved iterating on our in-app marketplace where users shop for financial products (credit cards, bank accounts, loans, etc.). Our KPI was to increase revenue — but if we focused on that metric alone, we might have ended up with bright, oversized CTAs everywhere.

In order to combat this, I partnered with my product manager and analyst to examine clicks by page. This exercise revealed that users who drill down into specific categories (for example, travel rewards credit cards) are more likely to apply for products — and most users prefer to read our unbiased reviews before making a decision. This data-informed approach helped guide subsequent iterations of our product cards in the app — ensuring we align the business goal (revenue) with users’ intent. If we had only thought about the KPI, we might have created an experience that provided short term benefits to the business with long-term negative consequences to users and our brand.

Data: can’t design without it, but can’t design with it alone

I hope these examples were useful in seeing a few of the ways that analytics has helped me and my team take a data-informed — but not data-driven — approach. Quantitative analysis may not provide all the answers, but it’s a critical piece of the puzzle — shedding light on what’s happening with our product and giving us the confidence to iterate intentionally, conduct user research strategically, speak with stakeholders, track our progress, and create better designs that align with our users’ mental models. Cheers to more data-informed design.

Want to be part of a team that writes about stuff like this? Check out open roles in Design + User Experience at NerdWallet.