Day 23 —Build, Measure the Right Thing, and Learn
In an Outcome-driven project/environment, it’s key to measure intended user behavioral change, so that we can learn and improve the quality of delivery.
Are we measuring the right thing?
With the Lean Startup’s concept “Learn Fast and Learn Often”, we entered a world of faster pace software product delivery. For example, Amazon pushes their software update every 11.6 seconds in order to constantly optimize the performance. However, how does software product performance links to intended business outcome? Are we optimizing something that actually brings user values that they appreciate? Based on Jeff Gothelf, the author of the book Lean UX, what we should measure is the intended user behavioral change. That means we want to define what’s important to the user first, before we can build and measure. Typically, a software update that allows customers complete a task faster sounds like a great idea, because it’s a common software performance criteria. However, is that really the top priority for the users to use the system fast? Or they’d rather take their time to “Think Through Things”? If it’s the latter, speed is not the top user criteria, but abundant information and support tools could be far more important.
Define user values without bias
Tools like Adobe Photoshop or AutoDesk AutoCad, they are designed for power users’ repeating usage, and “easy-to-find” is not necessarily the most important criteria. As you can probably imagine, most of the power users tend to customize their tools and using lots of keyboard shortcuts in order to suit their specific needs. Therefore, they don’t need everything to be easy-to-find since they’re going to customize it anyways. That said, in order to get the right metrics to measure, we need to start from understanding what’s valuable to the users, and what’s their day-to-day workflow. This is how upfront user research comes to help us heading to the right direction, so that we don’t have to suffer from “fail fast, and never effectively learn why we fail”.
Continuous learning through Design Thinking
So, in an Agile project/ organization, what is the right amount of upfront research, in order not to fall in analysis paralysis, or fail too often and lose funding? The secret is to adopt Design Thinking to strike that balance. Prioritize the business/product assumption you want to verify, determine the most efficient way to verify those assumptions and measure the metrics for meaningful outcome. For example, business assumption can be verified through participatory design workshop, general preferences could be gathered through survey, user interactions, usability, design concepts can be validated through prototyping and user testing. In other words, most of the assumption can be verified before writing a line of code. When you get those assumption verified, and you want to measure the actual software performance, then that’s the right time to start writing code.
ABC. Always be clappin’.
