Hypothesis driven UX design
The value of design changes when you enable whole teams to learn instead of just looking at pretty mockups
Shortly before I joined mobile.de (an eBay owned automotive platform in Germany with millions of visits per month) a smart product manager had an idea: The iPhone app should be redesigned not only to match current visual design guidelines and trends but also to get rid of usability issues that were caused by the current design approach.
Shortly after my start at the company I was handed the project. We tried to tackle the project with an iterative approach. Knowing that people are mostly reluctant to change we changed only parts of the app, released and when everthing went well we took on the next elements of the app. Everything went pretty well: numbers stayed stable, no “redesign drop” and feedback from users was overall positive.
Curious case of a result view
That changed when we released a version in which we had changed the search result page to the new design.
People complained a lot that there were too few items in the list, that they cannot find the important criteria and the interface was just not as clear as the old version.
Even though we were so “iterative and agile” users were still not liking it. On the one hand we were benefiting from the iterative approach since we knew that the result list was the root cause of the bad performance of our app. On the other hand had we made changes to the visual style and blended them together that we could not exactly define where we need to make changes in order to improve the experience. In retrospective we were not clear about the design hypotheses we had incorporated in our designs.
Redefining our design process
Inspecting status quo
To come to a better design process we took a step back and analyzed our current approach. Mostly we looked at the user/business problem to solve (which is good) and defined the problem space. Then we designed a solution (point where it gets worse) and shipped it to our customers to get feedback and to drive the business (pretty fine but what happens if it does not work out?)
Learning from Lean UX
Interestingly the problem space was mostly well defined. In our case we had usability issues through bad readability and almost zero whitespace throughout the app. The app felt compressed and users often complained that they cannot find what they are looking for. When we tried to solve those issues we certainly had some design hypotheses. What we did not do was to make those design assumptions or hypothesis explicit. Neither did we tell ourselves how our different design variants would solve for the specific problems.
So we looked at the work of Will Evans, Jeff Gothelf or Steve Blank. What they mostly incorporate in their thinking is the aspect of learning along the journey of product development, customer development or product discovery.
If you want to learn what works with your audience you must be able to build solution hypotheses for your designs. If, and only if, you can determine which solution was supposed to solve which problem then you have learned something a long the way. Then you can think about the next steps and become better as a designer, as a product and/or a business. We fell into the trap that we built something (Build) where we thought (Think) we improved the experience of our users. We even measured (Measure) the performance but when we looked at the numbers and comments (Check) we were not able to learn because the designs incorporated to many single problems into one.
Solution hypotheses to the rescue
After we had found out about our mistake we changed our process. We established a solution hypothesis format to have a unified approach and a common language for all team members.
If [action] then [outcome] because [customer need/problem]
Furthermore we added a step to our process. In this step, that we called “Solution hypotheses generation”, we forced ourselves to write down our hypotheses and how they would solve a customer problem. This has to happen even before one opens Sketch, Photoshop or draws on a whiteboard. This lead to a significant change. Design is now seen as a tool to validate or invalidate the solution hypotheses. The procedure changes the value of design from being one step in the process to something that enables your team to learn more about the customer. This ultimatively gives UX design a much higher business relevance.
Revisiting the curious case
For our redesign project we went back to the drawing board and defined solution hypotheses. After some prioritization we came up with three main hypotheses that we wanted to test:
More listings
If we provide more listings on the screen then we can provide better comparability and offer more diversity on our platform because users like to compare a lot of listings on the result page
Better structure
If we provide more structure to our listings then we achieve a better scanability because the user is able to scan the relevant information quicker
Better prioritization
If we prioritize information according to user needs then we achieve better guidance because the user can see all relevant information at a glance
We forced ourselves to let the designs be leaded by our hypotheses. This led to quite different designs.
Now we were able to test which design was most accepted by our customers and we were able to generate a lot more learnings.
5 takeaways for hypotheses driven UX design
- Take your assumptions and state them as explicit hypotheses
- Let your team buy into stated hypotheses
- Force yourself to design according to your hypotheses
- Force yourself to test against your hypotheses
- Create a culture of learning by showing stakeholders the benefits of validated customer hypotheses
