Data can make good design great, but it can’t make bad design good

I heard this phrase used by Margaret Gould Stewart in a Ted talk she gave last year. It became something I couldn’t stop thinking about, so I decided to explore the topic a little.

When designing at scale no detail should be overlooked. Small things can require huge consideration and it can be the culmination of these small, crafted details that can make a world of difference. We know that being able to measure the success or failure of a design and then iterating using knowledge gleaned from data sources can be a great formula for success, but what if the design is somehow fundamentally flawed? What if the design simply doesn’t make the grade and the designer has failed somehow to identify its shortcomings? What if something is shipped that isn’t actually solving the problem it is intended to? Can the data gathered from such situations actually make a bad design good?

“The first thing you need to know when designing at scale is, the little things really matter” — Margaret Gould Stewart

One of the best things I find about working with data is accepting failure as part of the process. There are many reasons that designs can produce underwhelming results, but having data to analyse to help me try and figure out why it has failed is alluring to say the least. This allows you to quickly learn, refine the design and try again in fast iterations until you ship something that is not only a significantly measurable improvement on what preceded it, but it is something we know the customers actually love and which improves their experience of your product.

“Don’t mistake speed for precocity: the world doesn’t need wrong answers in record time” — Cennydd Bowles

Good enough is never good enough

Sometimes though, us designers make something that simply falls short of its intended target. We either overlook a detail or use case. We get too focused on our velocity and release something too early, or we release something that just lacks craftsmanship. When this happens, it usually means one thing. That which we have produced, or the solution we have implemented, ends up not being as brilliant as we thought it was going to be. In this situation, I have occasionally heard people say “Let’s just test it like it is and get some data on it, its all about getting as much data as quickly as possible”. Except it isn’t. I feel this is misguided, and this impatient approach can be hugely detrimental and in some respects a waste of time. The issue with gathering data on a design that isn’t fully crafted or thought through is that the data you gather is tainted or biased by poor execution or a rushed concept and therefore shouldn’t be relied on to base informed decisions and further iterations. I think every designer should always try to ask themselves:
- Am I really happy with this?
- Am I comfortable to show this to millions of people?
- Is this my best attempt at the problem I’m attempting to solve?
- Can I live with it the way it is?
- Do some other people think so too?

I also think the answer should always be ‘yes’ to all of the above. Otherwise you shouldn’t ship the product or feature.

There are, of course, cases in which gathering data early can be useful. As I mentioned earlier, learning and iterating is a great formula, but if the design or product is lacking in overall craftsmanship, isn’t well executed, or is a poorly thought out step towards your intended goal, all the data in the world won’t fix the fact that it is a version that simply lacks the essence of the original, amazing thing you set out to do, which can be severely detrimental to a customers experience and your brand perception and can leave you in a place where you don’t know what to do next or which direction to take.

“Perfection is not attainable, but if we chase perfection we can catch excellence” — Vince Lombardi

When it just isn’t right

Data by itself doesn’t make great products. It takes skill and craft to take that data and use it to make something better. It also won’t show you what is wrong with your design or tell you how to fix it, it will only show you that you got it either right, wrong or that you had no significant impact. This can be challenging, especially for designers that have failed to realise that what they have produced isn’t up to scratch in the first place. They may iterate multiple times on an idea that was simply never going to work out. I know, I’ve been down that road, and it’s up to you as a designer to admit defeat, be humble and hold your hands up. Say you got it wrong. You have analysed the data and have multiple variations, but sometimes, nothing helps. It was destined to fail from the beginning and everything you have learned from the data is worthless because what you have produced will never solve the problem you want it to, regardless of how many times you iterate. You designed something that just isn’t very good, or that the world doesn’t need.

“Care and accuracy in the design process show respect to the user” — Dieter Rams

Data is our customers speaking to us and its up to us to know how to listen and respond appropriately to what they are saying. You owe it to them, as a designer, to give them your absolute best at all times, to give every design the care and attention it deserves, to agonise over every detail, to strive for the best solution you can possibly produce, every single time. This is what makes great design. Caring. Caring about what you are making, and caring about who you are making it for. Craftsmanship is important, iterating is important, having a vision is important and its lazy to not strive for that utopian ideal of perfect design every time, even if you’ll never reach it.

“If you don’t burn out at the end of each day, you’re a bum.” ― George Lois