Five things Canvas Conference taught us about product design
(Okay, spoilers: there’s actually six.)
Last Thursday, at very early o’clock, the Deliveroo design team boarded a train from Euston. And promptly fell asleep on it.
We were heading up to Birmingham, and to Canvas Conference, where we’d hear ten inside stories from the people building Netflix, Spotify, Slack, and other products and services the world loves to use.
It was a great event, full of really useful ideas to think about. Here are five things we came away with.
1. Our intuitions are usually wrong…
We’ve all done it. We’ve all, at some point, come up with a design based on some wonky assumptions, or that we’re slightly too big a fan of, downsides be damned.
Just for a second, think back to the 17th century, and to Galileo: famous for challenging beliefs and assumptions through scientific testing. Here’s one of his ideas you might have seen being proven.
As it happens, testing and proving ideas through experiments is a process that works consistently well, 400 years later, for Navin and the Netflix product design team. By defining both a hypothesis (based on clear metrics, and used as a compass throughout the design process) and an experiment (perhaps user sessions, or good old A/B testing), they use the results to conclusively prove or disprove new designs.
“If I had asked people what they wanted, they would have said faster horses.” ― Henry Ford
As you might expect, it turns out to be a far better way to design than just listening to what users say they want. Netflix’s own surveys say their users’ biggest request is to see the entire library before they start paying for the service. But when the team tested that using the method above, it was found to make no difference in signup numbers.
But even once you’ve established such a process for testing ideas, how do you make sure you’re considering all the possible ideas in the first place? By going wild with your designs. When rethinking the Netflix homepage, the team actively tried to design the most disparate things possible — different layouts, different hierarchies, different styles — to uncover as many ideas as possible. Because the chances are always good that the right idea isn’t the first one you thought of.
2. Although sometimes, our intuitions are right.
Similarly, we heard from Hilary about how the Skyscanner team run extensive A/B tests to improve the site. In fact, those tests use a similar format to Netflix: based on <insight>, we predict that <change> will result in <impact>.
A quick question, then, if you’re a Skyscanner user: did you know you can search for more than just flights?
Users tended not to notice that, so the team hypothesised that ‘if people are comparing flight prices for their trip, they’d also like to compare hotels more easily’. And they were right: in an experiment that, by default, showed new visitors a hotel comparison, hotel bookings increased by 20%, with a negligible impact on those for flights.
But the team had also suspected something else from the start: does a Skyscanner user really expected to be greeted with something other than flights? The data looked good, but was this really providing the best user experience…?
Ultimately, as you can see above, customers asked the same question. And so, despite their successful hypothesis and increased conversion, the team reverted the changes, declared the test invalid, and headed back to the drawing board. Teams working on any mature or successful product will ignore what the data says at their peril, particularly for a metric as important as conversion. But perhaps sometimes — not often, but sometimes — doing the right thing for the product should mean doing exactly that.
3. People are complex. And technology doesn’t always help simplify.
Ever stop and wonder whether, in racing to create technology, we cause greater problems than before? Look at how cars were invented to make travel easier… when they’re not causing congestion and traffic jams. Look at how the Enigma machine helped end war… but also how modern computing can help prolong it. How can we ethically decide who a self-driving car should prioritise saving in an accident? Isn’t life less stressful if we don’t have a smartwatch going ‘ding!’ every time an email shows up?
Tessa’s a Product Manager at FutureLearn, and gave us a lot to think about with that. Knowing that technology isn’t always the answer, she actively encourages physical, face-to-face collaboration as much as possible. No laptops in meetings. If you’re attending a one-to-one, leave your phone behind. And just talk.
This very open, honest form of communication may come more naturally to some people within a team than others, and so Tessa’s borrowed the concept of her childhood worry dolls for standups. Team members each write down an issue and have it randomly given to someone else. It can be easy for people in a team to internalise their own issues, but having everyone talk about each others’ broke down barriers and allowed stuff to get done.
On a similar note, understanding users can be tricky when you’re already so familiar with the product you work on. In her talk about working at Slack, Merci told us about the team’s approach to new features: embrace the mindset of a new user who’s totally unfamiliar with your product, and ‘pretend you’re a human being’.
In fact, Slack go one further: every person there is also part of the support team. Because users can be surprising. And if you’re designing for human beings, what’s the best way to understand their needs? Talk to them.
4. There’s a better way to make decisions about our products.
That product you’re designing. That code you’re typing. They’re the result of hundreds of decisions you’ve made, according to Jane Austin, MOO’s Director of Design & UX.
In fact, according to the Vroom-Jago Decision Model, there are two main ways to make them. You should try to be autocratic when you know more about the problem, you’re confident about acting alone, your team will accept your decision, and/or there’s little time available.
In all other cases, you should be consultative and collaborative. This might be because the problem isn’t clearly defined, you need information from others, you need buy-in from your team, or you have enough time to manage a group decision.
Tricky thing with decision-making, though: we’re also slaves to impulses and cognitive tricks. (We’re ‘meat robots’, to borrow Jane’s phrase.) Some of those might be:
- Anchoring. Humans tend to rely too heavily on the first piece of information we’re offered when making decisions — say, during price negotiations.
- Framing. We react to questions and choices in different ways, depending on how they’re presented — say, ‘you have a 90% chance of living’ versus ‘a 10% chance of dying’.
- Confirmation bias. We’re good at searching for and favouring information in a way that confirms our existing beliefs and hypotheses, while giving less consideration to other possibilities. So if you agree with viewpoint X, you’re more likely to agree with an article saying ‘viewpoint Y is bad’.
- Sunk costs. Companies can have a tendency to continue investing in a project to make up for costs they’ve already sunk into it. If something’s not going well, it can be tough to pull the plug and cut your losses.
So how can we, and our companies, be making better product decisions? We know that data can help us remove opinions and biases. We know reviewing our previous decisions can give us insight into the right path this time round. For Jane, it all comes back to diversity.
That’s diversity across disciplines (see Airbnb’s model, above). Diversity of skills. Diversity of people. It’s why Jane advocates cross-functional teams who make decisions at the right level with only lightweight governance: because they let everyone be heard.
5. Everything’s a mess. (And that’s ok.) 👍
“People […] look at the process and say, ‘you know, if you would just get the story right — just write the script and get it right the first time, before you make the film — it will be much easier and cheaper to make.” And they’re absolutely right.” — Ed Catmull, Pixar president
We really enjoyed Brian’s talk. It was a comforting reminder that, behind the scenes of any product or company, chances are good that things are a chaotic mess.
“But a chaotic mess is uncomfortable!” we cry. We like consistency. We like processes, thorough documentation and dependable workflows. They create standards. They stop everyone changing their minds all the time. They help us avoid… mess.
Well, yes. But perhaps messiness — to an extent — isn’t necessarily a bad thing. The industry moves fast. The world moves fast. And as Brian put it, “if things aren’t feeling messy, you’re probably on the road to irrelevance”. In fact, he suggested that one way of overcoming this discomfort and controlling the mess is to to keep sight of a stable constant, like the company’s prevailing vision and principles.
Individual project and tasks aren’t all approached in the same way. Think of anything you’re currently or soon to be working on, and see how your approach to it might map out on these spectrums:
Every project or task will have its own combination of those values — but Brian acknowledges there’s often a gravitational pull to the left. And perhaps that’s because he simplified it all to this:
According to Jeff Bezos, people who are right a lot often change their minds. Smart people revise their decisions because they gathered new information that wasn’t there before. Even though, yes, that can cause a mess.
We’ve all seen the below methodology before. And there are times when we can successfully make a project or task happen according to it. That’s great!
And the other times? Stuff changes. Shit happens. Everything’s finalised too late, without enough detail — and, ta-da, all the requirements have changed in the meantime.
We can help mitigate that. Making better, more informed product decisions (as Jane mentioned previously) can help. Making sure we’re considering a ton of possible ideas and outcomes early on (like the Netflix team) can help. But sometimes, Brian says, we just need to accept what we’re doing is inherently messy. It’s sometimes last-minute. It’s subject to change. And as long as everything doesn’t get too messy… that’s ok.
Because we need to keep moving. And sometimes, because you can’t cross a chasm in a series of small jumps.
Bonus 6. Deliveroo have a great research team.
Okay, that’s something we knew before Canvas. But a big shout-out to Charlotte for her talk on how she’s leading the research team here at Deliveroo. We’ve grown incredibly quickly over the last year or two, and the product team isn’t just designing and building for one set of users — we need to keep our customers, riders, restaurants and internal teams happy. Our research team are central to making sure we’re achieving that.
Onwards.
So, that was Canvas! Ten really insightful talks, and a ton of great things to think about. Also beer pong, which I won. 🍻
Also a quick note to say that we’re hiring at Deliveroo Design. Product designers, brand designers, user researchers, copywriters, just about everyone… coffee?
Finally, thanks to Charlotte and all the other speakers, and to the Canvas team for putting on such a great event — we’ll be keeping an eye out for the next one. Go follow Canvas on Twitter to see full videos of the talks (available soon), or if you’re interested in getting 2017 tickets once they’re out — we’d highly recommend going.
Until next year, then!
