What you measure determines what you achieve

Originally written as a guest article for InVision’s blog by Chris Thelwell, Head of UX and Design at Envato.

Editor’s note: We’ve asked a handful of design leaders to respond to prompts each week. This week’s prompt was “What key metrics do you use to track your design’s effectiveness?”

The UX designers at Envato are embedded within agile development teams. This means design has adopted methods for measuring our performance from our agile practices. However, alongside our engineers and product management colleagues we are working to change this. We believe it is also really important to measure the effectiveness of our work.

How you measure a team sets how a team behaves and how well it performs. A team will naturally optimize themselves around achieving the targets they set.

I believe this is where most agile practices go wrong.

“A team will naturally optimize themselves around achieving the targets they set.”

Agile is all about the delivery of value into the hands of the customer. “What’s wrong with that?” I hear you ask. Well, the problem lies in the fact that people focus only on measuring the delivery part. They ignore the value part or the impact that value has on the customer.

Envato’s team in action.

For example, Scrum measures velocity, burn, and story points. Kanban measures cycle time and flow. All these measures celebrate the act of shipping software to the customer. So the teams will naturally optimize themselves around shipping. High-fives all around.

When you measure delivery you expect deliverables

With the whole team focused on the shipping, the pressure on the designers is huge. Designers are expected to deliver the deliverables of design — wireframes, mockups, prototypes, and spec documents. And they can’t let the development team slow down. This is where most agile design teams get stuck feeding the development beast.

“We’re stuck in delivery mode because all we ever measure is shipping.”

Jeff Gothelf, author of Lean UX, has been reminding us for the last 5 years to “get out of the deliverables business.” Yet we’re still doing it. We’re stuck in delivery mode because all we ever measure is shipping.

Changing the way we measure

At Envato we still value shipping — it’s still really important. We measure how fast we ship working software into the hands of our customers, as well as the design team’s ability to support our developers in doing so.

But, taking a data-led approach, we’re now measuring the impact that the value we ship has on our customers. The teams are starting to measure key business metric that matter. They focus on making a difference to these metrics in every two week sprint. A success factor or hypothesis is decided at the start of an initiative. This defines when we are done, not just when the feature ships.

For example:

  • We measure the number of people who successfully sign up against those who drop out. We’ve seen significant improvements since we designed new ways to sign up, such as during checkout or by reducing the number of steps.
  • We look for an increase in conversion as we iterate on the way we communicate the benefits of our items — such as support
  • We are gradually reducing our page load speed. We discovered a link between slow page load speed and high exit rates. We believe this results in fewer purchases on those pages.

Because our designers are embedded in the development teams, they share the same metric-driven goal. So everybody on the team — not just the designer — works on making a positive difference to the impact our designs have on the customer.

Join the conversation

Write your own response to the prompt “What key metrics do you use to track your design’s effectiveness?” on Medium, and submit it to our publication.

Originally published at blog.invisionapp.com on March 3, 2016.

Like what you read? Give InVision a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.