What is valuable working software and how do you measure it? — The Agile Manifesto & user experience


Some smart people wrote a manifesto to help develop better software.

Back in 2001, a bunch of smart people got together on a ski holiday and came up with the Agile Manifesto and twelve complementary principles. These simple documents revolutionised the software industry.

The manifesto describes how to organise a team and prioritises early and continuous delivery of valuable software. It does not describe what you should build, why you are building it or what “working software” is.

The Agile Manifesto

Sourced from agilemanifesto.org

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.

Using “Agile” in vain

Many software companies are singing the virtues of a strong user-centred approach to agile software development. But yet, it’s not uncommon to hear of software companies that shortcut design processes. This is often presented as a way to increase the speed of delivery in the name of agile.

What is working software?

Agile promotes the idea of delivering working software early and continuously. The question is, what counts as working software? Is it functional, tested, documented, reliable, usable, convenient, aesthetic? Using something like the UX pyramid by Stephen P. Anderson can help with the conversation around this decision.

UX pyramid by Stephen P. Anderson

There are no shortcuts to success & competitiveness

There are three main consequences if you don’t strive to achieve the top level of the UX pyramid:

1. If a product fails, you won’t know if it’s because it didn’t fit the market or if the experience was poor.

To adapt an analogy from Brandon Schauer, CEO at Adaptive Path, imagine you want to test if a cake shop would be a good business idea. Now imagine that to test quickly and cheaply, you cut corners on the quality of the cake.

If sales are poor, how will you know if it’s because the cake was unappealing, or if customers aren’t interested in buying cakes? You always want to give everything the greatest possibility of success.

2. Design debt, like technical debt, needs to be paid down in the foreseeable future.

If it’s not, it will swallow productivity and irreversibly damage the user experience. The best time to fix something is immediately, before you lose the context. The greater the amount of debt you accumulate, the slower and harder it becomes to update and change a system.

The more things are delivered with a discounted experience, the greater the number of users affected and the quicker users will leave. Poor user experiences increase negative bias where “flaws in designs raises the bar for what they consider acceptable”. People are unlikely to keep something in their life that doesn’t spark joy.

3. Building up a large amount of design debt leaves you in a vulnerable position for a competitor to steal the market.

A new competitor that doesn’t have the overhead of design debt can build a product that appears more coherent and fresh to users. They won’t need all the features of an established product to attract users. If they have the right features and a meaningful experience, users will switch. It’s unlikely a team will be able to pay a large design debt quickly enough to stem the loss of users.

How can we deliver fast if we can’t compromise on the quality of the user experience?

Agile is about early and continuous delivery of valuable software. The best way to do this is by breaking down the deliverables to the smallest possible valuable thing. This is where the solution can stand independently of other deliverables. It’s also known as batching.

It’s a fallacy to believe through-put can be gained by compromising on user experience. Taking the steps to ensure things have the greatest possibility of success will ultimately save time and resources. It will reduce the engineering time required to rework poorly thought-out solutions. This leaves more engineering time to deliver a greater number of batches more quickly.

Eric Schmidt and Jonathan Rosenberg describe this in part in the book ‘How Google Works’.

…ship and iterate doesn’t mean you have license to push crummy products out the door and then hope to make them better.
In fact, Jonathan often warned his team not to launch crummy products and depend on the Google brand name to get early user traction. The products should be great at what they do, but it’s okay to limit functionality at launch.

How do we measure the quality of the user experience?

There are some things we can do to ensure a user experience is good, like user testing and a well defined design system. There are also some measurements we can make like time-to-completion, lostness, conversion and NPS. However, the level of user experience achieved is ultimately a qualitative judgement.

I propose that for each batch of value delivered, the team (including all the people involved in creating software) review the feedback and metrics available. Based on an informed opinion, they can vote on the level of user experience they believe has been delivered. This way, they can track the overall quality of a product and highlight areas where the team feels they haven’t hit the mark. Too many low scores is an indicator that too many corners have been cut and they need to be addressed.

The user experience estimation would work in a similar way to how teams estimate points for user stories. The team scores baseline experiences to benchmark future work. These baselines can be found in any product — it does not have to be their own, as long as everyone agrees. Then, for newly delivered items, each team member gives a qualitative score between 0–6 from the UX pyramid above.

An informed opinion of multiple experts will produce a score that is reasonably accurate. Any team member that tries to game the system for their own gain will likely be called out by the other team members.

What about MVP’s?

The terms MVP and “lean” are not part of the Agile Manifesto. However, they are often conflated. Lean is about learning fast, not delivering fast. I’ll quote directly from Does MVP mean throwing UX out the window? by Sam Enoka.

With our best attempts to become “lean”, it is all too common to forget that the point of an MVP is to learn how we can make products that users will find both valuable and delightful. Focus is easily tipped toward figuring out how fast we can ship, rather than how fast we can learn.
Some common (but hard to detect) traps I see people fall into revolve around justifying poor usability and design with the rhetoric that “it’s an MVP, if users complain then we’ll fix it”. The close cousin of this statement is “not many users are using this right now, so we shouldn’t spend too much time on it, let’s get something out there first”.

What Sam describes is what I call the Functional Fallacy. This is where it’s perceived that some functionality delivered is more valuable than no functionality at all. This is not necessarily the case. A poor experience can turn people off a product for life, which can be worse than the experience of no functionality.

Agreeing on a teams interpretation of agile.

It’s worthwhile having an ongoing discussion with the team (all the people involved in creating software) to define, publish and evolve how the agile principles should be applied. This should determine when to make the tradeoffs on each item defined in the manifesto and how the team will measure the effects.