How the digital sausage gets made — creating successful digital products

This is the 3rd article in a series about how to get digital right (part one, part two).

In the previous article, we established why agile software development is important and we situated it in its historical context. In doing so we also gained an understanding of what agile means, at least in broad strokes. In this part, we will dive deeper into the how and the what.

Because agility is only a means to an end, I frame the challenge in terms of a popular purpose; the creation of successful digital products. We know why agile matters, but how does one put it into practice? How do successful digital products see the light of day? What is the secret sauce of high-performing product teams?

At In The Pocket, we coined a number of principles to help our teams in the quest for business value and agility.

Start with why: find a shared mission to work towards

The impact of digital products is larger than we think. Consider the simple notification on your cell phone. When we receive likes or texts, our brain releases dopamine into our neural system. Dopamine is the same neural transmitter that is released after eating or having sex. This molecule evolved to make us feel good when we meet a survival need.

Likes and text messages may not seem essential for our survival but they hack into a hardwired need for social connection. In a way we have awarded root access to our brain to all kinds of digital products by carrying cell phones around. This explains the enormous impact of the digital revolution on society.

With great power comes great responsibility. As creators of digital products we have a moral duty to think through the effects of these products on people and on society.

Although human-centered design is a laudable approach, we cannot forget the business aspect either. Digital products are significant investments and people will only take the risk of creating them when they see some kind of economic upside.

Whether you want to change the world and / or make a buck, the starting point for any digital product should be its value proposition: why would someone choose to use your product?

Establishing a “true north” is vital to set up product teams for success. It creates ownership and a shared purpose for teams. This helps teams navigate tradeoffs, prioritize ideas, make decisions and improve their modus operandi throughout the entire product lifecycle.

Mission vs objectives

As it might take years to realize the product’s full mission, objectives help the team gain focus on a shorter term. Well-considered objectives offer a tangible way of tracking progress towards the product’s mission. They define a specific milestone in product success, along with the metrics to check whether a release is moving the needle.

Although objectives may be re-prioritized, they should not change often. Underlying initiatives can vary more regularly, based on what the team learns from releasing intermediary versions. The overarching mission is rarely updated, only when there is a significant pivot. Depending on the product’s life-cycle stage, the mission and the objectives can take different forms and shapes, however. Let’s explore this life cycle in some more detail.

Explore: from challenge statements to a value proposition

The first part of a product’s life cycle is a time of exploration. An organization faces a challenge or wants to explore an idea but is not quite sure how to address it. During this phase, the mission should clarify the challenge to be solved.

Strong missions constrain the challenge space and broaden the solution space. Constraining the challenge space means being specific in what you want to tackle. To give a negative example, it’s usually a bad sign if a mission contains many conjunctions:

We are creating a personal finance app to manage your expenses and also acts as a productivity tool that helps users discover professional services.

This mission sets a team up for confusion. Choose one challenge space and focus.

All too often, companies fall into the trap of falling in love with a specific technology (“we need an app”, “we need a blockchain”) or taking the default position of complying with the highest-paid person’s opinion on what the value proposition should be. Broadening the solution space requires an open mind in terms of solutions. We’ll come back on this in the “design for value” section.

Build: planting the flag

The build phase of the product life cycle starts with making the product available to users: the minimum viable product. The purpose of the MVP is to gauge interest in your value proposition and capture feedback to shape the product further.

In the explore phase the mission and objectives existed mainly to clarify and inspire. As you enter the build phase, they should become more objective and measurable.

At the end of the day, the purpose of the build phase is to establish product-market fit. Product teams need a way to check if their efforts are helping the product get traction.

Of course, products are multifaceted. There is always a risk of developing blind spots if you rely on a single success metric. There are a few frameworks out there that aim to help product managers in making success measurable. Google’s Heart and Dave McClure’s AARRR frameworks come to mind.

These frameworks are a good place to start, but they should not be viewed as dogma. Shreyas Doshi, former product lead at Stripe, points out the importance of using metrics that cover all the following categories:

• Health metrics: is the product available to users and performing in the manner that users would expect? Examples: latency, initial load time, uptime,…
• Usage metrics: how are users interacting with the product? Eg: funnel metrics, help docs usage, retention,…
• Adoption metrics: is the product being used? And is it used in the way its creators intended? Examples: DAU, MAU, free-to-paid conversions, feature adoption trends,…
• Satisfaction metrics: what is the user’s sentiment on a product? Example: customer satisfaction (for support, new features, overall..)
• Ecosystem metrics: what is the macrostate of the domain in which the product lives? Examples: share of wallet, 3rd party integrations, industry rank,…
• Outcome metrics: how is this product improving our bottom line? Example: revenue, margin, revenue per user,…

While the product manager may take the lead in establishing the metrics, they rely on the product team to help with implementation as well as with interpretation.

Grow: data-driven decision-making

Growth is what should be happening after you establish product-market fit. We want to increase the product’s success further and grow it sustainably. In this phase, we should meticulously analyze where users experience friction and remove that by launching targeted experiments and updates.

If the build phase made the mission and objectives measurable, the growth phase is when an outcome-based focus should become second nature. A high-performing team in the growth stage tracks and visualizes success metrics across the board. At every release, they check how they impacted the agreed indicators and they adapt accordingly. What can be added/removed/changed that would benefit both the user and move the gauges in the right direction?

Note that these metrics are not immune to improvement either. Teams rarely get them right from the first try. In systems thinking, this is the difference between single-loop and double-loop learning. We’ll explore these concepts in more detail in the articles on organizing for complexity.

Value: the art of balancing business needs with user needs

While one of the merits of the agile revolution was the focus on users, there was initially little attention for the business aspects of building software products. The agile manifesto states: “Working software is the primary measure of progress”. This statement is only partly helpful; it is perfectly possible to deliver working software that does nothing for your business goals. Or worse, software that’s harmful to your bottom line.

The Agile principles accompanying the manifesto do mention “valuable software” but we’re not told what value means or how to get there. Which agile methodology has artifacts or ceremonies that track results? Scrum delegates the entire value question to the product owner, which is a bit of a cop-out. As John Cutler put it; the rightmost column is missing in most agile boards:

Disregarding the value aspect may lead an organization into the so-called feature factory anti-pattern, where teams mindlessly implement features requested by another silo, dubbed “business”. Feature factories measure the software team’s performance in metrics that only measure the output. These metrics ignore what matters most: whether this output achieved the desired outcome. Examples of output metrics are story points or lines of code.

Understanding the difference between output and outcome metrics is crucial if you want to ingrain the value focus in an organization. Take velocity for example; a useful instrument out of the Scrum playbook but often misused. The concept of velocity is useful in helping teams make predictions about when an item can be delivered. It is a measure of predictability, nothing more and nothing less.

If this metric is misused by extending it to team performance, you’ll see adverse effects. You are effectively incentivizing people to game the system. Many well-intentioned managers have discovered that this path leads to hyperinflation of story points. When that happens, story points are no help in measuring performance or predictability.

“When a measure becomes a target, it ceases to be a good measure.”

— Goodhart’s law

Reporting on the outcome is a lot harder than measuring output. First, the team has to understand what constitutes value. Superficially this sounds easy but translating product value into meaningful metrics is hard work. By changing the focus from output to outcome, you measure the impact a team has on the bottom line.

In this section, we covered the importance of stating why you want to achieve something and how to articulate this ambition. In the next section, we will explore how to give teams the best chance at actually producing value.

Dual-track agile: combining product discovery and delivery

Quibi was an American venture that aimed to become a streaming platform for short-form content. The company was founded in August 2018 and raised $1.75 billion from investors. It was shut down in October 2020, only 6 months after launching. No user shed a tear over Quibi’s demise. Their mistake? Quibi tried to go from idea to scaling a product, without any contact with the customer.

Creating digital products is not a stage-gate process. Exploring, building, and shipping software are intertwined and should occur in parallel. This is the crux of dual-track agile development. In this section, we’ll explore these parallel processes.

Never stop discovering

“Software development is complex and unpredictable. Because of this, [modern product teams] begin with the idea that everything is an assumption until we prove otherwise. As we work, we gain clarity. Thus, we are always moving from a position of doubt to one of certainty.”

— Jeff Gothelf and Josh Seiden (Lean UX)

We established that a clear north star and outcome-based objectives help give focus to a team. Dual-track agile aims to attain those objectives by stringing two concepts together: product discovery and product delivery.

Everything starts with product discovery. A feature is a gamble that a specific idea will materialize into business value if the team turns it into software. Many things can go wrong in that last sentence. How does a team decide what ideas to act on? And how do you translate an idea into software?

This is the domain of product discovery; framing challenges and defining solutions that will help product teams to reach their objectives. By elaborating ideas, it becomes clearer what direction to take. This is not a one-off practice; it should happen continuously and in parallel with the software delivery.

Before deciding to build a feature, a process of refinement and validation can help teams to reduce assumptions and decide what ideas to act on:

• desirability: does the solution address the user’s needs, and will they choose to use it?
• usability: will users figure out how to use it?
• feasibility: can we build it with reasonable effort?
• viability: will the solution provide business value, and is it aligned with our strategic focus?

You’ll want people on your product team who ask these questions and look both inward and outwards to find answers. The team itself is well-placed to help answer feasibility questions, but they’ll have to look outward and involve actual customers to answer questions on value. Some people in the team will be more focused on the product and the users while others will be more focused on the internal engineering.

Analysis is not a goal in itself. The goal is to filter out bad ideas and retain the most promising ones. To help this selection process, it may pay dividends to validate specific ideas or assumptions before betting the farm.

Building software is one way to see if an idea was any good, but it’s an expensive method. Validation is a lot cheaper upstream. As soon as an idea crystallizes, consider creating a prototype and test how users react to that. Another validation technique is user interviews; these are exploratory interviews with typical end-users to better understand their needs, frustrations, and context.

Combining discovery and delivery

High-performing teams frequently release product updates with confidence. The result is a steady flow of increments that make the product better. During delivery, the team focuses on the list of items that have made it through the discovery funnel, resulting in the decision to implement the ideas. At this point in the funnel, the ideas have been refined and they are detailed enough to be implemented.

Even though the process of discovering what to build and then building it may seem like 2 distinct steps, they are interwoven and happen continuously. Output from the discovery phase is input for the delivery phase, and vice versa. This is what is commonly known as dual-track agile. Or even more commonly: “learning”.

There’s a multitude of delivery methodologies, each with their own ceremonies and best practices, that product teams can apply for organizing their delivery flow: I am not religious about any delivery methodology. Teams should pick the best tool for the job. At In The Pocket, product teams often select a specific methodology based on the experience level and culture of the customer. At the time of writing our most popular approaches are Scrum and Kanban.

One thing to keep in mind in combining discovery and delivery is the flow of the value stream. Flow is a concept from Kanban that tracks how work moves through the system. It is possible (and very common) to overwhelm the delivery part of the system if the ideation process upstream is not in sync with the throughput capacity in development. In Kanban terms: the arrival rate of items in the system is higher than the departure rate.

Allowing too much work into the system actually slows the system down. Kanban recommends to manage this by visualizing the workflow (including work-in-progress) and actively managing it. When 5 people are working on 50 user stories at the same time, user stories spend a long time in the system waiting for people to get round to working on them. The more tickets in the system, the more people have to switch contexts which slows them down even more.

Saying that WIP is high is another way of saying that a product team has high “utilization”. Or from the team’s perspective: the team has very little slack. While not intuitive, it is absolutely necessary to create slack in a team’s planning. There is math to support this this but a simple illustration does the job as well: rush hour traffic.

Imagine a crowded highway with 4 lanes. During peak hour the system operates near 100% capacity utilisation. When 1 lane is closed because of an accident, we see a non-linear effect on the cycle time — the time it takes to travel the highway. Reducing the capacity with 25% causes the cycle time to rise by a lot more than 25%. When an accident occurs at night, the cycle time is not impacted because of lower capacity utilisation. The same holds true for a software team. If they operate at 100% utilization, any unforeseen request (a production bug, a question from another partner in the ecosystem, unexpected discovery work,…) will wreak havoc on the cycle time of the system.

Systems theory aside, we help clients understand that it is counterproductive to try and check every box on every stakeholder’s wishlist. This is crucial in a service business; suppliers that just do as they are told, lose out in the long run. Successful agile teams understand that ruthless prioritization is key in the race to a minimum viable product where the user can be the ultimate judge. If it’s not in service of the mission, leave it out. If it’s not the highest priority to deliver it now, deliver it later.

Design for value

So far, we approached discovery through the lens of the product manager. While it’s tempting to focus on the functional aspects of a solution, seasoned product managers know that they need to take a more holistic approach and consider all the requirements when they assess a solution.

In this section, we’ll explore in more detail how teams, architects, and product managers can identify the most promising solutions within a certain problem domain. This approach is based on the following principles:

• Outcome-driven approach: what is the expected contribution of a solution?
• Systemic approach: where does the solution fit in the bigger picture?
• Tradeoff analysis: how do the inherent design decisions of a solution affect the realization of the various requirements, how do multiple solutions compare?

Outcome-driven approach

In order to be able to assess a solution, one needs to know the requirements it has to fulfill. There are three categories of requirements:

• Functional requirements: what the system should do,
• Quality attributes: how well the system satisfies the non-functional needs (security, performance, reusability, reliability, scalability…), and
• Constraints: non-negotiable requirements, i.e. what must be adhered to.

The aim is to design a product that will deliver the functionality in such a way that the quality attribute requirements are met and that it fits within the constraints. A good understanding of the requirements is a big step in choosing the right solution. What then, makes good requirements?

First of all, they should address the full scope and nothing but the scope. In other words, there should be a clear distinction between the scope and the constraints. Good product managers and architects derive requirements from the intended outcome for the problem.

Requirements should be solution-neutral and constraints should be external — things that are not under the control of the designers. There is virtually no limit to the requirements that can be stated but they should not unnecessarily constrain possible solutions. Disguising a solution as a requirement can seriously undermine a product’s chance of success (“the product should capture Bluetooth signals”).

It is not easy to formulate requirements in a specific and measurable way. Techniques like quality attribute scenarios exist but are notoriously time-consuming and therefore only applied to the most significant requirements. Depending on the complexity of the product a team may choose to seek clarity in formulation with less emphasis on measurability.

For most real-world problems, a single solution does not exist that meets all requirements (see trade-off analysis). The design approach is then to design for the most significant requirements — the ones that yield the most value and have the highest impact.

Systemic approach

Any technological solution has to fit in a broader ‘system’. This system includes participants and their interactions, resources and their allocation, processes and the technology supporting their operation, the relationships between all components and the way value is generated and measured.

This holistic approach is necessary to align the components with the objectives of the whole, and to avoid local optimization in one component at the expense of global optimization of the system as a whole. We’ll expand on these risks and how to tackle them in later articles.

Trade-off analysis

The design space of software is virtually unbounded. There is never one perfect solution for any given set of requirements. Trade-offs (i.e. a decision that favorably impacts one requirement at the expense of another requirement) are unavoidable.

For complex problems the analysis of the trade-offs should be front-and-center. This requires a complete view of the requirements, understanding their relative priorities, and designing the system to meet the highest priority requirements first. The approach is then to generate multiple solution sets that meet the prioritized requirements in different ways.

Doing this will reveal the trade-offs inherent to the solutions. Decision-making is then about carefully choosing trade-offs. Keep in mind that there is always the possibility that there is no solution with acceptable trade-offs. In that case, the only option is to revisit the requirements or to forego the solution.

There can be no agility without an engineering culture

So far we introduced dual-track agile and touched on product delivery. While we mentioned methodologies like Scrum and Kanban, we have not gone into great detail. Plenty of writers have addressed this topic and I won’t add to an already crowded bookshelf. This section explores engineering culture instead.

Many agile implementations focus on enabling rituals but we’ve always felt that a strong technical game is at least as important for product teams. If it is a painful and frightening process to ship changes to the market, there isn’t an agile ceremony that will make a difference until your organization has fixed that problem.

Deploy or die

Researchers Forsgren, Humble, and Kim confirmed our gut feeling on the importance of an engineering culture in their seminal book “Accelerate”.

There was never a shortage of books touting the advantages of DevOps. Accelerate stands out because they compared engineering practices with business performance of the organizations where these practices are applied. Moreover, their scientific approach is the result of years of research based on data from 31.000 IT professionals worldwide.

The authors start by breaking software delivery performance into four metrics.

Two metrics track tempo:

• Cycle time: the time it takes between the start of development and the availability of this change to an end-user. Shorter cycle times enable faster feedback loops.
• Deployment frequency: this is a proxy metric for batch size. Shipping in small batches correlates with a high frequency of interacting with users and the market. It also decreases the risk associated with a release.

Two metrics track quality & stability:

• Mean time to restore: failures are a given in complex systems (i.e. where all modern software products live), so the authors have elected to measure how quickly teams recover from failure.
• Change fail percentage: this measures how many changes to production result in degraded service or require fixing (patch, rollback, hotfix,…).

Through the yearly State of DevOps Report, the authors gathered data from respondents scoring their organizations on the above metrics as well as their ability to achieve business goals. The statistical analysis of the researchers confirmed two significant conclusions

• There is no trade-off between tempo, stability, and quality. High performers tend to score well on all these attributes.
• There is a strong correlation between software delivery capability and business performance.

This should not come as a surprise. If organization A takes an hour to get code from main to production and organization B takes several months, company A will obviously do a better job of using software to interact with the market. The authors go on to identify specific capabilities to improve organizational performance, visualized below. Arrows between the constructs indicate a predictive relationship.

At In The Pocket we do not prescribe these practices but we do hold teams accountable for the end results of tempo & quality. At the same time, we make it as easy as possible for teams to adopt the practices and capabilities that benefit tempo & quality.

Deliver quality

In The Pocket positions itself as a digital product studio. The word “product” is not chosen by accident. In the IT service industry, the term “project” is more common.

A project implies that both the scope and the end date are known beforehand. This makes no sense from the product perspective. Any successful digital product should be able to evolve. This requires continuity.

Digital products are often judged by their design and UX, and for good reason: it is key in attracting, retaining, and engaging users. But quality aspects that are less visible towards end-users, are just as important.

Aspects such as clean architecture, clean code, continuous integration & deployment, test automation, evolutionary architecture, and controlling technical debt, are key in creating successful digital products. To have a shot at thriving in the long term, the codebase must be able to evolve almost effortlessly.

At this point, there is an elephant in the room that we need to address. In many traditional enterprises, business stakeholders are blissfully unaware of what happens under the hood of a digital product. This misalignment between business and IT creates a risk of stakeholders forcing teams to neglect technical best practices in favor of functional improvements. In that case, the product starts to incur technical debt. When such a situation is allowed to go on long enough, the product becomes impossible to maintain or update.

In other words, there is more to business agility than agile development alone; this is the topic of the next article in this essay.