Discovery — an uncomfortable place for finding options
Discovery, “the fact of discovering what was previously unknown”, is such a great word. Rich enough but not overly precise, lending itself to many uses.
The concept of discovery is often used at the beginning of a software development project or product development as a phase concerned with discovering the work that needs doing — a set of activities dedicated to shaping the future of a product or a feature.
Over the years, I’ve noticed that this critical set of discovery activities is misused, robbing many teams of opportunities that can make the difference between building a great product from an average one.
On the surface, what should happen during discovery should be clear. Still, the concept is misapplied, leading to confusion and missed opportunities — hurting the development process in the long run.
Discovery is not visioning, and it is not refinement. And certainly, it is not mobilisation. Discovery is the process of identifying and surfacing options for the future.
Discovery done poorly is a lost opportunity to shape the future of a product or service development. Typical mistakes that I’ve seen are teams jumping too quickly at creating feature backlogs and focusing on prioritisation (closing options too soon) — or concentrating on visioning once they realise their vision and aspirations are unclear (not doing the right type of work). Or focusing on project mobilisation (again, not doing the right type of work). Once discovery is skipped, teams rarely return to discovery since the project time pressures will pile on — they’ll jump into built.
Visioning, backlog creation, prioritisations, and team mobilisations are all important activities — but they are not discovery activities.
Why are discoveries misused? I believe it is due to the nature of what discoveries should be. Discovery should be an uncomfortable place, a place of uncertainty. It is, in my view, similar to Aporia, which Dave Snowden defines in his Cynefin framework  — as “an irresolvable internal contradiction or logical disjunction in a text, argument, or theory”. In philosophy, Aporia is a philosophical puzzle on a seeming impasse in an inquiry.
And, for us humans, staying in an uncomfortable place is not easy. We quickly shift towards what we feel is solid ground — creating backlogs and tickets. And plans. And project mobilisation activities, ramping up teams.
Why does it matter to get discoveries right in software development?
It goes back to the nature of software development. Intrinsically, software development as a whole is an act of discovery. We solve a problem through exploration, analysis and design. We examine the opportunities and constraints of our technology stack and our business domain as we interact with it. In the end, we have learned all the lessons and know exactly how we’d do things differently next time, except we never do that same project again. And on the next, we start all over again.
Therefore, getting discoveries right is intrinsically linked to success in software development.
How can we do it?
The solution has to revolve around creating the conditions and the space for discovery to happen, focusing on:
- knowledge and aptitude
- organisation’s culture
From a process point of view, we can focus on creating a system of work where Discovery is a first-class citizen:
- Start by defining the major activities of a delivery cycle. I like to run the following system: Visioning, Discovery, Elaboration, Build and Run. Visioning is where we define the ‘why of purpose’ and ‘why of movement’. It is the space of strategy where we want to get an answer about what to do when there is no clear direction. Discovery builds on Visioning and is the space where we focus on finding options to start executing the strategy. Here we redefine problems and design solutions. We aim to define a tactic: know what to do next. Elaboration will continue the Discovery and is the space where we elaborate on viable solutions. The Build is where we build and run them (execute the tactic identified in Discovery).
- Define these activities' inputs and outputs and what happens with each cycle rigorously and explicitly. For instance, Discovery inputs could be a collection of prioritised OKRs. Discovery activities include experiments, user and market research, or product design. Outputs could be a set of Initiatives, with the viable ones needing further elaboration (turned into Features, Epics, backlog items, etc.).
- Define the cadence (how often do we enter or revisit each activity). While the activities of Visioning, Discovery, Elaboration, Build and Run can be sequential for a short period, they need to be cyclical. It is a good idea to define their cadence — for instance, we could revisit Visioning every 6–12 months, where Discovery could be done continuously or agreed to a cadence (e.g., once per quarter). Cadence will create a rhythm, and we can use the rhythm to create a good product development tempo.
From a knowledge point of view, we will need people with the right knowledge and aptitude to make Discovery successful. It would be best to combine multiple disciplines, such as product management, design, and software development. Since in Discovery, the process is of opening and closing options, some of these options will get abandoned and never will see the light of the day. This can be again an uncomfortable place, and we will need people who are hungry to try new things and comfortable abandoning ideas which are not proving to be that promising. We will need to find those people with the right aptitude.
Lastly, the organisational culture plays a major role in running discoveries. Leaders must give permission, resources and space for Discoveries to take place. They must embrace the nature of uncertainty, encourage their people to do the right things, and not rush them into closing options and asking for plans too early.
Discoveries done well greatly impact the future shape of a product. New opportunities are explored. Biases are reduced, pet projects eliminated, and new solutions and paths are identified. It is a liberating feeling that we need to sample more often.
 Cynefin — Weaving Sense-Making into the Fabric of Our World, by Dave Snowden, Zhen Goh, Sue Borchardt (Illustrator), Riva Greenberg (Editor), Boudewijn Bertsch (Editor), Sonja Blignaut (Introduction)