Software is a way to solve problems, it’s not the only way, and building it in increments often doesn’t tell us whether there might be bigger fish to fry. Sometimes there’s a larger pie to be had, if only someone had thought to ask better questions. In suggesting that we first explore the value that we’re going after (see here), that Agile might keep us a little too heads-down focused on building, I have been accused of being inexperienced, of ignoring that in most orgs the hard problem to solve is precisely how to build software.
I think this is a non sequitur. Of course orgs need to continuously improve how software is built — it’s not an either/or. They also need to improve their ability to quickly discover the right things to do, to quickly surface legitimate user needs and stakeholder goals. If we were any good at predicting what will create value, then whether our prioritization decisions are based on requests, HiPPOs, or our own gut instincts, then we should be able to predict the results of A/B tests. It turns out we’re really lousy at it (see Kohavi et al., 2009).
When I say Agile keeps us too focused on building, I’ve also been accused of making a mountain out of a molehill. As one person put it, the Manifesto is about software, and there is nothing in it that says not to consider other ways to solve problems. So, this person might say, in some rare instances we don’t have to build anything to solve problems…so what? Where is the conflict with Agile? Well, what if we think these exceptions rarer than they are precisely because Agile has kept us all so laser focused on building? To see how this might be, let’s go back in time, before the Manifesto. Let’s go back to Gerald Weinberg’s The Secrets of Consulting (1985). It’s quiz time. What are Weinberg’s first two laws of consulting?
In other words, in any situation, there is always, always, ALWAYS a way to create value. You just have to find it. Managers or stakeholders might not want to admit there’s a problem, especially if doing so involves losing face (and remember, a good consultant always allows people to save face). As Esther Derby (2019) councils, give them an out and respect their journey, or they won’t want to move forward with you. But this doesn’t mean the problems aren’t there. They are hidden in the stories we tell, the way we narratize how we’ve arrived where we are, in the way we even conceive of the situation, of what we think is “wrong” and what needs “fixing.”
To narratize, we must frame the situation, pick out what will even be noticed as “things” in the story. This unconscious process of “naming and framing” makes certain aspects salient as issues, which analogically suggests what we do in response (Schön & Rein, 1995). If we ignore this process of “problem setting,” if we don’t recognize a frame for what it is, if we cannot step out of it and explore alternatives, then the current narrative will remain fixed. When we can’t identify and check our assumptions, we are blind to our frame. This is true regardless of how suboptimal it might be. This is true regardless of whether we frame everything as a “technical problem,” which Weinberg stresses is a mistake. This brings us to the second law.
Business value is created by getting people to alter their behavior in a way that creates business value (Anderson, 2011). So, unless our software is written by software, even our “software problems” are people problems. Whether it is by purchasing a product, fixing buggy code, or improving the usability of an online store, we’re still talking about people doing certain things that either increase return, protect existing return, reduce cost, or avoid added cost (Arnold, 2014).
Weinberg notes that some will object to this. Managers, he says, will prefer framing everything as a “technical problem,” often as a form of self-defense. The underlying issue, however, can always be traced back to people and their behavior, and software is just one way to change what people do. The real questions then are, what value are you going to go after, what do you need people to do to create it, and what’s the best way to generate this change?
By capturing and probing your assumptions, exploring the frame, and generating lateral options, you enable the perspective needed to consider how much value you might be leaving behind by solving any particular problem. This is important because the way you frame the situation determines the size of the pie you’re going after; and, whether you’re deciding what problem to solve or determining how to solve it, your decision is only ever as good as its best alternative. As decision scientist Robyn Dawes (2001) argued, the number one reason we make poor decisions is failing to generate alternatives and sticking with our initial narrative, what is sometimes called “cognitive sloth.”
Here’s an example. Say a team’s customer wants it to software-enable a process. Doing so will save around $20k a month. Not bad! Maybe they do it and the customer is happy. They get a recognition for their work and are featured in the IT newsletter. All is well and good. Now, what if it turns out if they’d simply asked a few more questions, done a little more digging, and made a few introductions between the right stakeholders, the process could have been done away with altogether, saving $1m a month? Well, that oversight has a “cost” of $11.8m per year. This changes what you feel about the team’s accomplishments, doesn’t it? Of course it does. With different comparisons, you’ll see the same result differently.
There’s an analogy in experimental design. When the team software-enabled the process without exploring other options, it’s like they placed themselves in a between-subjects design. This is where subjects are only shown one condition of a study. In a within-subjects design, where they’re shown the other conditions, subjects have an expanded view. Well, if people are being asked to make a judgment about what they’re shown, it’s likely going to differ depending on which type of design they’re in. As an amusing example, Michael Birnbaum (1999) famously found that in a between-subjects design people rated the number 9 as being greater than the number 221. It makes sense too — the subjects were likely thinking of different contexts:
This is why we need to be really careful when we’re thinking about value — we’re anchoring ourselves. If all we’re comparing is saving $20k a month ($240k a year) to saving $0, then $240k a year seems like a big win. By exploring the frame and generating options, we’re kind of moving to more of a within-subjects situation. We open up our perspectives to include other comparisons, thereby widening context. Saving $240k a year is fine compared to $0 but compared to $12m it’s small potatoes. In general, if we focus on one option and decide to do it or not without eliciting comparisons, we’re going to make worse decisions than if we force ourselves to always consider at least three alternatives (Nutt, 1999; see also Torres, 2016).
If you need to set a rule to force you to always generate alternatives, then do it! An example might be, “I will always dovetail by pitching a potential problem with two alternative problems to solve.” Weinberg calls this his “Rule of Three,” which I believe he got from Virginia Satir. Paraphrasing for present purposes, if you cannot list three ways your understanding of the situation might be wrong, there is something wrong with your thinking.)
If instead we’re just taking requests, well, just because a customer is satisfied doesn’t mean the opportunity cost wasn’t large. If we’re doing one-plan-at-a-time solution thinking, then even if we “iterate” toward better ways of achieving “whatever-it-is,” we’re still locked in the same frame! Agile doesn’t make up for the opportunity cost of ignoring discovery. Conflating discovery with “BDUF,” as many agilists do, is an expensive mistake. As Andy Grove (1983) said, saying “Yes” to one option says “No” to the others. And we’re not talking about the solution items in a backlog here; we’re talking about the other problems we could have solved. What is the potential value left on the table? What are the alternative “costs of delay” being ignoring?
As Weinberg notes, “cost-benefit analysis” is often just a euphemism for “cost analysis.” Generally, no one is even looking at the possible benefit left behind. In sum, just because we have demand doesn’t mean we don’t need discovery. We can’t compare options if we don’t do the discovery work necessary to bring them to our attention in the first place.
Anderson, S.P. (2011). Seductive interaction design: Creating playful, fun, and effective user experiences. Berkeley, CA: New Riders.
Arnold, J. (2014). Understanding value. Black Swan Farming. Retrieved on March 8, 2017 from: http://blackswanfarming.com/understanding-value/.
Birnbaum, M. (1999). How to show that 9 > 221: Collect judgments in a between-subjects design. Psychological Methods, 4, 243–249.
Dawes, R. (2001). Everyday irrationality: How pseudo-scientists, lunatics, and the rest of us systematically fail to think rationally. Boulder, CO: Westview.
Derby, E. (2019). 7 rules for positive, productive change: Micro shifts, macro results. Oakland, CA: Berrett-Koehler Publishers, Inc.
Grove, A. S. (1983). High output management. New York: Vintage Books.
Kohavi, R., Crook, T., Longbotham, R., Grasca, B., Henne, R., Ferres, J. L. & Melamed, T. (2009). Online experimentation at Microsoft. Retrieved on October 18, 2016 from: http://ai.stanford.edu/~ronnyk/ExPThinkWeek2009Public.pdf.
Nutt, P. C. (1999). Surprising but true: Half the decisions in organizations fail. Academy of Management Executive, 13, 75–90.
Schön, D. & Rein, M. (1995). Frame reflection: Toward the resolution of intractable policy controversies. New York: Basic Books.
Torres, T. (2016). Why this opportunity solution tree is changing the way product teams work. Product Talk. Retrieved on September 9, 2019 from: https://www.producttalk.org/2016/08/opportunity-solution-tree/.
Weinberg, G. M. (1985). The secrets of consulting: A guide to giving & getting advice successfully. NY: Dorset House Publishing.