Core Concepts #4: Benefits
The “why” of tradespace exploration
This is part four of a seven-part series of posts on the core concepts of tradespace exploration, designed to help beginners become familiar with terminology and the general structure of tradespace data. Click here to find the other posts in this series, as well as other Tradespace 101 posts.
So we know that stakeholders make a decision when they spend resources in order to acquire an alternative. But “why” do they do that? As you might expect, they do it for the benefits that they stand to gain from having the alternative. Benefits correspond to the desires or goals of the stakeholders and, similar to resources, should be elicited from the stakeholders directly in order to capture the metrics that they use to judge the goodness of an alternative.
As you might expect, stakeholders generally try to maximize benefits while minimizing resources spent: but therein lies the rub! Usually more expensive alternatives generate more benefit, making it necessary to explore the tradeoffs between benefits and resources to uncover high-value solutions. Hence, why we do tradespace exploration!
Benefits are typically much more application-specific than resources: there aren’t any categories of benefit like time or money that are relevant to a majority of decisions. Benefits will most often reflect what an alternative does for the stakeholder: how it contributes to their objectives for using the alternative. So a stakeholder buying a phone will use benefits like screen size and battery life, while one making a decision about where to go on vacation will use benefits like quality of accommodations and number of nearby places of interest. The eclectic nature of benefits prevents us from establishing a cut-and-dry list of brainstorming prompts, so you’ll have to work with the stakeholders to capture their specific needs. However, just as with resources, a value model can be used to aggregate the “goodness” of multiple different sources of benefit into a single score.
Following the principles of value-focused thinking (VFT), it is better if the benefit metrics used by stakeholders are ends-based and not means-based. Ideally we should be capturing “why” the stakeholder wants the alternative to work (the ends). However, in practice stakeholders will often be biased towards using means-based metrics that describe “how” the alternative works — because they are correlated with the end goals but are typically less abstract. We’ll have more detailed discussion of VFT in the future, but this one bit of advice is incredibly relevant when working with stakeholders to capture useful benefit metrics: a task referred to as operationalizing their preferences. (Technically, we operationalize resource metrics as well, but it is usually much more straightforward)
Let’s do a quick example of eliciting and operationalizing benefit metrics: perhaps you are helping a friend buy a new car and they tell you that they have a preference for safer vehicles. You ask “How would you judge the safety of a car?” and they reply with the number of airbags. It’s true — a car with more airbags is likely to be safer than a car with fewer. However, this is clearly a means-based metric because airbags make a car safer, not measure its safety. After all, it wouldn’t matter how many airbags a car had if it were unsafe for other reasons! Instead, you recommend using the NHTSA 5-Star Safety Rating for a more complete, ends-based safety metric. Now that’s good operationalizing!
Now that we’ve covered stakeholders, alternatives, resources, and benefits — the who, what, how, and why of tradespace exploration — you have been introduced to all the elements needed to construct a simple value-focused tradespace and perform a basic MATE study. Congratulations! Remember: stakeholders spend resources to acquire alternatives that generate benefits. This is the skeleton of every decision and, if you understand that framework, you’ll have a head start when it comes time to set up a decision problem for tradespace exploration.
We’re also now far enough along to introduce the tradespace scatterplot: a visualization so fundamental that it is often referred to colloquially as “the tradespace”. Each alternative is marked on a graph with benefits on the y-axis and resources on the x-axis (reduced to one dimension each using value models). Each stakeholder has their own scatterplot corresponding to the benefits/resources they care about. Generally the Pareto set — alternatives with the most benefit for a given amount of resources — are the most desirable alternatives and the ones most likely to be chosen. This view of the data is where almost every dive into a tradespace begins, so getting familiar with creating and reading this plot is one of the first skills you’ll develop as a tradespace analyst!
Benefit is a rather abstract concept, at least in comparison to stakeholders, alternatives, and resources. As such, there are different ways to think about benefits that can result in changes in the framing of a decision problem. Here are a few examples that you may have encountered if you are familiar with the topic:
- Decision vs Experience. When we analyze a decision in an attempt to find the best solution, we compare benefits through a decisional lens: the expectation of how alternatives will perform and the expectation of how satisfied we think the stakeholders will be. After a decision is made, the benefit that is ultimately experienced may not match: perhaps our models or estimates about the alternative were inaccurate, or perhaps the requirements of the stakeholder changed unexpectedly. The experienced value is affected by the perceived positive and negative gaps between what was anticipated (during the decision process) and reality, due to the natural human tendency to judge based on reference points built from expectation. For example, when you decide on an entrée at a restaurant, presumably you believe that it will be delicious — but we’ve all experienced disappointment when the reality turned out different than expected. We can’t do much to preemptively capture the difference between decision/experienced benefit when exploring a tradespace other than to try to minimize it through rigorous modeling and data collection.
- Rational vs Emotional. Many stakeholders will deliberately choose to consider only rational benefits when analyzing a decision. Rational benefits are typically clearly measurable, physical properties of the alternative that describe how good it is at its intended purpose, e.g. screen size and contrast ratio for a TV. It’s not hard to explain why a larger screen would be a positive aspect of a TV! That said, not all benefit is accrued rationally in this way: most people also have an emotional response to their decision that impacts their ultimate satisfaction. The emotional benefit is often involuntary and sometimes subconscious, making it difficult to capture until experienced. However, some stakeholders will know and acknowledge the impact of emotion on their benefit during the decision process and should be encouraged to actively consider such attributes. For example, a stakeholder may know that color preference for their new car is not “rational” in that it doesn’t affect how “good” the car is at being a car — except they personally will have much more fun in a bright red car (even if it maybe costs more resources!). Emotional benefits are valid and worth understanding if they could tip a decision in one direction or another.
- Objectives vs Regret. Most benefits are identified because they contribute to the objectives the stakeholder has: what they want the alternative to do. However, some problems will consider not only the desire to maximize objectives but also to minimize regret: the negative feelings associated with realizing that some objectives are not being completed as well as desired. Regret is similar to the difference between decision and experienced value, in that it is often based on a backwards comparison, this time between what is had and what could have been had (rather than what is had and what was expected to be had). This categorization is often just a matter of framing — it is relatively easy to model the benefit from an attribute as either objective-maximized or regret-minimized depending on the preferences of the decision maker. However, objectives and regret are distinct enough that it is possible to both gain from an objective and feel regret about it at the same time: for example, you may directly receive benefit from a larger screen on your TV while simultaneously feeling the “negative” benefit of regretting you didn’t get the largest size. Indirect costs such as externalities and opportunity costs are often modeled as regrets if they are considered as benefit attributes rather than resources.
Not all stakeholders spend resources AND gain benefits: sometimes costs are paid on behalf of someone else. These one-dimensional stakeholders can create undesirable outcomes if they fail to coordinate, especially when there is a significant disparity in power. When the benefit-gaining stakeholder has power, the “not my money” rationale can lead to spiraling costs in pursuit of extreme capability — like when the military effectively strongarms Congress into pouring more money into expensive acquisition programs. When the resource-paying stakeholder has power, the alternative may end up barely functional — like when a parent buys their child their first car, but saves cash with a beater just one pothole away from the junkyard. Both of these examples failed to properly explore the cost-benefit tradeoffs and resulted in a decision that is extremely inefficient.
