Behavioral Science as a lens to solve problems
Why Hindsight Bias isn’t 20/20, and what is a better frame
By Jared Peterson and Ellis Morlock
Upon reading Darwin’s “On the Origin of Species” and finally understanding the central idea, Thomas Henry Huxley reportedly said to himself, “How extremely stupid not to have thought of that”¹.
But yet, he didn’t think of it.
The common saying goes that “hindsight is 20/20”. In hindsight, everything just seems so straightforward and obvious, and we feel we should have been able to see it in advance.
The obviousness of hindsight isn’t just about recognizing the solution to a problem, but also in recognizing a problem to begin with. Perhaps you once dated someone who was a terrible choice for you. As you look back and reflect on the relationship, the warning signs were all there. How could you be so dumb? How could you overlook what was so obvious?
Why did it take so much time and effort to do the obvious thing?
The obviousness of both problems and solutions in hindsight can be a source of doubt for Behavioral Scientists. You look back at all the work you did to change your population’s behavior, and you realize that all you did was make things a little easier. Shouldn’t it have been obvious to do that in the first place? Why did it take so much time and effort to do the obvious thing?
The feeling that a now past problem, outcome, or decision was obvious is not just a popular aphorism but also an empirical phenomenon studied under the label of Hindsight Bias². Oftentimes, when we look to the past, we invent explanations for why something happened, or feel we “knew it all along”. But this is a false sense of knowledge that cannot be relied on, which is where the “bias” comes in.
However, and perhaps more interestingly, hindsight isn’t always a bias. Sometimes, the obviousness that comes with hindsight just means you did the work to look at the problem in a way that surfaced the most appropriate solution.
In psychology, this is called framing, but in the domain of Behavioral Science, we borrow from Michael Hallsworth’s metaphor and call this using Behavioral Science as a lens³.
Insight Problems and Frames
The feeling of “hindsight” is more apparent in some contexts than others. Consider the class of problems known as Insight Problems: a type of problem that requires a sudden and often creative solution. Unlike analytical problems that can be solved step-by-step using algorithms or logical reasoning, insight problems involve a reorganization of one’s understanding of a situation, leading to an “aha” moment when the solution becomes clear. Insight problems are often studied using simple games, almost like riddles, such as the 9-Dot Problem, the Candle Problem, or a personal favorite, the Mutilated Chess Board⁴, described below.
Imagine a standard 8x8 chessboard from which two diagonally opposite corner squares have been removed, resulting in a board with 62 squares. The task is to determine if it is possible to cover the entire mutilated chessboard with 31 dominoes, where each domino can cover exactly two adjacent squares.
One way to approach this problem is through brute force. You could try every possible combination of dominoes. But if you try this and keep track of every possible combination, you better get comfortable. There are billions, and perhaps trillions of possible combinations to try (We couldn’t figure out the exact number, let us know if you do). Far too many for one lifetime. Even if you tried a combination every second (not possible), the average human only lives for 2.3 billion seconds.
After the first few attempts at covering the chessboard, you might become convinced it’s not possible. But can you prove it’s not possible? Take a second to see if you can figure it out. You only get one chance at having this insight (then it becomes hindsight!).
Here’s the answer: a domino must cover one black and one white square, and since two white squares have been removed, there are more black squares than white squares. Thus, it is impossible.
The real challenge in making decisions in real life lies not so much in applying a standard decision-making process (that is, a standard decision frame) correctly — as the research on biases might imply — but in determining how to even think about a problem in the first place.
When you first hear about the Mutilated Chess Board, you might first frame the problem as a guess-and-check problem. But once that frame doesn’t work, you are left a little bit lost, and may not know how else to solve it. But, if you look at it for long enough, the color of the squares may just stand out to you, and suddenly you have the right frame to prove that there is no solution. In fact, with this new frame, it is hard not to see the answer as obvious. The two black squares are staring you in the face, how could you not see it?!
The difficulty of insight problems is this: the frame enables the insight, and the insight enables the frame⁵. It’s circular reasoning. How could you know how to frame the problem without knowing the key insight? How could you know the key insight without the right frame? It seems impossible. If you knew “color” was relevant, the solution would be obvious. But how could you possibly know that “color” was the relevant variable? And even worse, the board could have been uniform in color, and you would have had to realize the importance of adding color to the board.
(There are also additional solutions that don’t involve color. We will leave it to you to discover those on your own.)
Most problems in life are insight problems which have this seeming circularity at the center, which is why real-world decision-making is so hard. The process of realizing what context is relevant and forming a frame (through a process called Sensemaking⁶⁷) isn’t straightforward. You cannot just apply a formula to a situation and know what is and isn’t relevant, and even if you know what is relevant, you may not know why it is relevant. You have to do the hard work of creating a frame yourself, without knowing in advance what the frame will be. Decisions such as whether to have kids, where to stay for your next vacation, or how to redesign a product, don’t have clear, logical solutions, but instead require you to frame these problems in a way that makes sense given your context, values, and priorities.
The real challenge in making decisions in real life lies not so much in applying a standard decision-making process (that is, a standard decision frame) correctly — as the research on biases might imply — but in determining how to even think about a problem in the first place. If there were one true frame for solving problems (e.g., just apply Expected Utility Theory), then deciding whether to have kids would be easy. But no such single frame exists, as we all have different criteria for what we consider relevant and why those criteria are relevant. Like with insight problems, you have to figure out not just which answer is correct, but also how to even come up with an answer. As John Schmidt⁸ has said, “Problem formulation and problem-solving are not merely linked cognitive processes. They are the same cognitive process.”
The solution to an insight problem is almost always obvious once you have framed the problem; it is framing the problem that is difficult.
The problems we face in Behavioral Science are insight problems. Solving a Behavior Change Problem is not as simple as plugging everything into a formula and calculating. Instead, we have to go through a process of discovery where we try to understand the relevant aspects of the context⁹ (the goals, constraints, segments, motivations, etc.) in order to illuminate possible solutions that might work. Just as with the Mutilated Chessboard, the hard part is developing a frame that helps us understand what is relevant and how it is relevant. But once you have appropriately framed the problem, the solution may be obvious in a way that makes you feel like you might have known it all along.
This is worth repeating; the solution to an insight problem is almost always obvious once you have framed the problem; it is framing the problem that is difficult. But this is where Behavioral Science excels — in framing problems. By developing the right frame with the tools of Behavioral Science, we are able to achieve the obviousness of hindsight while we are still in the thick of the problem. Behavioral Science as a lens is thus the process of transforming hard problems into something a little more obvious.
Behavioral Science as a Lens
How do we develop the right frame so that we can see the solution with foresight, rather than experiencing the “aha” only in hindsight?
Michael Hallsworth’s recent manifesto on Behavioral Science had ten great suggestions for behavioral scientists to consider. He even has one on hindsight bias, but that is not our focus here. Instead, the focus is on his first principle; using Behavioral Science as a lens. In calling Behavioral Science a lens, Hallsworth is engaging in a framing of his own and is trying to make something intuitive for us that might not otherwise be. Very meta.
Behavioral Science has often been described as a toolbox; we have the “almighty” biases and nudges, such as defaults, loss aversion, reminders, anchors, social proof, etc. Along with this toolbox, there is a pretty standard process that involves tweaking some elements of the Choice Architecture by adding one of the tools and then testing to see if there was an impact.
But there is something very unsatisfying about this toolbox metaphor. It doesn’t fully capture what it is that we do as Behavioral Scientists. Lists of interventions and concepts divorced from context are of little use once you are in the middle of a problem and need to figure out how the list relates to the problem you are tackling. Knowing the names of the tools in the toolbox isn’t the same as knowing how to swing a hammer, or build a house. The important thing isn’t the list of concepts, but whether we are able to use Behavioral Science as a lens in order to have and create insights.
This is where Behavioral Scientists excel — in making behavior change just a little bit more common sense so that solutions reveal themselves.
When given a problem, such as improving the onboarding flow of a website, expert Behavioral Scientists are constantly on the lookout for the right concepts to frame the problem. They scan the literature, look to frameworks, interview, survey, and research until they feel they have the right representation of the problem. Because once they have the right representation, the problem transforms into something tractable. To paraphrase Cash et al. (2022)¹⁰— they analyze the behavioral problem with such depth that the relevant Behavioral Science principles become obvious in principle, even if the intervention itself still requires extensive design work and iteration. This is where Behavioral Scientists excel — in making behavior change just a little bit more common sense so that solutions reveal themselves.
Calling Behavioral Science a “lens” may be a hard sell for clients who want impact, not insights. But Behavioral Science as a lens does not mean the approach is purely academic or intangible. Framing for insights is just strategy by another name — behavioral strategy.
Such behavioral strategies are so much more than advice on how to tweak the letterhead on an email, but instead can inform every facet of a solution to ensure it is behaviorally informed at every step in the process.
Behavioral Science, at its core, is a systematic approach to producing insights relevant to changing behavior.
This is an important distinction between academic and applied Behavioral Science. While academics might be interested in understanding the exact mechanism that will have an impact in a particular context, in the field we rarely have the opportunity to conduct psychological research. Instead, clients just want us to provide something for their teams so that they can execute on it. By providing them a lens through which to see problems, we can do that. We provide much more than one-off insights and nudges — we provide an entire strategy and approach for framing and tackling behavioral problems. As Michael Hallsworth told us when we asked him to review this article, using Behavioral Science as a lens may mean not even designing an intervention.
To see a problem through the lens of Behavioral Science is to use the Behavioral Science frameworks, theories, and research approaches to understand and represent a problem, even a wicked one¹¹, in a way that makes solutions just a little more obvious. Behavioral Science, at its core, is a systematic approach to producing insights relevant to changing behavior.
All of our frameworks and theories are lenses that help us to understand and frame problems in a way that illuminates and sparks insights.
Conclusion
Imagine a photographer taking photos of a family event. She puts the camera to her eye only to see a blurry image through the glass. Knowing the distances, angles, and lighting of the shot, she adjusts the sensors, and what was previously obscure becomes crystal clear. Equipped with the knowledge of the clarified scene now in her view, she can never return to the understanding of this same scene in its clouded state. What was previously obscure can now not be unseen, giving an illusion that everything must have been obvious from the outset. Similarly, using Behavioral Science as a lens transforms problems from obscure and cloudy into actionable paths forward.
What was previously obscure can now not be unseen, giving an illusion that everything must have been obvious from the outset.
In thinking about Behavioral Science as a lens, or as a sensemaking process, we extend ourselves beyond just designing single interventions to nudge small changes in behavior using a defined toolkit and instead are able to see how Behavioral Science can be applied to any problem at any level. We are not just engineers “tinkering around the edges”. That metaphor doesn’t capture the essence of what we do. We are strategists, not just tacticians. Problem framers, not just problem solvers. Everything is common sense once you have a frame to solve the problem. The problem is getting that right frame, and that is where Behavioral Science can have a meaningful impact.
By employing empirical methodologies, with empirically informed frameworks and theories, we are able to develop psychologically rich understandings of a situation and determine what’s relevant in order to frame problems in a way that makes behavior change just a little more obvious. The wicked problems of reality become tractable behavior change problems.
This idea is not original to us but is embedded in the very origins of design thinking. Herbert Simon, Nobel Laureate and originator of design thinking, made the same point many years ago by analogy to mathematics. We will end with a quote from him¹².
“All mathematical derivation can be viewed simply as change in representation, making evident what was previously true but obscure. This view can be extended to all problem solving — solving a problem simply means representing it so as to make the solution transparent. If the problem solving could actually be organized in these terms, the issue of representation would indeed become central. But even if it cannot — if this is too exaggerated a view — a deeper understanding of how representations are created and how they contribute to the solution of problems will become an essential component in the future theory of design.”
Jared Peterson is a co-founder of Nuance Behavior where he blends Behavioral Science with the study of expert intuition to enhance decision-making and drive behavior change in tech and beyond. He also dedicates part of his time as a researcher at Shadowbox Training where he collaborates with experts in high-stake domains to illuminate and train expert decision-making and cue detection. He has a Masters of Behavioral and Decision Sciences from the University of Pennsylvania. If you are interested in connecting with him, reach out on LinkedIn!
Ellis Morlock is a behavioral science consultant at Behavioralize, where she applies the latest thinking, methods, and frameworks in behavioral science to solve real-world human and business challenges. She focuses on what motivates people in specific contexts and how to create positive and lasting behavior change. Ellis holds a Master of Behavioral and Decision Sciences from the University of Pennsylvania. If you are interested in connecting with her, reach out on LinkedIn!
References
1. Huxley, Thomas Henry. 1887. On the Reception of the ‘Origin of Species,’ in Darwin, Francis (ed.). 1887. The Life and Letters of Charles Darwin, Including an Autobiographical Chapter (London: John Murray), volume 2, pp. 179–204. Quote is on v2:p197.
2. Baruch Fischhoff, & Beyth, R. (1975). I knew it would happen. Organizational Behavior and Human Performance, 13(1), 1–16. https://doi.org/10.1016/0030-5073(75)90002-1
3. Hallsworth, M. (2023). A manifesto for applying behavioural science. Nature Human Behaviour, 1–13. https://doi.org/10.1038/s41562-023-01555-3
4. Chu, Y., & MacGregor, J. N. (2011). Human Performance on Insight Problem Solving: A Review. The Journal of Problem Solving, 3(2). https://doi.org/10.7771/1932-6246.1094
5. Vervaeke, J., Lillicrap, T. P., & Richards, B. A. (2012). Relevance Realization and the Emerging Framework in Cognitive Science. Journal of Logic and Computation, 22(1), 79–99. https://doi.org/10.1093/logcom/exp067
6. Klein, G., Phillips, J. K., Rall, E. L., & Peluso, D. A. (2007). A data-frame theory of sensemaking. ResearchGate
7. Chater, N., & Loewenstein, G. (2016). The under-appreciated drive for sense-making. Journal of Economic Behavior & Organization, 126, 137–154. https://doi.org/10.1016/j.jebo.2015.10.016
8. Focusing on the Problem — Part Two — ShadowBox Training. (2021). Shadowboxtraining.com. https://www.shadowboxtraining.com/news/2021/10/22/focusing-on-the-problem-part-two/
9. Peterson, J. (2023, February 27). The Science of Context — Behavioral Design Hub — Medium. Medium; Behavioral Design Hub. https://medium.com/behavior-design-hub/the-science-of-context-e6cc50252709
10. Cash, P., Vallès, X., Echstrøm, I., & Daalhuizen, J. (2022). Method use in behavioural design: What, how, and why? https://doi.org/10.57698/V16I1.01
11. Wikipedia Contributors. (2024, January 3). Wicked problem. Wikipedia; Wikimedia Foundation. https://en.wikipedia.org/wiki/Wicked_problem
12. Simon, H. A. (1996). The Sciences of the Artificial (3rd ed.). Cambridge, MA: The MIT Press.
