A Good Question is Better than a Great Answer

In Douglas Adams’ science fiction series, The Hitchhiker’s Guide to The Galaxy, a group of pan-dimensional beings design a supercomputer known as Deep Thought for the sole purpose of providing the “Answer to the Ultimate Question of Life, the Universe, and Everything.” After 7½ million years of computation and thought, the ancestors of the original question askers, trained and selected from birth to be the ultimate answer receivers, return to Deep Thought to hear the answer. After an extensive amount of hesitation, Deep Thought declares that the answer to The Great Question is 42, describing that this answer seems meaningless because the original askers never actually knew what The Great Question was.

Whether you’re a scientist trying to form a hypothesis to be experimentally tested, a business leader seeking the ever elusive product/market fit, a thinker suffering from existential angst or simply a person who wants to be a better conversationalist, the act of forming a good question is one that, despite its foundational importance in our lives, often goes overlooked. A good question is better than a great answer, but this is only half of the story. Many of us fall victim to one of most fundamental processes underlying cognitive bias, something known as question substitution. When faced with a difficult question, we often substitute a related, but easier question with more readily available information without ever knowing that the substitution took place. Others follow the pan-dimensional beings in suit. We answer or seek answers to questions that haven’t actually been asked. Both of these phenomena can be a source of constant dissatisfaction. The answers we surface through introspection or receive from others never quite seem to quench the urge that launched us towards their pursuit.


The purpose of this post is to provide a set of strategies and tools that can be used to circumvent subtle cognitive errors that are present in all of us. Our goal is to start asking questions before providing answers. This ultimately means getting to the heart of our problems by asking questions that are better equipped to drive satisfactory answers in a given context.Part 1 is about understanding this context and building a strategy to stay within it while forming questions. Part 2 will be specifically about cognitive errors and the strategies we use to overcome them.


Questions of Context

Before jumping into the realm of question substitution and cognitive bias, it is important to take a step back and think about the purpose of a given question. Our questions can operate as logical knives, reducing our answer space and applying structure to complexity, or they can give us new vantage points, allowing us to expand our answer space. Some questions are meant to provoke thought — rhetorical tools not intended to be answered — while others are entirely description-oriented with only a handful of possible answers. Some can be incredibly effective tools used to guide others through our trains of thought. Many of our questions imply compliance with a certain discipline, a set of rules, or a body of definitions. “Is the man guilty of burglary?” and “How should he be punished?” imply accordance with the set of rules, definitions, and processes that constitute the American Legal System when asked in a United States court of law. In this context, an answer of “Yes, he is guilty because I don’t like the tone of his voice” is completely irrelevant. Being aware of the context in which a given question is asked should inform the types of answers we expect.


The Subtleties of Hat Wearing

As product developers, we are often thrown into the abyss. We are sailing in a fog of unknown constraints and market needs in an effort to produce something that people will love and use. As engineers, we are faced with the chaotic, unforgiving forces of nature that will take all of our unaddressed assumptions, chew them up, and throw them right back in our faces. Our goal as engineers is to leverage these forces in order to construct objects to withstand them and create utility from them. Depending on the hat we choose to wear on a given day, these may compose our respective environments. The questions we choose to ask are like divining rods. They can surface information beneath layers of chaos like water beneath an Australian desert, but with one major caveat. Our divining rods only reveal water in an environment that is palatable for hat-wearers relevant to that environment.

All environments may be inextricably linked. The mechanics of a given material may influence the price of a product which will affect the end user. The properties of quantum mechanics will underlie the mechanics of that given material. The best information diviners, however, are the ones who can ask questions in a way that addresses the environment at the appropriate level of resolution and context. They understand the hat they wear and they can form questions to serve the needs of people wearing similar or different hats. They can identify answers relevant to their given context.


Our goal as information diviners is to ask questions in a way that addresses the environment at the appropriate level of resolution and context.


We simply don’t have the bandwidth to passively absorb data at all levels in the hierarchy, from the granular to the abstract. Ultimately, the only way to make sense of our respective environments, if we are truly pioneering new paths, is to build a priori systems that allow us to selectively choose information that is relevant or valuable to us. The questions we choose are the funnels for this information, the a priori systems we put in place.


The Minimum Viable Question

If we are avid information diviners, one of the things we are routinely terrified of is something known as scope creep. It refers to the insidious tendency for projects or ideas to grow beyond their bounds and budget, and it is often a result of poor communication between parties, lack of initial success criteria or project objectives, and a weak change control system. Scope creep is both a result of and a metaphor for poor context awareness in question formation.

What does this mean? In the product world, imagine our goal was to design something to help blind people navigate a crowded city more effectively. We’ve done our homework and we’ve found that it’s very difficult to add more handheld devices to our target user’s repertoire — a white cane and a guide dog are already widely used and quite sufficient for immediate, local navigational needs. These local navigational needs include avoiding objects and people in the immediate vicinity. In this imaginary scenario, let’s say what is really needed is a global navigation system — getting from point A to point B — that minimizes sensory bandwidth needed for local navigation. Let’s say our success criteria are as follows:

  1. Our target user is able to use this system to get from arbitrary point A to arbitrary point B across three streets and through a busy market.
  2. They have to do (1) while using their existing tools.
  3. This whole system has to be just as safe as it would have been without our product.

In accordance with these goals, is it reasonable to build our own web mapping service akin to Google Maps? What about the addition of a beautifully fading LED on the white cane? Should we create navigational shoes?

Having awareness for the scope at which we are trying to answer a question or meet a set of criteria should allow us to decide which questions to consider. There are an infinite number of questions and hypothesis about any given subject. Having the context awareness to selectively choose the questions that will give us the most information about our system at the appropriate level of resolution with the least amount of effort is key. Building a new web mapping service might help, but it certainly is not lower effort than using an existing one. LED fading might be useful for getting the attention of other people in our user’s vicinity, but it doesn’t directly fit in with our above goals. Navigational shoes are a good start. What additional advantage do they provide over our user’s existing tools? Will they occupy the necessary sensory bandwidth that our user requires to stay safe?

There is a term in the product development world known as the minimum viable product or MVP that refers to this concept for product definition and objectives. To generalize and apply this towards better questions, let’s use the term minimum viable question or MVQ.


A minimum viable question is one that will give us the most information about our system at the appropriate level of resolution with the least amount of effort.


In order to begin our journey towards becoming MVQ Jedi, it is important to consider three things.

  1. Is there an implicit set of rules that this question is intended to be answered in accordance with?
  2. What level of resolution is this question speaking to?
  3. Is the intent of this question to converge on a smaller set of answers or to diverge into a larger answer space?

Is there an implicit set of rules that this question is intended to be answered in accordance with?

This may seem obvious, but in practice these rules are quite subtle. These rules are the reference frames for our questions. Without a reference frame, most of our questions are basically meaningless. “Is she moving?” doesn’t actually mean anything unless we specify motion relative to a given reference frame. When we ask, “Is she moving?,” we implicitly mean “Is she moving relative to the earth’s inertial reference frame?” Many of our most basic concepts are relative to some reference frame, whether that frame is an inertial reference frame, a gravitational field, a period of history, or the United States Constitution. Forming our questions with this frame made explicitly clear is the best way to avoid the trap of slipping into other implicit reference frames and losing ourselves in that transition. If we climb high enough up the question hierarchy towards the abstract or deep enough into the strange realms of physics, we may see these reference frames blend together or even disappear. What is “Quality?” may hold the same answer for a Tibetan monk as it does for an auto mechanic. “What is the speed of light in a vacuum?” has the same answer whether the light comes from a satellite moving through a vacuum at 30km/s as it does from a rocket in a vacuum moving at 60km/s. These types of questions — those that generally hold relevance across reference frames — are the most fundamental and highly treasured of our knowledge systems. It is important, however, to keep reference frames attached to our questions for as long as possible on this climb. This allows us to compare answers. When they happen to be the same across reference frames, we can begin to develop new fundamental truths.


What level of resolution is this question speaking to?

Level of resolution ties back to our informational divining rod thought. This was the rod’s ability to find water that is only palatable for hat-wearers from a specific environment. Understanding and making the scope of our questions clear is the main point here. A butterfly flapping its wings in San Francisco may, through some incredibly complex chain of events, affect the weather in Tokyo, but it has little relevance in a conversation about building technology to actively remove CO2 emissions from a Japanese factory. A question involving the purpose of a product or the vision of a company should explicitly address the stakeholders or relevant hat-wearers at play. “From a marketing perspective, what is the best way for our company to position itself?” may be better than “What is the best way for our company to position itself?”


Is the intent of this question to converge on a smaller set of answers or to diverge into a larger answer space?

This is rarely made explicit, but it is important to consider how the answer space will be affected by a given question before asking. In a brainstorming session, it might be more useful to ask divergent questions in the beginning. These are questions that expand the answer space, shining light onto new avenues of thought. The type of question that a hypothesis seeks to answer is convergent in nature. Following the scientific method, we typically seek a binary result. The hypothesis was true or it wasn’t. If we are in scientific method mode, we may ask “What is the temperature at which liquid x boils at 1 atmosphere of pressure?” Our hypothesis may be that liquid x boils at the boiling temperature of water at the same pressure. Our experiment should land us on one side of the hypothesis fence or the other. It wouldn’t make sense in scientific method mode to ask “what is the meaning of liquid x?” This is a divergent type of question with many paths that each branch off into their own respective thought trees.


Our goal is to get to the heart of our problems by asking questions that are better equipped to drive satisfactory answers in a given context. By asking the above three questions before verbalizing our own questions, thinking about the minimum viable question, and looking at the world through the eyes of an information diviner, I believe we can begin to find answers that truly do quench the urge that launched us towards their pursuit. Stay tuned for Asking the Right Question: Part 2 which will focus on question substitution, answering unasked questions, and building strategies to overcome some of our most pervasive cognitive errors.