Estimation Methods in Policy Making & Product Management

The Magic of Estimation

Imagine you’re a magician, not pulling rabbits out of hats but numbers out of thin air. You’re not just any magician; you’re an estimator, conjuring up data-driven insights that guide crucial decisions. Welcome to the whimsical world of estimation, where the art of the guesstimate reigns supreme.

In both public policy making and product management, estimation is the secret sauce that transforms ambiguous ideas into actionable plans. Whether you’re crafting a new government policy or developing the latest tech product, the ability to estimate potential impacts is your crystal ball, revealing the future in a haze of probabilities and possibilities.

Sounds like a click-bait introduction? Well, read on and find out more.

Enter the Fermi Estimation

Our story begins with a renowned physicist, Enrico Fermi, who had a knack for making educated guesses. Picture this: the first atomic bomb has just been detonated in the New Mexico desert, and Fermi is observing the blast from a safe distance. Armed with nothing but a piece of paper and his sharp mind, he performs an estimation that would go down in history.

As the blast wave from the explosion approaches, Fermi quickly drops a small piece of paper from his hand. He observes how far the paper is blown away by the blast wave. Knowing his distance from the explosion and using the displacement of the paper, he performs a quick mental calculation to estimate the bomb’s yield. This seemingly simple act, rooted in keen observation and logical reasoning, birthed what we now call Fermi estimation.

How Fermi Did It

Fermi’s method involved a few key steps:

1. Observation: He carefully observed the displacement of the paper.
2. Assumptions: He made assumptions about the energy required to move the paper a certain distance.
3. Calculation: Using basic physics, he estimated the energy released by the bomb.

Fermi knew the approximate distance from the blast center and assumed that the energy of the blast wave decreased with the square of the distance. He also estimated the energy required to move the piece of paper a specific distance. By combining these pieces of information, he made a rough but remarkably accurate estimate of the bomb’s yield.

This technique of breaking down a complex problem into smaller, more manageable parts, making reasonable assumptions, and performing logical calculations is the essence of Fermi estimation. It’s not about precision; it’s about being directionally correct and gaining insights quickly.

Applying Fermi Estimation

Fermi estimation isn’t just a historical curiosity; it’s a powerful tool used in various fields, including product management and public policy. It allows us to make educated guesses about complex problems by breaking them down into smaller, more manageable components.

Let’s say you’re tasked with estimating the number of teachers in Singapore. Here’s how you might approach it using Fermi estimation:

1. Population of Singapore: Start with the known population of Singapore, which is approximately 5.7 million people as of recent estimates.
2. School-Aged Population: Estimate the proportion of the population that is school-aged. Let’s assume that about 20% of the population is between the ages of 5 and 18, the typical school-going age range.
3. Class Size: Estimate the average class size. Let’s assume an average class size of 25 students.
4. Teachers per Class: Assume that each class has one teacher.
5. Number of Classes per Teacher: Consider that teachers might handle multiple classes, but for simplicity, let’s assume each teacher handles one class to start.

By breaking down the problem, you can approximate the number of teachers needed:

1. School-aged population = 5,700,000 × 0.20 = 1,140,000 students
2. Number of classes = 1,140,000 / 25 = 45,600 classes
3. Number of teachers = 45,600

And for a fact, I do know that Singapore has about 33,000 teachers.

The Whimsical Art of Being Approximately Right

Now, I wasn’t exactly correct in my estimation. But the key is to get the ballpark right, and not the ball exactly into the hoop.

Both policy makers and product managers must embrace the whimsical art of being approximately right. In both fields, over-precision can be a trap. Spending too much time trying to get exact numbers can lead to analysis paralysis, where decisions are delayed and opportunities are missed.

Instead, the goal is to be directionally correct. In product management, knowing whether a feature will generate $1 million or $10 million is more important than knowing the exact number down to the last dollar. In policy making, understanding whether a policy will have a positive or negative impact on employment is more valuable than knowing the exact number of jobs created or lost.

I’m not making a statement about how policies or products are measured or tracked as part of the evaluation cycle , or how policies or products have no need to be data-driven — I’m talking about how decisions at the margin are made at the Senior Management heuristic level.

Product Management: The Land of Innovation

Shifting our focus back to product management, product managers are often tasked with questions like, “How many users will this new feature attract?” or “What revenue can we expect from this product launch?” Again, these questions are big and complex, but Fermi estimation provides a way to tackle them.

Take the example of estimating revenue from a new product. A product manager might start by estimating the number of potential users, the percentage of those users likely to buy the product, and the average revenue per user. By breaking the problem down into these smaller pieces, the product manager can arrive at a reasonable estimate of the total revenue.

So, any tips? Here’s a good toolkit for any product manager:

1. Break Down the Problem: Start with the big question and break it down into smaller, more manageable parts. Each part should be something you can reasonably estimate.
2. Use Available Data: Leverage public data, historical data, and expert opinions to make your estimates. The more informed your guesses, the more accurate your overall estimate will be.
3. Embrace Uncertainty: Accept that your estimates won’t be perfect. The goal is to be directionally correct and to use estimation as a tool to guide decision-making.
4. Iterate and Refine: Estimation is an iterative process. As you gather more data and insights, refine your estimates and update your conclusions.
5. Communicate Clearly: When presenting your estimates, be transparent about the assumptions and data you used. Clear communication helps build trust and ensures that stakeholders understand the basis for your estimates.

Remember, when making broad company-level or broad-level decisions, it is far more important for your estimates to be directionally correct, because resource allocation and strategic planning are typically big high-level moves. So don’t sweat your details too much.