Why do we always underestimate time?
And what can we do about it
“How long do you think it will take?”
A question we often tried to answer when we are planning what to do. We estimate almost every day whether we notice it or not, from estimating how much time we should plan for our travels, how long it will take for us finish our work and do we have enough time to run our errands. So how come with all that practice, we are still bad at estimating how much time we need. One would argue that it doesn’t matter if we can estimate correctly, but if we can’t estimate correctly we won’t be able to plan realistically and that would make achieving our goals more difficult. In this article, we will go through some reasons why we underestimate and how we can improve our estimation.
We have a habit of underestimating time because it is natural for us to do so. Psychologists labelled this phenomenon “Planning Fallacy” and while it is important to understand how the underlying psychology works, we are not going to discuss that in details here because there is a lot of content out there already. However, we will discuss why planning fallacy happens and more importantly, what we can do to combat those biases.
Biases everywhere
Biases are at the centre of planning fallacy, but it is not as simple as we may think because there are many different biases at play here and they all contribute towards the overall tendency to underestimate how much time we need. Note that I am focusing on individual work estimation as we need to understand that before we can extend the logic for groups.
Optimism bias
We are inherently optimistic and while that is an endearing quality that enables us to work through difficult situations, it is not helpful when we are trying to estimate how much time we need. Our innate optimism encourages us to think in best-case scenarios but unfortunately, that’s usually not the case in reality and whenever we face an obstacle, we are shooting ourselves in the foot as we didn’t plan for that adequately. This optimism bias has a long-lasting effect even when we are made aware of our previous mistakes in the estimation, we are still likely to insist that our current estimate is realistic.
Self-serving bias
Optimism bias describes our view on externalities, but the other side of that coin is how we see ourselves. We naturally attribute success to our abilities and actions and we tend to attribute failures to external factors like dependencies on others and bad luck. This skewed view encourages us to estimate aggressively because it is a way for us to boost our self-esteem and show that we have a high output and can get a lot of work done quickly.
Memory bias
So the reasonable follow-up question is wouldn’t we just learn after we failed to estimate correctly the first time? While that might be obvious and we hope that we are learning from our mistakes, the matter of fact is that our memories are extremely unreliable and we just don’t recall details very accurately and combining that with self-serving bias, we end up recalling a biased set of events in the past and mistakenly used them to come up with an equally biased view of the future.
Focalism
What if we do have an accurate recollection of past events? Unfortunately, we will still be affected by a cognitive bias known as focalism, which describes the tendency for us to give too much weight on a single piece of information and ends up making an inaccurate judgment. Focalism can come in two ways, the first is when we focus too much on the future and ignore past events and learnings. Also, we are likely to magnify the impact of certain factors that might not be as important as we think. It can be a new skill we learnt, a recent hire to the team, or even past wins. While they are all positive indicators that we are likely to be able to deliver, it will be a mistake to think that any of them will make a significant impact to the outcome as many other factors influence the overall deliverables.
Bringing them together
Here are just a few key biases that affect how we estimate and most of the time we are facing a combination of them which makes estimating a single task difficult enough. In addition to that, we tend to see every task independently and when we start planning multiple things together, the difference between reality and our plan multiplies and we end up being way off from what we initially estimated. All these biases work together and no wonder our estimates end up wildly off.
Focus on the fact
So what can we do to combat our natural tendency to underestimate? There are many techniques out there that can help, from adding a multiplier to your estimate to breaking down your tasks into small tasks that are easier to estimate. Depending on your preference, you can find a technique that works for you, but I want to explore the underlying thread that makes those techniques work. Ultimately, we are trying to avoid being affected by these biases and it seems like the key factor that underpins most techniques out there is to shift the focus from how much time we feel we need to how much time we think we will need. The difference might be subtle, but by bringing in some objectivity, it helps us to switch from deciding based on emotions to deciding based on data. If we can detach our estimates from who we are, we should be able to take an ‘outsider’ view which gives us back some perspective.
While it might seem straight forward, trying to focus on concrete data is not easy or intuitive for us. In a way, we avoid measuring our estimates against reality because we don’t want to face the fact that we are bad at estimating. However, I hope by shedding some light on the biases that underlie this problem, we can learn to accept that it is natural for us to underestimate and the best path forward is to lean in and learn from our mistakes. Until we can build up the courage and discipline to learn to face reality, we will always be chasing an unrealistic plan and it will make achieving our goals more difficult than it needs to be.
Finding the right tool
It’s not enough to just have the intention to change, we also need tools to help us get there. As I couldn’t find a tool that I like, I decided to build Escapement for this purpose. Escapement is a tool that helps you start on that journey, it helps you track your actual time spent against your initial plans. It can provide granular insights on how well you are estimating your tasks and remind you to adjust your estimates going forward. It also offers many other interesting insights about your productivity to help you achieve your goals. Here are a few key insights that I have gotten from Escapement.
Estimation difference converges over time
As we would expect, our estimates are widely off when we are just starting out but luckily we know that they start to converge as we spend more and more time on similar type of tasks and eventually it converges close to 0%. As you can see for me, even after almost 300 hours spent on building Escapement, I am still underestimating by 8%. While the goal is to get that difference as close to 0% as possible, knowing that systematically I am off by 8% meant that I can adjust my plan easily next time I work on the product.
It is important to be honest with ourselves and while I hate to over or underestimate how much time I need, it is much better to face the reality and make adjustments than to waste time and energy working on a plan that is unrealistic to start with.
Shorter tasks get done quicker
This chart shows how quickly we get things done split by the estimated time required. The hypothesis is that we tend to finish smaller tasks quicker and this insight certainly supports our hypothesis. Here we can see that almost 45% of tasks that takes 1 hour or less is completed within 24 hours, that percentage drops as we increase the task duration. On the far right, we can see that over 35% of tasks that takes 2 hours or more is completed after 11 days. This shows that we are generally bad at finding a big enough block of time to focus on long tasks and they tend to take much longer to complete. So next time when we are planning to do something, breaking it down to smaller chunks give us the best chance to finish quicker.
It’s worth noting that it’s not about finishing tasks with a really short cycle, but it’s about finishing them on a target cadence. If you are planning a week in advance then you should expect tasks to be completed within 7 days.
Knowing is half the battle
These are just two of the many insights that Escapement helps me understand my work. Of course, your mileage will vary, but without trying to understand what you are doing, it is hard to improve how we spend our time. At the end of the day, it is not about getting a nice chart from a tool, it is about understanding our strengths and weaknesses and plan accordingly. Taking control of your time is not easy to do, but I hope Escapement makes it a bit easier for you to do so.
