Shake It Up — The Fantastic World of Forecasting
Mastering Monte Carlo Forecasting: Transforming Spreadsheets into Powerful Project Delivery Tools
This article aims to introduce key concepts on the topic of Monte Carlo Simulation, and will be followed more complex and technical articles in a series I’m naming Shake It Up!
Pinning dates to project or feature delivery is perhaps the least exciting part of the work. Yet, when talking to sponsors and potential users, this is one piece of information you cannot avoid sharing. In this series I will share with you my experience in learning about Monte Carlo Forecasting, and then how I turned spreadsheets into forecasting tools.
The Average Mistake
What do you do to forecast your projects?
If you look at historical data, and draw an average, or a trend line of some sort, then you are partially doing it right. But you can do better.
To understand this, we need to take a little trip to number-land, specifically to the values of central tendency. These are:
- the Mean,
- the Median
- and the Mode.
We call them “values of central tendency” because all three are used to indicate a moderate behavior, or even a compromise, within a set of data. Here’s a brief explanation about them, one by one.
The Mean is the value you get when you sum up all the data, and divide the sum by the number of values in the data set. In other words, the Mean is the Average.
The good thing about the average is that it allows equal participation in the outcome to every element in the data set, irrespective of its size and position. It is the democratic center, without the need of sorting the data set. The bad thing with the average is that very small or very large numbers influence the outcome “unfairly”. We can reinforce our analogy on democracies, but to be less controversial consider the popular example of Jeff Bezos going for a drink in a pub. If I had to be in the same bar asking people about their salary while Mr. Bezos is enjoying his lager, and draw an average, my results would show me that people in that bar are, on average, millionaires.
That is why other values of central tendency are often used to provide a more complete picture of what is happening in the data. The Median, for example, is the mid-most number in the ordered data set. Just sort the set and pick the middle number. That’s the median.
The Mode, then, is the most popular number in the data set. If you are dealing with the Hollywood of datasets, where all values are equally popular, then you might either have multiple modes, or no mode at all.
When you use an average, or a trend line, for your forecast, as Sam L. Savage says in “The Flaw of Averages”, your forecast will be wrong on average.
Welcome to Monte Carlo
The most exciting thing about the future is that it is yet to happen.
That said, when an event happens, the possibility of it happening again in the future are not to be ignored. This is what I think is so useful when you bring in historical data to your forecasting.
Monte Carlo simulations bring unpredictability and possibility together. The basic premise is the following:
If we had to look at past events and randomly change and combine the steps building towards them, how would this information inform our insights on a similar future event?
If the above premise is still confusing, the Monte Carlo analogy comes in handy.
Monte Carlo is a European hub for casinos, and when you think of casinos you think of randomly picked numbers, slot machines, and dice.
If you roll a fair 6-sided dice you will notice that, in the long run (say after 100 rolls), all 6 values would have appeared (outcome) almost the same number of times, and so your probability of guessing the next number would be 1/6.
When you roll two dice, and plot the sum as an outcome, you realize that some outcomes are more popular than others. For example, while there is only two ways how you can get 2 as an outcome (Dice A : 1 + Dice B : 1, and vice versa), 5 is a little bit more advantaged in that aspect (1+4,2+3,3+2,4+1). In that sense, the probability of getting 5 as an outcome is higher than that of getting 2.
Now imagine you repeat the experiment with 20, 40, 300 dice. Imagine if each of the dice has a different number of sides, or different values on it. But the basic mechanics remain: if you roll the dice together, and add them up as an outcome. The emerging become even more interesting.
These patterns can be best seen on a probabilistic distribution. In easier terms, for each of the values in the outcome you ask the following question: How often has this outcome occurred in my test run?
Back to the Future
So how can randomly rolled dice help in forecasting?
I like to think of dice as variables that have a known scope of behavior. Just like a 6-sided dice can produce a number between 1 and 6, a random pick from collection of historical throughput data, for example, can produce any of the values within the collection.
Remember, though, that the outcome depends on the inputs. In the 6-sided fair dice, all six numbers shared the same probability of them occurring as an outcome, because each shared the same level of participation. If you run a Monte Carlo simulation on a single data set, the probability distribution in the long run will match the distribution pattern of the numbers in your data set. That is why Monte Carlo simulation is mostly useful when you combine different variables.
Taking throughput forecasting as an example, what would be other independent variables that can provide a more realistic outcome from a simulation?
Think of team capacity, bugs produced, urgent incidents, scope creeps. If you manage to represent these events into data collections (or collections of data as impacting the throughput data) and combine it with the simulation’s outcome, then you will be able to have some surprises!
In the words, once again, of Sam L. Savage, the thousands of permutations that make up a Monte Carlo simulation are similar to when we set up a ladder before we climb it. We shake it up. Each of the angles produced in the shake are angles that can happen, and the wisest of us put their first step up that ladder only once all (randomly generated) angles provide assurance that the the ladder is firm and safe to climb.
So what is the probability?
Finally forecasting capability is best paired with clear communication.
Communicating a forecast needs to produce not only the outcome (for example, delivery date), but also a context of risk or probability. The future is not guaranteed, and the more information presented around a forecast, the better the decision taken out of that forecast could be.
That is why very often an outcome of a Monte Carlo simulation is communicated with a percentage probability. “There is 85% probability that the project will be delivered in 10 weeks”.
The idea here is not to sound robotic, but to clarify that there is no thing like a certain forecast. Also, the “other 15%” can open up an interesting discussion with stakeholders, such as, how bad can it go and, perhaps, what can we do to avoid worse outcomes. Keep in mind that farther a forecast is projected, the higher the uncertainty around it and therefore the riskier it would be. The best bet is to use forecasting to trigger the right discussions.
What’s next?
With the foundations on Monte Carlo simulations laid out, I will share how I have built forecasting tools from spreadsheets through which more lessons about Monte Carlo simulations will be revealed.
Matthew Croker is a Team Process and Data Coach, specialized in the software development industry. Through his work he helps teams within companies focus on learning how to work best together by analyzing their data, simplifying the setup of processes and creating habits that boost their productivity.
