E2E Planning: What comes after the Bottleneck Economy?

Think demand planning during a period of constrained supply is difficult? Just wait for the period that follows. When supply once again exceeds demand, as it surely will, statistical forecasts trained on constrained-demand data will no longer work. How then will companies accurately forecast the future?

By: Daniel Sack, Olivier Bouffault, Marcel Sieke, Rajesh Shetty, and Slava Bazaliy

The holidays are upon us, and we can expect a continuation of markets in which demand exceeds supply. But what happens when supply chain constraints are relaxed? CPG, Fashion and Luxury, and Retail companies need to start upping their game so they can accurately plan ahead for when supply exceeds demand, for that day will surely come. And when it does, those companies that have not prepared themselves will lose sales and customers. For example, CPG players may find themselves deprioritized by e-tailors due to poor service levels or lose customers who switch brands and/or stores when the products they want are not on the shelves. This is even more relevant today, given large channel shifts in many categories from brick-and-mortar to e-commerce. Accurate forecasting is essential to manage the high volatility in demand in the years to come. BCG’s PLAN AI is designed to give companies the advanced tools they need to look ahead with confidence.

Planning when “true” demand is not easily measured

Modern end-to-end (E2E) planning frequently relies on decision-support and planning-collaboration tools offered in off-the-shelf software suites. These tools use sophisticated statistical forecasts and, often, machine learning to construct demand forecasts. The forecasts are then used to generate recommendations on quantities to produce or buy and where to allocate them, and to support other key decisions.

Prior to COVID, consumer brands had a long-standing bias toward overproduction and overbuying. This bias was driven by a myriad of internal factors such as growth aspirations, incentive structures, and process design. While this often resulted in unsold inventory and lower full-price realization, it did provide an easy training ground for statistical forecasting tools built into the off-the-shelf planning suites.

As supply is increasingly constrained, actual sales diverge from “unconstrained” demand signal which demand forecasts attempt to predict.

Decisions to overbuy and overproduce result in more supply than needed to meet demand, which means that “true” demand opportunity in the market is more easily observable, and stockouts are relatively limited. Statistical forecasts are capable of “uncensoring” or “unconstraining” historical-demand signals, but that becomes increasingly difficult as the proportion of stockouts gets closer to 100%. Our current supply-constrained period means that statistical forecasts will have insufficient training data to generate these unconstrained demand forecasts.

Statistical forecasting models included in most off-the-shelf planning software suites underestimate actual sales when trained on supply-constrained historical sales data — the more constrained, the greater the difference between predicted and actual.

At present, the inability to create accurate forecasts is less important since most companies are opting to produce/buy as much as possible to make up for pandemic-induced shortages. But going forward, once supply chains are again able to fulfill ordered quantities, produce/buy decisions will become far more important. The statistical forecasts underpinning these recommendations will be derived from planning tools that have been trained on systemically constrained demand data. They will, therefore, underestimate demand, which can lead to underbuying, underproduction, and tremendous amounts of uncaptured value. This, in turn, will result in out-of-stocks and lost sales, which will only slow the process of generating the “observations” upon which these statistical models depend — and further impede the recovery.

Building accurate demand models

In periods when the “unconstrained” data typically available during periods of overbuying and overproduction are absent, companies must draw on a more diverse and timely data set to generate recommendations. Adding data from external sources that may have correlations with demand, but are not supply constrained, can give valuable insight into what the “true” demand–the “counterfactual” — might have been. Complementing those external data sources with more real-time internal data, such as signals from short-term demand sensing models, can also bring additional value and speed to decision making.

One of the most powerful sources of demand planning information is embedded institutional knowledge — the human intelligence that resides among people whose job it is to make important decisions such as buying quantification, assortment localization, allocation, or replenishment. This intelligence, which companies have not typically been able to leverage, can be extremely useful when systematically captured and fed into machine learning models. In our experience, in fact, data generated from institutional knowledge are among the most, if not the most, predictive features in statistical forecasts.

Data scientists must do several things to leverage human intelligence fully:

1. Revisit the machine learning underpinning the statistical forecasts, putting in more explicit unconstraining mechanisms and allowing for explicitly injecting additional assumptions by a human planner.
2. Incorporate more granularity (particularly on the time dimension) to allow for more accurate extrapolation after stockouts, including the possibility of considering different future-demand scenarios.
3. Integrate the richer set of data mentioned in the previous two points.

Alternatively, data scientists can embrace Bayesian methods to build bespoke models enabling easier adjustments of macro assumptions.

PLAN-ing for the future

And they can turn to BCG’s PLAN AI. The PLAN AI framework addresses these three points, uses Bayesian methods to transform the end-to-end planning processes — and automates two critical tasks. First, it integrates with BCG Lighthouse, a high-frequency data and analytics platform that captures real-time consumer and market-level signals to anticipate future demand. Lighthouse can introduce into the planning process any number of diverse, external datasets to help forecast uncensored or unconstrained demand. Second, it leverages custom-made BCG products such as our patent-pending Ranking Application to integrate human intelligence into planning software and the algorithms on which they are built. With the two resources built into PLAN AI, BCG can help companies conduct highly accurate demand forecasting regardless of the supply/demand balance.

In a perfect world with perfect data, this forecasting would be automated by the out-of-the-box features of off-the-shelf planning tools. But, alas, the world is more complicated than that. Especially in times of great market volatility, the algorithms that drive demand forecasting must be tailored to the specifics of your business and must incorporate the most relevant external data and as much human intelligence as is available.

Here in the real world, where business challenges always seem to outpace the technical solutions we devise to solve them, PLAN AI — supported by BCG Gamma’s proven ability to customize forecasting algorithms — can provide a high level of confidence and flexibility companies need to successfully plan for the future.

In the meantime, and until supply chains return to their former strength, BCG can assist in “de-bottlenecking” the supply side by:

1. Simplifying product portfolios using our value-driven algorithms to optimize product mix and help companies move to the next step and create optimal produce/buy plans
2. Working closer with company suppliers and taking the integration of supply constraints to the next level in supply plan scenarios
3. Strengthening demand shaping by matching price and price/promo information with the supply and inventory situation to grow market share

Companies that begin now to strengthen their demand-planning capabilities will be able to accurately match supply to demand once supply returns — regardless of biased input data. In doing so, they can look forward to both reducing waste and maximizing value.