Zero-Based Demand: An Innovative Model Framework for Forecasting Travel Demand during COVID… and Beyond
Written by: Serena Zou, Alex Fernandez, Aaron Arnoldsen, Mike Beyer, Andrew Simon and Arun Ravindran
COVID-19 dealt the airline industry a stunning blow it’s still recovering from. In 2020, airlines experienced more than 75% drop in international demand and a nearly 50% drop domestically¹, putting it in the record books as the worst year ever in the history of air travel. Today, a full two years later, emerging COVID variants are still forcing workers to take sick leaves, as a shifting patchwork of quarantine policies makes travel from one country to another a veritable obstacle course of changing rules and regulations. And as if this weren’t enough, airlines and other travel companies have discovered that the years of historical data and relatively predictable demand patterns are now much less useful for predicting future demand. No longer able to rely on extrapolations of past customer behavior or on “gut feel,” many airline planners are searching for ways to predict travel demand amid what can seem like an endlessly volatile market.
As BCG has demonstrated with Lighthouse by BCG, it is possible to forecast the future without relying solely on the past. Developed specifically for the travel demand market, BCG’s zero-based demand (ZBD) forecasting model essentially sits atop Lighthouse by BCG, utilizing high-frequency data and a dynamically weighted unobserved component model (“DW-UCM”) engine to provide maximum flexibility to the project of understanding future demand. ZBD consists of three components: An initial module configured to predict if a travel segment is open or closed (e.g., due to government restrictions), a second module that predicts near-term recovery, and a component that forecasts long-term recovery using a probabilistic model.
These elements of the DW-UCM framework enable the forecasting engine to compute both near-term movements in demand and long-term scenario planning. Based on numerous real-world client engagements, ZBD has shown a superior ability to forecast airline demand amid market volatility, and when historical data has lost its predictive power. Looking ahead, companies will be able to leverage this multi-horizon, high-granularity, modular dynamic-modeling framework to respond more quickly when the next great shock comes.
ZBD Use Cases: The Airlines and Beyond
Demand forecasting is a fundamental building block for nearly every business organization. Whether used for capacity planning, supplier selection in supply chain optimization, or campaign design in marketing, companies rely on these predictions for critical decision-making. Getting those decisions right can make or break a company.
Never has demand forecasting been more critical than since the outbreak of COVID-19. But this need comes not just during pandemics. In recent years, markets have been rocked by everything from hurricanes, floods, and droughts to earthquakes and volcanic eruptions (such as the one in 2010 that grounded the European airline industry). So too, other non-natural shocks such as 9/11 or the Great Recession of 2007–08 have suddenly upended standard ways of predicting demand.
The ability of ZBD to accurately forecast demand amid market turmoil can help the airline industry continue its post-pandemic recovery. But the benefits of such forecasting can go well beyond the airlines themselves and extend to tangential sectors, which include (but are not limited to):
- Hotel Revenue and Labor Management: Demand forecasting plays a key role in the ways that hotels manage both their revenue and their staffing. By leveraging outputs from the ZBD forecast model and airline-travel patterns, hotel owners can improve their existing forecasting algorithms to create a more accurate and time-sensitive foundation for overall revenue- and staff-management systems.
- Network Planning: For industry sectors with strong competition, such as the airline industry, slight improvements in network planning can lead to significant cost reductions and/or revenue increases. Demand forecasting can help airlines to optimize their fleets, increasing capacity on routes with greater demand, while decreasing it those with lesser demand.
- Air Cargo: This sector has also been hit heavily by COVID-19. Forecasting demand can improve air-cargo capacity planning by estimating the belly space that will be available in commercial flights during the upcoming months.
- Tourism Marketing: Government tourism departments can look to ZBD predictions to improve their revenue management.
- Car Rental: Due to its high correlation with the airline industry market, the car rental industry can benefit from more accurate forecasting of passenger demand.
- Oil Price Prediction: With the airline industry being one of the largest consumers of oil, changes in demand among airline and other travel industries can be used to forecast changes in oil prices.
The Perennial Benefits of Granularity and Understandability
The benefits of using ZBD come not only in periods of uncertainty. Regardless of market conditions, ZBD’s ability to predict demand at high granularity and with high frequency — and to do so quickly and without the need for a team of highly skilled (and highly paid) experts — can always help companies optimize resource allocation.
BCG believes that the combination of “Human plus AI” is the best way to bring the benefits of AI to the world. Rather than creating black box technologies that are impenetrable to business users, we use tools such as SHAP analysis and BCG’s FACET to help clients understand the drivers of model outputs. By incorporating ZBD into our proprietary visualization tool DAIS, we provide business users with a global view of how market demand is evolving. This transparency enables users to add their human insight and experience to further improve the value of the model outputs.
The Prime Ingredient: Forward-looking, High-Frequency Data
ZBD is an airline-specific forecasting model that sits atop the highly acclaimed Lighthouse by BCG, introduced in 2020. Traditionally, demand-forecasting models have relied on historical data to capture consistent, reoccurring patterns such as trend and seasonality. Since this approach to forecasting essentially became obsolete with the appearance of COVID-19, BCG turned to close-to-real-time (high-frequency) data sources. These sources include consumer-mobility trends, economic activity tracking, web traffic and search trends, and government and publicly available spending and unemployment figures. These data sources are at the core of Lighthouse by BCG, which also utilizes both historical and forward-looking data generated by its own internal models. These models make it possible to track everything from disease spread to trends in unemployment, consumer mobility, and consumer spending.
The advantage of using Lighthouse by BCG data is three-fold:
- Many of these data sources are travel-demand drivers. Some, such as total number of cases and deaths obtained from the epidemiological forecasts, played a more important role at the height of the pandemic. Others, such as consumer mobility trends and the Consumer Activity Index, can have more impact during normal times. Lighthouse by BCG’s forward-looking orientation can capture market signals early and thus, provide more reliable forecasts — even with uncertainties.
- Lighthouse by BCG allows us to use 2020 and 2021 data in training, instead of excluding them. This, in turn, enables ZBD to maintain its relevancy during both normal and abnormal periods to provide short-term, medium-term, and long-term forecasts.
- The broad geographic data coverage on which Lighthouse by BCG is based helps the AI in ZBD learn from different countries worldwide. The resulting global “collaborative learning” environment helps the model understand the travel patterns of countries that are recovering faster than the others.
The Recipe: A Flexible Modular Modeling Approach
To handle rapid changes due to events such as secondary infection waves, local outbreaks, and COVID lockdowns, or to any other natural or economic influences that can affect the airline industry, ZBD adopts a dynamically weighted, unobserved component model (DW-UCM) structure that addresses three key questions:
- When will domestic and international air travel be possible?
- How will domestic and international air travel demand recover?
- What will be the long-term, new-normal demand levels for domestic and international air travel?
For each of the questions, we built several ML models, each of which features a variety of engineering techniques.
The first component in the DW-UCM structure, the restriction model, tracks both virus-evolution forecasts and changing government restrictions. It uses this information to forecast when a specific country pair will open their borders to each other and resume mutual international flights. The virus-evolution forecast is one of the multiple datasets within Lighthouse by BCG. The government-restriction data is extracted from the Oxford Tracker dataset, which combines different containment and closure policies such as restrictions on internal movement or stay-at-home requirements. We built the restriction model using a binary classification structure, the output of which includes the probability and given data of the opening of a country pair. This framework also allows users to construct various scenarios using different probability thresholds.
To address the issue of demand recovery, we used an XGBoost regression model to create short-term demand forecasts, and a Random Forest model for medium-term forecasts. The short-term model leverages airline booking, searching data and various external Lighthouse by BCG data to capture recent changes in the market and provide strong predictive power amid volatile demand.
The medium-term model uses Lighthouse by BCG’s CAI forecast suite to capture potential downturns linked to pandemic progression and consumer activity. The CAI forecasts are generated from a nonstationary Markov chain model that conforms to index movements over time. In this way, the near-term demand model uses a UCM in which unobserved components such as seasonality are dynamically weighted to improve the overall accuracy of the forecasts.
To prepare the long-term forecasts, the model looks to long-term CAI forecasts. Studying the correlation between the CAI values and airline demand (together with more external signals from Lighthouse by BCG such as Point-of-Interest data), we can use ZBD to predict how the market will evolve directionally.
In keeping with BCG’s Human + AI orientation, the model also allows users to put in their estimates for optimistic, normal, and pessimistic states. As a result, ZBD can accommodate the human uncertainties of various business situations as it conducts scenario planning.
Using a flexible model structure such as this provides a few obvious advantages:
- A dynamic demand-forecast model can incrementally rebalance each component’s weight upon the addition of new information, thus improving overall accuracy.
- For volatile travel demand during uncertain times such as during a pandemic, the model’s pattern can vary significantly for differing forecasting horizons — rather than trying to use a one-size-fits-all model. Using numerous features and algorithms enables each model to perform at its best for each given period of time. By learning across multiple model components and times, the demand-forecasting engine can balance weights along different time granularities and capture changes in market behavior.
- ZBD’s modular model structure makes it easier for end-users to track performance and tune models. This is particularly important during uncertain times, when conditions can change quickly and more detailed attention is required to assure constant, reliable forecasts. Since each module is relatively independent, users can track down problems more easily: Even if one model breaks, other models won’t be directly impacted.
- When the world enters a period of post-pandemic normality, end-users can easily update date-input features of selected models. Lighthouse by BCG data sources such as the epidemiological forecasts, for example, could be replaced by forward-looking booking data and CAI forecasts.
A Brief Caveat or Two
It should be noted that we have made some assumptions when constructing the ZBD model. Furthermore, we note that there is some degree of bias in the data, such as the data the model uses to predict border-restriction openings. We have observed that when the R0 value was declining, governments in general tended to open borders. But there are exceptions. In Brazil and the U.S., the airline market has been kept alive despite high R0. In countries such as Australia where the R0 has remained low, borders remained closed.
Initially, the opening of a border was a one-time event: once the border opened, it would remain open for the foreseeable future. But as new COVID variants continue to appear, borders that were once open may close again, only to reopen at some future time. Because of ZBD’s modular structure, these inconsistencies can be addressed without affecting short-term recoveries. For the medium-term forecasts, using a stochastic model enabled us to predict an increasing probability that new vaccines would be developed to treat emerging variants. This has proved to be the case.
For data scientists interested in using this zero-based demand-forecasting framework, we should note that, unlike traditional ML models, ZBD does not always perform better with more data. It can sometimes be the case that more data points — those that do not represent current status — may simply introduce extra noise. Some degree of customization in the training data may be required, together with giving more weight to recent observations.
ZBD can provide more reliable forecasts than traditional ML models during uncertain times by capturing multiple waves and providing more accurate directional forecasts. But ZBD will not always provide 100% accuracy and will require careful monitoring and, when needed, equally careful adjustments.
An added benefit of BCG’s “Human + AI” orientation is that ZBD was developed to be “explainable AI.” As such, business users are more able to conduct monitoring and make adjustments without input from domain experts or data scientists.
The Path Forward
Travel demand is typically driven by business needs, vacation plans, and activities such as visiting family and friends. Forecasting the trends and seasonality of these demands could once be based on historical data. But, alas, unforeseen events (such as global pandemics) can create volatile markets in which it is fruitless to try to plan ahead by looking back. ZBD addresses this reality by ingesting high-frequency, forward-looking data, and a flexible, modular model structure to create outputs that help companies act faster than their competitors the moment a new disruption unfolds. But even during periods of relative normalcy, ZBD’s dynamically weighted modular framework allows companies to consider these new data sources to gain a much more accurate picture of the market. A company’s ability to correctly anticipate demand is, perhaps more than any other factor, fundamental to its success.
Reference:
