Demand-Planning Framework in the Gaming Industry
The global video game market has increased by nearly 15% in the past four years. As it has grown, it has created a huge, untouched market into which few companies have ventured. One of the hallmarks of this new market is demand volatility. As such, effective supply chain management — particularly demand forecasting — can help companies gain competitive advantage to win these new customers. By improving their ability to correctly predict demand, gaming enterprises are better positioned to maintain synergy with their upstream and downstream suppliers and customers. This, in turn, enables them to deliver superior customer value, and to do so at less cost to the supply chain. Boston Consulting Group has created a framework to improve your ability to predict sales and optimize your marketing spend.
Advantages of Demand Forecasting
Gaming companies are very familiar with illegal websites that allow free download of games, and with online e-commerce websites that provide huge discounts on game prices. They may be less aware of additional factors that make it difficult to correctly anticipate customer demand. Intensive data crunching and insight generation helped us gain a deep understanding of the following factors that can impact sales:
1. Dearth of quality data: On average, 750 games are released every year across varied consoles (PS4, Xbox, Switch). The number falls by 15% for every console, if only games with reasonable reservations (>1,000) and standard editions are considered. This gives the model very few data points to uncover trends explaining variability in demand patterns.
2. Lack of promotion/marketing data: Game sales are boosted by the offline marketing and promotional events run by the store. However, the majority of gaming companies do not capture this data into their systems and, thus, there is a significant loss of useful information that could otherwise be leveraged to forecast demand.
3. Evolving consumer trends: Technological advancements, such as Virtual Reality (VR) and Augmented Reality (AR), are creating a revolution in the gaming industry. To fully capture the dynamic nature of this niche segment it is imperative to have a stronghold of industry experience and business understanding.
4. Buzz vs. Hype: In the vast ocean of digital content, it is difficult to distinguish between buzz (organic mentions) or hype (artificially induced awareness) created by paid online reviews. The framework should assign the proper importance to these varied forms of digital content, and use it to estimate future demand.
A Battle-Tested Framework to Predict Demand
Recently, Boston Consulting Group (BCG) transformed the demand planning methodology of one of the world’s largest video game retailer, which operates over 5,700 stores across 14 countries offering a variety of new and pre-owned video gaming consoles. The methodology, developed on the back of advanced ML models, is able to predict sales 42 days before the launch of the game. This insight into market demand has helped the company save $14 million that would otherwise have been lost due to poor market sizing based on archaic demand models.
Data preparation and exploration
Six primary sets of data sources were mined to stitch together a 360-degree game view. Some of these data sources were created at a customer level. For example, is a customer who has honored his or her reservations in the past more likely than a customer with poorer past conversion history to honor a new reservation they’ve made for the game of interest. This metric is initially created at a customer level and then aggregated (by taking average across all customers) to a game level.
A few sources, such as genre, #reservations, and cancellation rate, are directly calculated at a game level:
Note that the majority of the video game retail shops allow customers to reserve a copy of a new game before the game itself launches. When the game is released, customers can then procure their reserved copy (known as “pick up” in data language). These reservation trends are very useful in forecasting the sales of any game
Crunching the Data: The EDA frameworks employed include Decision Tree, Trellis Plot, Factor Analysis, Bi- variate analysis, and Correlation. The 12 most significant variables in forecasting video game sales are:
Key EDA findings include:
- Reservation Trend: First 2 days since launch are the most crucial in determining inventory requirements.
2. Cancellation Trend: Games with higher cancellation rate have a lower pick-up rate.
3. Customer Past-Reservation Behavior: A customers’ current conversion rate is correlated with his conversion history.
4. Age of Reservation: Customers who reserve closer to launch date are more likely to pick-up.
5. Tuesday vs Thursday launch: In general, Tuesday launches require more inventory (as a multiple of reservations) due to having to cover two weekends post-launch.
Machine Learning Models
Only standard edition games that released post 2016 and had at least 1,000 reservations were considered for the analysis (~600 games). The training dataset consisted of all the games that released in the period between January 2016 to December 2018. The validation dataset included the period between January 2019 and July 2019.
The sales pattern of a video game does not follow a time-series or a cyclic curve. There is, however, a seasonal component, with demand increasing significantly on days such as Black Friday and throughout the holiday seasons. Accordingly, we developed a Random Forest Regression model, instead of a typical time-series-forecasting model, to handle this unique trend.
The dependent variable for the model was total 28 days sales of a game, while the independent variables were all the factors described above. Special care was taken while designing the model to handle outliers, missing data, correlation, overfitting, under fitting etc. (through parameter tuning).
The model had a MAPE of approximately 12% on validation datasets and performance was not skewed towards one cohort of games, which suggests the model was quite robust.
Model Validation (Pilot Testing)
To test the model’s efficacy against the client’s heuristic algorithm, it was predicted on a random sample of 58 games.
Key observations from the graph:
1. The fact that most of the model predictions are centered around the value of 1.0 indicates that majority of the predictions are closer to the actual sales value.
2. The sparse distribution to the right and left of 1.0 indicates that the model under- or over-predicts for very few games.
Predictions for Live Games
Convinced by its accuracy on a wide range of games, the model was put into production and used to predict anticipated sales of upcoming games. The algorithm picked up games from the data base that are releasing in the next 42 days and predicted potential sales. Twenty-eight days after a game’s release, the model’s performance was again measured against the actual sales and the client’s algorithm. For the model’s 12 predictions over the last 2 months, the BCG algorithm outperformed the client’s algorithm in 7 instances and matched client’s algorithm’s performance in the remaining 5 instances.
Estimated annual incremental profit: Approximately $14 Million
Business Impact of Accurate Forecasting
Merchants require that purchase orders be locked in approximately 21 days before the launch of a game. This can vary depending on the game and the merchant’s relationship with the supplier. Additional orders can be placed at any point and take on average 10 days to reach stores (assuming supplier has units available, which is typically the case).
Having just one 21-day order window, however, did not provide sufficient flexibility to adjust the number of units ordered when there were sudden changes in reservations and/or cancellation trends. This resulted in repeated over- or under-ordering of units. To correct this, we moved away from the traditional 2-order window method and instead deployed a 6-order window method and increased the forecast duration by 14 days.
This methodology is applicable in all scenarios where the concept of “reservation” holds good, such as when estimating the sell-out probability of a concert, estimating movie performance upon release, or forecasting the number of enrollments for an online or offline course.
Conclusion
Given the highly volatile nature of the gaming industry, market sizing is essential to prevent huge financial losses. Demand forecasting is central to such sizing, though it can be difficult to perform accurately due to a number of factors. Accuracy can be improved, however, by capturing the right set of variables and choosing the right modeling technique. Variables that improve potential game sales include reservation and cancellation trends for that game, past conversion trends of customers who have reserved the game, the reservation window size of the game, and the game genre.
Furthermore, having only a single order window is highly detrimental to demand forecasting because the inflexibility it causes can lead to significant losses due to under- or over-ordering. By opting instead for a multiple-order window method while increasing the forecast duration, demand accuracy can be improved and new markets won.
