Advanced Demand Prediction powered by Machine Learning to create the future
Predicting the future
Following up on my last post, I’m going to talk about Isaac Asimov. Isaac Asimov’s masterpiece, the Foundation Series, is coming to Apple TV+ in 2021.
Beginning with the story of Hari Seldan, a psychohistorian who is able to predict the future of humanity, the Foundation series is also a thought-provoking work about the nature of “predicting the future” through the crisis of prophecy collapse that is brought about during the course of the series and how it can be overcome.
AI is about “prediction.”
Prof. Ajay Agrawal of the Rotman School of Management, University of Toronto, co-authored the book “Prediction Machines: The Simple Economics of Artificial Intelligence (Harvard Business School Press, April 2018).” In the book, he pointed out that the essence of AI at the present is to dramatically lower the cost of making “predictions” based on a huge amount of data. And, by reflecting that concept, he invented the method and workshop for handling AI in management, named “AI Canvas.”
AI is “information processing by prediction”. It predicts information that it does not have from information that it does have. This is the intrinsic value of modern AI.
“Prediction” can have its value in many areas. For example, how popular will a new gaming device be and how many orders will be received? What’s the availability of hospital beds next week? What’s the availability of hotel rooms in three months’ time? Accurate demand forecasting can help companies, regardless of industry, to avoid lost opportunities and to potentially generate huge profits.
How accurate are the demand estimates? How certain is it? How fluctuating are they? This is a problem that plagues corporate management every day.
What data from within your company will you use to accurately estimate demand? How do we reflect the signals that demand may change? What data from outside the company should be used?
From weather-related effects to influencer posts on social media, influencing consumer sentiment, there are a number of factors that influence consumers to buy products. Furthermore, the unexpected happens. For instance, home quarantine as a countermeasure to the COVID-19 outbreak has resulted in bread machines flying off the shelves around the world and, conversely, a sharp decline in demand for travel suitcases.
Like many other solutions, there is no silver bullet for forecasting demand. However, there are many ways to improve the accuracy of forecasts. In this article, we’re going to discuss the modernization of demand forecasting and some of the things to keep in mind.
Demand prediction and supply chain
Speaking of supply chain, demand forecasting and demand planning based on it play a diverse role. It goes beyond mere forecasting to optimize procurement, inventory, warehousing, shipping and pricing. The demand forecasting process can even apply for the detection of anomalies by monitoring the difference between forecasts and actual sales. Hence, demand forecasting should be as close to reality as possible throughout the supply chain. It’s important to ensure of the overall optimum.
How do you achieve the best demand forecast that is realistically possible? The answer, of course, is that the approach depends on the industry, the available resources and the objectives you want to achieve with your demand forecast.
Statistical approach
Traditional statistical forecasting approaches still work in stable markets. Therefore, they continue to be an important method. It can be done with basic software such as Excel. As long as you include time series data, modern data analysis tools can almost automatically generate models.
The base of the traditional method is historical performance data. Typically, to look at seasonality, you collect at least two years of sales data. The situation that occurred two or three years ago could happen again. However, the traditional statistical approach is in some ways vulnerable to changes in the increasingly complex market environment. It is difficult to predict when market saturation will occur. And of course it is also difficult to predict illogical changes in customer preferences. The approach is also not agile enough to support an increasingly personalized and individualized product strategy due to the impact of the Internet.
Modernization of demand forecasting will be critical. And increased demand volatility will accelerate efforts to use machine learning to improve forecasting accuracy.
Modernizing Demand Forecasting with Machine Learning
The first important part of modernizing demand forecasting is the integration of data with your own internal systems. State-of-the-art systems integrate seamlessly with a variety of systems including ERP, CRM and POS . In addition to taking into account seasonality and market trends, you also apply a variety of methods to improve the precision of machine learning-based demand forecasting.
Moreover, external data will be used. Historical financial and sales reports, marketing research, macroeconomic indicators, and social media signals (retweets, shares, spikes in followers), weather forecasts, such as information on campaigns, promotions, promotional activities. Big data, i.e., both structured and unstructured data, is handled to provide more accurate data-driven predictions.
Changes in weather can cause significant demand fluctuations, especially in seasonal products (from swimwear to umbrellas to fur coats), cosmetics, food, and automobiles. Weather has a noticeable impact on digital channels such as ecommerce. Generally, rain or snowfall increases sales due to so-called nesting consumption (which is purchasing behavior at home.) Unusal weather information, such as unusually high temperatures in winter, can also be taken into account to reduce errors in forecasting.
Forecasting demand for new products
The use of diverse data allows you to address the problems associated with forecasting the demand for new products, the so-called cold start problem. Traditional projections require a minimum of two years of sales data. For new products, however, there is no sales history. In that case, besides market research and expert opinion gathering, you can try to identify clusters of prior products with similar characteristics and lifecycle curves and to use those data as a substitute in order to make predictions.
In some industries, companies update their assortment every few months. For example, fashion companies launch new products at least twice a year, and they have to sell them quickly so as to be ready for the next collection. In such cases, the demand forecasting process must consider fashion trends, seasonality, and other external factors, along with data related to past collections.
Perceiving a change by demand sensing
There are ways to use demand sensing to capture real-time fluctuations in buying behavior, adjust existing forecasts and reduce near-time errors. For companies operating in a rapidly changing market, this approach can be of great help. You can extract daily signals such as the number of accesses and orders from POS systems and e-commerce sites, and keywords in social media, and detect increases or decreases in sales by comparing them with past patterns. And, You will automatically assess the significance of each deviation, analyze the impact factors, and suggest adjustments to the plan in the near term.
Notes on Machine Learning-Based Demand Forecasting
While modernized demand forecasting can provide more granularity and higher accuracy in forecasting than traditional methods, you may want to be aware of some points. Machine learning-based approaches continue to make predictions without human intervention, depending on how the system is built. However, quality checks by manual are vital to ensure robustness. In demand sensing, there are cases in which false signals are received. Therefore, it is important to take a measure of repetitive human in the loop during the rising phase.
Various techniques of machine learning can handle larger scale data and more complex combinations and patterns. Smart demand forecasting not only analyzes vast amounts of information, but also continuously re-trains the model to respond to volatility by adapting to changing conditions. Hence, more accurate and reliable forecasts can be made in complex business scenarios.
But that doesn’t mean all companies should immediately jump into complex, intelligent technology. You can start with small enhancements to existing systems to address problems that are difficult to solve by using traditional methods. For example, the machine learning module could be used to change short-term planning to a data-driven approach, while leaving long-term forecasting to the conventional statistics, etc.
No matter how smart a predictive solution is, the key decisions depend on people. It is human beings who evaluate the performance of predictive model to ensure that it continues to maintain adequate accuracy in a complex and changing business environment. As written earlier, It’s also important to involve a variety of experts in the forecasting process like taking a measure of Human In the loop. You will be able to identify and plan for a better future for our businesses only by making the best use of both AI and human intelligence.
Asimov’s masterpiece “Foundation,” which is mentioned at the beginning of this article, also contained a significant theme of what people should be like if the future were to be predicted. It’s not just about predicting the future while fully exploiting the potential of machine learning-based modernization. You have an insight into the future, and you will make it. We would like to value such a way of being.
Appendix
For an article related to demand forecasting, see “using blockchain to reshape the supply chain against a disaster.” You may also be interested in this one.
