Dynamic Pricing using Machine Learning

• Have you ever looked up a plane ticket, and decided to buy later, only to come back and see the price has gone up a few hundred dollars?
• Have you ever experienced the same with concerts, hotels, or games?
• Have you ever thought about why Uber ride costs vary day to day?

All of the above is done using Dynamic Pricing techniques.

Dynamic pricing is adjusting prices based on external elements such as demand, supply, market, and customer behavior. It involves setting flexible prices that can change frequently to optimize revenue, maximize profits, or achieve other business objectives. Dynamic pricing allows businesses to optimize their pricing strategies based on real-time market conditions and customer behavior, helping them to remain competitive and maximize their revenue potential.

Generally speaking, dynamic pricing can be applied to most verticals and industries and is commonly used in e-commerce, retail, travel and hospitality, ride-sharing, sports, and entertainment industries. However, there are other sectors, such as highly regulated markets and commodities, that a single vendor couldn’t control the supply and demand. Therefore, it will not be as beneficial as highly scarce supply markets like airfare or concert tickets.

In this article, we will review the use of machine learning in dynamic pricing. At first, we look at how traditionally dynamic pricing is done. Next, we review some of the strategic applications of machine learning methods in dynamic pricing, then we review some of the models used and a few papers published in each case.

Traditional methods:

Traditionally, dynamic pricing involved employing several experts trained on the job with lots of trials and errors and constant A/B testing to adjust the prices of goods or services based on a multitude of factors such as business strategy, supply and demand, competition, market appetite, time to market, and customer sentiment. These traditional methods involved several Mathematical and Statistical methods and approaches to calculate the risk, predict the outcome and measure the success.

If we want to list traditional activities involved in dynamic pricing from a Machine Learning lens, we can introduce them as follows:

1. Monitoring the Sales performance and pricing: This is the obvious step apart from any pricing plan a team has to take to know a complete picture of the top of the revenue funnel. This generally involves documenting and analyzing sales data daily and creating statistical reports, including various moving averages, trajectory calculations, etc. The same data creates a baseline for performance comparison after implementing the dynamic pricing strategies and models and applying the necessary adjustments to gain better results.
   > This is our baseline stochastic process identification/definition in statistical pattern recognition language. As a result of this step, we will know the descriptive identifiers of our random process, including measures of central tendency, dispersion, and shape, e.g., mean, std, variance, and skewness. Even we can go to the extent of deciding what the closest random process that models our sales is. Spoiler alert, Poisson is one of the most used processes to model sales, as well as ARIMA, and to some extent, Markov chain and Gaussian process.
2. Monitoring the Market: After looking into the mirror and investigating ourselves, we must look out of the window into the world to understand how everyone else looks like. As a monopoly in the market is scarce, and we always have some form of at least indirect competition, we have to analyze duopoly and oligopoly situations and gain a clear picture of the competition. Therefore, we must gather and analyze as much information as possible about the market conditions, including our competition, direct or indirect, total market historical sales data, customer demand and sentiment, and other relevant factors.
   > This will be closely similar to the previous point. We are defining our problem domain and learning more about its statistical features. However, when working on our competitor data, that we don’t have access to their actual sales details, we can use methods like Monte Carlo simulation to generate estimates of their significant statistical features.
3. Predefinition of Price Factors: After statistically understanding our sales, and market patterns, along with statistical identifiers, we can calculate which factors are the most influential in changing our major identifiers, for example, shifting our mean or skewing our distribution, etc. These factors can be anything internal or external such as our demand patterns, competition price changes, inventory levels, seasonality, and focused segments.
   > This step will be instrumental later for our feature engineering. You see the language is changing towards machine learning quite quickly.
4. The business strategy and policy on pricing: Companies generally have their own pricing philosophy and principles. So, the next step will be setting forth those strategies considering the data and analysis results from the previous steps and making sure those strategies are sound on paper. Usually, business strategies evolve around product offerings and product types. Therefore, not all a business’s products should follow the same strategy. In addition, these rules might be based on predefined thresholds, such as increasing prices when demand exceeds a certain level or reducing prices during periods of low demand.
5. Applying the policy and monitoring: After defining the pricing strategy and processes, it’s time to apply those policies and generate the new price package for our products. This also can include A/B schemas of price implementations, which sometimes involve rebranding and other retail-specific approaches. Once the prices are set, we must constantly monitor our market and sales patterns to change and adapt accordingly. To monitor and record feedback from the market, we usually define markers and analysis windows to get meaningful information and decide correctly.
6. The last step overlaps and loops us back to the first step, so these steps are not a one-time process. We analyze, devise a plan, apply it, monitor it, analyze the results, revise it, and apply it again.

If we go back and look at the above iterative steps, we can see how closely they resemble the Expectation Maximization (EM) process in machine learning. We set our expectations, run through the process, review to maximize a posteriori, set again, and run again.

It’s important to note that traditional dynamic pricing approaches are generally assumed to be repetitive and seasonal Wide Sense Stationary (WSS) processes. Therefore, artifacts and impulses in the market can temporarily throw them off the optimum path. Still, with short-term artifacts, these models emerge to a new optimum point after some time. However, if, for any reason other than temporal artifacts, the spectrum of market dynamics changes drastically, e.g., the Covid-19 Pandemic, we must re-analyze the emerging market thoroughly and create a new model.

As we mentioned, historically, many statistical methods have been used for dynamic pricing. However, we’re going to go over a few of those methods that have an actual counterpart in the machine learning world.

• Regression-based: This group includes many models and methods, such as Linear regression and polynomial regression. For years many of these methods have been, and are still, used to model the relationship between pricing variables and demand. These models help predict optimal prices based on historical data.
• Time series analysis: Time series forecasting techniques, such as the famous autoregressive integrated moving average (ARIMA) or exponential smoothing methods, capture seasonality, trends, and patterns in pricing and demand data.
• Bayesian-based: Bayesian regression is utilized to incorporate prior knowledge and update pricing models with new data. These models provide a probabilistic framework for pricing decisions.
• Clustering-based: Clustering algorithms, such as k-means, segment customers based on their purchasing behavior, preferences, or other relevant characteristics. These clusters can inform pricing strategies tailored to different customer segments.

Machine Learning in Dynamic Pricing

Nowadays, machine learning is one of the crucial tools in dynamic pricing strategies, enabling businesses to optimize pricing decisions based on real-time data and market conditions. We can approach the problem of Dynamic Pricing using ML from many angles. The following list is some of those angles following the same order as the steps that we introduced previously.

• Forecasting the demand: Based on our sales data, and market trends, we can use machine learning methods to forecast future demand accurately. This will, later on, be handy in deciding on the optimum price.
• Analyzing the competition: Looking out of the window, as we mentioned, we have to monitor our competition, and machine learning can help in analyzing our competitor’s pricing strategies, their promotion trends, and the general market dynamics. Although we won’t have access to our competition’s details of sales data, we will be able to fit models on their pricing and learn their pricing strategies. Predicting the competition’s pricing is essential to have educated decisions about our price trends and increase our revenue.
• Dynamic pricing: Based on the output of the previous steps and all of the statistical information, we can use machine learning methods to change our product price dynamically. We can create complex schemas using deep learning or reinforcement learning, constantly learning and adapting to new market elements.
• Personalizing the prices: This can be called an ultimate dynamic pricing solution, which can be used in some commerce sectors, like loan offerings but is not generally applicable to every vertical, especially mass product retail. Based on our customer profile, purchase, and browsing history, we can generate a personalized price using these algorithms, mostly one-time promotional prices in abandoned carts. This approach allows businesses to tailor prices to individual customers, improving customer satisfaction and driving sales.

One of the examples of the application of machine learning is the 2009 work of Levina et al., in their paper, they discuss the dynamic pricing of a monopolistic company, with no competition, of their perishable products while learning about customer demand. The company uses an aggregating algorithm to predict demand based on sales data and adjusts its pricing policy accordingly. The methodology is independent of specific distributional assumptions. The paper presents numerical experiments that demonstrate the robustness of the learning procedure to deviations from the model. The experiments show that an informed company, aware of strategic consumer behavior, performs better than an uninformed company. Furthermore, the frequency of learning and the uncertainty in valuation affect the performance of the learning procedure.

Dynamic Pricing with Online Learning and Strategic Consumers: An Application of the Aggregating…

Several annual conferences focus on revenue optimization, and one approach is dynamic pricing. For example, the Institute for Operations Research and the Management Sciences (INFORMS) has an annual Revenue Management & Pricing conference. In their 2017 conference, they had a dynamic pricing competition. Ruben van de Geer et al. have published a paper discussing the results of the Dynamic Pricing Challenge conducted during the conference. Participants submitted algorithms for pricing and demand learning, and their performance was evaluated in simulated market environments. The study found that algorithm performance varied significantly across different market dynamics, highlighting the complexity of pricing and learning in the presence of competition. The winning algorithm, “logit,” performed well in the oligopoly competitions, while other algorithms had varying performances across oligopoly and duopoly competitions. The paper analyzes competitors’ sales, prices, and revenue generation, highlighting the importance of targeting different customer segments and adapting pricing strategies based on market conditions.

Dynamic pricing and learning with competition: insights from the dynamic pricing challenge at the…

There are also various works utilizing Bayesian models; for example, in 2019, Argawal et al. proposed a dynamic pricing and learning framework where a seller sets prices and advertising schemes to maximize revenue. The seller provides signals to buyers about the quality of the product. Using Bayesian persuasion, the paper formulates the problem and aims to design an online algorithm that adaptively learns the optimal pricing and advertising strategy. The main result is an efficient algorithm with a regret bound. The algorithm achieves this bound without assumptions on the demand function but assumes linearity and specific properties of the valuation function. The paper also presents improved results for the case of additive valuations. The proposed algorithm discretizes the price and type space and optimizes the upper confidence bound on the revenue function. The computational bottleneck is solving a high-dimensional program at each time step, but the paper shows an efficient method exists to solve it.

Dynamic pricing with Bayesian demand learning and reference price effect

Another example that utilizes the Bayesian concepts is the work of Lie et al., published in 2020. They explore the coordination of dynamic pricing and inventory management strategies for a retail firm selling a durable product in a volatile market with unknown demand distribution. The firm uses a Bayesian approach to learn about the demand distribution over time. The study formulates the problem as a stochastic dynamic program and identifies a state-dependent base-stock list-price policy as the optimal strategy under certain conditions. A dimensionality reduction approach is employed to simplify the computation and implementation of the model. The analysis also considers the effects of demand learning on the optimal policy and extends it to cases with unobserved lost sales and additive demand. The paper concludes by outlining limitations and suggesting future research directions, including alternative approaches like reinforcement learning and empirical validation of the model using real-world data.

Dynamic pricing and inventory management with demand learning: A bayesian approach

It is an excellent time to segway toward reinforcement learning, but let’s first look at a dynamic programming approach using the same Q learning concept. In 2019, Dutta investigated a pricing approach for online ticket sales in the context of game tickets. Their paper combined dynamic programming and empirical data analysis to determine demand functions for university football game tickets. The study finds that dynamic pricing strategies generate higher revenues than fixed pricing. Additionally, it identifies the optimal capacity allocation of tickets based on game intensity. High-intensity games are more profitable when all tickets are sold by the club, while less popular games require a harder optimization challenge for revenue maximization. The paper also discusses the application of dynamic pricing in the real world, with examples of popular secondary market online retailers like Ticketmaster and Vivid Seats. The mathematical models used in the study involve a standard Bernoulli model of dynamic programming for profit maximization and estimation of demand functions. The results show that the average ticket price increases as the game day approaches, and the dynamic pricing strategy generates the maximum revenues. The optimal capacity allocation between dynamic pricing and fixed pricing depends on various factors, such as the percentage of tickets sold using optimal fixed pricing and the total capacity.

Capacity Allocation of Game Tickets Using Dynamic Pricing

As an example of the reinforcement learning we can mention the work of Kastius, et al. in 2021. They studied the performance of two reinforcement learning algorithms, Deep Q-Networks (DQN) and Soft Actor Critic (SAC), in different market models for dynamic pricing. Their study focuses on duopoly and oligopoly settings. The results show that both algorithms provide good results, with SAC performing better than DQN. The paper also highlights that competitors can force RL algorithms into cooperation under certain conditions without direct communication. The experiments are conducted using a logistic model to represent customer behavior and various manually tuned competitor strategies. The findings suggest that DQN is more reliable in situations where fixed price strategies are optimal, while SAC is more adept at handling complex strategies. The performance of the algorithms is evaluated based on their expected long-term rewards compared to optimal strategies. The paper also explores the performance of the algorithms against deterministic and non-deterministic competitor strategies. Overall, the study demonstrates the potential of RL algorithms for dynamic pricing in competitive markets.

Dynamic pricing under competition using reinforcement learning - Journal of Revenue and Pricing…

Another research using deep reinforcement learning (DRL) for eCommerce was done by Liu et al. They present an end-to-end framework for dynamic pricing on an e-commerce platform using DRL. The framework models the dynamic pricing problem as a Markov Decision Process (MDP) and makes several contributions compared to existing DRL-based algorithms. These contributions include extending the problem from a discrete set to a continuous price set, introducing a new reward function called the difference of revenue conversion rates (DRCR), and addressing the cold-start problem of MDP through pre-training and evaluation using historical sales data.

The framework is evaluated through offline experiments using real dataset from Alibaba Inc. and online field experiments conducted on Tmall.com. The offline evaluation compares different reward functions and different parameter settings for the DRL models, showing that DRCR performs better than revenue and that a DQN model with K=100 performs best. The online experiments involve pricing luxury products during a markdown season and pricing fast-moving consumer goods (FMCGs) on a daily basis. The results demonstrate that the DRL-based pricing policies outperform manual pricing by operation experts in terms of revenue and profit conversion rates.

The paper discusses the challenges and limitations of the proposed framework, such as the need for more training data for low-sales-volume products, pricing different products together to form marketing strategies, and incorporating additional features for pricing under specific scenarios.

Overall, the paper presents a novel approach using DRL for dynamic pricing in e-commerce, achieving significant improvements over manual pricing strategies in both offline and online experiments.

Dynamic Pricing on E-commerce Platform with Deep Reinforcement Learning: A Field Experiment

As we mentioned earlier, one of the leading industries that constantly exploit dynamic pricing is the ride-sharing industry. In 2019, Garg and Nazerzadeh reviewed driver-side payment mechanisms in ride-hailing marketplaces like Uber and Lyft. They focused on surge pricing and compared the effectiveness of multiplicative surges (historically used) and additive surge (proposed by Uber). Their study presents a dynamic stochastic model to analyze driver earnings and their strategies to maximize earnings. The authors show that multiplicative surge is not incentive compatible in a dynamic setting and propose an incentive-compatible pricing mechanism based on the additive surge. The paper provides empirical comparisons using data from a ride-hailing company, showing that additive surge balances the value of short and long-surged trips better than a multiplicative surge. The analysis demonstrates the impact of surge mechanisms on driver earnings and highlights the benefits of the additive surge in practice.

Driver Surge Pricing

Another paper that focuses on ride-sharing is the 2022 work of Huang, et al. This paper proposes a deep reinforcement learning framework for dynamic pricing in ride-hailing platforms. It uses a soft actor-critic algorithm and formulates the dynamic pricing problem as a continuous action Markov Decision Process. The reward function incorporates the order response rate and the KL divergence between supply and demand distributions. The proposed method outperforms baseline algorithms regarding order response rate and total revenue. Experimental results show that methods considering drivers’ schedules and future demand achieve higher revenue and order response rates. The learning curves demonstrate that modified reward functions lead to faster convergence and higher stability. The proposed method is sensitive to the hyperparameter λ1, which controls the contribution of the order response rate. The analysis of policies reveals that the proposed method achieves better demand-supply balance than the baseline method.

Deep Reinforcement Learning-based Trajectory Pricing on Ride-hailing Platforms | ACM Transactions…

Before concluding this article, we must mention the important Sports and Entertainment (S&E) ticket industry. You may have seen all the outcries against Ticketmaster’s dynamic pricing systems a few years ago when Billie Eilish’s $199 tickets suddenly became $399, and Bruce Springsteen’s went up to $699. Unfortunately, not many papers are published on S&E vertical, and based on a survey published by Bouchet et al. in 2016, at least 30% of Sports and Entertainment companies in the US use daily price adjustments, and over 50% have some automation in their systems.

One of the papers that looks at dynamic pricing as a revenue management (RM) technique in the Sports and Entertainment industry is the work of Phumchusri and Swann. They also mention that this sector has received less attention in the literature than in the travel industry. The authors develop dynamic pricing models for S&E tickets based on stochastic demand and incorporate Bayesian updates to learn about demand uncertainty. The study uses data from a major performance venue in the U.S. to test the models and evaluate their performance under different scenarios.

The findings suggest that demand learning is particularly beneficial when initial demand estimates are inaccurate, leading to 8–11% potential revenue improvements. The study also reveals that underestimating the base demand rate causes less revenue loss than overestimation, indicating a risk-averse approach’s advantage. Additionally, the research highlights the importance of considering price and time sensitivity in ticket demand.

The paper suggests several possible extensions, such as incorporating imperfect information on price and timing sensitivities, exploring pricing models for substitutable products, and investigating dynamic pricing models with limited price adjustments. The authors also suggest the need for empirical research to examine the short-term and long-term effects of dynamic pricing in the S&E ticket industry and encourage further research in this area.

DYNAMIC PRICING WITH UPDATED DEMAND FOR THE SPORTS AND ENTERTAINMENT TICKET INDUSTRY

In this article, we explored the concept of dynamic pricing, which involves adjusting prices based on factors such as demand, supply, market conditions, and customer behavior. Dynamic pricing allows businesses to optimize their pricing strategies in real-time to maximize revenue and remain competitive.

We also discussed traditional methods of dynamic pricing, which involve expert analysis, statistical calculations, and monitoring of sales performance and market conditions. We highlighted the iterative nature of dynamic pricing, where businesses analyze, devise a plan, apply it, monitor the results, and make necessary adjustments.

Machine learning is presented as a crucial tool in dynamic pricing strategies. We outlined several applications of machine learning in dynamic pricing, including forecasting demand, analyzing competition, dynamically changing product prices, and personalizing prices based on customer profiles. Machine learning algorithms can help optimize pricing decisions based on real-time data and market conditions.

And at the end, we referenced various research papers that explore the use of machine learning in dynamic pricing. These papers discuss different approaches, such as regression-based models, time series analysis, Bayesian models, clustering algorithms, and reinforcement learning. The examples highlight the potential of machine learning in improving pricing strategies and revenue optimization in various market environments.

Overall, we emphasized the importance of dynamic pricing in maximizing revenue and staying competitive in industries such as e-commerce, retail, travel and hospitality, ride-sharing, sports, and entertainment. Machine learning techniques offer new opportunities for businesses to leverage real-time data and market insights to make more informed pricing decisions.

Thank you for taking the time and reading this article!