Hey everyone 👋, the main focus of this article is to explain how we can implement a Dynamic Price Optimization Agent Using Reinforcement Learning.

Pricing is one of the most critical aspects of any business, and with AI-powered techniques like Reinforcement Learning (RL), companies can adapt their prices dynamically based on real-time market conditions.

In this blog, we will cover the fundamentals of RL in pricing, compare it with traditional AI techniques, and guide you through an implementation that optimizes pricing decisions using machine learning and data-driven strategies.

Let’s dive in! 🚀

💡Understanding Dynamic Pricing

Dynamic pricing is a strategy where prices change in real time based on factors such as demand, inventory, competitor pricing, and customer behavior. This technique is widely used in industries like e-commerce, transportation, and hospitality.

Why AI for Dynamic Pricing?

AI enables businesses to make data-driven pricing decisions, adapting to market conditions instantly. Reinforcement Learning (RL) stands out as an advanced AI technique because it can learn optimal pricing strategies through continuous interactions with the environment.

📈 Comparing AI Techniques for Dynamic Pricing

Before implementing RL, let’s compare it with two other AI techniques: Regression Analysis and Time Series Analysis.

Conclusion: While Regression and Time Series models provide useful insights, RL is the best choice for real-time price optimization due to its ability to adapt dynamically.

⚒ How Reinforcement Learning Works in Pricing

Reinforcement Learning (RL) is a type of machine learning where an agent learns by interacting with an environment. In dynamic pricing, the RL agent adjusts prices based on customer reactions and receives rewards in the form of revenue. Over time, it learns the best pricing strategy to maximize long-term profits.

Key Components of RL in Pricing:

• State: Current market conditions (e.g., competitor price, inventory level)
• Action: Price adjustment decision
• Reward: Revenue generated from the pricing decision
• Policy: Strategy that the agent learns to optimize pricing decisions

Important RL Hyperparameters

1. Alpha (α) — Learning Rate 🧠

Think of alpha as how much attention you give to new information versus old information.

• Suppose you’re learning a new way to solve a math problem. If you completely ignore your old method and only use the new one, you might make mistakes because you haven’t fully understood it yet. This is like having a very high alpha — you learn fast, but you might make a lot of errors along the way.
• On the other hand, if you stick too much to what you already know and don’t accept new ideas, your learning becomes slow. This is like having a very low alpha — you’re more stable, but you don’t improve quickly.

So, alpha controls how quickly the RL agent updates its knowledge. If alpha is too high, it keeps changing its decisions too much. If it’s too low, it learns very slowly.

2. Gamma (γ) — Discount Factor ⏳

Gamma helps the RL agent decide how much importance to give to future rewards.

• Imagine you have two options:

1. Get $5 now (short-term reward)
2. Get $20 next week (long-term reward)

• If you really need money right now, you’ll take the $5 immediately — this is like having a low gamma, where the agent focuses only on immediate rewards.
• However, if you’re thinking long-term and planning for a bigger gain, you’ll wait for the $20. This is like having a high gamma, where the agent is willing to be patient and wait for better rewards in the future.

So, a high gamma value makes the agent focus on long-term gains, while a low gamma makes it only care about short-term rewards.

3. Epsilon (ε) — Exploration Rate 🎲

Epsilon controls the balance between exploring new things and using what we already know.

• Let’s say you always go to the same restaurant because you know the food is good. But what if there’s a much better restaurant nearby that you never tried? If you never explore, you’ll never find the best option!
• However, if you try a new restaurant every single day, you might never settle on the best one and waste time eating at bad places.

This is exactly what happens with an RL agent:

• A high epsilon means it explores a lot and tries new actions often. This prevents it from getting stuck with a bad strategy.
• A low epsilon means it mostly sticks to what it already knows and stops experimenting.

So, we usually start with a high epsilon (lots of exploring) and slowly decrease it over time so that the agent settles on the best strategy.

📌Final Summary of RL Hyperparameters

Alpha (α) = Learning speed →

• High α = fast but unstable learning,
• Low α = slow but steady learning

Gamma (γ) = Future thinking →

• High γ = plans long-term,
• Low γ = focuses on short-term

Epsilon (ε) = Exploration →

• High ε = tries new things,
• Low ε = sticks to what it knows

By adjusting these values properly, we can make our RL agent learn smarter and make better pricing decisions over time.

👨‍💻 Let’s see the Implementation

To demonstrate how RL can be applied to pricing, we used a dataset containing sales, inventory levels, competitor prices, and other market variables.

retail_store_inventory.csv

💰 Steps to Implement RL for Pricing:

1. Data Preprocessing: Load sales data and clean missing values.

1. Feature Engineering: Extract key variables like demand trends and seasonal effects.
2. Reinforcement Learning Model:

• Use Q-Learning, a popular RL algorithm, to train the model.

class QLearningAgent:
    def __init__(self, actions, alpha=0.1, gamma=0.9, epsilon=0.1):
        self.q_table = defaultdict(lambda: np.zeros(len(actions)))
        self.actions = actions
        self.alpha = alpha  # learning rate
        self.gamma = gamma  # discount factor
        self.epsilon = epsilon  # exploration rate
        
    def choose_action(self, state):
        # Epsilon-greedy action selection
        if np.random.uniform(0, 1) < self.epsilon:
            return np.random.choice(len(self.actions))
        state_action = self.q_table[state]
        return np.random.choice(len(self.actions)) if np.all(state_action == state_action[0]) else np.argmax(state_action)
            
    def learn(self, state, action, reward, next_state):
        # Q-learning update rule
        predict = self.q_table[state][action]
        target = reward + self.gamma * np.max(self.q_table[next_state])
        self.q_table[state][action] += self.alpha * (target - predict)

• The agent experiments with different price points and observes the effect on demand.
• Over time, it learns the pricing strategy that maximizes revenue.

4. Training & Evaluation: Test the model in simulated market conditions and analyze revenue improvements.

Results (According to the implementation done by me👀):

• Baseline Revenue (without RL): ~$12.8 per transaction
• Revenue After RL Optimization: ~$19.6 per transaction
• Price Accuracy: 82% correct price predictions
• Revenue Accuracy: 86% alignment with optimal revenue

These results indicate that RL significantly improves pricing decisions compared to static methods.

🧩 Challenges and Considerations

While RL offers significant advantages, it comes with challenges:

• Computational Cost: Training RL models requires powerful hardware and time.
• Data Requirements: A large dataset is needed for the model to learn effectively.
• Market Uncertainty: RL models may struggle with unexpected economic shifts or sudden customer behavior changes.

🔑 Future Improvements:

• Hybrid Models: Combining RL with Time Series models for better forecasting.
• Automated Feature Engineering: Using deep learning to extract hidden patterns in customer behavior.
• Explainability: Enhancing model transparency to understand why certain pricing decisions are made.

🏁 Conclusion

Reinforcement Learning is a powerful tool for dynamic pricing, offering real-time adaptability and maximizing revenue. While traditional AI techniques like Regression and Time Series models provide useful insights, RL outperforms them by continuously learning and adjusting prices based on actual market conditions.

With further advancements in AI, dynamic pricing models will continue to evolve, helping businesses stay competitive and profitable in rapidly changing markets.

📍Final Thoughts

For students and professionals interested in AI-driven pricing strategies, exploring RL can provide valuable insights into real-world business applications. As computational power and data availability increase, RL-based pricing will become an essential component of modern retail and e-commerce strategies.

If you’re passionate about AI and business applications, dive into reinforcement learning and explore its potential in various industries beyond retail!

Have thoughts or questions? Leave a comment below!

GitHub link to the project : https://github.com/sithum20210380/RL-Based-Dynamic-Price-Optimisation

What is Serverless Architecture?

Serverless architecture (also known as serverless computing) represents a paradigm shift in how we think about developing and deploying applications. Unlike traditional server-based approaches, serverless computing enables developers to concentrate solely on writing code, without having to worry about the underlying infrastructure. This means no more server provisioning, OS management, or dealing with hardware failures.

There are two key components in Serverless Architecture,

• Functions as a Service (FaaS)
• Backend as a Service (BaaS)

How it differs from traditional server-based architectures

1. Infrastructure Management:

• Traditional Server-Based Architecture: Requires manual provisioning, configuration, and management of physical or virtual servers. Administrators are responsible for patching, scaling, and maintaining the servers.
• AWS Serverless Architecture: The cloud provider automatically handles server management, including provisioning, scaling, and maintenance. Developers focus solely on writing and deploying code.

2. Scalability:

• Traditional Server-Based Architecture: Scaling typically involves adding more servers (vertical or horizontal scaling) and often requires manual intervention or complex auto-scaling configurations.
• AWS Serverless Architecture: Automatically scales up or down in response to demand. Functions and resources automatically scale to handle the load without any manual intervention.

3. Cost:

• Traditional Server-Based Architecture: Costs are incurred for running servers 24/7, regardless of usage. Costs include server hardware, power, cooling, and maintenance.
• AWS Serverless Architecture: Pay-as-you-go pricing model, where you only pay for the actual compute time used by your code. No costs are incurred when the application is idle.

Implementation

Step 1: Create and Build a React app

Step 2: Create Node.js backend

Step 3: Deploy Lambda functions in the AWS Lambda Console

Step 4: Create an S3 Bucket and Enable Static Website Hosting

Step 5: Setting Up API Gateway to Expose the Lambda Function

Step 6: Configure CloudFront infront of S3

Step 1: Create and Build a React app

AddStudentForm.js

import React, { useState } from 'react';
import axios from 'axios';

const AddStudentForm = () => {
    const [studentId, setStudentId] = useState('');
    const [name, setName] = useState('');
    const [email, setEmail] = useState('');

    const handleSubmit = async (e) => {
        e.preventDefault();
        try {
            await axios.post('https://y7gs0ssz03.execute-api.ap-south-1.amazonaws.com/prod/', { studentId, name, email });
            console.log('Submitting Data:', { studentId, name, email });
            alert('Student added successfully');
            setStudentId('');
            setName('');
            setEmail('');
        } catch (error) {
            console.error('Error adding student:', error);
            alert('Error adding student');
        }
    };

    return (
        <form onSubmit={handleSubmit}>
            <input type="text" placeholder="ID" value={studentId} onChange={(e) => setStudentId(e.target.value)} />
            <input type="text" placeholder="Name" value={name} onChange={(e) => setName(e.target.value)} />
            <input type="email" placeholder="Email" value={email} onChange={(e) => setEmail(e.target.value)} />
            <button type="submit">Add Student</button>
        </form>
    );
};

export default AddStudentForm;

StudentList.js

import React, { useState, useEffect } from 'react';
import axios from 'axios';

const APIURL = 'https://y7gs0ssz03.execute-api.ap-south-1.amazonaws.com/prod/';

const StudentList = () => {
    const [students, setStudents] = useState([]);

    useEffect(() => {
        const fetchStudents = async () => {
            try {
                const response = await axios.get(APIURL);
                console.log('Response:', response);
                const responseData = response.data.body; 
                const students = JSON.parse(responseData);
                console.log('Response Data:', students);
                setStudents(students);
            } catch (error) {
                console.error('Error fetching students:', error);
            }
        };
        fetchStudents();
    }, []);

    return (
        <div>
            <h2>Student List</h2>
            {students.length > 0 ? (
                <ul>
                    {students.map((student) => (
                        <li key={student.studentId}>
                            {student.name} - {student.email}
                        </li>
                    ))}
                </ul>
            ) : (
                <p>No students found</p>
            )}
        </div>
    );
};

export default StudentList;

Step 2: Create Node.js backend

handler for addStudent.js

const AWS = require('aws-sdk');
const dynamoDB = new AWS.DynamoDB.DocumentClient();

exports.handler = async (event) => {
    const { studentId, name, email } = event;

    const params = {
        TableName: 'StudentDetails',
        Item: {
            studentId: studentId,
            name: name,
            email: email
        }
    };

    try {
        await dynamoDB.put(params).promise();
        return {
            statusCode: 200,
            body: JSON.stringify({ message: 'Student details added successfully' }),
            headers: {
                'Content-Type': 'application/json'
            }
        };
    } catch (error) {
        return {
            statusCode: 500,
            body: JSON.stringify({ message: 'Error adding student details', error: error.message }),
            headers: {
                'Content-Type': 'application/json'
            }
        };
    }
};

handler for getStudent.js

const AWS = require('aws-sdk');
const dynamoDB = new AWS.DynamoDB.DocumentClient();

exports.handler = async (event) => {
    const params = {
        TableName: 'StudentDetails',
    };

    try {
        const data = await dynamoDB.scan(params).promise();
        return {
            statusCode: 200,
            body: JSON.stringify(data.Items),
            headers: {
                'Content-Type': 'application/json'
            }
        };
    } catch (error) {
        return {
            statusCode: 500,
            body: JSON.stringify({ message: 'Error getting student details', error: error.message }),
            headers: {
                'Content-Type': 'application/json'
            }
        };
    }
};

Step 3: Deploy Lambda functions in the AWS Lambda Console

First, navigate to the AWS Lambda service and click on the “Create function” button. You’ll be prompted to choose a method to create your function. Select “Author from scratch,” which allows you to manually configure all the necessary details. Give your function a name, such as NodeJsServerlessFunction, and choose Node.js as the runtime environment. For this example, you might select Node.js 16.x. Optionally, you can configure an execution role, either by creating a new role with basic Lambda permissions or by using an existing role.

After writing your function and configuring triggers, deploy the function by clicking on the “Deploy” button in the upper right corner of the Lambda console. Once deployed, you can test your function directly in the console. Click on the “Test” button, create a new test event with a sample payload, and run it to see how your function responds. The results will be displayed in the console, allowing you to verify that your function is working as expected.

Step 4: Create an S3 Bucket and Enable Static Website Hosting

First, sign in to the AWS Management Console and navigate to the S3 service. Once there, click the “Create bucket” button. You’ll need to enter a unique bucket name, select the AWS region where you want the bucket to reside, and optionally configure settings such as versioning, tags, and default encryption. After configuring these details, click “Create bucket” to finalize the creation.

With the bucket created, you can now enable static website hosting. Open your bucket’s properties by clicking on its name in the S3 console. In the “Properties” tab, locate the “Static website hosting” section and click “Edit.” Choose the “Enable” option to activate static website hosting. You’ll need to specify an index document (e.g., index.html) and an error document (e.g., error.html). These documents will serve as the entry point and error page for your website. Once you’ve specified these, click "Save changes" to apply the configuration.

Next, you need to upload your website content. Navigate back to the bucket overview and click the “Upload” button. Add all the files necessary for your website, in react application, upload the files in the build folder, then click “Upload” to transfer the files to your S3 bucket. By default, these files are private, so to make your website publicly accessible, you need to adjust their permissions. Select the uploaded files, click the “Actions” button, choose “Make public,” and confirm the action.

Finally, to access your static website, go to the “Properties” tab of your bucket again. In the “Static website hosting” section, you’ll find the “Bucket website endpoint,” which is the URL of your site (e.g., http://your-bucket-name.s3-website-your-region.amazonaws.com). Open this URL in a web browser to see your website live. This setup is ideal for hosting simple static websites without the need for backend server infrastructure.

Step 5: Setting Up API Gateway to Expose the Lambda Function

Navigate to API Gateway in the AWS Management Console to create a new API. Depending on your use case, you can choose either an HTTP API or a REST API. HTTP APIs are simpler and more cost-effective for most use cases. After selecting the type of API, fill in the API name and description, then proceed to create the API. For REST APIs, create a new resource (e.g., /greet) by clicking "Actions" and selecting "Create Resource." For HTTP APIs, add a new route (e.g., GET /greet).

To integrate your Lambda function with API Gateway, set up the Lambda integration. For REST APIs, select the created method (e.g., GET), choose "Lambda Function" as the integration type, select the Lambda region, enter the Lambda function name, and save the settings. For HTTP APIs, add an integration to the route, choose "Lambda Function," select your Lambda function, and save the integration.

Deploying the API is the next crucial step. For REST APIs, click “Actions” and select “Deploy API,” then create a new deployment stage (e.g., dev) and click "Deploy." For HTTP APIs, create a new deployment in the "Deployments" section and deploy it to a new stage (e.g., dev). Note the Invoke URL from the deployment stage settings and use it to test the API with tools like Postman or your browser. For example, accessing https://<api-id>.execute-api.<region>.amazonaws.com/dev/greet?name=John should return a greeting.

Step 6: Configure CloudFront in front of S3

Using Amazon CloudFront in front of an S3 bucket can significantly improve the performance, security, and scalability of your web content. Here’s a detailed guide on how to set it up.

Create a CloudFront Distribution

Next, navigate to the CloudFront service in the AWS Management Console and click “Create Distribution.” Select “Get Started” under the “Web” delivery method. In the “Origin Settings,” specify your S3 bucket as the origin. You can either type the S3 bucket’s name or use the dropdown to select it.

Configure Distribution Settings

In the distribution settings, you can configure various parameters to suit your needs. For the “Origin Settings,” set the origin domain name to your S3 bucket’s URL and configure the origin path if your content is stored in a specific folder within the bucket. Enable “Restrict Bucket Access” to ensure that CloudFront is the only entity allowed to fetch content from your S3 bucket directly.

In the “Default Cache Behavior Settings,” set the “Viewer Protocol Policy” to “Redirect HTTP to HTTPS” to enforce secure connections. Configure caching settings, such as the allowed HTTP methods and cache TTL (Time to Live) values. For the “Distribution Settings,” set the price class according to your budget and expected geographic distribution of your users. Enable logging if you want to track access logs for your distribution and add alternate domain names (CNAMEs) if you have a custom domain name.

Set Up Permissions

If you enabled “Restrict Bucket Access,” you need to configure an origin access identity (OAI) to grant CloudFront permissions to access your S3 bucket. Create a new OAI or use an existing one, and update the bucket policy to allow CloudFront to fetch objects. You can do this by attaching a bucket policy that grants the OAI read access to the S3 bucket.

Deploy the CloudFront Distribution

Review your settings and click “Create Distribution.” CloudFront will start creating the distribution, which may take several minutes. Once deployed, you’ll receive a CloudFront domain name, such as d1234567890abcdef.cloudfront.net. Use this domain name to access your content.

Conclusion:

In this blog, we covered the essential steps to implement a serverless architecture using React and Node.js. Starting with creating and building a React app, we then set up a Node.js backend. The deployment of Lambda functions in the AWS Lambda Console followed, which is a core component of our serverless setup. Next, we created an S3 bucket and enabled static website hosting, making our frontend accessible over the web. To connect our backend logic with the frontend, we set up API Gateway to expose the Lambda functions. Finally, we configured CloudFront in front of our S3 bucket to enhance performance and security.

By following these steps, you can effectively deploy a serverless application that is robust, scalable, and cost-efficient. This architecture not only simplifies the deployment process but also provides flexibility and resilience, enabling you to build modern web applications with ease. Whether you are a seasoned developer or new to serverless technologies, these steps will guide you through creating and deploying your serverless applications successfully.

GitHub Repo: https://github.com/sithum20210380/my-serverless-app.git