Price Monitoring with Python and Beautiful Soup

Introduction

Price monitoring is a crucial aspect of staying competitive in today’s market. In this article, we will show you how to build a simple price monitoring script using Python and Beautiful Soup to track product prices on e-commerce websites. This information can help you make informed decisions about pricing strategies and identify potential opportunities in the market.

Photo by Markus Winkler on Unsplash

If you are not able to visualise the content until the end, I invite you to take a look here to catch-up!

Requirements

To follow along with this tutorial, you will need the following:

1. Python 3.x installed on your system.
2. Beautiful Soup 4 and the Requests library installed. You can install them using pip:

pip install beautifulsoup4 requests

Step 1: Setting Up the Project

First, create a new directory for your project and navigate to it:

mkdir price_monitor
cd price_monitor

Next, create a Python file to write your code:

touch price_monitor.py

Step 2: Fetching Product Information

To fetch product information, you need to specify the URL of the product page you want to monitor. For this example, we will use a dummy URL:

import requests
product_url = "https://www.example.com/product-page"
response = requests.get(product_url, headers={'User-Agent': 'Mozilla/5.0'})
# Check if the request was successful
if response.status_code == 200:
    page_content = response.text
    print(page_content)
else:
    print(f"Failed to fetch content from {product_url}")

Step 3: Parsing HTML Content with Beautiful Soup

Now that we have the web page content, we can use Beautiful Soup to parse the HTML and extract product information.

from bs4 import BeautifulSoup
soup = BeautifulSoup(page_content, "html.parser")
print(soup.prettify())

After inspecting the HTML structure of the product page, we can see that the product price is contained within a specific HTML element. In this example, we assume the price is within a ‘span’ element with the class ‘price’. You will need to modify this according to the actual structure of the website you are monitoring.

price_element = soup.find("span", class_="price")
price = float(price_element.text.strip().replace("$", ""))
print(f"Product price: ${price:.2f}")

Step 5: Setting Up Price Monitoring

Now that we can extract the product price, we can set up a monitoring script to check the price at regular intervals. We will use Python’s time.sleep() function to add a delay between requests.

import time
def monitor_price():
    while True:
        # Fetch and parse the page content, then extract the product price (steps 2-4)
        # Sleep for a specified interval (in seconds) before checking the price again
        time.sleep(3600)  # Check the price every hour
if __name__ == "__main__":
    monitor_price()

Step 6: Notifying When the Price Drops (Optional)

You can enhance your price monitoring script by sending a notification when the price drops below a specified threshold. One way to send notifications is through email using Python’s smtplib library:

import smtplib
def send_email(price):
    # Set up email server and login information
    server = smtplib.SMTP("smtp.example.com", 587)
    server.starttls()
    server.login("you@example.com", "your_password")
    # Create email content
    subject = "Price Drop Alert"
    body = f"The product price has dropped to ${price:.2f}. Check it out at {product_url}"
    message = f message = f"Subject: {subject}\n\n{body}"

    # Send email
    server.sendmail("you@example.com", "recipient@example.com", message)

    # Close the server connection
    server.quit()

def monitor_price():
    target_price = 50.00  # Set your desired target price here

    while True:
        # Fetch and parse the page content, then extract the product price (steps 2-4)

        if price <= target_price:
            print("Price drop detected!")
            send_email(price)

        # Sleep for a specified interval (in seconds) before checking the price again
        time.sleep(3600)  # Check the price every hour

if __name__ == "__main__":
    monitor_price()

Conclusion

In this article, we demonstrated how to build a simple price monitoring script using Python and Beautiful Soup to track product prices on e-commerce websites. This information can help you make informed decisions about pricing strategies and identify potential opportunities in the market.

Please remember to respect the target website’s terms of service and robots.txt file when using web scraping, and do not use the extracted data for commercial purposes without permission.

Additionally, you can enhance this script by monitoring multiple products or websites, storing historical price data in a database, or integrating it with other notification methods like SMS or push notifications. Happy coding!

If you found this article valuable or insightful, I’d greatly appreciate your support by following me, Jonathan Mondaut, here on Medium. I’m committed to sharing more engaging and practical content that will help you stay ahead in today’s fast-paced world. Don’t miss out on my upcoming articles — follow me now, and let’s learn and grow together!