Never Miss an Update: How to Send RSS Feeds to a Telegram Channel with Python and AWS

RSS feeds are a great way to receive updates from websites and blogs in a structured format. In this article, we will go through the steps to create an application that sends news updates from an RSS feed to a Telegram channel.

Prerequisites

• A Telegram account
• A bot on Telegram
• A Telegram channel
• Python 3 installed on your system
• An Amazon Web Services (AWS) account

Step 1: Create a Bot on Telegram and Obtain its API Token

To create a bot on Telegram, you will need to talk to the BotFather on Telegram and follow the instructions provided. The BotFather is a bot that helps you create new bots and manage them. Once you have created the bot, you will receive an API token that you will need to use in your code to access the Telegram API.

Step 2: Create a Channel on Telegram

To create a channel on Telegram, go to Telegram Settings -> Create a new channel. You can invite people to the channel by sharing the invite link with them.

Step 3: Install Required Python Libraries

You will need to install the following Python libraries to run the code:

• requests: for making HTTP requests
• dateutil: for parsing and manipulating dates
• feedparser: for parsing RSS feeds
• boto3: for working with Amazon S3 (we will need to save the last time when we read the RSS)

You can install these libraries using pip install library_name.

Step 4: Specify the API Token and Channel IDs

In the code, you will need to specify the API token for your bot and the IDs for the channels where the updates will be sent:

BOT_TOKEN = ''
NEWS_CHANNEL_ID_1 = ''
NEWS_CHANNEL_ID_2 = ''

Step 5: Create an Amazon S3 Bucket

Create an Amazon S3 bucket where the last update date for each RSS feed will be stored. This will help ensure that you do not send duplicate updates to the Telegram channel.

Step 6: Specify the URLs for the RSS Feeds, Channels, and S3 Object Keys

In the code, you will need to specify the URLs for the RSS feeds that you want to receive updates from, as well as the channels where the updates will be sent and the object keys for the last update dates in the S3 bucket.

S3_BUCKET_NAME = ''
S3_CHANNEL_1_LASTUPDATEDATE_OBJECT_KEY = 'rss_parser_storage/news_channelId_1.txt'
S3_CHANNEL_2_LASTUPDATEDATE_OBJECT_KEY = 'rss_parser_storage/news_channelId_2.txt'

FEEDURL_CHANNEL_MAP = {
    'here should be the link to RSS channel 1': (S3_CHANNEL_1_LASTUPDATEDATE_OBJECT_KEY, [NEWS_CHANNEL_ID_1]),
    'here should be the link to RSS channel 2': (S3_CHANNEL_2_LASTUPDATEDATE_OBJECT_KEY, [NEWS_CHANNEL_ID_2])
}
TELEGRAM_TOO_MANY_REQUESTS_ERROR_CODE = 429

Step 7: Parse the RSS Feed and Get the News Updates

Use the feedparser library to parse the RSS feed and get the news updates.

rss_feed = feedparser.parse(feed_url)

Step 8: Send the News Updates to the Telegram Channels

For each news update, use the send_message function to send a message to the specified Telegram channels. This function makes a request to the Telegram API to send a message to the specified channel.

def send_message(channel_id, message, image_url=None):
    """
    Sends a message or image with caption to the specified Telegram channel.

    Parameters:
    channel_id (str): ID of the Telegram channel to send the message to.
    message (str): Message or caption to send.
    image_url (str, optional): URL of the image to send. If not provided, only sends the message.

    Returns:
    None
    """
    if image_url and len(message) <= TELEGRAM_PHOTO_CAPTION_LENGTH_LIMIT:
        response = requests.get(f'https://api.telegram.org/bot{BOT_TOKEN}/sendPhoto?chat_id={channel_id}&photo={image_url}&caption={message}&parse_mode=html')
    else:
        response = requests.get(f'https://api.telegram.org/bot{BOT_TOKEN}/sendMessage?chat_id={channel_id}&text={message}&parse_mode=html')
    if not response.ok:
        if response.status_code == TELEGRAM_TOO_MANY_REQUESTS_ERROR_CODE:
            print(f'status code: {response.status_code}')
            retry_in_seconds = int(response.headers['Retry-After'])
            print(f'Waiting {retry_in_seconds} seconds before retrying...')
            sleep(retry_in_seconds)
            send_message(channel_id, message)
            return
        raise Exception(response.text)

Step 9: Handle Too Many Requests Error

If the API returns a 429 status code, which indicates that there are too many requests being made, you will need to wait for a few seconds before sending the next message.

Step 10: Update the Last Update Date for Each RSS Feed in the S3 Bucket

After sending the news updates, update the last update date for each RSS feed in the S3 bucket.

last_published_date = max(last_published_date, entry_published_date)

Step 11: Set Up a Scheduled Event Using AWS Lambda

You can set up a scheduled event using AWS Lambda to run the script at a specified interval to receive the latest news updates. To do this, follow the below steps:

1. Go to the AWS Lambda console and create a new function.
2. Choose Python 3.8 as the runtime.
3. In the Function code section, copy and paste the code that we have written to send the news updates to the Telegram channels. Or you can use AWS sam to build a template to sync with AWS every time you did a change
4. In the Basic settings section, set the Timeout to 5 minutes or more, depending on your needs.
5. In the Triggers section, click on the Add trigger button and choose CloudWatch Events - Schedule. Or you can use EventBridge to configure events that will be sent with some interval.
6. Set the Schedule expression to the desired interval at which you want the function to run. For example, to run the function every hour, you can set the expression as rate(1 hour).
7. Click on the Save button to save the function and the trigger.

Your application is now set up to send the latest news updates from the specified RSS feed to the Telegram channel at the specified interval.

The final code will look:

import json
import requests
from dateutil import parser
from time import sleep
import feedparser
import boto3

BOT_TOKEN = ''
NEWS_CHANNEL_ID_1 = ''
NEWS_CHANNEL_ID_2 = ''

S3_BUCKET_NAME = ''
S3_CHANNEL_1_LASTUPDATEDATE_OBJECT_KEY = 'rss_parser_storage/news_channelId_1.txt'
S3_CHANNEL_2_LASTUPDATEDATE_OBJECT_KEY = 'rss_parser_storage/news_channelId_2.txt'

FEEDURL_CHANNEL_MAP = {
    'here should be the link to RSS channel 1': (S3_CHANNEL_1_LASTUPDATEDATE_OBJECT_KEY, [NEWS_CHANNEL_ID_1]),
    'here should be the link to RSS channel 2': (S3_CHANNEL_2_LASTUPDATEDATE_OBJECT_KEY, [NEWS_CHANNEL_ID_2])
}
TELEGRAM_TOO_MANY_REQUESTS_ERROR_CODE = 429
TELEGRAM_PHOTO_CAPTION_LENGTH_LIMIT = 1024

s3_client = boto3.client("s3")

def send_message(channel_id, message, image_url=None):
    """
    Sends a message or image with caption to the specified Telegram channel.

    Parameters:
    channel_id (str): ID of the Telegram channel to send the message to.
    message (str): Message or caption to send.
    image_url (str, optional): URL of the image to send. If not provided, only sends the message.

    Returns:
    None
    """
    if image_url and len(message) <= TELEGRAM_PHOTO_CAPTION_LENGTH_LIMIT:
        response = requests.get(f'https://api.telegram.org/bot{BOT_TOKEN}/sendPhoto?chat_id={channel_id}&photo={image_url}&caption={message}&parse_mode=html')
    else:
        response = requests.get(f'https://api.telegram.org/bot{BOT_TOKEN}/sendMessage?chat_id={channel_id}&text={message}&parse_mode=html')
    if not response.ok:
        if response.status_code == TELEGRAM_TOO_MANY_REQUESTS_ERROR_CODE:
            print(f'status code: {response.status_code}')
            retry_in_seconds = int(response.headers['Retry-After'])
            print(f'Waiting {retry_in_seconds} seconds before retrying...')
            sleep(retry_in_seconds)
            send_message(channel_id, message)
            return
        raise Exception(response.text)
def lambda_handler(event, context):
    """
    Main function that gets triggered at a specified interval to send news updates from the specified RSS feeds to the specified Telegram channels.

    Parameters:
    event (dict): Event data passed by the trigger.
    context (LambdaContext): Information about the current execution context.

    Returns:
    None
    """
    for feed_url, (s3_last_update_date_obj_key, channel_ids) in FEEDURL_CHANNEL_MAP.items():
        # Get last update date from S3
        date_str = s3_client.get_object(Bucket=S3_BUCKET_NAME, Key=s3_last_update_date_obj_key)["Body"].read()
        last_update = parser.parse(date_str)
        # Parse RSS feed
        rss_feed = feedparser.parse(feed_url)
        # Iterate through entries in reverse order (most recent first)
        for entry in reversed(rss_feed.entries):
            # Get entry published date
            entry_published_date = parser.parse(entry.published)
            # Check if entry is newer than the last update date
            if entry_published_date > last_update:
                # Build message to send
                msg = f'<b>{entry.title}</b>\n\n{entry.description}\n\n <a href="{entry.link}">Детальніше</a>'
                # Send message to each channel
                for channel_id in channel_ids:
                    # Check if entry has an image and send image with caption if possible
                    if len(entry.links) > 1 and entry.links[1].href:
                        send_message(channel_id, msg, entry.links[1].href)
                    else:
                        send_message(channel_id, msg)
                print(f'sent {entry.links[0].href}')
            # Update last published date
            last_published_date = max(last_published_date, entry_published_date)
        # Update last update date in S3 if necessary
        if last_published_date > last_update:
            s3_client.put_object(Bucket=S3_BUCKET_NAME, Key=s3_last_update_date_obj_key, Body=str(last_published_date))

Conclusion

In this article, we have gone through the steps to create an application that sends news updates from an RSS feed to a Telegram channel. We have used Python libraries such as feedparser, requests, and boto3 to parse the RSS feed, send the updates to the Telegram channels, and store the last update dates in an Amazon S3 bucket. We have also set up a scheduled event using AWS Lambda to run the script at a specified interval to receive the latest news updates. With these steps, you should be able to create your own application to receive updates from an RSS feed and send them to a Telegram channel.

Thanks for reading and good luck!