Creating an Email Service with AWS Lambda, S3 and SNS

Photo by Kenny Eliason on Unsplash

In this blog post, I’ll demonstrate how to utilize AWS Lambda, S3 and SNS to send emails to subscribers by extracting content from a CSV file uploaded to an S3 bucket.

Architecture of this project

Prerequisites

• Email address
• Programing language skill e.g., Python, Typescript, etc
• Basic understanding of Lambda, S3 and SNS

Agenda

• Create S3 Bucket
• Create Lambda IAM Role
• Configure SNS for notification
• Create Lambda Function (Python)
• Configure Lambda Function Trigger
• Test Service

Create S3 Bucket

Create an S3 bucket to store CSV files containing email content, which will be read by a Lambda function.

→ Go to S3 menu.

→ Click Create bucket.

→ AWS Region: In my case, it’s ap-southeast-1 chosen based on your specific use case.

→ Bucket name: lambda-emailing-service

→ Object Ownership: Check ACLs disabled (recommended) as we do not require granular permission settings.

→ Block Public Access settings for this bucket: Check Block all public access as we aim to prevent unauthorized users from accessing the bucket.

→ Bucket Versioning: Select Disable as we do not utilize bucket versioning here; it is only for testing purposes. Enabling versioning is suitable if your use case is concerned with retrieving accidentally deleted objects.

→ Leave the other settings at their default configurations..

→ Click Create bucket.

Created Bucket

Create Lambda Role

We will create a role that will be assumed by the Lambda function, allowing it to do these following actions:

• Read CSV files from the S3 bucket.
• Broadcast emails using SNS.
• Generate a CloudWatch log of the Lambda function execution.

→ Go to IAM menu.

→ Click on Roles in the left sidebar.

→ Click Create Role.

→ Trusted entity type: AWS Service (only AWS service can assume this role).

→ Service or use case: Lambda (only Lambda function can assume this role).

→ Permission policies: Select these two AWS managed policies:

- AmazonS3ReadOnlyAccess

- AmazonSNSFullAccess

→ Click Next.

→ Role name: Lambda-Emailing-Role

→ Description: Allows Lambda functions to read CSV file from S3 bucket, send email using SNS and push event to CloudWatch log.

→ Click Create role

→ Next, edit the role and add the following inline policies to Lambda-Emailing-Role to allow the Lambda function to create CloudWatch logs. These logs are crucial for debugging, performance tracking, and other essential functions related to Lambda function execution.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:logs:*:*:*"
    }
  ]
}

→ Policy name: CloudWatchLogPolicy

→ Click Create policy

Configure SNS for notification

Amazon SNS is a messaging service that delivers messages to various subscribers, It delivers messages to subscribing endpoints, e.g., email addresses.

→ Go to Simple Notification Service menu.

→ Click Next step.

→ Select Standard (This is suitable for situations where message order is not essential, such as mass notifications or alerts.)

→ Name: new-product-topic or other desired name.

→ Click Create topic

Created SNS Topic

To receive notifications, you must subscribe to the topic. You can choose to receive notifications via various methods, including SMS and email.

→ Click Create subscription

→ Protocol: Email

→ Endpoint: Your desired email address.

→ Click Create subscription.

→ Then, you will receive a confirmation email for your subscription.

→ Click Confirm subscription.

SNS topic with subscriber email address.

Create Lambda Function

We will use Python, along with the following libraries: boto3, csv, and StringIO from io, to read a CSV file from the previously created S3 bucket, generate email content, and send emails to subscribers using an SNS topic.

→ Go to Lambda menu.

→ Click Create function.

→ Function name: Lambda-Emailing-SNS.

→ Runtime: Python 3.12

→ Architecture: X86_64

→ Expand Change default execution role.

→ Execution role: Lambda-Emailing-Role

→ Click Create function.

→ You will be redirected to the created function.

→ Click on Code tab.

→ Paste this code into index.py.

→ Replace SNS Topic’s ARN based on your case.

import boto3
import csv
from io import StringIO

s3 = boto3.client('s3')
sns = boto3.client('sns')

def handler(event, context):
    bucket_name = event['Records'][0]['s3']['bucket']['name']
    object_key = event['Records'][0]['s3']['object']['key']
    
    print("Bucket name:", bucket_name)
    print("Object key:", object_key)

    if bucket_name == 'lambda-emailing-service':
        email_data = read_csv_from_s3(bucket_name, object_key)

        if email_data:
            send_emails(email_data)

def read_csv_from_s3(bucket_name, object_key):
    email_data = []

    try:
        response = s3.get_object(Bucket=bucket_name, Key=object_key)
        csv_data = response['Body'].read().decode('utf-8')

        reader = csv.DictReader(StringIO(csv_data))
        for row in reader:
            topic = row['topic']
            message = row['message']

            email = {
                'topic': topic,
                'message': message
            }
            
            print("email: ", email)

            email_data.append(email)
    except Exception as e:
        print('Error reading CSV file from S3:', e)

    return email_data

def send_emails(email_data):
    try:
        for email in email_data:
            message = f"Subject: {email['topic']}\n\n{email['message']}"
            response = sns.publish(
                TopicArn='<your_topic_ARN>',
                Message=message
            )
            print('Email sent to SNS topic:', response['MessageId'])
    except Exception as e:
        print('Error sending email:', e)

CSV file structure and content

Note: For the purpose of this quick guide and testing, we’ll proceed without implementing environment variables or using AWS Secrets Manager. However, in a production environment, it’s recommended to enhance security by storing the SNS topic ARN in a Lambda function environment variable or using AWS Secrets Manager.

Configure Lambda Function Trigger

We need our Lambda function to run every time a CSV file is uploaded into the S3 bucket. Let’s configure an S3 event to trigger the Lambda function.

→ On Lambda function

→ Click Add trigger.

→ Source: Select S3

→ Bucket: s3/lambda-emailing-service

→ Event type: All object create events

→ Suffix: .csv

→ Check on I acknowledge that using …

→ Click Add.

Lambda function after configure S3 event trigger.

Test Service

We’re all set. Let’s test the service by uploading a CSV file into the S3 bucket.

→ Go to S3 bucket.

→ Click Upload.

→ Click Add files.

→ Check your email inbox.

Note: Every time Lambda is triggered, the event is logged into CloudWatch log.

CloudWatch logs contain the execution history of the Lambda function.

That’s it! We’ve successfully created an emailing service using Lambda functions, S3 buckets, and SNS topics to send emails to subscriber email addresses. One use case for this example could be producing stock checking reports for stakeholders or generating reports on other stored data in S3. Another use case involves automating data processing pipelines with Lambda. For example, Lambda can process IoT data streams, triggering email alerts to stakeholders for real-time insights and decision-making

Thank you for reading! Your feedback is valuable to me. Please feel free to leave comments, as I appreciate any suggestions to improve my blog.