Scaling Email Sending with Node.js and AWS SES: A Guide to Sending 50,000 Emails at Once

Photo by Firmbee.com on Unsplash

Email marketing is a powerful tool for businesses of all sizes. However, as the number of recipients grows, sending emails can quickly become a bottleneck for your application. In this article, we will explore how to scale email sending using Node.js and AWS SES (Simple Email Service) to send 50,000 emails at once.

Before we dive into the technical details, let’s first understand the basics of email sending and the limitations of traditional methods. Typically, when sending emails, a server will establish a connection with the recipient’s email server, send the email, and then wait for a response. This process is repeated for each email, which can take a significant amount of time when sending to a large number of recipients.

To overcome these limitations, we can use a queuing system like BullMQ to handle the email sending process. Instead of sending the emails directly, we can add the emails to a queue and let BullMQ handle the sending process in the background. This allows us to send multiple emails at once and significantly speed up the process.

Let’s take a look at how to implement this with Node.js and AWS SES. First, we will need to set up an SES account and verify the email addresses that we want to send emails from. Once this is done, we can install the required packages: bullmq, aws-sdk, and nodemailer.

Next, we will need to create a new queue for our email sending process. We can do this using the BullMQ class and passing in the name of the queue as a parameter.

const BullMQ = require('bullmq');
const queue = new BullMQ('email-sending');

We can then use the add method to add emails to the queue. In this example, we will be adding an object that contains the recipient's email address, subject, and message.

queue.add('email', {
    to: 'recipient@example.com',
    subject: 'Hello',
    message: 'Hello, World!'
});

Next, we need to process the emails in the queue. To do this, we will use the process method and pass in a callback function that will be called for each email in the queue.

queue.process(async (job) => {
    // Send email using AWS SES
});

Inside the callback function, we can use the nodemailer package to send the email using AWS SES. We will need to configure nodemailer with our SES credentials and then use the sendMail method to send the email.

const nodemailer = require('nodemailer');
const AWS = require('aws-sdk');

const ses = new AWS.SES({
    accessKeyId: 'ACCESS_KEY',
    secretAccessKey: 'SECRET_KEY',
    region: 'REGION'
});

const transporter = nodemailer.createTransport({ SES: ses });

transporter.sendMail({
    from: 'sender@example.com',
    to: job.data.to,
    subject: job.data.subject,
    text: job.data.message
}, (err, info) => {
    if (err) {
        console.log(err);
        return;
    }
    console.log(info);
});

// Add emails to the queue
for (let i = 0; i < 50000; i++) {
queue.add({
to: 'recipient@example.com',
subject: 'Test Email',
message: 'This is a test email'
});
}

console.log("50,000 emails added to the queue. Emails will be sent in the background.");

// Wait for the queue to be empty before exiting the program
queue.on('empty', () => {
console.log("All emails have been sent.");
process.exit();
});

Now, we’ll create a worker process that will handle sending the emails. The worker process is a function that will be called for each job added to the queue. It will use the nodemailer library to send the email using AWS SES. We’ll also need to configure nodemailer with our SES credentials.

queue.process((job, done) => {
const nodemailer = require('nodemailer');
const AWS = require('aws-sdk');

const ses = new AWS.SES({
    accessKeyId: 'ACCESS_KEY',
    secretAccessKey: 'SECRET_KEY',
    region: 'REGION'
});

const transporter = nodemailer.createTransport({ SES: ses });

transporter.sendMail({
    from: 'sender@example.com',
    to: job.data.to,
    subject: job.data.subject,
    text: job.data.message
}, (err, info) => {
    if (err) {
        console.log(err);
        return;
    }
    console.log(info);
  return done()
});
});

// Add emails to the queue
for (let i = 0; i < 50000; i++) {
queue.add({
to: 'recipient@example.com',
subject: 'Test Email',
message: 'This is a test email'
});
}

console.log("50,000 emails added to the queue. Emails will be sent in the background.");

// Wait for the queue to be empty before exiting the program
queue.on('empty', () => {
console.log("All emails have been sent.");
process.exit();
});

Now, let’s take a look at an example of how to build an email sender using Node.js and AWS SES. In this example, we will use the Bull library to create a queue of emails that need to be sent, and we will use the nodemailer package to send the emails using AWS SES. The example also demonstrates how to send 50,000 emails at once using this architecture.

The first step is to install the required packages, Bull and nodemailer, and import them in our code. We then create a new instance of the Queue class, passing in the name of our queue, ‘emails’, and the connection details to our Redis server.

Next, we define a function that will be called every time a new job is added to the queue. Inside the function, we use the nodemailer package to send the email using AWS SES. We need to configure nodemailer with our SES credentials and then use the sendMail method to send the email.

Finally, we add 50,000 emails to the queue and wait for the queue to be empty before exiting the program. With this approach, we can easily scale our email sending process to handle large amounts of emails without overwhelming the system or risking timeouts.

It’s important to note that, it’s not recommended to send that many emails as it may be marked as spam and can be blacklisted, also it’s better to segment the emails into smaller chunks and distribute over a period of time.

In conclusion, by using Node.js and AWS SES, we can create a robust and scalable email sending architecture that can handle large amounts of emails. With the help of the Bull library, we can easily manage the process of sending emails in the background, ensuring that our program doesn’t hang or time out while sending a large number of emails at once. This approach can be used in a variety of applications, such as sending newsletters, notifications, or confirmation emails to users.