Scheduling events in AWS in a Serverless way - Pt1. EventBridge
In this series of blog posts, I’m going to take you through different AWS services that can schedule events in some form, the serverless way. In this first edition, we’ll be taking a deeper look into AWS EventBridge.
In each blog post, we’ll test the ability of each service to schedule events, based on three scenarios. These scenarios are a single event, an hourly event, and second-level accuracy events. We’ll also evaluate if the solution scales, and how much it costs to use this service for each scenario.
What is EventBridge: A Quick Introduction
Amazon EventBridge is a serverless event bus service that makes it easy to connect your applications with data from a variety of sources. EventBridge delivers a stream of real-time data from your applications, software-as-a-service (SaaS) applications, and AWS services and routes that data to targets such as AWS Lambda. You can set up routing rules to determine where to send your data to build application architectures that react in real-time to all of your data sources. EventBridge enables you to build event-driven architectures that are loosely coupled and distributed.
Some key components of EventBridge that are important for this blog include the Event Bus, Rules & Scheduler.
Event Bus: “A router that receives events and delivers them to zero or more destinations, or targets”
Rules: “To specify what EventBridge does with the events delivered to each event bus, you create rules”
Scheduler: “A serverless scheduler that allows you to create, run, and manage tasks from one central, managed service”
Scenario 1: Scheduling a single-time event to invoke a Lambda
Let’s start with the most basic scenario, we have to schedule a one-off event to invoke a Lambda, tomorrow at 13:00. This can be solved by EventBridge Scheduler. This service enables you to schedule events at a certain time, or on a certain schedule (which can be cron-based or rate-based).
After going to the next page, EventBridge allows you to send this event to an AWS service, without having to write the integration for this.
Sweet!
They offer the choice between templated targets that are often used, or a choice of all API integrations currently available, which at this time of writing is an impressive 320+ available integrations.
This is exactly what we need. Now, if you create these schedules more often, there is one problem that you run into: the schedules that are in the past and executed are piling up in your account, cluttering your environment.
AWS has thought of this, as they provide the option to have the schedule delete itself after completion. Additionally, EventBridge scheduler also offers retry mechanisms, DLQ integration, and encryption using CMKs.
Costs: The first 14 million invocations per month are free. After that, you pay $1.15 (eu-central-1) per 1 million invocations.
Scenario 2: Send a message to SNS, every hour.
It seems that EventBridge Scheduler is the right tool for the job again.
When you want to create a recurring event, there are two options: cron-based and rate-based. With rate-based events, you can schedule them to happen every X minute, hour, or day. If you need more specific control, such as on which time to run the event or only run it on a specific day, the cron-based schedule is your best friend.
For our scenario, the rate-based event is the perfect fit. We set the rate at 60 minutes, define the event that we want to send to SNS, and put the correct topic as the destination.
Costs: The first 14 million invocations per month are free. After that, you pay $1.15 (eu-central-1) per 1 million invocations. Since our schedule runs every hour, we invoke this schedule 731 times (for an average month) per month. This is within the Free Tier, so it would cost us $0.00.
Scenario 3: Scheduling events with second-level accuracy, sending them through Lambda
Now, this is where the trouble starts for EventBridge. EventBridge is great for scheduling one-time & recurring events, but when you need to schedule events with second-level accuracy, EventBridge comes up short. All methods of scheduling that we discussed before allow you to schedule events up to the minute.
So, for this, we would be looking at more custom solutions. Let’s make an example.
I have to make an HTTP call to a client using AWS Lambda. But, when the client doesn’t respond, I have to adhere to a retry schedule. Each time the client doesn’t respond, the waiting time increases to make sure we don’t overload the client with requests. This will go as follows:
- 5 seconds; then
- 10 seconds; then
- 30 seconds; then
- 1 minute; then
- 5 minutes; then
- 10 minutes
Using EventBridge the last 3 parts of the retry schedule are easily done. We create a schedule that is 1, 5, or 10 minutes from now that will invoke the Lambda. If the client still doesn’t respond, Lambda creates a new schedule with the next waiting time and this is repeated until the schedule is done.
However, for the first three parts, we have to get more creative. The easy solution would be to handle this in the Lambda itself. We would wait for 5 seconds for the client to respond. If there is no response, we send it again and wait for 10 seconds again. And repeat.
One problem with this solution is that we just created a Lambda anti-pattern: Synchronous waiting within a single Lambda function. Therefore, this solution will not be suitable.
Unfortunately, it seems like this problem is not solvable if we only want to keep using EventBridge, and don’t want to implement any anti-patterns in Lambda. We have to use different AWS services to solve this problem (such as Step Functions, SQS, or others. Stay tuned for the next blog in this series)
Costs: The first 14 million invocations per month are free. After that, you pay $1.15 (eu-central-1) per 1 million invocations. Since we cannot satisfy the first part of the retry schedule, we can maximally create 3 schedules per retry (1, 5 & 10 minutes). Therefore, to stay within the Free Tier, we create this pattern 4.6 million times per month.
BONUS: Does it scale?
Since we are only using EventBridge Scheduler, we’ll only consider this part of EventBridge. Let’s look at the most important quotas
- We can create 50 schedules per second
- We can have 500 invocations from a schedule per second
- We can have 1,000,000 million schedules at the same time.
Note: all these limits are adjustable. Source: AWS
So, does it scale?
Yes
But also not that much.
If you are keeping the default quotas, it is easy to see how you would reach the ‘50 scheduled creations per second’ quota. Especially if your platform is built to handle hundreds of events per second.
Therefore you should be aware of the quotas if you adopt EventBridge Scheduler in your platform if there is a chance you’ll be hitting the quotas easily. If you think you’ll hit these quotas easily, reach out to AWS Support to have the quotas raised.
So, EventBridge, yay or nay?
While EventBridge offers powerful capabilities for scheduling events in a serverless environment, it’s important to recognize its strengths and limitations. EventBridge shines when you don’t need to have second-level accuracy for scheduling (recurring) events, especially with the introduction of EventBridge Scheduler.
However, if you’re looking for a way to schedule events on a second-level granularity, EventBridge will not satisfy your needs.
Stay tuned for the next edition of this series, where we’ll explore a service that you might not expect in this series around scheduling serverless events: SQS
