Building a Simple Scheduled Task with AWS using Lambda Function and Amazon Cloudwatch Event

Published in

TheLorry Data, Tech & Product

6 min readApr 4, 2021

Let’s build a simple scheduled task on AWS using Lambda Function and Amazon Cloudwatch Event Rule.

Photo by Emma Matthews Digital Content Production on Unsplash

Automating repeated tasks in a scheduled manner is also known for so-called cron jobs. It really helps in reducing the resource costs, maintaining the system availability, and removing manual workloads efficiently overtime.

CronJobs can be used for single commands or for the automated execution of periodically recurring sequential tasks. — Seobility Wiki

These scheduled tasks are implemented for various different purposes.

For instance, sending notifications and emails to end-users, performing route optimization, training machine learning models, monitoring status of the system, system backups, data analytics reporting, managing server instances, cleaning up unused resources, or removing obsolete entries in the database.

Cron jobs are indeed very useful and efficient as they unnoticedly run in the background and can even perform highly computational tasks on a timely basis.

In this article, we will bring you to quick walkthrough on how to create a simple scheduled task using Amazon CloudWatch and AWS Lambda.

Without further ado, let’s dive in!

The AWS Resources

There are two AWS resources involved in building the scheduled task on AWS which is:

Amazon Cloudwatch Event (Rules): It offers a near real-time stream of system events that able to takes corrective action as necessary, by sending messages to respond to the environment, activating functions, making changes, and capturing state information.
AWS Lambda (Lambda Function): A serverless compute service that runs code only when triggered and using only the compute resources needed. Technically, it will perform a specific task defined in the function.

Use Case Scenario

Every morning, a data analyst needs to send reports to managers about the Company Performance Analysis. Technically, what he will do is calling an API endpoint that will fetch data from a database and it will perform some data transformation to produce the reporting analysis. Once the process is done, he will send the generated report to the managers through email. He needs to do this literally every morning at 8.00 AM GMT+8.

From this scenario, we can see that the process flow of sending the email notifications to the managers in the morning can be automated by creating a scheduled task.

Thus, the diagram below shows how it can be done on AWS.

Let’s breakdown the above process flow:

The Cloudwatch Rules will activate the lambda function according to the scheduled time defined in the cron expression. In this case, the rule is configured to be triggered every day at 8:00 AM GMT+8.
Lambda Function will be activated once it is invoked by Cloudwatch Events. It will call the Reporting Email API endpoint to perform the required tasks.
Then, the Reporting Email API will perform the analysis and generate the report. Once it is done, it will automatically send the email to managers.

It seems like the process is very straightforward and easy.

Actually, yes it is indeed very easy and it can be done with just a few steps!

Now, let’s get our hands dirty by setting up this scheduled tasks pipeline on AWS.

Create AWS Lambda Function

Configure Lambda Function

Go to AWS Management Console and log in using our AWS Account. Then, proceed to AWS Lambda in the console and click on the Create function.

Then, choose author from scratch and define your function name. In this tutorial, we will be creating python function. Thus, select python 3.7 as runtime.

You will need to choose or create an execution role before creating the function. Once it is done, click on the Create function.

Once done, we can see our function already created.

Lambda Code Editor

In the Code tab below, we can see the Code source where we can write our own custom code. Any API calls can be done or even executing a complex and highly computational task in the Lambda function. For this example, we will write a code where this function will call an API endpoint.

Click on Deploy once you have completed the code.

It’s done! Let’s create Amazon Cloudwatch Event to trigger this lambda function in timely basis.

Create Scheduled Event in Amazon Cloudwatch Console

Go to AWS Management Console and log in using our AWS Account. Then, proceed to Amazon Cloudwatch and click Rules under event section in the left navigation menu. We will see the Events list and then click on Create Rule.

Event Source

In create rule page, Event pattern is selected by default in the event source section. We will creating scheduled events, therefore we need to select Schedule option. As we selected the Scheduleoption, we can see there are two kinds of event rules that we can define:

Fixed rate schedule
Schedule based on cron expression

For fixed rate, we need to define the interval and its unit. The units are minutes, hours and days.

In our case, let’s define our schedule event rule using cron expression . A cron expression is a string composed of 6 or 7 fields separated by white spaces. The cron expression field can consists of any of the allowed values, together with various combination of the allowed special characters for that field. Kindly checkout this documentation about the Cron Syntax for detail understanding about the cron expression.

We will define our expression that will trigger event in everyday at 8:00 AM GMT+8 time zone. This is how the cron expression will look like.

We actually can construct a very detail schedule events using the cron expression and it is depending on how intricate is the use case that we are working on at the moment.

Add Targets

Click on the Add target and we will see that lambda function is selected by default. Now, select the function that will be our target and for this example we will be selecting lambda-scheduled-task function that we have created earlier.

Click on Configure details once everything is done.

Configure Rule Details

We need to provide a name and description to our event rule. Make sure that Enabled is checked and Click on Create Rule.

The event rule will be created and we will be redirected to the Rules list page. From here, we can see that our scheduled event is successfully created.

Now this event will be triggered everyday at 8:00 AM in GMT+8 and it will also triggers our Lambda function accordingly.

Congratulations! Now we have completed the Scheduled Task Pipeline on AWS. :)

Summary

In this article, we have discussed about the related components for scheduled task on AWS, create and configure the Lambda function, and create scheduled event rule using Amazon Cloudwatch Event.

We hope this simple article will help you to build your own customized scheduled tasks on AWS.

Peace! ✌️