ECS Task Runner with Lambda
When you need to execute a long-running or periodic jobs in AWS, it’s been very conventional to write your scripts and configure them as cron job on EC2 instances. However, you risk the failure of EC2 instances and you may also have problem in its scalability.
An interesting approach is to wrap your business logic of periodic long-running jobs into Docker images and compile them into AWS ECS Task Definitions and use the ECS CLI or SDK to run-task with them. The ECS scheduler will try to allocate resources for the task on proper ECS instance and execute the Docker command you wrapped in.
Let’s assume you already wrap your codes of business logic into the docker image and created your ECS task definition. It’s very easy to run the task for one-shot just like this:
aws --region ap-northeast-1 ecs run-task --task-definition my_task_definition
ECS, Cloudwatch Events and Lambda
Moreover, if you prefer the serverless fashion, you can configure Cloudwatch Events with scheduling rules, invoking Lambda to run the ECS task for you.
Considering the following example(ecs-task-runner.js):
In CloudWatch Events, configure the schedule expression like this
50 6 ? * SAT *
The above means execute the custom event Every Saturday 6:50AM UTC. Then configure the constant JSON input payload just like this
When invoking Lambda function, the input payload above will construct itself as the event object for Lambda, and Lambda will hence perform the one-shot task running on ECS as you expect.