How to use AWS Fargate and Lambda for long-running processes in a Serverless app

AWS dropped so many serverless announcements at re:Invent 2017, the community is still scrambling to make sense of them all. This post is all about AWS Fargate.

In this article, I will show you how to create an end-to-end serverless application that extracts thumbnails from video files. But, oh no, processing video files is a long-running process! Whatever will we do?

This is where Fargate comes in.

TL;DR A Docker container does the processing -> The container extracts the thumbnail and uploads the image to an S3 bucket -> The container is managed by AWS Fargate. All functionality is triggered from AWS Lambda functions and contained within a serverless application written with the Serverless Framework.

Excited? Me too! Read on. 💥

Some background & overview

With the execution time limits of the AWS Lambda platform, there are a lot of use cases involving long-running processes that are hard to implement.

On the flip side, serverless computing offers benefits (like zero-administration, pay-per-execution, and auto-scaling capabilities) that are hard to ignore. AWS recently announced AWS Fargate to bridge that gap.