Triggering Lambda Functions via Text Messages
by Sunil Mallya, Solutions Architect, Amazon Web Services
Time is the most critical component in the startup equation. Startups should maximize their time with customers and focus on building a better product, not spending their time managing infrastructure and servers. Amazon continues to build services that abstract complex components and focus on making it easy to run your application. AWS has taken this a step further with its serverless computing services, namely AWS Lambda and Amazon API Gateway: Now AWS manages your code and servers!
This post explores a serverless application that applies filters to images sent via MMS. Users send images via MMS, the application applies the filter on demand, and ultimately responds back to the user with the filter applied to the image. Twilio is used as the text message endpoint. We first explore this process from a service-orientated architecture and then look at potential improvements by going serverless using AWS’s new services, Lambda and API Gateway.
Motivation for Serverless
Many startups today benefit from leveraging a service-oriented architecture (SOA). The idea of segmenting your application by service and decoupling them achieves a more manageable environment. Because it’s distributed, a decoupled application is more reliable. In addition it’s easier to deploy changes on a service-by-service basis and scale individual services more effectively.
With this architecture in mind, we deploy our core application that handles the ingestion point to a fleet of Amazon EC2 instances and use another fleet of EC2 instances to act as our filter service. The following diagram illustrates this design. The application uses a queue, Amazon SQS in our case, to maintain the image filter jobs. When a user sends a text message with a photo, Twilio hits our API endpoint and subsequently adds the image to the queue. The second fleet, responsible for the filter process, dequeues images and applies the filter. This actually works quite well. We can scale the ingest service containing the endpoint based on user activity and the filter process based on number of images in the queue.
The issue here is server management — we still have to manage two different fleets and a queue! That means thinking about instance family, type, Availability Zone, etc. In the next section, we show you how moving to a serverless architecture alleviates these pain points.
Instance-based Architecture
The following totals are based on processing 1 million messages a day with a 1 MB image, including Free Tier. We assume that the application compresses the image and reduces it to 100KB to reduce the outbound transfer.
Monthly Cost Analysis
Note the following:
• Data ingest in 1 month = 1 MB * 1 M messages * 30 = 30,000 GB
• Data out in 1 month = 100 KB * 1 M messages * 30 = 3000 GB
In this scenario, the total cost is $803 per month. For detailed pricing, click the link: https://calculator.s3.amazonaws.com/index.html#r=IAD&key=calc-CC319210-B287-4485-B6EA-4F096F6D08B5
Serverless Deep Dive
AWS Lambda is a compute service that runs your code in response to events. Resources in your AWS environment generate events. Each time an event is generated, your code is executed. You are billed only for the time that your code is executed, measured in increments of 100 milliseconds.
Lambda can currently be triggered by events in Amazon S3, Amazon DynamoDB, Amazon Kinesis, Amazon Simple Notification Service, Amazon Simple Email Service , Amazon Cognito, Amazon Cloudwatch logs, Amazon Cloudwatch Events, Scheduled Events. Currently, Lambda can’t be triggered by SMS, we need a creative solution. In this post, we want to showcase how we can use text messages via Twilio to invoke the Lambda function. Enter API Gateway, which can be leveraged as the connector.
API Gateway is a fully managed API as a service where one can create, publish, maintain, monitor, and secure APIs at any scale. API Gateway extends the functionality of Lambda by helping you to create REST endpoints that trigger functions. These endpoints pass data from the client to a Lambda function. API Gateway supports mapping templates, an important feature with which you can transform data that comes in through API Gateway, as well as data for the response.
Serverless Architecture
By combining these services you can build powerful REST applications that are completely serverless. We can replace the image ingest fleet, filter process fleet, and queue used in the earlier example with these services to achieve a serverless architecture.
Twilio is a great use case for a serverless architecture because very little code is needed to run in response to each message. Twilio sends one HTTP request per message to our API Gateway endpoint, which then invokes a Lambda function that handles computing required to apply the filter. The Lambda function uses a third- party library to apply the filter, writes the image to S3, and responds to Twilio with the filtered images. API Gateway takes this output, builds an XML object, and finally sends the response back to Twilio to respond to the user via MMS.
A few things to note: Twilio uses an HTTP URL-encoded GET request, so an API Gateway mapping template is key here because Lambda requires a JSON object. This mapping template builds a JSON object using the necessary parameters from the URL-encoded string.
We also use a mapping template to build a proper response to Twilio, which requires an XML object. A mapping template is a great workaround here because Lambda can’t return an XML object. Lambda returns a string with the image URL to API Gateway, which builds an XML object.
The pricing structure is also shifted: Now you pay per request to API Gateway and time spent running Lambda functions as opposed to paying for servers and load balancing on an hourly basis. This helps us to get even closer to paying for only what we use. Running EC2 instances that aren’t at peak utilization creates cost inefficiencies. Keeping a fleet of EC2 instances at peak utilization can be resource intensive, depending on your application. With Lambda, you can achieve full utilization by paying for only the resources consumed and not having to keep your infrastructure running idle.
Monthly Cost Analysis
With AWS Lambda you are charged for the number of requests for your functions and the time your code executes. The compute time is calculated in GB-seconds, which is measured by the amount of memory allocated to your function multiplied by the time your code executes for, rounded up to the nearest 100ms. Note that the compute capacity allocated to your Lambda function is dependent on the memory allocation set. The monthly compute price is $0.00001667 per GigaByte per second (GB-s) and the free tier provides 400,000 GB-s. The Lambda free tier doesn’t expire at the end of your 12 month AWS Free Tier term but is available to existing and new AWS customers indefinitely.
Total compute (seconds) = 1 M requests * 30 days * (0.1 s) = 3,000,000 seconds
Total compute (GB-s) = 3,000,000 * 128 MB / 1024 = 375,000 GB-s
375,000 Gbps is within the 400,000 GB-s free tier limits
Monthly compute charges = $0 (free)
Lambda Request charges
The monthly request price is $0.20 per 1 million requests and the free tier provides 1 M requests per month.
Monthly request charges = 29 M * $0.2 / 1 M = $5.80
Total Lambda cost: $0 + $5.80 = $5.80 per month
The Amazon API Gateway free tier includes one million API calls received per month for up to 12 months.
API Gateway API call charges: 29 M API calls * $3.50 = $101.50
Total size of data transfers = 28.6 GB (assuming 1KB response payload size)
API Gateway data transfer charges: 28.6 GB * $0.09 = $2.57
Total API Gateway cost: $101.50 + $2.57 = $104.07 per month
Storage: 3000GB = $89.02
30 M PUT Requests = $150
30 M GET Requests = $12
Total S3 Cost = 251.02 per month
Total cost: $5.80 + $104.07 + $251.02 = 360.89 per month
Paradigm Shift
Using serverless resources fundamentally changes computing to become even closer to a shared resource. Instead of provisioning virtual resources to customers, AWS can handle the inner management by abstracting the underlying servers and running code for customers.
“Computing may someday be organized as a public utility just as the telephone system is a public utility… The computer utility could become the basis of a new and important industry.” — John McCarthy
Visit this Github repo for a complete tutorial walkthrough and code sample: https://github.com/awslabs/lambda-apigateway-twilio-tutorial
