Ddosify
Published in

Ddosify

Cold Start Comparison of AWS Lambda, Cloudflare Workers and AWS Cloudfront Function

AWS Lambda vs. Cloudflare Workers vs. AWS Cloudfront Function cold start comparison with Ddosify Cloud.

There are many serverless platforms such as AWS Lambda, Azure Functions, Google Cloud Functions, and Cloudflare Workers. Most serverless platforms spin up a container after the first request and use the same container for the following requests. After inactivity and a period of time, the container will be closed. Spinning up a container after a first request takes time, called the “cold start problem.”

To solve this famous cold start problem, Cloudflare designed a different approach and they claim 0ms cold starts all around the world. Internally, Cloudflare Workers runtime uses the fast and powerful Chrome V8 Engine to run Javascript code. Each process is sandboxed using “isolates” so that the different users’ codes are run in a single process in an isolated and secure way. This reduces the overhead of processes.

Figure 1: Isolates Diagram. Ref: Cloudflare — Containers on the Edge

Cloudflare Workers functions are deployed on the global Edge Network in 100+ countries and 200+ cities across the globe within seconds. Thanks to Anycast, the incoming request will be routed to the nearest data center. Thus, despite the traditional serverless platforms, you don’t need to create a new endpoint for each location to achieve the same latency all around the world. And you shouldn’t face the cold start problem.

This article compares Cloudflare Workers, AWS Lambda and AWS Cloudfront Function for cold start problems. While Ddosify Cloud can send requests from 100+ countries, AWS Lambda region support is limited, so testing the latency in relatively high traffic worldwide wouldn’t be fair. For this reason, we decided to use only Europe as a request origin with focusing on the cold-start problem.

Test Bed

AWS Lambda

On the AWS Lambda side, we created a basic Node.js 14.x that returns HTTP 200 for all the requests and enabled Function URL which is a new feature that you can call lambda without an API gateway. We changed the Auth type of Function URL to None for public access. This serverless function is deployed in the Frankfurt (eu-central-1) region.

exports.handler = async (event) => {
const response = {
statusCode: 200,
body: JSON.stringify('Hello from AWS Lambda!'),
};
return response;
};

Cloudflare Workers

We created a basic HTTP Handler that returns HTTP 200 for all the requests on the Cloudflare Workers side. Cloudflare functions are deployed automatically across the world. We used the Free plan for Cloudflare Workers.

addEventListener("fetch", event => {
event.respondWith(handleRequest(event.request))
})
async function handleRequest(request) {
return new Response("Hello from Cloudflare Workers!")
}

AWS Cloudfront Function

We created a basic HTTP Handler that returns HTTP 200 for all the requests on the AWS Cloudfront Function side. CloudFront functions are deployed automatically across 200+ CloudFront Edge Locations.

function handler(event) {
var response = {
statusCode: 200,
statusDescription: 'OK',
};
return response;
}

Now, we can generate requests for the AWS Lambda and Cloudflare Workers. We sent 10,000 requests linearly in 10 seconds (1,000 RPS) with the Ddosify Cloud load testing tool into both functions separately. Ddosify sent all the requests from 17 countries evenly from the Europe continent.

Results

Figure 2: AWS Lambda vs. Cloudflare Workers vs. AWS Cloudfront Function cold start comparison. We sent 10,000 requests linearly in 10 seconds (1,000 RPS) using the Ddosify Cloud.

AWS Lambda, Cloudflare Worker and AWS Cloudfront function handled all requests without a problem. The results show that the AWS Cloudfront Function is better than the Cloudflare Worker and AWS Lambda on the response times side. The results are shown in Figure 2 and Table 1.

AWS Lambda vs. Cloudflare Workers vs. AWS Cloudfront Function response times comparison using Ddosify Cloud.
Table 1: AWS Lambda vs. Cloudflare Workers vs. AWS Cloudfront Function response times (avg, max, min) comparison using Ddosify Cloud.

Cold Start Comparison

Since the Average calculation includes anomalies, we used the P90 Response Time metric to make a more accurate comparison.

The AWS Lambda obtained 333 ms response time on the P90 metric in the first second and 217 ms after 3 seconds. P90 continued for about 210 ms after 3 seconds. You may think 333 ms is not a big deal, but it is good to know that this result is the aggregated value of the 1K requests. You’ll see higher response times for the first individual requests.

The Cloudflare Workers obtained 219 ms on the P90 metric in the first second and then continued for about 213 ms until the end of the load test.

The AWS Cloudfront Function obtained 220 ms on the P90 metric in the first second and then continued for about 198 ms until the end of the load test.

These results show that there is still a cold start problem in the AWS Lambda. Cloudflare Workers and Cloudfront functions’ initial P90 response time is very similar to the response time until the end of the load test. According to the results, the Cloudflare Workers function and AWS Cloudfront Function do not have a cold start problem. AWS Cloudfront function is also faster than Cloudflare Workers.

Summary

We compared AWS Lambda, Cloudflare Workers and AWS Cloudfront Function for the cold start problem. The results show that the AWS Lambda suffers from the well-known cold start problem. Even if we choose the Free plan for Cloudflare workers, we did not observe any cold start problem. We also did not observe any cold start problem in AWS Cloudfront Function.

Cloudflare Workers is not the complete replacement for AWS Lambda. You can not use many packages that require Node dependencies like net/http in Cloudflare Workers because it is built on the V8 engine.

You can compare your serverless functions like AWS Lambda, Cloudflare Workers, Azure Functions, and Google Cloud Functions using Ddosify Cloud. You can see all the response times (i.e., Avg, Max, Min, P90, Stddev) and location statistics on a multi-location-based, open-source-centric, and codeless load testing platform.

--

--

--

Ddosify is a high-performance load testing solution. The mission of the Ddosify project is to create generic, no code and the most powerful load testing experience for developers, quality assurance engineers, and even non-technical people.

Recommended from Medium

How to raise money in web3 // DAO

Sending emails in Flask

What Sets CodeCast Apart

Get Ready to Feel SASS-Y

Caveats in C++ #2: Implicit conversions: size is not an int!

WSO2 IoTS Plugin: Building Visualizer

Bigtable backup for disaster recovery

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Fatih Baltacı

Fatih Baltacı

Co-Founder at Ddosify — High-performance, multi location-based, codeless load testing

More from Medium

Build an Email Sending Service with AWS Lambda

User sign-up example with Mantil, Go, JWT, AWS Lambda, SES and DynamoDB

Behind the Scenes: Argonaut and AWS Lambda

Understanding Lambda Execution Models