GA in November 2019, Cloud Run quickly established itself as a game changer for serverless applications in the public cloud. In the case of Cloud Functions, the number of multiple language runtimes was insufficient, so it felt like a busy situation to follow other companies’ FaaS (Functions-as-a-Service). Meanwhile, Cloud Run appeared. Cloud Run has a larger instance size compared to Cloud Functions. In other words, CPU and memory can be used more abundantly. In addition, with a few clicks, Cloud Run can split and process traffic by version and control the number of requests (maximum requests per container) that a function instance can handle. Comparing several characteristics, Cloud Run is backward compatible with Cloud Functions. Cloud Functions 2nd generation has been released so that Cloud Functions users can enjoy the infrastructure of Cloud Run. And on August 3, 2022, it became GA.
Cloud Functions (2nd gen) is Google’s next-generation FaaS. Works based on Cloud Run and Eventarc! Therefore, when creating with Cloud Functions 2nd-gen, service is also created in the Cloud Run menu. In effect, the user is using the Cloud Run service through the Cloud Functions menu. Applications previously operated as Cloud Functions can also be easily transferred. In this article, we will look at the differences between Cloud Functions 1st gen and 2nd gen. ( * When talking about the environment of Cloud Run, it is called container instance, and in the case of Cloud Functions, it is called function instance. In this article, Cloud Functions 2nd-gen, which we are talking about, operates in the Cloud Run environment. It is correct, but refer to the name of the service as both 1st and 2nd are marked as function instances regardless of the service name)
Instance size
For 1st gen, 2 vCPU, up to 8GB.
2nd gen can be up to 8 vCPU 32 GiB (preview).
Excluding preview, 4 vCPUs up to 16 GiB are supported. The interesting thing is that 2nd runs in the Cloud Run environment, so there is a difference in memory calculation method from Cloud Functions 1st gen. That would be the difference between GB and GiB.
Timeout
1st gen runs for up to 9 minutes. For the 2nd, just like Cloud Run, it runs for 60 minutes.
Cold Start
Even the 2nd can’t avoid a cold start. Compared to the 1st gen, there is not much difference (in theory). Also, both can specify a standby instance like below. In this case, you only need to set this up for businesses where responsiveness is important, as you will be charged for idle instances.
When created with Cloud Functions 2nd-gen, CPU is allocated and used with “CPU is only allocated during request processing” status. This means that when there are no requests, the CPU is not allocated, so you are not charged.
2nd-gen operates based on Cloud Run, so if the minimum number of instances is not specified, the idle state does not last more than 15 minutes. That is, the function instance in it is terminated, and subsequent requests become a cold start. The 1st-gen is not significantly different from the 2nd-gen, although there is no official guide on how long it will remain idle.
concurrency
1st gen handles only one request per function instance. That is, when a single request comes in, one function instance is started to process the request. If two requests come in sequentially, both will be processed by one function instance. At this point, the first request will be cold start, but the second request will be processed immediately. If two requests come in at the same time rather than sequentially, two function instances are executed to handle them. In other words, concurrency is 1. Below are the images used in Cloud Run, but can also be applied to Cloud Functions.
On the other hand, 2st gen is based on Cloud Run, so one function instance handles up to 1,000 requests. This value is CPU dependent. If created with 2nd gen, concurrency is set to 1. It follows the default values of Cloud Functions. To change this setting, go to the Cloud Run menu and edit it. As mentioned in the introduction, functions created with the 2nd gen are visible in both the Cloud Functions and Cloud Run menus. However, for detailed settings that are not on the Cloud Functions side, you need to go to the Cloud Run menu.
The default value of “maximum requests per container” for services created in the Cloud Run menu is 80. This value is concurrency.
Anyway, the fact that one function instance can perform multiple processing is a huge advantage depending on the situation. Of course, if one request uses most of the resources of the function instance, “maximum requests per container” should not be set to 1 or more, even for the 2nd gen. It should be flexibly adjusted and used according to the nature of the application.
concurrency test
Generate traffic through Apache JMeter and check the concurrency of 1st and 2nd gen. There are tests in the official documentation as well, but the results will be different. The computing power that generates the traffic will also be different, and the size of the function instance will also be different. The specifications used for testing are as follows.
- Instance size: 2GB / 2 GiB (concurrency: 10)
- Number of threads (users): 400
- Number of calls per second: 3
- Ramp-Up Time: 1 second
- Traffic sending location: Republic of Korea
That means 400 users are sending 3 requests per second. First, we prepared a function for testing. The source code is the basic example code that appears when “CREATE FUNCTION” is pressed. Changed only Runtime from default Node.js to python (no specific reason to change Runtime)
Apache JMeter was also prepared.
Now that we’re all set, we can start streaming HTTP traffic. Here we generated traffic for about 15 minutes and compared the results. Below are the changes in Active instances.
Since it is a simple application that responds “hello world” when a request comes in, you can see the instance increase by the maximum (number of users) only during the cold start period. From then on, you can see that the function instances are only keeping up to a level that can handle all incoming requests. Processing is completed in approximately 4–15ms, so in the case of 1st-gen, 40–50 function instances are sufficient to process 1,200 requests per second. On the other hand, 2nd-gen is possible with about 10 function instances. This will definitely lead to cost savings!
What was the cold start situation in the beginning when 400 clients were connected simultaneously in 1 second? Let’s check it through JMeter’s Summary Report. The first image is 1st-gen and the second image is 2st-gen. (* Don’t worry about Error %. This is an error that occurred because the socket was abnormally terminated while JMeter was shutting down in the middle.)
Let’s take a look at some meanings. First, we can look at how the cold start affected the Max value. 1st-gen sometimes took up to 10 seconds, while 2st-gen took up to 3.5 seconds. The average processing time was 200 ms, which was not different between the two versions.
Wrap-up
In early 2022, when Cloud Function 2nd-gen first came out, cold start was not optimized, making it difficult to use in production. However, looking at the current test results, it is difficult to find a reason to use 1st-gen because it has already surpassed 1st-gen. Of course, if you are using Container Registry, you need to do a little effort, such as having to go to Artifact Registry to use 2nd-gen. I remember raving about Cloud Run as a game changer when it first came out, but now Cloud Functions is on the shoulders of giants. The future of Google’s serverless applications is even more curious.
