Slalom Build
Published in

Slalom Build

Solving cold starts using .NET Core — A Deeper Dive

Down the rabbit hole on what impacts cold start times

1) Code and related configuration

These includes actual code changes and compilation options like enabling Ready to Run.

2) Host configuration

Configurations that are specific to the hosting technology utilized. For example, in AWS we’d choose the runtime for our Lambdas, and for functions in Azure we’d configure if they are hosted in Linux or Windows.

Code and related configuration

Ready To Run (R2R)

If you are using Serverless and not using Ready To Run (R2R) yet, please start using it right away. It’s an easy way to reduce your cold starts by 30-80%. If you don’t know how to apply it to your project, you can read the first part of this this post here. Also you can find example code here. I’ll assume you are using R2R for the rest of the guide.

Which libraries we use

Not every library takes the same time to load. If you are referencing any library that is not capable of being compiled ahead of time (AOT), the time saving benefits of R2R will not apply to those libraries. Try to avoid referencing libraries that are using .NET Core 2.1 and older (or .NET Standard 1.2 and older).

About AutoMapper

Even though I personally enjoy working with AutoMapper, it is tuned to deliver multiple mappings during a long period of time. Another way to say this is that it is not tuned to do the first mapping fast.

Example start times for manual mappings, AutoMapper and Mapster

Newtonsoft

Newtonsoft.json is a fantastic library, but its slower performance is the reason Microsoft created their own implementation. If you replace Newtonsoft.json with System.Text.Json, you will also see a reduction in start times!

Initial calls

The .NET assemblies are written in intermediate language (CIL) and need to be compiled before execution. The “traditional” compilation is done using the Just In Time compiler (JIT).

Example with Newtonsoft

Here is an example of how to initialize Newtonsoft:

var initializationString = JsonConvert.SerializeObject(new { property = 1 });

Example with Entity Framework Core (EF Core)

Using the same idea as before, here is an example on how to initialize EF utilizing the MySQL provider, Pomelo. We just open and close the connection without actually loading any data.

var mySqlServiceRequestOptions = new DbContextOptionsBuilder<MyContextClass>();mySqlServiceRequestOptions.UseMySql(myConnectionString);var ctx = new MyContextClass();ctx.Database.OpenConnection();ctx.Database.CloseConnection();

Host configuration

The primary focus here is on code and code configuration, but I want to take some time to explain how different settings or hosting options can affect cold starts as well.

Operating system in Azure

When creating and configuring an Azure Function, you can choose if you want to use Windows or Linux as your operating system.

Memory configured in AWS

AWS Lambdas scale CPU based on the amount of memory that is configured. This will impact cold start times and are directly impacted by CPU processing power. But there is a downside, configuring more memory incurs a higher price per hour.

Specific runtime selected

You have multiple options to choose from for .NET runtimes:

  • .NET 3.1: Now that AWS supports .NET 3.1 natively, this is the go-to option for both Azure and AWS in most cases. A big caveat is that the AWS framework uses the System.Text.Json library, and if you add Newtonsoft as well, it may add unnecessary extra time.
  • Create your own container or framework: This is the most complicated option but it can definitely bring great results. I explain how to build your own framework for AWS Lambdas in this article.

Dedicated and pre-warmed instances

Having dedicated instances reduces the amount of cold starts, not the time they take. This lowers the average time that a user needs to wait, but when a cold start happens, it takes exactly the same amount as before. The idea is that we have instances that are always on since they will not incur cold starts.

Conclusion

When you’re working with Serverless Computing, it’s paramount to address cold start times to get a responsive application. If you approach Serverless Computing to create an API in the same way you approach a traditional deployment (eg: A Kubernetes cluster), you’ll probably have inconsistent response times that can annoy users.

Glossary

Tiered compilation

https://docs.microsoft.com/en-us/dotnet/core/whats-new/dotnet-core-3-0#tiered-compilation

Run-time compilation settings

https://docs.microsoft.com/en-us/dotnet/core/run-time-config/compilation

Explanation of Ready To Run:

https://docs.microsoft.com/en-us/dotnet/core/whats-new/dotnet-core-3-0#readytorun-images

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Diego Garber

Solutions Architect for Slalom Build Chicago currently focused on cloud technologies and .Net Core