How We Sped Up Our TypeScript Serverless Builds Ten Times (70 Lambdas Under 1 Minute)

TL;DR: Previously, we used serverless-webpack to bundle our TypeScript lambdas, and it took around 10 minutes to package our 70 lambdas service. We switched to serverless-esbuild, which sped up our builds almost ten times, packaging the 70 lambdas service in under a minute. We are using decorators, but emitting decorator metadata is not supported by esbuild. We fixed a bug and submitted a PR to the esbuild plugin that emits the decorator metadata called esbuild-plugin-tsc, but until the PR is merged and a new version is released, we’re using our fork called @emarketeer/esbuild-plugin-tsc.

We’re using Serverless Framework with AWS, and our lambdas are written in TypeScript. One important thing to mention when talking about building lambdas is that bundle sizes affect cold starts as shown here. That’s why we wanted to make each lambda an individual bundle (setting package.individually flag to true), only packaging the required dependencies using tree shaking.

We used serverless-webpack, a Serverless Framework plugin that uses webpack to package lambdas, and it was fine at first. We were familiar with the tool, and it did its job.

The problems started when our services started growing. The build process started using too much memory, and our pipelines started crashing. We’re not the only people who had these issues, so we applied some workarounds from the related issue on GitHub. Our builds stopped failing, but when one of our services became as big as 70 lambdas we found ourselves waiting for more than 10 minutes to deploy the service.

One could argue we should’ve split the service into several smaller ones and my answer is maybe. I don’t believe that build tools should dictate the size of a microservice, though.

We tried many different things with webpack, like switching loaders and minifiers. We also tried using webpack 5 cache, but nothing gave us a significant speed boost because the limiting factor was the bundler itself.

Switching to a Different Bundler

We tried looking at alternative bundlers, and several looked promising. SWC is fast, but its bundling is still in beta and will be replaced in the future. Esbuild, on the other hand, is fast, already has bundling, and doesn’t require any configuration.

When I tried using it, it did bundle the 70 lambdas service in a reasonable time, but almost no tests have passed. That’s when I learned that esbuild doesn’t support emitting decorator metadata. And we are using decorators a lot for our ORM mappings, and dependency injection. We also needed to bundle a native binary for some lambdas. We sat aside some time in one of the sprints to investigate ways of solving these issues and migrating to esbuild.

There already was an esbuild plugin that promised to solve the decorator metadata issue esbuild-plugin-tsc, but I knew it didn’t work from my previous testing. Thanks to a nice person, there was already an issue opened on GitHub that clearly stated what the problem was. I forked the project, fixed the bug, and created a pull request. Since I needed that fix ASAP, I also published the fixed version of the plugin under @emarketeer/esbuild-plugin-tsc. Including a native binary was surprisingly easy, I just added a package pattern as described here for specific functions. We also had to tweak the concurrency setting in the serverless-esbuild so that we don’t get out of memory errors on our CI runners.

The speed increase is amazing. What previously took almost 10 minutes to be packaged on my laptop was now taking just 50 seconds. Our CI builds went from 12 minutes to around 1 minute (including the deployment). The resulting bundle sizes are smaller or on par with what we got with webpack. The total deployment time for the 70 lambdas service on CI now is ~70 seconds.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store