Reducing Lambda bundle size with EsBuild and Lambda layers

Serverless is becoming increasingly popular as it lifts the burden of having to manage your servers and is more economical than cloud containers or on-premise servers. Popular BE frameworks like NestJS allow you to switch from containerized to lambda quite easily. https://docs.nestjs.com/faq/serverless

While by just transforming your monolithic backend into one lambda, you’d already see the benefits like easier management, less cost, and, maybe, improved performance. But the performance aspect is the most difficult, as having one “fat” lambda with a huge bundle size will lead to slow cold start times. Since during cold starts, Lambda downloads all your code, and the bigger your bundle is — the longer it’ll take to download.

In this article, we’ll explore the approaches we use at MiKi to optimize that using Lambda layers, EsBuild, and smarter BE framework choices.

Choosing the right framework

First, let’s see how much overhead a modern fully-fledged framework adds. Let’s say we want to build an AppSync Api with a lambda behind it, along with some basic authorization.

I’ve left out quite a few dependencies; only included the core ones, to make the comparison easier.

But do we need all those dependencies, or can we write something from scratch using mostly what NodeJS gives us? Instead of NestJS’s dependency injection, we can use tools like tsyringe or typedi, etc. We can recreate the graphql annotations that NestJS gives us with type-graphql. And like in the NestJS example, we’ll also need graphql and typeorm.

I’ve left out quite a few dependencies; only included the core ones, to make the comparison easier.

Already, it’s three times smaller than before, but we can do better

Splitting into multiple smaller lambdas

We’ll need to do some re-architecting to decouple our code and make many small, independent lambdas. This will reduce the bundle size even further, since every lambda will only import the dependencies it needs. For example, auth dependencies can only be imported by your auth lambda. Or, if you have a number of big libraries for some specific functionality, you can extract this into a separate lambda, to reduce the bundle size of other lambdas.

Note that I’m not talking about microservice architecture here because all our lambdas can still use the same database.

Making our bundling speed extremely fast

NestJS uses webpack under the hood, which might be one of the slowest bundlers. Since we got rid of NestJS, we can choose a more performant bundler. One of the best options is esbuild — “An extremely fast bundler for the web” ©.

Faster bundling will improve the performance of your CI/CD by 10–100x allowing your developers to work more productively.

Using Lambda layers

Our final step on the way to the tiniest bundle you’ve ever seen is to leverage Lambda layers. It allows you to move your common dependencies to a “layer”, then every lambda would only bundle its unique dependencies, accessing everything else from the layer. A layer is just a zip file, where we can put our node_modules and package.json . Or, we can put only the package.json and create a step in our CI/CD pipeline that would install the dependencies.

Due to the way node_modules resolution works

Node.js starts at the directory of the current module, adds /node_modules, and attempts to load the module from that location.

If our lambda will not find a dependency in local node_modules, it’ll go one level up and search in our layer. Let’s adjust our esbuild config to only bundle the unique dependencies.

const { build } = require('esbuild');
// Needed to make decorators work
const { esbuildDecorators } = require('@anatine/esbuild-decorators');
const pkg = require('./layer/nodejs/package.json');

const fs = require('fs');
const path = require('path');
const externals = [
  ...Object.keys(pkg.dependencies || {}),
  ...Object.keys(pkg.peerDependencies || {}),
];

function getEndpoints() {
  const lambdasFolder = path.join(__dirname, './src/lambdas');
  const files = fs.readdirSync(lambdasFolder);
  return files
    .filter((file) => file.endsWith('.ts'))
    .map((file) => path.join(lambdasFolder, file));
}

async function bundle() {
  const tsconfig = path.join(__dirname, './tsconfig.json');
  return build({
    platform: 'node',
    target: 'node14',
    bundle: true,
    sourcemap: false,
    plugins: [await esbuildDecorators({ tsconfig })],
    entryPoints: getEndpoints(),
    outdir: path.join(__dirname, 'dist'),
    tsconfig,
    format: 'cjs',
    treeShaking: true,
    external: externals,
  }).then((value) => {
    // eslint-disable-next-line no-console
    console.log('Build completed: ', value);
  });
}

bundle().catch((err) => {
  // eslint-disable-next-line no-console
  console.error(err.message);
  process.exit(1);
});Conclusion

Conclusion

I went through the same transformation in an averagely-sized commercial project, and, compared to one fat lambda, the bundle size of each of our small lambdas was on average 20 times smaller. So next time you build a project from scratch, or will want to improve the performance of an existing application, I advise you to re-evaluate the need for a fully-fledged BE framework. Since most of the time, a more minimalist approach will give you way better performance and, consequently, lesser costs.

At MiKi(https://miki.digital/) we help businesses make stunning websites with a blazing-fast and cost-efficient cloud backend.
Interested?
Feel free to give us a call at +447397590078, book a meeting at https://calendly.com/miki-digital/30-minute-consultation or feel out the contact form at our website https://www.miki.digital/contact