How to make serverless framework boilerplates customizable

The Serverless framework lets you reference external JSON, YML and JS files using the syntax ${file:(fileName):propertyName}. However, you can't customize these external config files with runtime arguments.

A reader asked me:

“I have this boilerplate that is repeated many times in my serverless.yml, how do I reuse the boilerplate but override only specific fields?”

It’s an interesting question and here are some potential solutions.

Disclaimer: the below examples use the HTTP event source for the sole purpose of illustrating the solutions with a trivial configuration object.

YML anchors

The simplest solution is to use YML anchors. You can define and apply the anchor and override specific fields. For example:

If the boilerplate is large, then you might wish to put the boilerplate into a separate file. Unfortunately, I haven’t found a way to make anchors work with external config files in the serverless.yml. I'm not able to import the content of an anchor from another file (as below). Please let me know in the comments if you know what I have done wrong here.

A bigger problem with anchors is that it’s difficult to replace nested fields. For example, given the following anchor:

defaultPerson: &defaultPerson
  location: 
    city: Amsterdam
    country: Netherlands
  occupation:
    title: independent consultant
    expertise:
      - serverless
      - aws

Suppose I want to change location.city and insert a new field occupation.company. Look what happens if I apply the overrides before and after the anchor:

AFAIK, the only way to make this work is to declare multiple anchors:

defaultPerson: &defaultPerson
  location: &defaultLocation
    city: Amsterdam
    country: Netherlands
  occupation: &defaultOccupation
    title: independent consultant
    expertise:
      - serverless
      - aws
Yan: 
  <<: *defaultPerson
  location: 
    <<: *defaultLocation
    city: Amstelveen
  occupation:
    <<: *defaultOccupation
    company: theburningmonk ltd

It’s getting a bit messy though.

JS Proxy objects

The Serverless framework lets you reference an external JS file too. But you can only reference properties and not invoke methods. This makes it difficult to make a customizable boilerplate config.

For example, if I want to template the HTTP event source configuration but allow the method to be customizable. Ideally, I would like to be able to do something like this:

index:
  handler: index.handler
  events:
    - http: ${file(defaultHttp.js):withMethod("get")}
    - http: ${file(defaultHttp.js):withMethod("post")}

Unfortunately, since I can’t invoke methods, I would need to wrap them into properties, e.g.

const generateConfig = (method) => ({
  path: '/',
  method
});
module.exports.get = () => generateConfig('get');
module.exports.post = () => generateConfig('post');

Then I would be able to reference them in the serverless.yml:

index:
  handler: index.handler
  events:
    - http: ${file(defaultHttp.js):get}
    - http: ${file(defaultHttp.js):post}

That’s a two-step process. Fortunately, with ES6 Proxy we can do better!

We can return a Proxy that traps any attempt to access a property on the exported object. The attempted property name (e.g. get) is used to construct the actual config object we will return.

const handler = {
  get: function (obj, method) {
    return () => ({
      path: '/',
      method
    });
  }
}
module.exports = new Proxy({}, handler);

And this is what happens when I reference ${file(xyz.js):get}:

With this simple technique, I can create a boilerplate config which you can customize a field even if it’s deeply nested. For example:

const handler = {
  get: function (obj, city) {
    return () => ({
      location: {
        city, // customize a nested field
        country: 'Netherlands'
      },
      occupation: {
        title: "independent consultant",
        expertise: ["serverless", "aws"]
      }
    });
  }
}
module.exports = new Proxy({}, handler);

And import the config like this:

custom: 
  yan: ${file(defaultPerson.js):Amstelveen}

But wait! Can I insert/update a property at an arbitrary location, like in the defaultPerson example earlier?

Well, yes, you can do that by returning another Proxy from the first Proxy.

For example, the following config object lets you update either path or method, or insert a new field.

const _ = require('lodash');
const template = {
  path: '/',
  method: 'get'
};
const handler = {
  get: function (obj, path) {
    const x = _.cloneDeep(obj);
    return () => new Proxy(x, {
      get: function (obj, value) {
        _.set(obj, path, value);
        return obj;
      }
    });
  }
}
module.exports = new Proxy(template, handler);

You can then reference it in the serverless.yml like this:

functions:
  index:
    handler: index.handler
    events:
      - http: ${file(defaultHttp.js):path./index}
      - http: ${file(defaultHttp.js):method.post}

Which creates the endpoints as you’d expect:

endpoints:
  GET -https://xxx.execute-api.us-east-1.amazonaws.com/dev/index
  POST -https://xxx.execute-api.us-east-1.amazonaws.com/dev/

This is what happens when I reference ${file(xyz.js}:method.post}:

But wait again! Can you customize the config with more than one variable? E.g. what if I want to customize both path and method?

You can have a Proxy that returns another Proxy that returns another Proxy and so on. So conceivably it's possible.

As suggested by Philipp Beau, you can also create a scheme to support key-value pairs in a comma-separated fashion. For example, ${file(xyz.js):key0_value0-key1_value1}

I’ll leave that as an exercise to you, the readers. Let your imaginations go wild and let me know what practical problems you are trying to solve. (I love a good hack, but solving actual problems are always more rewarding ;-))

Hi, my name is Yan Cui. I’m an AWS Serverless Hero and the author of Production-Ready Serverless. I have run production workload at scale in AWS for nearly 10 years and I have been an architect or principal engineer with a variety of industries ranging from banking, e-commerce, sports streaming to mobile gaming. I currently work as an independent consultant focused on AWS and serverless.

You can contact me via Email, Twitter and LinkedIn.

Check out my new course, Complete Guide to AWS Step Functions.

In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. Including basic concepts, HTTP and event triggers, activities, design patterns and best practices.

Get your copy here.

Come learn about operational BEST PRACTICES for AWS Lambda: CI/CD, testing & debugging functions locally, logging, monitoring, distributed tracing, canary deployments, config management, authentication & authorization, VPC, security, error handling, and more.

You can also get 40% off the face price with the code ytcui.

Get your copy here.

Originally published at https://theburningmonk.com on August 5, 2019.