Serverless: Reusing common configurations across functions

In the final part of the series, We’ll build on top of our previous post and look into how we can break down our template files further by reusing common configurations across multiple functions instead of defining them repeatedly.

Configurations such as VPCs, Cloudwatch Alarms, Layers, IAM Role Permissions (and much more) which can be common across functions can be reused for better and efficient management. This would help in case the configuration has to change. You could simply change the configuration in one place and it would be reflected in all the functions!

Lets see how we can do that with the Serverless Framework!

Setting up the base code

Since we’re continuing from where we left off last time, We can clone this repository from the last part. We should have the following project structure:

- functions
    - randomGenerator
          - node_modules
          - package.json
          - package-lock.json
          - randomGenerator.js
          - serverless.yml    
    - uuidGenerator
          - node_modules
          - package.json
          - package-lock.json
          - serverless.yml
          - uuidGenerator.js
- .gitignore
- serverless.yml

Adding a VPC and Layer to the functions

For the purpose of this post, I’ll be reusing an existing VPC and an existing layer but the same principle should be applicable on other common configurations as well.

So, go ahead and add the VPC and Layer configuration in the configs of both the functions randomGenerator and uuidGenerator:

// functions/randomGenerator/serverless.yml
randomGenerator:
  /** previous config **/
  layers:
    - arn:aws:lambda:ap-southeast-1:XXXXXXXXXXXX:layer:lambda-commons-layer:21
  vpc:
    securityGroupIds:
      - sg-XXXXXXXXXXXXXXXXX
    subnetIds:
      - subnet-XXXXXXXXXXXXXXXXX
// functions/uuidGenerator/serverless.yml
uuidGenerator:
  /** previous config **/
  layers:
      - arn:aws:lambda:ap-southeast-1:XXXXXXXXXXXX:layer:lambda-commons-layer:21
    vpc:
      securityGroupIds:
        - sg-XXXXXXXXXXXXXXXXX
      subnetIds:
        - subnet-XXXXXXXXXXXXXXXXX

With layers, we’re specifying the ARN of the Layer that we want to attach to our lambdas and with vpc, we’re specifying the security group and the subnet ids. (Be sure to replace the dummy ids with the correct ones in your AWS Account).

Run a serverless print and this point and save the console output somewhere. We’ll need this for future comparisons.

Next, Run serverless deploy and we should be able to see the layer and the vpc settings attached to both of our lambda functions.

While this approach is okay for a small project, Its not very efficient and manageable when you have hundreds of functions and therefore, re usability will help alot in the long run!

Creating Common Configurations

1. Start by creating a folder “configurations” in the root project folder.
2. Create 2 sub folders inside: “vpc” and “layers”.
3. Create a file config.yml inside each of the sub folders.
4. Move the vpc related config from both the functions to configurations/vpc/config.yml.
5. Move the layer related config from both the functions to configurations/layers/config.yml.

// configurations/vpc/config.yml
vpc:
  securityGroupIds:
    - sg-XXXXXXXXXXXXXXXXX
  subnetIds:
    - subnet-XXXXXXXXXXXXXXXXX

// configurations/layers/config.yml
layers:
  - arn:aws:lambda:ap-southeast-1:XXXXXXXXXXXX:layer:lambda-commons-layer:21

The above steps should result in the following project structure:

- configurations
    - layers
          - config.yml 
    - vpc
          - config.yml
- functions
   /* files inside the function folder */
- .gitignore
- serverless.yml

6. Update the configs of both of our functions:

// functions/randomGenerator/serverless.yml
randomGenerator:
  /** previous config **/
  layers: ${file(./configurations/layers/config.yml):layers}
  vpc: ${file(./configurations/vpc/config.yml):vpc}
// functions/uuidGenerator/serverless.yml
uuidGenerator:
  /** previous config **/
  layers: ${file(./configurations/layers/config.yml):layers}
  vpc: ${file(./configurations/vpc/config.yml):vpc}

And with this, we’ve referenced the newly created configuration files in our function configs. We had to specify the configurations/ ARNs only once in our common configuration files and then reused those wherever needed.

Run serverless print at this point and compare the output with the previous one. Both of them should be the same. Run serverless deploy to verify the lambdas as well.

Taking a step further with stages!

We can take this a step further by splitting up our configuration based on different stages. For example, we might have different network settings for different environments and so on. So based on the stage that we’re deploying to, we can load the configurations relevant to that stage / environment.

To do this, we need to make some minor changes in our folder structure:

1. create 2 new files in resources/vpc folder: config.dev.yml and config.staging.yml (or any other stage that you wish to deploy to)
2. Copy the contents of config.yml to config.dev.yml and config.staging.yml. Remove config.yml. We wont need it anymore.
3. Update the Security Groups and the Subnet Ids in the config files based on the environment.
4. Update the configs of both of our functions:

// functions/randomGenerator/serverless.yml
randomGenerator:
  /** previous config **/
  vpc: ${file(./configurations/vpc/config.${opt:stage, self:provider.stage}.yml):vpc}
// functions/uuidGenerator/serverless.yml
uuidGenerator:
  /** previous config **/
  vpc: ${file(./configurations/vpc/config.${opt:stage, self:provider.stage}.yml):vpc}

Here, we’ve used Serverless variables “opt:stage” and “self.provider.stage” to reference our configuration file names.

If for example, “staging” is passed during deployment as the stage argument, the opt:stage variable would be set to “staging” and our respective config file (configurations/vpc/config.staging.yml) will be loaded. If no stage is provided then the default stage in self:provider.stage which is “dev” will be loaded and our dev configuration file will be referenced.

Run serverless print and serverless print -- stage staging to compare the outputs with the previous ones. The only difference you should see should be different vpc settings loading up for different stages. The rest of the output should be the same. Run serverless deploy and serverless print -- stage staging to verify the lambdas as well.

And Thats it!

This is yet again, a very simple concept but one thats extremely powerful, can be extremely useful in the long run as projects grow and easily portable to many other similar use cases.

Conclusion

This concludes my series of articles on the Serverless Framework. I had a wonderful time exploring the framework and an even better time documenting it! Feel free to reach out in case of any questions, concerns or feedback!

PS: The Source Code for this part has been uploaded on Github for reference.

This article is a part of my 5 Article Series on the Serverless Framework!

Part 1: Serverless: Managing environment variables efficiently with stages

Part 2: Serverless: Managing config for different environments with S3 and Bash Scripts

Part 3: Serverless: Creating light and lean function packages

Part 4: Serverless: Breaking a large serverless.yml into manageable chunks

Part 5: Serverless: Reusing common configurations across functions