Implementing Serverless Framework for your existing AWS lambdas without breaking them!

Introduction

Imaging you have built a project and have multiple lambda functions created on AWS with triggers like API gateway setup and everything works fine. BUT! Now you have to implement some CI/CD solution to the project, This is where serverless framework comes in handy. Serverless allows you to deploy your AWS lambda functions along with the resources and helps you manage your code and the infrastructure. You can read more about serverless from here.

Now why i’m writing this article?. Well! if you implemented your lambda functions along with serverless framework and managed your resources through cloud formation you have nothing to worry, but if you are in a situation as stated before where you have all your lambdas deployed and now you want to implement serverless without breaking the deployed code specially if they are python lambda functions you are in the right place. I will be guiding you through everything you need to be doing right in this kind of a situation.

Step 01 - Setup Serverless framework and plugins

First thing you need to be doing is setting up serverless framework. If you have npm installed already then this is straight forward. Run

npm install g -serverless

Then we will be using two plugins they are :

• serverless-python-requirements
• serverless-deployment-bucket

we will be using python requirements plugin to help us correctly package the python lambda functions. this plugin will make sure all the files are bundled into the root path of the deployment zip file

Serverless deployment bucket plugin is used to upload and maintain our artifacts in a s3 bucket

To install these plugins first you need to go to your repository path where you have your serverless.yml file (Which we will create in our next step) then run below two commands :

serverless plugin install -n serverless-python-requirements

serverless plugin install -n serverless-deployment-bucket

How our Existing project looks

Deployed Funtions on AWS without being managed by aws cloudformation

Project structure

Step 02 — Creating Serverless.yml file

Alright its time to create our serverless.yml file. As you can see from above screenshots, we have three lambda functions deployed on aws which are not managed by cloudformation. We will create one serverless file to manage all our lambdas through a single cloudformation stack.

service: test-lambda-service
package:
  individually: true

frameworkVersion: '3'
plugins:
  - serverless-deployment-bucket
  - serverless-python-requirements
provider:
  stackName: test-lambda-stack
  name: aws
  vpc:
    securityGroupIds:
      - ${ssm:test_vpc_security_grp_variable}
    subnetIds:
      - ${ssm:test_vpc_subnet_01_variable}
  runtime: python3.8
  deploymentBucket:
    name: test-deployment-bucket
    serverSideEncryption: AES256
  iam:
    role: ${ssm:test_lambda_role}
  DeletionPolicy: Retain
functions:
    - ${file(./test-lambda-one/serverless_config.yml)}
    - ${file(./test-lambda-two/serverless_config.yml)}
    - ${file(./test-lambda-three/serverless_config.yml)}

create your serverless.yml file in the project base path where all the lambda funtion code is present.

alright! we have created our serverless.yml file to our lambdas. lets talk about what we have on it.

service - this is your serverless service name you can give anything

package individually -this packages our lambdas into individual zip files to be deployed

plugins -These are the plugins which i mentioned earlier in the setup section

ssm -you can see in my yammel file i have used ssm then followed by a variable name. using ssm you can access aws parameter store variables. This is done to prevent putting sensitive information into our repo.

provider -Under provider section you can have all your global variables which needs to be applyed to all the lambda functions such as iam role,python version, deployment bucket etc. (Note: you can override these properties under function wise configurations)

functions -under functions section we have to define configurations for our three lambda functions. You can see in my yammel file i have refferenced to three separate configuration files, this is done to make our yammel file looks clean. below is a code snippet of a function wise configuration in serverless_config.yml file

test-lambda-one-handler:
  handler: lambda_handler.lambda_handler #handler function
  name: test-lambda-one  #lambda function name
  module: test-lambda-one
  memorySize: 128
  environment:
    CONFIG: >-
      ${ssm:test_lambda_one_env_variable}
  package:
    patterns:
      - '!./**'
      - './test-lambda-one/**'

In the above configuration file there are couple of key things which is specific to python lambda functions . That is package patterns this tells serverless exactly what module to be packaged and the other properties are straight forward.

Step — 03 Creating the cloud formation stack

The next step is to create the cloudformation stack to onboard our deployed lambda functions.

First thing is we need to create our cloudformation template. We can achieve this by using the below command

sls package

Once you run this command it will generate .serverless file path inside your project. Inside this we will get a generated cloudformation json template. We can use this generated cloudformation template to create our cloudformation stack in AWS.

generated template

We can then go to AWS create cloudformation stack and use the generated template to import your existing resources. Once you have done importing the existing lambda functions. You are all set. Then you can run the below command.

sls deploy

Now if you go and check your created cloudformation stack. you will see that it has deployed your lambda functions smoothly without breaking your existing functions.

Conclusion

Ideally when you are going serverless. you should maintain your infrastructure as code from the get go. But if you are in a situation where you have multiple lambdas deployed having things like API gateways setup. You can onboard serverless framework to existing lambdas using this approach for onboarding an IaC framework to streamline your ci/cd process without breaking anything.