Serverless-Flow: A CI/CD Branching Workflow Optimized for Speed and Quality

The ephemerality of Serverless architectural components, coupled with the pay-per-use pricing model, allows us to have rapidly deployable, cheap and disposable environments when working with Serverless-First architectures. No longer are we constricted to Dev, Staging, UAT and Prod environments.

Instead, we can have any number of environments that are isomorphic to the real production infrastructure. This allows us to reimagine how our CI/CD can work, reducing the lead-time to release to production, while maintaining quality assurance.

“Serverless-Flow” is similar to GitHub Flow, but takes advantage of the ephemeral pay-per-use environments Serverless offers. There are 4 steps to getting to Serverless-Flow on your CI/CD:

1. Isolating Environment Deployments 🏝
2. Integration Test Coverage 👷‍♀️👷‍♂️
3. Branching Workflow — Serverless Flow 🌳
4. Cleanup ♻️

1. Isolating Environment Deployments 🏝

The basic pre-requisite to using the Serverless-Flow approach to CI/CD is to have service(s) that are deployable to isolated environments based on a single change to a deployment argument.

Note: These examples will focus on the Serverless Framework

In the Serverless Framework, people often make use of the stage option for this: serverless deploy -s [STAGE NAME] -r [REGION NAME] -v

The stage variable can be used by the Serverless Framework to make environment variables change, and also included in the naming of other resources to ensure resources don’t clash between stages.

The key is that the environments are isolated and don’t clash. If you deploy a Dev1 environment, you don’t want to have issues when deploying Dev2 because an S3 resource created as part of the service has a naming conflict. The simple solution to this is to include the variable you’re using for environment declaration in the naming of any collateral resource. Alternatively, some teams stop naming additional resources at all and let AWS generate unique names.

Some projects avoid stages all together to make this issue clearer to the team. API Gateway natively supports the concept of staging (generating per stage URLs), whereas services like S3 do not. It’s important that teams ensure collateral resources are made dynamic per-environment to prevent accidental impact to other environments and to allow deployments to proceed. One tactic to avoid stages can be to add a custom option to your Serverless deploy: serverless deploy --infraStackName staging

Just like a stage, this deployment argument can be included in resource naming to avoid conflicts:

service:
    name: my-service-${opt:infraStackName}
...
Resources:
    StaticBucket:
        Type: AWS::S3::Bucket
        Properties:
            BucketName: ${self:service}-bucket

2. Integration Test Coverage 👷‍♀️👷‍♂️

To have trust in CI/CD all the way to production we need to verify our changes have not broken existing functionality, and that they deliver the new functionality. Teams should no longer rely on manual regression testing and instead, automation should be the primary verification check.

Serverless architectures are best tested via Integration Testing on the deployed infrastructure. Mocking is rarely appropriate, and instead, real scenarios should run on the real underlying services.

It’s key that the integration test suite can be run on any environment, automatically conducting all required setup and teardown of data.

For example, if a test runner like jest is used, arguments should be passed for the stack/stage name and AWS Profile to use for verification (e.g. asserting the content of an S3 bucket is correct).

Note, a future deep dive article will demonstrate how integration testing of Serverless Architectures can be achieved in Jest.

3. Branching Workflow — Serverless Flow🌳

Serverless-Flow, as the name suggests is based on many of the same principles as GitHub Flow. GitHub Flow in short proposes:

1. Create a branch from main with a descriptive name.
2. Add commits onto that branch locally, pushing regularly.
3. Open a Pull Request to main
4. Code Review
5. Deployment to production before final merge to main

https://guides.github.com/introduction/flow/

Serverless-Flow avoids the verification in production step by making use of the ephemeral environments available to us in Serverless.

1. Create a branch from main with a descriptive name.
2. Add commits onto that branch locally, pushing regularly.
3. Open a Pull Request to main
4. Code Review
5. Automatic deployment to new environment named after Pull Request number. e.g. serverless deploy --infraStackName pr-42 (triggered by PR open or push to existing PR)
6. Integration Test Suite runs against PR-x environment to verify changes.
7. Optional: Functional Review on PR environment (environment will have a subdomain for the PR number, allowing a preview of the feature)
8. Deployment to Production (before or after the merge to main)

Note: GitHub Flow is actually not strict on where verification happens. “Different teams may have different deployment strategies. For some, it may be best to deploy to a specially provisioned testing environment” — therefore Serverless-Flow is a form of GitHub Flow, where we are strict on where verification happens.

4. Cleanup ♻️

Finally, it’s not great to have many stacks building up so we need to do some garbage stack collection. This can be achieved by a Lambda cron, automatic cleanup on merge to main or a lightweight stack management API that allows stacks to be claimed, released and even reused*.

Read more about stack reuse and management APIs with our open-source Table Lock project.

Reducing your Serverless Integration test runtime on CI from 40 minutes to 7

Conclusion

Serverless allows an infinite number of environments. If we structure our infrastructure well we can create innovative CI/CD processes that allow teams to deploy faster, with better verification & QA processes.

Serverless Transformation is a Theodo initiative. Theodo helps companies leverage Serverless through expert delivery and training teams.
Learn more about Theodo and speak to one of our teams today!

If you like content like this consider subscribing to our weekly newsletter!