How to Create Free Serverless CI/CD Pipeline: 3 Easy Examples
Is it even possible to make CI/CD flow serverless?
Just to be sure that we’re on the same page. Here is a short explanation what serverless means, taken from Martin Fowler’s blog:
Serverless architectures are application designs that incorporate third-party “Backend as a Service” (BaaS) services, and/or that include custom code run in managed, ephemeral containers on a “Functions as a Service” (FaaS) platform. By using these ideas, and related ones like single-page applications, such architectures remove much of the need for a traditional always-on server component. Serverless architectures may benefit from significantly reduced operational cost, complexity, and engineering lead time, at a cost of increased reliance on vendor dependencies and comparatively immature supporting services.
Also, you might want to check out my previous posts on serverless:
- Make any java web app serverless
- You serve lambdas too cold. Warm it up!
- You don’t need frameworks for a serverless API in Java
Should I even consider making the deployment serverless, or just use the plain old Jenkins or TeamCity? This is the first question I’ve asked myself when I had to somehow deploy a serverless application. Nowadays, everything may be serverless, CI/CD is not an exception. In fact, there are lots of ways of doing that.
Disclamer:
Many of the provided examples are based on AWS. I’m pretty sure there are as many solutions for Azure and Google Cloud Platform. For example, check out these:
1. Create a CI/CD pipeline for your existing code with the Azure DevOps Project
2. Serverless, DevOps, and CI/CD: Part 1, Part 2 and Part 3.
3. Bringing the best of serverless to you
The beginning. Not serverless.
This is not a serverless approach at all, but for the complete picture I want to mention these two, because before the serverless approach, I’ve tried to use them.
Bitbucket
If you’re hosting a project on Bitbucket, I think this is the easiest way to get CI/CD set up quickly.
Also, they have a free plan for really small projects limited by 50 minutes per month. The other plans are not too pricey, so I didn’t look at it as a problem.
In my case, the project is hosted on GitHub, so the real issue was the need of migrating the whole project to Bitbucket, just to have CI/CD which is an overkill in my opinion. However, if you’re already using bitbucket for hosting your git repository, consider using their pipelines, it looks awesome.
Shippable
This one was actually nice and simple, and I wouldn’t have to migrate my repository anywhere (as I would with Bitbucket). However, I quickly used all the free tier limits, and their minimum paid plan is $25/month, which I considered being a little high for a small project.
This was actually the point when I decided to find a serverless solution for CI/CD. Meaning, I don’t want to pay when it’s not used.
Serverless for the Rescue
This approach will save you tons of money at the beginning of any project when you have no paying customers yet. For more details on that, check out this article: How To Make Multiregional Application (And Pay Zero).
LambCI
If you try to search for a serverless solution for CI/CD on AWS, the first thing you’ll find is LambCI.
That is an ultra canonical serverless way of building, testing and deploying your project because it even uses the core serverless AWS service for executing your builds — AWS Lambda.
The installation is simple: create a GitHub token, and then launch the given CloudFormation template. I won’t duplicate all the installation instructions here since you can just use the project’s up-to-date docs for that.
One more thing, that is worth mentioning — it has Slack integration. We’ve tried it. Works perfectly:
We’ve tried using it at the beginning of a project, it worked fine for some time, but then we quickly hit the AWS Lambda limits. Here is the full list of limits you should definitely consider:
- No root access
- 5 min max build time
- Bring-your-own-binaries — Lambda has a limited selection of installed software
- 3008Mb max memory
- Linux only
The main problem for us has been the 5-minute limit. I assume it’s quite enough for many small projects. If so, give it a try!
LambCI + ECS Cluster
I didn’t want to give up on LambCI so quickly. The only easy solution I found to work around the limits is using a related to LambCI project.
The idea is to have a pre-configured ECS cluster for running your build tasks. This way you will not have the AWS Lambda limits at all. However, you will have servers (i.e. ECS cluster)… It’s not that serverless now.
Although, it can be fixed by tweaking LambCI a little bit, currently it’s still not.
AWS CodePipeline
AWS CodePipeline is a fully managed continuous delivery service that helps you automate your release pipelines for fast and reliable application and infrastructure updates. CodePipeline automates the build, test, and deploy phases of your release process every time there is a code change, based on the release model you define. This enables you to rapidly and reliably deliver features and updates. You can easily integrate AWS CodePipeline with third-party services such as GitHub or with your own custom plugin. With AWS CodePipeline, you only pay for what you use. There are no upfront fees or long-term commitments.
Now, it’s not the most famous AWS service, but it’s just highly underestimated.
CodePipeline is a very flexible way of building, testing and delivering your code to your users. You may separate your delivery process into stages, and a stage may also have multiple actions. Each action may accept any input and produce some outcome for further stages. Here is how our process looks like:
It reads code from GitHub and some sensitive credentials from S3, then tests and build everything, migrates databases and so on and so forth. You may even have a stage when a dedicated person must manually approve a build before it goes further (e.g. the approving people may be QA team). Also, it’s very easy to setup a multiregional deployment process. Just a very powerful tool.
The attractive part of CodePipeline is that you pay only for what you actually use while building your projects (except the CodePipeline subscription fee: $1/pipeline/month).
What’s Next?
Here I’ve described services for serverless CI/CD, but there is a kinda related topic that I’ve intentionally ignored: automatic serverless tests using AWS for your GitHub pull requests. This is what we’ll discuss further.
Comments, likes, and shares are highly appreciated. Cheers!❤️
