Bitbucket pipelines allows you to run your tasks directly in containers as docker-in-docker, so you can leverage containers from trusted sources like open source official container images.
It is very easy and fast to configure.
Perhaps is not as flexible as its competitors GitLab, Azure DevOps, Github Actions etc.
But if you are already using Atlassian tools, it will put the pipelines very close to the developers. With out-of-the-box integration with Jira software.
It has integration with VS Code, so you see after pushing code the status of the pipeline without leaving the IDE.
To make the solution complete, sooner or later you will need to run pipelines in your own infrastructure with self-hosted runners. Since August 2021 you can do that in Bitbucket cloud pipelines with self-hosted runners.
I am focusing here in docker runners, but new types of runners are being added constantly, Windows runners and Mac are also available. There is an open beta of a Linux shell for high load jobs which improves performance over the default docker runner.
This is the architectural diagram on how to use cheap self-hosted runners in AWS for your CI/CD jobs.
Creating the runners in Bitbucket
For these tests I created runners at workspace level, these runners will be available to pipelines in all the repositories of the workspace.
Runners can also be configured at repository level.
In workspace settings, there is a workspace runners section, simply add runners there.
For docker runners, it will display a docker runcommand that is used to start a runner.
The original command requires a bit of tweaking to run it as a daemon, it is intended to be run interactively.
So I simply add nohupat the start and &at the end, so the process remains on background and remove -it from the docker command we don’t need an interactive terminal here. This same command can be converted to a systemd service easily, I’m not doing that because my runners will be disposable in my case.
This command contains secrets, so we require a secure place to store it.
So we save these commands in SSM parameter store as a secure string, one per runner in this case.
This is the only manual part in this process, there is no api for bitbucker runners at this time.
This issue request could use some up votes :-)
AWS Spot fleet
Spot fleet provides you with a way to run groups of spot instances with some rules and maintain that capacity from multiple instance pools and AZs.
Optionally, you can run on-demand spot mixed capacity for stateless critical apps.
In this case, I am using only spot with diversifiedas the fleet allocation strategy, I want to minimize the interruptions. So If I distribute my load between instance families and availability zones, the probabilities of the whole fleet being interrupted should go down.
The spot fleet will make sure of maintaining fleet_type="mantain”the capacity from availability zones and instances families available.
AWS Parameter Store
As I showed before, the configuration of the Bitbucket runners is stored in SSM parameter store as a secure string encrypted with a customer key.
This secures the communication of the keys to register as a runner in Bitbucket pipelines.
Terraform template files for user data.
With a bit of Ansible help, to prepare the instances to run docker in a vanilla Redhat machine using this awesome Ansible role.
I’m calling the role directly with some extra-vars and -m include_role.
The complete userdata.sh template looks like this:
The Bitbucket pipeline
With public Bitbucket runners, we can create the self-hosted runners executing terraform.
Here is a very simple custom pipeline where you can pass an action variable to execute the terraform command plan, apply, destroy.
It assumes you can access your AWS account from Bitbucket pipelines, you should have at least your AWS keys as repository variables.
You can execute by run pipeline, select custom and pass the action to execute.
After running the pipeline you should see your runners registered and start using them.
Then adding runs-on your tasks and a label that you added in the runners the tasks will run in your infrastructure.
