A Gentle Introduction to CI/CD Pipeline with Github Actions
Improving your productivity by get rid of boring and time-consuming code deployment
Hi there! Hope you are doing well. In the recent 2021 Stack Overflow Developer Survey confirms that Docker is the #2 most loved tool beside Git and become a fundamental tool to being a developer. In the old-fashioned way, developers have to write code and then build Docker image base on the Dockerfile in local environment before push to Docker Hub. Most people agree with me that it took a lot of time and effort to do since the whole process is repeated whenever the source code changes (aka “integration hell"). But don't worry, CI/CD come to rescue our life.
So, what is CI/CD?
CI and CD are two acronyms frequently used in modern development practices and DevOps. CI stands for continuous integration, a fundamental DevOps best practice where developers frequently merge code changes into a central repository where automated builds and tests run. But CD can either mean continuous delivery or continuous deployment.
When we combine these 2 things together, we will have the most powerful weapon to deal with “integration hell” called CI/CD pipeline. Nowadays there are many tools that help us automate tasks within your software development life cycle like TravisCI, CircleCI, Jenkins, Buddy, FinalBuilder,… If you are familiar Github, they also provides Github Actions and you don’t need to use third party tool for your CI/CD pipeline.
GitHub Actions help you automate your software development workflows in the same place you store code and collaborate on pull requests and issues. You can write individual tasks, called actions, and combine them to create a custom workflow. Workflows are custom automated processes that you can set up in your repository to build, test, package, release, or deploy any code project on GitHub.
In this article, I will show you how to setup a full Github Actions CI/CD pipeline to build image and push to Docker Hub and finally deploy to server by
ssh. But before that, we need to understand what the components of Github Actions are and how they work.
Components of Github Actions
We come to detail of the components inside of Github Actions workflow. Github Actions use
YAML file for definitions of these components.
A workflow is a custom automated process that we can include in our repository to build, test, and deploy our source codes. We can have more than one workflows and it must be stored in
.github/workflows folder in the root directory of the source code and can be scheduled or triggered by an event.
An event is a specific activity that triggers a workflow. For example, activity can originate from GitHub when someone pushes a commit to a repository or when an issue or pull request is created. You can also use the repository dispatch webhook to trigger a workflow when an external event occurs. For a complete list of events that can be used to trigger workflows, see Events that trigger workflows.
A job is a set of steps that execute on the same runner. By default, a workflow with multiple jobs will run those jobs in parallel. You can also configure a workflow to run jobs sequentially. For example, a workflow can have two sequential jobs that build and test code, where the test job is dependent on the status of the build job. If the build job fails, the test job will not run.
A step is an individual task that can run commands in a job. A step can be either an action or a shell command. Each step in a job executes on the same runner, allowing the actions in that job to share data with each other.
Actions are standalone commands that are combined into steps to create a job. Actions are the smallest portable building block of a workflow. You can create your own actions, or use actions created by the GitHub community. To use an action in a workflow, you must include it as a step.
A runner is a server that has the GitHub Actions runner application installed. You can use a runner hosted by GitHub, or you can host your own. A runner listens for available jobs, runs one job at a time, and reports the progress, logs, and results back to GitHub. GitHub-hosted runners are based on Ubuntu Linux, Windows, and macOS, and each job in a workflow runs in a fresh virtual environment.
Let’s get it started
We use docker to build simple hello-world RESTful service that written by Golang. Dockerfile is an instruction step by step to build image. We use multi-stage builds in order to keep the image size as minimum as possible.
Add the env to repo secret
Encrypted secrets allow you to store sensitive information in your organization, repository, or repository environments. To make a secret available to an action, you must set the secret as an input or environment variable in the workflow file. In this project, we need add some these secrets below
DOCKER_HUB_USERNAME- Docker Hub username
DOCKER_HUB_ACCESS_TOKEN- you can either create an access token or use directly Docker Hub password to login for pushing image.
HOST- ssh host
PORT- ssh port, default is
USERNAME- ssh username
KEY- content of ssh private key. ex raw content of ~/.ssh/id_rsa
SLACK_WEBHOOK_URL- Slack webhook url for sending notifications about the status of the CI/CD pipeline after running.
Create workflow and configure events to trigger
Firstly, create a yaml file stored in
.github/workflows folder in the root directory of the source code. This example workflow will be trigger on push and pull requests on branches “main”, “staging” and “dev”. Besides workflow also can be trigger based on a schedule (aka cronjob).
We split our workflow into 2 main jobs: the “build” and “deploy”. The “build” similar to CI pipeline and the “deploy" similar to CD pipeline. All the jobs will run on the specific runners that you choose. So in this example, I will use
The “build" job
To simplify the build and push Docker image things, we just use existed action
docker/build-push-action@v2. It is built based on BuildKit which is a new Docker image builder. It brings many improvements and better performance. One of the most fanciest things that make me impressed is using a remote repository as cache. It is especially useful for your CI build where a cache folder might not be available and you would have cold builds for every pipeline.
With BuildKit, in addition to the local build cache, the builder can reuse the cache generated from previous builds with the
--cache-from flag pointing to an image in the registry. This also supports a lot of type cache exporters and in this i just
type=inline for our simple need.
The “deploy” job
To deploy our code to server, we need to execute remote ssh and run docker command. The action
appleboy/ssh-action can help us do the job and you don’t need to worry about security threats because it also provides many argument to protect your connection and prevent Person-in-the-Middle attacks.
At last but not least, after the running you need to know whether the CI/CD pipeline is success or not. We will setup simple Slack Webhook to alert the status of the build using this action
And voila! Everything works like a charm. We just have done setup our Github Actions CI/CD pipeline.
Here is the full configuration of Github Actions workflow:
Great ! Isn’t it ?
With just a few simple setup, we have an automated processes which trigger jobs based on different events. This solution may help you get rid of boring, time-consuming code deployment and moreover concentrate on business and code logic. The source code of this example project can be found here:
GitHub - toannhu96/go-github-actions: Simple go rest service with github actions to push image to…
Simple go rest service with github actions to push image to docker hub and deploy to server - GitHub …
Thanks for reading this article, I hope it was useful, feel free to discuss more.