Building a Docker CI/CD Pipeline with GitHub Actions

Add a Must-Have Tool to Your DevOps Toolbelt

Edoardo Nosotti
Jan 4 · 10 min read
Photo by Isaque Pereira from Pexels

I will come clean: I began using GitHub Actions only recently and I want to make amends. I missed out, possibly because CI/CD Actions became generally available only in late 2019 and by then I was already well-acquainted and satisfied with other tools I used for ~5 years. Not trying something new just because the good ol’ stuff works is, indeed, against my own principles and overall a very bad idea. So eventually I tried the GitHub Actions and found them to be amazing!

GitHub Actions offer a complete and powerful set of functionalities, a lot of extensions are available on the Marketplace and GitHub offers a generous free tier also for private repositories (at the time of writing).

Actions are also very easy to enable. In this article, I will show how to use the Actions to:

  • Lint and test the code when new commits are pushed to GitHub
  • Build and push a Docker image to the Docker Hub when a release is published (in addition to linting and testing)

Getting Started With GitHub Actions

  1. A working knowledge of Git and Docker.
  2. A basic understanding of the YAML syntax is also required. You can learn YAML from its official website, or this basic tutorial kindly provided by the Ansible people.
  3. Fork my test application repository and remove all files under .github/workflows/ to practice with the code in this article.
  4. A valid user account on the Docker Hub is required.

Adding Actions to a repository is as simple as adding a YAML file and storing it under the .github/workflows/ directory. As soon as the YAML file will be committed and pushed, the Action will be enabled. Cool.

Your first Action: linting the code and running tests

The following YAML file contains a simple Action to run a linter on the Python source code and execute the tests every time a commit is pushed to GitHub:

If you store the code above into into a .github/workflows/run_tests.yml file, then commit & push the change to GitHub, you will see the Action running:

Go to your repository page on GitHub and click on Actions.

First, please review the comments and the name elements in the code above. They provide useful insights to better understand the process.

It is worth noting that several elements in the Actions YAML file can take:

  • Single values: on: push
  • Arrays of values: on: [push, pull_request, release]
  • Maps of values and subsettings:

Filters are also supported:

With the configuration above:

  • On push all changes to branches without a / in the name will trigger the Action ('*'), plus changes to all branches with a hotfix/ prefix in the name.
  • Changes to all branches, regardless of their name, will trigger the Action on a pull_request ('**').

To better understand how values and filters work, take some time to review the Workflow Syntax reference and the Filter pattern cheat sheet. The list of Events that trigger workflows will also come in handy. In this tutorial, we will only use the push and release events.

The steps in the code above will be executed once for each python-version listed in the matrix. It is also possible to run the same jobs on different OS environments and set exclusion patterns to avoid running the tests on specific combinations of OS & runtime version. The official Python guide for GitHub Actions provides a dedicated paragraph on the subject, in case you are interested in implementing complex testing scenarios.
To get an up-to-date list of supported OS environments, check the official runs-on documentation.

Another very important thing to note are the uses elements. The base runs-on environments are deliberately “bare bones”, so they can be customized at will. Even very basic functions, such as checking out the code from your repository, need to be “imported”. This indeed keeps the Actions very flexible and interoperable.

Let’s review this code snippet:

uses: actions/setup-python will import and run this Action from the Marketplace, at runtime. @v2 targets a specific release of the Action. Adding a release version ensures your Action will not be broken by changes to the imported Action. The Actions you import might also support configuration options, so remember to check their documentation. You just need to navigate to: https://github.com/{action_name}, such as: https://github.com/actions/setup-python, to get it.

Also, in the code block above, the actions/setup-python step will be executed once per each python-version specified in the matrix.

To find more Actions to import into your own workflow, visit the GitHub Martketplace. Always verify the imported Actions code before using them in your workflow, especially if they don’t come from “trusted” sources!

You can also share your own Actions with the community on the Marketplace.

Finally, the steps with the run elements are really easy to understand: they just run the commands they are provided with on their host container.

If you have followed all the instructions so far, you should have a copy of my test application with a single YAML file under .github/workflows, the .github/workflows/run_tests.yml file containing the Action code shown above. Now:

  1. Commit the change
  2. Push the commit to the GitHub remote
  3. Navigate to the Actions tab of your repository page on GitHub

Wait for the workflow to complete its execution, then click on its “title” (it should be the same as the commit message, in this case):

A list of jobs, as described in the Action code, will be presented. In this case, we have only 1 job. Click on it:

Next, a list of all executions of the steps list will be presented. As explained above, if you provided a matrix with multiple Python versions, you should have an entry for each Python version. Pick one:

A list of all the steps will be presented, named as in the name elements in the Action code above. Each “step” line can be expanded to get its output:

Also, the command lines can be furtherly expanded to get some more useful information:

Congratulations! You have just created your first Action.

Adding Actions for Docker and the Docker Hub

In the previous chapter, we used an Action to lint the code & run unit tests whenever a change in the code is pushed to GitHub. Now, I want GitHub to build a Docker image and push it to the Docker Hub each time I publish a release on GitHub.

Add a new .github/workflows/build_push_docker.yml file to the application sources and paste this code into it:

Note the ${{ secrets.* }} variables in the Action code. Actions can use variables and expressions. In order to push an image to the Docker Hub (or any other registry indeed) we need to:

Credentials absolutely don’t belong into code files that are committed to a repository. The registry and repository names are not sensitive data, but I’d rather not hardcode such details into the code as well. Hardcoded configuration sometimes makes it easier to break things and harder to pinpoint the issue.

So we will add the following variables:

  • DOCKERHUB_USERNAME (username of your Docker Hub account)
  • DOCKERHUB_PASSWORD (password for said Docker Hub account)
  • DOCKERHUB_REPOSITORY (path to the Docker repository)

as GitHub Repository Secrets. Such values are stored in encrypted form by GitHub and are safe to use with Actions. Secrets can be created at repository, environment or organization level. Let’s keep it simple and create three repository secrets for the variables above: from the Settings / Secrets repository configuration panel on GitHub:

Ensure that all Secrets are created:

Note: since the introduction of “rate limits”, authenticating on the Docker Hub is always recommended, also for just pulling base images.

Now, go back to the Action code and note the following lines:

All of those elements are configuration options offered by the Docker build-push-action, version 1:

  • The tags element allows you to specify arbitrary tags for the image.
  • The tag_with_ref element will automatically determine proper tags according to specs that I’d suggest to review in the official documentation. Please note that if you enable the Docker build/push job also for push events, tag_with_ref will tag as latest code committed to the master branch.
  • The tag_with_sha element will also tag an image with the SHA of the Git commit from which the image was built. This is a good practice because it allows to trace each image back to the code version from which it was built.

We are ready to run the new Action:

  1. Commit the change
  2. Push the commit to the GitHub remote
  3. Create and publish a new release of the source code
  4. Navigate to the Actions tab of your repository page on GitHub

This time, the workflow execution will be named after the release, instead of the commit:

If the workflow execution was successfully completed, the image should also be listed into the Docker repository, with all the three tags as specified above:

The Actions we built so far are a good starting point to understand how GitHub Workflows and Actions work, but they have a design flaw.
The lint/test Action and the Docker build/push Action run indepentently. Even if we set the same on triggers for both, they would still run independently. This means that nothing prevents us to release broken code which would result in a broken Docker image pushed to the Docker Hub.

At this time, there are no options to chain Actions that I know of.

The simplest fix for this would be to merge the Actions, putting both the build and push_to_registry jobs into the same Action, so if the build job (lint/test) fails, the workflow will be halted and push_to_registry will not happen. In its most basic implementation, this solution would force us to make a though choice for the on triggers:

  • If we set on: [push] or on: [push, release] we build/push a Docker image every time a Git commit/push happens.
  • If we set on: [release]we lint/test only when a release is published

Neither of these options would work for me. Different scenarios might require different workflows, but as a rule of thumb I want all changes to my code to be tested right away and only releases become new Docker images.

Putting it all together

Remove the 2 YAML files we have created in the previous chapters, replace them with a single file, and paste this code into it:

First, note the on elements and the release elements in particular:

I used types to filter out a few events. I am not interested in deletion events and I want to ignore the created events. When a new release is published in GitHub, a created event is also fired. So without any filter the Action would be executed twice.
Filtering with [published, edited] the Docker image will not be built if a release is just “drafted”. This works for me, but you can replace that filter with [created, edited] if you want drafts to be built, too, and use the expressions provided by GitHub Actions to set the tags accordingly.

Now check the push_to_registry job elements. I added a conditional to skip this job if the event which triggered the Action was not a release:

Conditionals can also be more fine-grained:

but we don’t need this now, because we have already filtered out the release types in the on element.

Commit and push the change to GitHub. The Action will be executed, but only the build job will be completed, push_to_registry will be skipped:

Now publish a new release from this commit. The Action will be executed again, this time with both jobs enabled:

Congratulations! Your first GitHub — Docker pipeline is ready ;)

To dive deeper into GitHub Actions, Secrets and Environments, you could add another job to deploy the newly created Docker images to a container orchestration platform, such as Kubernetes or AWS ECS.

RockedScience

Tutorials, tips and fast news on Cloud, DevOps and Code

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store