CI/CD with GitHub Actions v2

GitHub announced GitHub Actions with CI/CD included on August 8th 2019. That reminded me of my earlier blog post CI/CD with GitHub Actions in which I modeled a CI/CD workflow with GitHub Actions in May 2019. I consider the update GitHub Actions with CI/CD version 2 / v2 of GitHub Actions.
In this post I share the differences of CI/CD with GitHub Actions in May 2019 and August 2019. You can find the code on GitHub: github.com/lotharschulz/hello-github-actions/blob/master/.github/workflows/cicd.yml

GitHub Actions v2 is still in beta phase — I got access to v2 from August 15th 2019 onwards and was impressed by the changes I experienced.

My favorite changes

These are my favorite GitHub Action v2 changes:

• new yaml syntax with migration support
• workflow files .github/workflows directory as part of the core-concepts-for-github-actions
• self hosted runners announced — that probably helps with AWS VPC and similar local tasks
• Action start up time is much quicker compared with version 1 of GitHub Actions — as in my first GitHub actions post, I did not measure it
• contexts available on run time

Nat Friedman’s post about GitHub Actions v2 contains additional changes as well as some of the ones listed above.

CI/CD transition — v1 to v2

User interface / parallel jobs

My initial CI/CD setup (v1) back in May 2019 was:

Three jobs run in parallel in the beginning

• benchmark go code
• post gif on failed job
• test go code

followed by 3 sequential jobs that complete the workflow

• build docker image
• login to docker hub
• push docker image to docker hub

The basic structure remained the same with CI/CD v2. There are two jobs run in parallel in the beginning:

• benchmark go code
• test go code

Building a binary and a docker image as well login to docker hub and push to docker hub — all that runs in one job as separate, sequential steps.
The interface looks similar to terminal environments

in particular with opened steps details.

The job core concept defines that jobs can run in parallel by default in GitHub Actions v2:

Jobs can run at the same time in parallel or be dependent on the status of a previous job and run sequentially.

(source)

One job being dependent from another is defined using needs keyword. I used this to make the build-and-dockerhub-push-if-linux job dependent from benchmark and test job

needs: [benchmark, test]

(source)

The needs keyword did exist in GitHub action v1 already. However the semantic was different. Jobs did run in parallel if several jobs were defined on the same level and only one had a successor job.

Matrix Builds

I experimented with matrix builds. This concept allows you to run the same job in different environments. The classical example would be to build on different platforms:

• linux
• mac
• windows

I did implement a matrix job for Linux and Mac only:

strategy:
      fail-fast: false
      matrix:
        os: [macOS-10.14, ubuntu-18.04]
        goos: [linux, darwin]
        exclude:
          - os: macOS-10.14
            goos: linux
          - os: ubuntu-18.04
            goos: darwin
    runs-on: ${{ matrix.os }}

(source)

Disabling fail-fast for matrix builds is recommended to verify how jobs behave on different platforms. See also https://github.com/jonico/libzengithub/blob/master/.github/workflows/package.yml#L5-L33 for a matrix build definition with more details.

Conditionals

The matrix job builds the binary on both, linux and mac. However only linux supports version1 Action: actions/docker/login@master.

Probably there are three options to circumvent that:

1. use conditionals and login to docker hub on linux only
2. use the docker cli to login to docker hub
3. run docker image build & push docker images in another job that runs on linux only

I went for the first option and at a later stage for the second options as well. This is the conditional I use to build and push docker images only on linux:

if: matrix.os == 'ubuntu-18.04'

(source)

Contexts

There are several contexts available at job runtime that I use to construct the docker tag:

tag=$d-${{ matrix.os }}-${{ github.sha }}

(source)

Please note: Expressions in an if conditional do not require the ${{ }} sytnax. I learnt it the hard way and had to change my code. I wish I could use the ${{ }} syntax in conditionals as well.

Observations

Invalid Workflow

Some Actions initially defined for v1 caused an invalid workflow error message, when I defined a name attribute.

• Error message:

- Your workflow file was invalid:
.github/workflows/dockerimage.yml (Line: 44, Col: 7):
There's not enough info to determine what you meant.
Add one of these properties:
run, shell, uses, with, working-directory

• Fix

remove the name attribute: github.com/ls/hello-github-actions/pull/9/commits/ef337…

Usage Limits

I did run into usage limits before I restricted the number matrix builds with conditionals. Every run that contains matrix builds removes more credits from your virtual account than a job without matrix builds.

Matrix include mapping requires extra values

The exclude definition:

exclude:
          - os: macOS-10.14
            goos: linux
          - os: ubuntu-18.04
            goos: darwin

could probably be defined as well as include definition:

include:
          - os: macOS-10.14
            goos: darwin
          - os: ubuntu-18.04
            goos: linux

However, the latter definition does not work as expected. Actually it cause this error message:

- Your workflow file was invalid:
.github/workflows/cicd.yml (Line: 28, Col: 13):
Matrix include mapping does not contain any extra
values to include

Only items outside of the set defined by the matrix can be included, defining a subset of the matrix set must be done with exclude.
One way to deal with this is to use conditionals and together with a matrix without any include or exclude

matrix:
        os: [macOS-10.14, ubuntu-18.04]
    runs-on: ${{ matrix.os }}
    ......
    - uses: actions/checkout@v1
    - name: builds the binary on linux
      if: matrix.os == 'ubuntu-18.04'
      run: CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-w -extldflags "-static"' -o hello_world${{ matrix.os }}${{ matrix.goos }} .
    - uses: actions/checkout@v1
    - name: builds the binary on mac
      if: matrix.os == 'macOS-10.14'
      run: CGO_ENABLED=0 GOOS=darwin go build -a -installsuffix cgo -ldflags '-w -extldflags "-static"' -o hello_world${{ matrix.os }}${{ matrix.goos }} .

In case the conditionals are not true, the steps are greyed out.

As a matter of taste I like the matrix build with exclude best. Thanks Johannes Nicolai for making me explore this case.

Job sequence as checks

Once my first two parallel jobs finish successful, there is a short moment in time, when all checks including Actions are marked in the web interface as successfully completed.

This can be fixed with required status checks within GitHub branch protection settings.

This way, the checks in question are always visible in the web interface.

Conclusion

GitHub Actions with CI/CD included brought a lot of improvements and changes that makes it easier to define CI/CD workflows. It is still public beta. I hope it is generally available soon including self hosted runners.

Further reading

• GitHub Actions now supports CI/CD, free for public repositories
• Matrix builds
• Core concepts for GitHub Actions
• Choosing the type of actions for your workflow
• Example printing context information to the log file
• Contexts
• About context and expressions
• steps if
• Usage Limits
• Runner
• Migrating GitHub Actions from HCL syntax to YAML syntax

Thanks to Johannes Nicolai for reviewing this post

Originally published at www.lotharschulz.info.