Revolutionize Your Next Generation CI with GitHub Actions: Unlocking the Potential of Reusable Workflows and Composite Actions

As software development practices continue to evolve, so too do the tools and platforms we use to support them. Continuous Integration (CI) is an essential part of the development process, allowing developers to streamline their workflows, identify and fix errors quickly, and deliver high-quality code more efficiently. There are a multitude of options when looking at CI tools from Jenkins, GitLab, GitHub Actions, Circle CI, Bamboo and more which have all been widely used in the past few years.

In this blog post, we’ll explore why we chose GitHub Actions and how it can take your CI process to the next level. We’ll start by comparing it to our core CI platform at project44 and explain why we chose to make the switch. Then, we’ll dive into the unique features that make GitHub Actions stand out, including reusable workflows and composite actions. By the end of this post, you’ll have a solid understanding of what GitHub Actions can offer and how it can help you optimize your development process.

The evolution of CI platforms: from Jenkins to GitHub Actions

The world of CI platforms has come a long way since the early days of Jenkins. While Jenkins was once the go-to option for many developers, it’s now just one of many CI tools available. At project44, our core CI platform, outside of a couple acquisitions using GitLab and CircleCI, has been Jenkins. As we’ve grown and become almost 100% Kubernetes native, we’ve run into a multitude of issues with Jenkins

• Scaling Jenkins to handle our workloads
• Keeping plugins up to date
• Keeping Jenkins itself up to date
• Having to write groovy pipelines (a DSL not many folks are experts in)
• and much more (just read the common industry complaints about Jenkins)

Even with our recent focus on improving the Jenkins experience around creating some generic blueprint pipelines for teams to use, and improving access controls, plugins, security, etc., we still run into too many problems which require folks who have intimate knowledge of Jenkins.

This prompted us to sit down and think about overhauling our CI process and unifying on one tool for all of our engineering teams.

Why we chose to switch to GitHub Actions?

In the past couple years GitHub Actions has emerged as a leading contender in the CI space as they’ve continued to innovate and add features. As a platform that’s tightly integrated with GitHub, GitHub Actions has the advantage of being built on top of a code collaboration platform that millions of developers already use and love.

In addition to its tight integration with GitHub, it’s a platform-agnostic solution that can run workflows on any environment, whether that’s Linux, macOS, or Windows.

Another key advantage of GitHub Actions is its focus on extensibility. Developers can create and share custom actions and reusable workflows that can be used across projects, which can help teams save time and avoid duplicating effort. This, in turn, can help speed up the development process and improve overall productivity.

The main reasons why we chose GitHub Actions are:

• Tight integration with GitHub
• Platform-agnostic workflows
• Built-in support for containerization
• Customizable and extensible actions
• Cost (we already pay for GitHub, we don’t need more licenses for GitLab or other tools)

Reusable Workflows

Reusable workflows are a key feature of GitHub Actions that allow you to create a single workflow and reuse it across multiple repositories. This can be a huge time-saver for teams that work on multiple projects with similar build, test, and deploy requirements.

To create a reusable workflow, you define the workflow in a separate repository that contains only your workflows. This repository can then be referenced by any other repository that wants to use the workflow. When you define a workflow as reusable, you can specify the inputs and outputs that it expects, which allows it to be more flexible and customizable. As with other workflow files, you locate reusable workflows in the .github/workflows directory of a repository. Subdirectories of the workflows directory are not supported.​ For a workflow to be reusable, the values for on must include workflow_call:​

When you create a new workflow in a repository, you can reference a reusable workflow using a uses statement in your YAML configuration. This statement points to the location of the reusable workflow in your organization and specifies any inputs that need to be passed to the workflow.

The main advantage of reusable workflows is that they allow you to centralize your workflow definitions in a single location, which can make it easier to manage and maintain your workflows. Instead of duplicating the same workflow across multiple repositories, you can make changes to the reusable workflow in one place and have those changes automatically applied to all of the repositories that use it.

Composite Actions

A composite action is one of three different types custom GitHub Actions that could be created (composite, JavaScript and Docker). The main difference is that a composite action’s action.yaml runs property contains a list of steps to run as opposed to a program to execute.​

Main benefits of composite actions are:

• split large workflows into multiple files​
• reducing duplication​
• many steps are condensed into a single one within the Actions view on GitHub, improving the ability to track a workflow’s progress​
• the descriptive nature of an action.yaml file improves the readability of a GitHub workflow when understanding necessary inputs and outputs.​

But also there are some disadvantages:

• can’t read GitHub Secrets — you have to pass them in​
• have to define the shell on each step​

Here’s how one simple composite action for pushing git tag could look like:

name: Push Git Tag
description: Creates and pushes tag to remote
inputs:
  tag:
    required: true
    description: Tag to add and push to remote
  git_username:
    required: false
    default: GitHub Actions
    description: The git username to use for git commands
  git_email:
    required: false
    default: <>
    description: The git email to use for git commands
runs:
  using: composite
  steps:
    - run: |
        git config user.name "${{ inputs.git_username }}"
        git config user.email "${{ inputs.git_email }}"
        git tag -a ${{ inputs.tag }} -m "Version ${{ inputs.tag }}"
        git push origin ${{ inputs.tag }}
      shell: bash

Best Practices and Tips

We made decision to create a central repository where we will keep all of our reusable workflows and composite actions. One of the reasons for this is so our DevOps, Developer Experience and Site Reliability engineering teams could easily maintain GitHub Actions workflows.

We also made decision to use private GitHub runners so that we can autoscale them to support our workloads as well as for security purposes (we would highly recommend avoiding using public runners for private organizations).

Like we already mentioned for reusable workflows you can specify the inputs that it expects, which allows it to be more flexible and customizable but if you want to use single workflow for example for all java or python projects you would need to provide a lot of input variables which could become unreadable and complex. To alleviate this issue we implemented a similar feature that is available in other CI tools and came up with idea to create workflow configuration YAML file where we could define all required inputs so that we don’t need to pass all of them as workflow inputs. We created our action read-config which we are using to read .github/p44-config/ci-config.yaml in each repository and use output for different steps in our workflow. Below you can see a few examples of our ci-config.yaml:

# for java applications it will contain java object so we could setup proper java version
java:
  version: '17'
  ... # other java related properties

# for node applications it will contain node object so we could setup proper node version
node:
  version: '16'
  ... # other node properties

helm_charts:
  ... # helm charts related properties

docker_images:
    - ... # properties for each docker image

# configuration for slack notifications
slack:
  notifications:
    - channel_name: ...
      notify_on:
        - success
        - failure
        - cancelled

once this config file is parsed by our read-config action it can be used in workflow steps like this:

- name: Setup java
  uses: actions/setup-java@v3
  with:
    distribution: ...
    java-version: ${{ fromJson(steps.ci-config.outputs.config).java.version }}
    cache: ...

and then in your application repository you just need to add .github/p44-config/ci-config.yaml and your workflows will look like this:

name: Feature Branch Build
on:
  push:
    branches-ignore:
      - master

jobs:
  feature-branch-build:
    uses: <organization_name>/<central-github-actions-repository>/.github/workflows/feature-branch.yaml@master
    secrets: inherit

This makes it simple and easy to use by developers!

GitHub Actions supports alerting for failed pipeline runs where you can receive email notifications, but we also wanted to support slack notifications. So we created our own action for this purpose, which reports the specific job status and sends an alert to specific team channel (sample config is provided in example above).

Structure of centralized repository with shared reusable workflows

How are we using secrets in GitHub Actions?

GitHub Actions allows you to store and manage secrets securely using the repository settings. Secrets are encrypted and can only be accessed by authorized users, ensuring that sensitive information such as passwords and API keys are kept safe. You can add secrets at an organization and/or repository level.

Here are the steps to handle secrets in GitHub Actions:

1. Create a new secret: In the repository or organization settings, navigate to the Secrets → Actions and click on New Secret/New Organization Secret. Enter the name and value of the secret you want to create, and click Add Secret.
2. Reference the secret in your workflow: In your YAML configuration, you can reference the secret using the ${{ secrets.SECRET_NAME }} syntax, where SECRET_NAME is the name of the secret you created in step 1.
   For example, if you have a secret named FOO_API_KEY, you can use it in your YAML configuration as follows:
   env: API_KEY: ${{ secrets.FOO_API_KEY }}
3. Use the secret in your workflow: Once you have referenced the secret in your YAML configuration, you can use it in your workflow as needed.

It’s important to note that secrets are encrypted and cannot be viewed once they have been created.

In addition, it’s important to ensure that only authorized users have access to the secrets in your repository. You can control access to secrets by managing the permissions of the users and teams that have access to your repository.

What about Dependabot and secrets in GitHub Actions?

Dependabot is a popular tool for managing dependencies in your GitHub repositories and we are using it to find and fix vulnerabilities in our projects dependencies. It can automatically create pull requests to update your dependencies to their latest versions and helps you stay up-to-date with security patches and bug fixes.

When using Dependabot with GitHub Actions, it’s important to be careful with how you handle secrets. Dependabot can trigger workflows that contain sensitive information, such as API keys or credentials, and you want to make sure that this information is kept secure.

Here are some best practices for handling secrets with Dependabot and GitHub Actions:

1. Use environment variables: Instead of hard-coding secrets directly into your YAML configuration, use environment variables that reference your secrets. This makes it easier to manage and update your secrets, and reduces the risk of exposing sensitive information in your configuration files.
2. Restrict access to secrets: Make sure that only dependabot have access to secrets with limited access, preferable only to secrets with read only access.
3. Use encrypted secrets: GitHub allows you to encrypt your secrets using the actions/secrets context. This helps protect your secrets from being exposed in plain text and ensures that they are only accessible by authorized workflows.
4. Test your workflows: Before deploying your workflows, test them thoroughly to ensure that they are working as expected and that your secrets are being handled securely. You can use the GitHub Actions simulator to test your workflows in a safe environment and catch any issues before they cause problems in production.

By following these best practices, you can use Dependabot and GitHub Actions together to manage your dependencies and keep your repositories secure.

Conclusion

GitHub Actions is a modern and flexible platform for automating your development workflows, allowing you to easily implement Continuous Integration (CI). It provides a powerful alternative to traditional CI tools and offers features such as reusable workflows and composite actions, making it easier to build more modular and flexible workflows with some additional features that you could implement to simplify them like we have shown.

To use GitHub Actions securely, you should follow best practices for handling secrets and use Dependabot with caution. By doing so, you can ensure that your workflows are secure and your secrets are protected.

Overall, GitHub Actions is a valuable tool for streamlining your development process and delivering high-quality code efficiently. Whether you are starting a new project or migrating an existing one to GitHub, GitHub Actions can help you automate your workflows and increase your team’s productivity.

We’ll have another post where we talk about our evolution around Continuous Delivery (CD) and how we pair that with GitHub Actions, our new CI tool of choice.