CI/CD for Java Maven using GitHub Actions

By: Alexander Volminger & Keivan Matinzadeh

GitHub Actions is an exciting new feature that have the potential to replace a lot of your previous development pipelines. Say you want to send build status messages to through Slack? Or implement a whole CI/CD solution? GitHub Actions can do all of this, with everything still being inside your repository.

This tutorial will show you how to set up a CI/CD solution for a Java Maven project. The tutorial will start of with a simple GitHub Action that builds the Maven project and displays the result. More advanced topics will be gradually explored; such as the use of secrets in GitHub Actions and running integration tests against an external service. By the end of this tutorial you should have a GitHub Action that publish your tested Java application as a container image at Docker Hub.

GitHub actions

GitHub have become they the largest host for source code in the world with over 40 million users and more than 100 million repos. Up until recently, users have been dependent on external software for implementing continuous integration into to their projects. Tools such as Travis, Jenkins and the more recent CircleCI have been popular options. But on November 13th GitHub launched their own Continuous Integration(CI) and Deployment (CD) system called GitHub Actions. You are now able to implement CI/CD directly inside the tool that store your code.

Actions themselves are individual jobs that can be combined together in a workflow to create a complex pipeline that handle your needs in the development life cycle. These workflow files can then be configured in YAML and triggered to run on specific events such as each time someone pushes code to the master branch. Several concepts used in GitHub Actions and workflows will be brought up in this tutorial and each one will be explained on first time they occur.

Fork Example Project

To start of you will need your own repository of our starting maven project. It is a simple application now in the beginning that we will gradually build out.

1. Start by going to our starting repository https://github.com/Volminger/github-action-maven-example-start
2. Make a fork of the repository

You should now have something like this, but under your own repositories.

The project consists of two Java classes, where App.java contains the main method which creates an instance of the class DeepThought and calls for an answer.

The pom file is used to configure the maven project. It has been configured so that it knows that App.java contains the main method and that the project should be packaged with its dependencies. This is done to simplify the execution of the jar file once we add library dependencies. The pom file has has also been pre-configured with a surefire plugin which enables tests to get triggered each time a maven build is ran.

Build maven Project

It is now time to create and configure your first workflow. The first thing you want to know is if your maven project builds without any errors. Thus that is exactly what the first job in our workflow will find out.

1. Start by going to your forked repo of github-action-maven-example-start
2. Click on the Actions tab

4. Click on Set up a workflow yourself. You will then be greeted by this screen

5. Replace the pre-populated workflow code with the one below

The first part in the workflow means that it is triggered every time a push or a pull request is made to the master branch. We then have the jobs keyword, it is under this that we will state the name of all the different jobs we would like to run. In this case we start of by stating build_and_test. The first thing we then declare is in what OS environment these job should run in. Here we declare that it should run on ubuntu-latest.

Under steps a sequence of steps are declared. The first step uses another action called checkout. With the use of this other action we give our job access to the code in our repository. The third step also uses another action called setup-java. This action sets up a Java environment that our job can run in. We also use the with keyword to tell setup-java to use a specific version of the JDK, which in this case is JDK-14. Lastly, we create a final step that will build our project using maven package. This is done by using the run keyword, it will execute its arguments in the command-line of the current job machine.

6. Commit your new workflow file once all the workflow configuration is in place.

Take a look at the GitHub Action result

We are now able to see the result of our action. If you go to the action tab again you will be able to see the status of each time the workflow gets run.

You should have gotten a successful run from the last commit of your workflow. If that is not the case the action log is the place to investigate. This can be reached by pressing on the commit text in the list of executed workflows. It will lead you to this screen.

By pressing on the job label build_and_test, over to the left, you are able to browse the logs of the run. You will there be able to see the logs for all the steps that are under a job. Each step can then be expanded by pressing on their name.

Speed up Build by Caching Dependencies

Since package and dependency management tools keep a local cache of downloaded dependencies, workflows will often reuse downloaded dependencies. But since jobs on GitHub-hosted runners start in clean virtual environments dependencies need to be downloaded each time a job is ran resulting in longer runtime. To help mitigate the runtime of the build in a GitHub-runner, the most frequently used dependencies in workflows can be cached by GitHub. This is simply achieved by adding a new step under steps that caches the dependencies.

1. Go to your workflow file under .github/workflows
2. Add a step with the name Cache the Maven packages to speed up build. It should be placed right under the job Set up JDK 14 and before Build project with Maven

In the new step we use an action called cache that enables the caching of dependencies. Each time cache action runs it looks for the dependencies you provide in the key arguments. If it finds a dependency in the cache they will be restored to the argument you provided as path. If it is instead a cache miss the action will download the dependency, create a new cache entry for it and place it in path. restore-keys are an optional argument that is used when we get a cache miss. The cache action then uses the argument provided in restore-keys to partially try to match cached dependencies.

To be able to provide the cache action with the right information we have to introduce the concept of context information. It is what enables us to get the runner OS by the expression

${{ runner.os }}.

To access a specific GitHub context you need to place it inside ${{}}

${{ <context_you_want_to_access> }}

In our case runner is an object that contains information about what the current job is running on. OS is then a string type stored inside the runner object stating what operating system the job is running on.

key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}

We also introduce the concept of GitHub Action Functions. There are several different types of functions available that can cast values to strings or compare values, but here we use the function hasshFiles(path). The function returns a hash for all the files given in its path argument. So in our case the current version of our pom.xml file will be hashed and placed as a part of the key. An example key is Ubuntu-18-m2-HashForPomFile.

3. git add, commit (and push) the updates to your workflow file

4. Check that the cache job is successfully run

Add Junit tests

Having your project build successfully is not enough, you also need to have code that works correctly. The main way to gain confidence in the correctness of your code is to use junit testing. Here we will use the Junit testing library.

Start by adding the junit-jupiter dependency to the POM file in the project.

1. Go to your pom.xml file
2. Go down to the project dependencies and add the following between <dependencies> PLACE_HERE </dependencies>

3. git add, commit (and push) updated pom.xml file

3. We also need to add a Junit test to test. Do this by creating the new file TestDeepThought.java in the folder src/test/java

4. git add, commit and push our new test case

Pushing the new code should trigger a new build and now each time the project is built the tests will run. The build will fail and stop if one of the tests fail. The result of the test can be observed inside the GitHub action logs

Publish build as a package

We have now compiled and ran tests on our maven project. But we have yet done anything with the actual file we build with maven. That will now change, it is time to publish your package. For this you will need to update your workflow file.

1. Go to your workflow file in .github/workflows
2. Add a new job called publish-job with the following content:

The configuration above will publish a package. It will only be executed if the job build_and_test have been executed successfully. This is achieved with using the new keyword needs. It defines the jobs that must successfully complete before the this new job will execute. It then runs the checkout and setup-java actions; downloading a copy of the repository on the runner and then setting up the Java JDK.

run: mvn -B package — file pom.xml -DskipTest

Here the -DskipTests flag is used to signal that we do not want to run the tests again, since they have already been ran in the build_and_test job. This will speed up the build process.

run: mkdir staging && cp target/*jar-with-dependencies.jar staging

In this statement a folder named staging is created and all files ending with the .jar extension in the target folder is copied into the staging folder.

- uses: actions/upload-artifact@v1
          with:
            name: Package
            path: staging

To enable the actual upload of a artifact we make use of a new action called upload-artifact. It enables us to upload our package with the name Package and and the in the path argument the file we want to upload is given.

3. git add, commit and publish your updated workflow

This should result in a package appearing in each run in the action tab, it can be seen here under the Artifact list.

Publish Java Application as a Container Image

Now when you have a buildable well tested application it is time to package it as a container and publish it so that other developers easily can make use of your awesome application. For this part you need to have a Docker Hub account. please go ahead and create an account if you do not already have one.

GitHub Secrets

In the process of uploading our container image we are going to use something called GitHub Secrets. They are encrypted environment variables that can be injected into your actions. A secret can not be seen once it have been created and they remain encrypted until they are used in a workflow. But even when they are used in a workflow they cannot be seen in the action logs since they are blurred out. They are thus suitable to be used to store sensitive information such as passwords and keys.

Create new Secrets

It is now time to create your own secrets that will be used to publish your container image on Docker Hub.

1. Go to the settings page of your project and go into Secrets
2. Click on Add a new secret

3. Add DOCKER_PASSWORD as the new secret’s name and proceed to add your Docker password in the value field.

4. Add another secret with the name DOCKER_USERNAME, the value field should contain your Docker username.

5. Add another secret with the name DOCKER_REPO where the value field contains the Docker Hub repository name you want to post your container image to. It should follow the fashion REPLACE_ME_WITH_YOUR_USERNAME/github-action-maven-tutorial

You should by now have 3 secrets with the following names:

Create Container Image (Dockerfile)

It is now time to create a Dockerfile that will be used to build your container image.

1. Create a new file straight in your repo with the name Dockerfile
2. Add the following content to the Dockerfile:

This Dockerfile configuration will create an container image from the maven project. The FROM instruction initialises a new build stage and sets the base image as maven:3.6.3-openjdk-14-slim for subsequent instructions. The slim edition is used to try to keep the image size down. RUN mkdir -p /workspace runs a command line script that creates a folder called workspace. WORKDIR /workspace sets the working directory to /workspace for any RUN, ENTRYPOINT or COPY that follows in the Dockerfile. The pom.xml file and the src directory is then copied to /workspace. After that the shell command mvn -B package — file pom.xml -DskipTests is ran to build the jar file.

FROM openjdk:14-slim sets the JDK as the base image. We then copy the jar we created in the build phase into a new jar file called app.jar. EXPOSE 6379 informs Docker that the container listens on port 6379. The last bit, ENTRYPOINT [“java”,”-jar”,”app.jar”] works almost like a CMD entry in that it tells the container to run java -jar app.jar once the container start up. But unlike CMD, ENTRYPOINT cannot be overwritten when the container is ran.

3. git add, commit and push your new Dockerfile

Publish to Docker Hub

It is now time to add the configuration to your workflow that will actually build and publish your container image to Docker Hub.

1. Go to your workflow file in .github/workflows
2. Add a new job called “build-docker-image” with the following content:

3. git add, commit and push your updated workflow file

We will once again use context information to get the information from the secrets we created. Remember that the syntax are
${{ <context_you_want_to_access> }}. In this case we first need to access the secret object which then stores the secrets we previously created.

- name: Login to Docker Hub
  run: docker login -u ${{ secrets.DOCKER_USERNAME }} -p ${{ secrets.DOCKER_PASSWORD }}

Logs us in to Docker. It does this by injecting the secrets DOCKER_USERNAME and DOCKER_PASSWORD into our docker login command.

- name: Build Container image
  run: docker build -t ${{ secrets.DOCKER_REPO }}:latest .

Builds the image by running the docker build command. The tag is set to the name of our Docker Hub repo with :latest at the end.

- name: Publish Docker image
  run: docker push ${{ secrets.DOCKER_REPO }}

Will finally publish the Docker Hub repo provided as a secret. If it is the first time you push something to the Docker Hub repo the repository will be created for you.

4. Let your updated workflow run and then go to Docker Hub and confirm that you container image have been published

Integration Test Against Cache

The application is growing and you find yourself needing to connect it to an external cache. Of course this new addition to the application needs to be tested, which should be doable in your CI. GitHub actions let you do exactly this! To achieve it we are going to use something called Service Containers.

Service Containers

Service Containers are Docker containers that can be used to provide an easy and portable way to host services that the users might need to test or operate an application. Each job in a workflow can have its own fresh Service Container that could be used for things like hosting a memory cache service to run tests against.

Steps can be run in two different ways in the context of Service Containers. Depending on which way you choose to run your jobs, the way you communicate with the service container will differ. The first way is to run your steps directly inside a service container. To access your Service Container in this case you simply use the service container’s label. No ports need to be opened up since you jobs are running inside the container.
The second way to run your steps are on the runner machine (which is the regular case when you do not use Service Containers). In this case the service container cannot be accessed by its label. Instead, we use localhost:port or 127.0.0.1:port. We also need to expose the Service Container’s ports to the runner so our steps can access it.

Update Java Project

First we need to update our maven project to use a cache service called Redis. For this we use a Java package called Jedis, which is a small Redis client for Java.

1. Go to your pom.xml file
2. Go down to the project dependencies and add the following between <dependencies> PLACE_HERE </dependencies>

3. git add, commit and push the updated pom.xml file

4. To make use of caching we need to add some new Java code. Add the new file RedisJava.java to src/main/java/ with the following content:

5. We also need to update our main method in App to make use of our new cache service. Do this by updating src/main/java/App.java to

6. We also need to add another Junit test to test our new caching service. Do this by creating a new Java class. Create the new file TestRedisJava.java in src/test/java

7. Git add, commit and push new Java files

Update test_and_run job

Is is now time to start modifying our workflow so that we have a Redis service to run our integration tests against. For this we will configure a job that uses a Service Container where the job’s steps are executed inside the runner, not inside the Service Container.

1. Go to your workflow file in .github/workflows
2. Update the old job build_and_test with:

Like you see the steps part is still the same, but the upper part is new.

services:
      redis:
        image: redis

services defines the services to be used in this job. redis: is the the label of our Service Container. image: redis defines the docker image used with the service container.

options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

options defines the container options which in this case is

• health-cmd “redis-cli ping”: Command to run to check health
• health-interval 10s: Time between running the check
• health-timeout 5s: Maximum time to allow one check to run
• health-retries 5: Consecutive failures needed to report unhealthy

ports:
 — 6379:6379

Because we are running our steps inside the runner and not inside the service container, we need to expose the Service Container port to the runner. Redis uses port 6379, so this is the port that we will expose.

3. git add, commit and publish your updated workflow

Conclusion

You have hopefully realized, by following along this tutorial, that GitHub actions are easy to use. They provide a valuable alternative in how you can set up your CI/CD since you do not need to connect to any external services (if you already have your code on GitHub). You can build simple workflows that will just build your application. But you have also seen that GitHub actions scale in complexity and can perform tasks in your workflow such as running integrations tests on a external services such as Redis.

GitHub Actions provide an easy and valuable alternative for setting up your own continuous integration pipeline. They enable the development of developer process workflows (such as CI/CD) straight inside the tool you use for storing your source code. You are no longer forced to go to an external service to be able develop CI/CD pipelines.

Final workflow configuration

All the code for this tutorial can be found here. Your workflow file should by the end look like this