Multi Version Kubernetes CI Pipeline for Atlassian Jira Server & Data Center Apps

Unleash the potential of the official Atlassian Docker Images in your CI Pipeline to test against a full fledged Jira with a real DB.

The aim of this blogpost is just to give a rough overview on how to achieve a setup to test your app against multiple Jira versions. You will need to invest a lot of time to actually get it working on your infrastructure — But it is worth it.

Atlassian started to release docker images on DockerHub August 2019.

  • Bundles AdoptOpenJDK 8 (hopefully 11 in the future)
  • Bundles Jira (also ready for Data Center)
  • Does NOT bundle all JDBC drivers (MySQL for example)
  • Provides ENV vars to configure database, jira-home a.s.o

So there are some things still missing to use the images directly in your CI pipeline.

What do we need to do, to make the Images “ready”?

  • Add missing JDBC driver
  • Exposing Jira via Ingress to outside world
  • Finish Setup Wizard and create user admin with password admin
  • Backup database and dbconfig.xml once done
  • Restore database and dbconfig.xml during startup
  • Adding timebomb licenses for Jira and tested Apps (REST API)
  • Install Apps into running Jira (REST API)

From here on out I will provide mostly codesnipptes of YAML files used to deploy the images in a Kubernetes Cluster. My Kubernetes Cluster uses a BareMetal NGINX Ingress Controller.

The Big Picture

We have a BareMetal NGINX Controller on the Master. And multiple Jira Nodes in different versions with their databases on the Slave Nodes.

Big Picture Kubernetes Setup

We want to have them setup automatically and also generate the Ingress automatically.

Step 1: Vanilla Setup

Under vanilla setup I understand that we use the official docker images without any prior database backups with an empty database. We use the vanilla setup to finish the setup wizard and save a backup of the config and the database in order to deploy the “ready setup” later in our CI Pipeline.

We deploy official MySQL Docker Image to our Kubernetes Cluster like so:

kubectl create -f jira-software-8-0-jdk8-with-mysql5-7-vanilla-db.yml

We give all our k8s things the label destroyBy so that we can easily tear down our setup during ci. To remove our deployment we can easily do.

kubectl delete pods,services,deployments,ingresses \
-l destroyBy=jira-software-8–0-jdk8-with-mysql5–7-vanilla-db \
--include-uninitialized

This snippet can also be used later to remove services, deployments and everything else that comes together with such a deployment.

We deploy Jira together with the latest MySQL JDBC driver like so:

kubectl create -f jira-software-8-0-jdk8-with-mysql5-7-vanilla-jira.yml

As you can see we mount a Volume from a ConfigMap entry that contains the MySQL JDBC Driver. Therefore we need to create that first:

kubectl create configmap jdbc-driver-mysql-5147 \
--from-file mysql-connector-java-5.1.47.jar

The cool thing is, we do not need to build our own Docker Image and Push it to our Repo just because one tiny thing is missing. You could do that, but for me this approach is more flexible (assume Atlassian pushes bugfixes to their images, you would also have to constantly update your extended images).

We can expose Jira now via Ingress like so.

kubectl create -f jira-software-8-0-jdk8-with-mysql5-7-vanilla-ingress.yml

It took me quite some time to get that working, and there are two things that matter:

  • a) the Timeouts are needed because the Jira Setup Wizard can take quite some time to load at some steps, and you would get timeouts then
  • b) the BodySize is needed in order to later deploy Apps via the UPM REST API. If your JAR files are big you will get a Request Entity too Large Error.

If your Kubernetes Cluster’s Hostname is myk8s your URL will now look like this if you also use a BareMetal NGINX Ingress with exposed NodePort.

http://jira-software-8-0-jdk8-with-mysql5-7-vanilla-ingress.myk8s:30080/

Now we finish the setup and use an Evaluation License and the following database:

Jira Database Setup working against MySQL 5.7

After we setup the user admin with password admin we backup the dbconfig.xml and the database.sql dump.

We connect to the MySQL Pod, therefore find out its actual name via the Kubernetes dashboard or the kubectl get pods command.

kubectl exec \
-it jira-software-8-0-jdk8-with-mysql5-7-vanilla-db-56bf852kllp \
-- /bin/bash
## inside container
mysqldump -ujira -pjira --default-character-set=utf8 jira > jira.sql
scp jira.sql 192.168.0.1:/home/thedude/Desktop/jira.sql

Save the jira.sql on your Desktop we need it later.

Now connect in the same way to the Jira Pod and backup the dbconfig.xml

kubectl exec \
-it jira-software-8-0-jdk8-with-mysql5-7-vanilla-jira-56b52kllp \
-- /bin/bash
## inside container
scp /var/atlassian/application-data/jira/dbconfig.xml \
192.168.0.1:/home/thedude/Desktop/dbconfig.xml

Step 2: Ready Setup

We now have the database dump and the database config from the vanilla setup. Now we need to put these pieces together and find a way to populate the database during startup and inject the database config.

Note: We cannot use the ENV Variables of the Jira Docker Image, since the dbconfig contains a lot more info e.g. that the setup was finished a.s.o

I decided to deploy my own DB-Helper Docker Image to my Repo which contains all database dumps and scripts to populate PostgreSQL and MySQL Databases. I will outline the important parts:

For the Docker Entry Point db-init.sh we use:

On now put the dumps into the db-dumps db-dumps directory and you should have:

/Dockerfile
/db-init.sh
/db-dumps/jira-software-8-0-jdk8-with-mysql5-7-db.sql

Now build and push v2 of the Docker Image to your Registry.

I decided to put the dbconfig.xml files on a config server. In Step 2.3 inside the YAML file we download it during startup. Put the files on a server like so:

https://myconfig-server/config/jira-software-8-0-jdk8-with-mysql5-7/dbconfig.xml

Now we can put everything together into one ready.yml file:

kubectl create -f jira-software-8-0-jdk8-with-mysql5-7-ready.yml

We have different initContainers for the dbconfig and the database dumps, everything else is mostly the same as in the vanilla setup.

Step 3: Running inside CI Pipeline

I will only provide the important snippets here in order for you to achieve what you need.

Startup script:

kubectl create -f jira-software-8-0-jdk8-with-mysql5-7-ready.yml

Shutdown script:

kubectl delete pods,services,deployments,ingresses \
-l destroyBy=jira-software-8–0-jdk8-with-mysql5–7-ready \
--include-uninitialized

Wait for Jira startup script:

bash install-util-wait-for-http-200.sh \
http://jira-software-8-0-jdk8-with-mysql5-7-ingress.myk8s:30080/secure/Dashboard.jspa

You can install a 3 hour Timebomb License for Jira Software like so:

curl --fail -s -u admin:admin \
-H "Content-Type: application/vnd.atl.plugins+json" \
-X POST -d '{"licenseKey":"AAABi....2j3"}' \
http://jira-software-8-0-jdk8-with-mysql5-7-ingress.myk8s:30080/rest/plugins/applications/1.0/installed/jira-software/license

You can Install any App.jar via the UPM (Universal Plugin Manager) like so:

TOKEN=$(curl -k -sI -u admin:admin "http://jira-software-8-0-jdk8-with-mysql5-7-vanilla-ingress.myk8s:30080/rest/plugins/1.0/?os_authType=basic" | grep upm-token | cut -d: -f2- | tr -d '[[:space:]]')
curl -u admin:admin -k -w "%{http_code}" -X POST \
--fail -F "plugin=@awesomeapp.jar" \
"http://jira-software-8-0-jdk8-with-mysql5-7-ingress.myk8s:30080/rest/plugins/1.0/?token=${TOKEN}"

Once you have waited for the app to be installed you can install a 3 hour Timebomb License for any App like so, assuming your appKey is awesomeapp:

curl --fail -s -u admin:admin \
-H "Content-Type: application/vnd.atl.plugins+json" \
-X PUT -d '{"rawLicense":"AAABCA...dh"}' \
http://jira-software-8-0-jdk8-with-mysql5-7-ingress.myk8s:30080/rest/plugins/1.0/awesomeapp-key/license

A successful run in my Jenkins that tests my app against all supported Jira Versions looks like this:

And that is simply pure joy 🙂 I hope you will get a similar setup running for you and the more tests are automated the better is the developer’s sleep.

This blogpost is published by Comsysto Reply GmbH

comsystoreply

Innovation through insight.