How to run e2e tests against a Kubernetes cluster with travis-ci

Working on several open-source projects that are meant to run in a kubernetes cluster, I was looking for an easy way to integrates end to end tests in my validation build pipeline. After some research, I think I found a nice setup to achieve it!

Solution requirements

I had only a few requirements:
- Since it was for an open-source project, the solution had to be free. No other account than the Github account had to be needed.
- It had to be able to simulate a multi-nodes cluster.

Looking at one solution

The project is hosted on Github and use Travis-ci.org for the continuous build. My first move was to look at available services in Travis-ci and to check if they had an offering as a "service" like they do for databases. Unfortunately, the K8s-cluster service was not yet available (as of October 2017) :(.

I also looked at external solutions that can offer a kubernetes-cluster as a service for testing purposes but I didn't find any.

My research conclusion was: if it doesn't exist yet, let's build it!

Travis-ci integrates and offers Docker as a service; It will ease the implementation: if I put one k8s node in a Docker container, it should be easy to create a multi-nodes cluster. Since I have already played a bit with "kubeadm", I was confident in using it to get all the pieces of the puzzle together.

By chance, before I started to implement my own DinD (Docker in Docker) Kubernetes Cluster configuration script, I saw during a kubernetes community meeting, a presentation done by a Mirantis developer about an equivalent solution. I gave it a try on my personal machine, and it was working perfectly.

Eventually my work will consist in configuring properly the mirantis solution in the .travis.yml in order to have the cluster setup during the job's initialisation step.

The Solution

Ok, so my solution for running a Kubernetes cluster in travis-ci.org consists of downloading the Mirandis DinD Cluster installer and also the Kubernetes CLI binary "kubectl". Then run the installer and configure properly the environment PATH variable and we are good to go!

The travis-ci configuration file (.travis.yml) is divided into several parts. In our case, the setup of the kubernetes cluster can be considered as a dependency like for other services (docker, database…), that is why we describe the configuration in the "install" section.

You can see the travis build result here: https://travis-ci.org/cedriclam/test-travis-k8s/builds/285231052

Source code available here: https://github.com/cedriclam/test-travis-k8s

I’m not a bash/shell expert, so you may be able to improve those few lines of bash code.

I hope this will be useful for you too!

My next publication will definitely be talking about a simple framework in golang (my second passion with k8s) to help to build end-to-end tests against a kubernetes cluster.