Trying out Jenkins X on Azure Kubernetes Service(AKS)

PC Chan
PC Chan
Oct 29, 2018 · 4 min read
Image for post
Image for post

When I was at Jenkins World 2018, everybody seemed to be talking about Jenkins X. An opinionated Kubernetes and Jenkins with automated CI/CD pipelines that allow you to focus on writing awesome applications for your customers. So I decided to jump on the high-speed rail and experience how Jenkins X is working on AKS.

Unfortunately, like a few of you, my train ride was not that smooth. I was not able to build the pipeline with the Docker registry created in my AKS cluster (GitHub issue.) So I switched the built-in Docker registry out with Azure Container Registry(ACR) and things started to work smoothly. While my team work on creating a PR to default to ACR for AKS, here’s how you can workaround the issue.

Switch to ACR and add secrets to Jenkins X and Kubernetes

  • Jenkins X did a great job documenting how to get started. I followed the steps here to install the jx binary on Windows. The jx version I have is 1.3.435. I also ran all jx, az cli, kubectl commands from my laptop.
  • Next, I installed Jenkins X on an existing AKS cluster by using the jx install command
  • Run jx console to open and sign into Jenkins
  • Switch to ACR instead of the built in Docker registry. Go to Manage Jenkins → Configure System and update the DOCKER_REGISTRY environment variable to your ACR.
Image for post
Image for post
Update global properties in Jenkins
  • You need to add ACR credentials to the jx namespace so your Jenkins agent can push images to ACR. Run:

jx create docker auth --host "yourACR.azurecr.io" --user "yourACRUser" --email "yourEmail@xxx.com" --secret "yourACRSecret" create

  • Because two other namespaces are created by Jenkins X, you need to add the credentials to jx-staging and jx-production as well so that you can pull the Docker images from ACR. Run the following commands with your ACR credentials:

kubectl create secret docker-registry jenkins-docker-cfg --docker-server="yourACR.azurecr.io" --docker-username="yourACRUser" --docker-password="yourACRSecret" --docker-email="youremail@xxx.com" --namespace=jx-staging

kubectl create secret docker-registry jenkins-docker-cfg --docker-server="yourACR.azurecr.io" --docker-username="yourUser" --docker-password="yourACRSecret" --docker-email="youremail@xxx.com" --namespace=jx-production

Create a Spring Boot app and test end to end

  1. Run jx create spring -d web -d actuator to create a new Spring boot app and add a test home page. i.e., just follow steps in this demo.
  2. Commit and push the test home page to the master branch.

My staging environment is successfully updated!

Image for post
Image for post

Notes:

If you create a PR, you will run into once again, the same authentication issue.

When a PR is created, Jenkins X creates a new namespace for the PR. You can list all the namespaces by running kubectl get ns

Image for post
Image for post
Namespaces in my AKS cluster

Suppose jx-XXXX-pr-X is your PR namespace, run kubectl describe pod --namespace=jx-XXXX-pr-X to find out what's wrong:

Image for post
Image for post
Authentication error in pod

To work around, manually add secret to the PR namespace.

kubectl create secret docker-registry jenkins-docker-cfg --docker-server="yourACR.azurecr.io" --docker-username="yourACRUser" --docker-password="yourACRSecret" --docker-email="youremail@xxx.com" --namespace=jx-XXXX-pr-X

I also ran into an issue when the (build) agent pod in AKS got stuck in pending state due to insufficient CPU. I solved the issue by scaling my AKS cluster: az aks scale --name yourAKSCluster --resource-group yourResourceGroup --node-count X (where x is the number of nodes.)

As mentioned, we are working on a PR to default to ACR when running Jenkins X on AKS. You won’t need to do the above manual steps and will be able to do as Kohsuke said “just go to the station and take the high-speed rail to your end destination.” Stay tuned!

We are always looking for feedback, please leave a comment here, or in Jenkins X Slack channel, or email Azure Jenkins Support.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store