Zero to Continuous Delivery with Google Cloud Platform

Introduction

I recently gave a talk at a local Docker meetup based in Manchester, UK. This blog accompanies that talk. Over the course of this article I hope to give the reader instructions for creating a continuous delivery pipeline that utilises Google Cloud Platform (GCP) and specifically Google Container Engine (GKE).

The tutorial will provide detail for creating your GKE cluster, utilising Kubernetes (k8s) and Docker to spin up a Jenkins container, using k8s to spin up Jenkins build agents, defining the build pipeline as code, creating a spring boot app as a docker container and finally deploying the application with k8s. Phew there is a lot to get through…so buckle in we’re in for the long haul. To help with adoption and helping people play with GCP, the tutorial also outlines GCP project creation and user creation. The only assumption I’ve made is that you’ve signed up to GCP and can access the GCP console.

TL;DR

All the code for this tutorial has been open sourced and can be found on my github account and the presentation can be found on slideshare.

GCP Project Creation

After signing up to Google Cloud Platform (at the time of writing they are offering a free $300/60 day trial) navigate to the cloud console.

1. Navigate to the project management screen
2. Click the ‘Create Project’ button (figure 1)
3. Enter the project name and make note of the project ID. This is important for later. Figure 1 shows an example project that has the project ID of ‘docker-meetup-1381’
4. Click ‘Create’ to create your GCP project

Figure 1 : GCP project creation

Thats the project created. Now we can configure a GCP service account for managing the project.

GCP Service Account Creation

In order to manage the resources we are about to create GCP has a concept of service accounts. The next steps will talk through creating the service account and downloading the associated key file.

1. Ensure your project is selected in the top right corner of the GCP console (shown in figure 2)
2. Click the top left burger menu and choose ‘IAM & Admin’
3. Click ‘Service Accounts’ from the left side menu
4. Click the ‘Create Service Account’ button
5. Provide your desired service account name and tick the ‘Furnish a new private key’ (Example shown in figure 3)
6. Once you click ‘Create’, it will also trigger a browser download of your new service account key file.
7. Make a note of the location and filename of your key file as this is utilised later.
8. Back on the ‘Service accounts’ screen ensure your new service account is ‘ticked’ and click the ‘Permissions’ button at the top of the screen.
9. To keep things simple but at the expense of security, we’ll give the service account ‘Editor’ access to our GCP resources. However I would advise reviewing the permissions for longer term projects. On the right hand permissions panel, type in your service account name in to the ‘Add Members’ field and click the corresponding service account when suggested. Choose ‘Editor’ from the role dropdown and click ‘Add’. (Example shown in figure 4)

We’ve now got our GCP project and credentials all ready to go.

Note

On Safari the key is sometimes downloaded as a file called ‘Unknown’ for some reason. Although not required I would recommend moving the file out of the downloads directory and renaming it to make it clear that is a JSON file. For example:

mkdir ~/keys
mv ~/Downloads/Unknown ~/keys/gcp-service-account.json

Figure 2: Selecting your GCP project

Figure 3 — Service account creation

Figure 4 — Permissions for service account

GKE Cluster Creation

Time to create our GKE (K8S) cluster.

1. Firstly install the Google Cloud SDK to your command line.
2. Once you have installed the GCloud SDK you can go ahead and install the kubernetes extensions

# gcloud components install kubectl

3. Clone the bootstrap repository

# mkdir ~/gcp-continuous-delivery
# cd ~/gcp-continuous-delivery
# git clone https://github.com/eggsy84/gcp-bootstrap-infrastructure.git

4. Change to the bootstrap directory

# cd ~/gcp-continuous-delivery/gcp-bootstrap-infrastructure/bootstrap/

5. The bootstrap script can now be executed. It takes 5 parameters:

Google project ID
The project that will contain your GKE cluster. (Taken from the ID you noted earlier)

GCP zone
Taken from the ‘Available Zones’ list provided by GCP

GCP machine type
Taken from the ‘Machine types’ provided by GCP. Each node in your GKE cluster will be provisioned as per this type.

Number of nodes for your k8s cluster
Numerical input for the amount of nodes to create.

Path to your service account file
Path to the service account JSON file you downloaded earlier in the tutorial.

# sh deploy.sh \
  docker-meetup-1381 \
  europe-west1-b \
  n1-standard-2 \
  1 \
  ~/keys/gcp-service-account.json

6. The script will open your browser and ask you to authenticate. Enter your GCP username and password details and click ‘Allow’. Return back to the command line terminal and you should see output similar to:

About to create a Container Cluster in the ‘docker-meetup-1381’ GCP project located in ‘europe-west1-b’ with 1 x ‘n1-standard-2’ node(s)
Press any key to continue…or Ctrl+C to exit

The confirms what action it will undertake and asks for confirmation.

Important Note: The script will now provision infrastructure on GCP which may be liable to you incurring costs.

7. Extract the Jenkins IP address

The script will output its progress and towards the end output an IP address that you can access Jenkins on. Such as:

Jenkins service up and running on 104.155.36.72

You should now be able to open up a browser, navigate to that URL and view your Jenkins server! Essentially thats it — you have provisioned a GKE cluster, used Kubernetes to deploy a pre-configured Jenkins master docker container on it and gave that service an external IP address to make it accessible. The Jenkins instance has been preconfigured with a single Jenkins job that produces a build pipeline of a sample spring boot application using the Jenkins JNLP agents and the Jenkins Pipeline plugin.

Important Note: Exercise for the reader to address security of the instance as it is currently unsecured and does not require any user credentials.

Application deployment

After around 10 mins (you may have to refresh your screen) your jenkins instance will have created a job called ‘spring-boot-pipeline’ and executed a build.

1. Navigate to the jenkins job. From here you should your build pipeline in full glory. (Example shown in figure 5)

Figure 5 — Build pipeline

2. Click on build number 1

3. Click Console output to view the output of the job.

4. Scroll down to the bottom of the output and you should see a message similar to:

[spring-boot-pipeline] Running shell script
+ echo Application up and running on 130.211.78.33
Application up and running on 130.211.78.33
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

5. The shows the spring boot application has been successfully built and also deployed in to our Kubernetes cluster. You can navigate to the IP address provided to view the running application.

What just happened?

• Created a GCP kubernetes cluster
• Used k8s and docker to deploy a pre-configured Jenkins master to that cluster
• Got Jenkins to execute a build of our sample spring boot application
• The Jenkins build spun up a Docker container containing a JNLP Jenkins slave for executing the build
• The build compiled the code using Maven
• It then went on to produce a Docker image of the application
• Push this to Google’s Container Registry
• Use k8s deployments to then deploy the docker image in to our k8s cluster
• Use k8s services to register a service infront of the docker container and set its type to LoadBalancer to ensure the application is accessible over the internet.

Next in the series

Follow up posts will go in to more detail on each of the elements that make this example work so that you can replicate it and begin your own projects in a similar form.

I’d also welcome input on any changes to the various Github projects involved so please do drop me a pull request.