Gitlab Pipelines, Build, Tests, and Deploy Private Images (GKE, Pulumi)

This is the third part of articles to use Pulumi as infrastructure as Code (IaC) to create and maintain the infrastructure and deploy applications on Kubernetes cluster on the Google Cloud Platform GKE Autopilot and pipelines to automated everything with GitLab Pipelines. You can check the first article here!

In this episode, we will create a GitLab pipeline for a simple Symfony PHP project that will build container images, run lint, unit tests and static analysis code and finally deploy the application in our GKE Autopilot Cluster with Pulumi.

Configuring the project

In our repository, we will change the structure we did in the previews article, to afford the application we will split between infrastructure and application.
The structure will be like this: (you can check the final repository files here)

.
├── README.md
├── application   
├── docker
│   ├── nginx
│   │   ├── Dockerfile
│   │   └── application.conf.template
│   └── php
│       ├── Dockerfile
│       └── Dockerfile.base
├── infrastructure
│   ├── Pulumi.dev.yaml
│   ├── Pulumi.yaml
│   ├── gcp
│   │   └── gke
│   │       └── index.ts
│   ├── index.ts
│   ├── k8s
│   │   ├── external-dns
│   │   │   ├── index.ts
│   │   │   └── namespace.ts
│   │   ├── greeting
│   │   │   ├── config.ts
│   │   │   ├── deployment.ts
│   │   │   ├── hpa.ts
│   │   │   ├── index.ts
│   │   │   ├── ingress.ts
│   │   │   ├── namespace.ts
│   │   │   ├── pod.ts
│   │   │   ├── regsecret.ts
│   │   │   └── service.ts
│   │   ├── guestbook
│   │   │   ├── config.ts
│   │   │   ├── index.ts
│   │   │   ├── namespace.ts
│   │   │   └── redis.ts
│   │   └── ingress-nginx
│   │       ├── index.ts
│   │       └── namespace.ts
│   ├── package-lock.json
│   ├── package.json
│   └── tsconfig.json

Basically, we move all files related to infrastructure to the new directory "infrastructure", create new directories:

• application regarding our application (simple Symfony app)
• docker regarding the docker files for our application
• infrastructure regarding all stuff related to infrastructure as code (Pulumi)

Gitlab Container Registry

Gitlab offers a free private container registry, so we have a free git private repository, free pipeline, free container registry in one place, that simplify a lot of work.

Check this article for the firsts steps: https://about.gitlab.com/blog/2016/05/23/gitlab-container-registry/

For PHP application is good to have a base image different from your current container image, usually, we need to install extensions and this takes time, the reason we have a file Dockerfile.base, for now, we just build manually and push to the registry.

docker build -f docker/php/Dockerfile.base -t registry.gitlab.com/felipe.girotti/gke-pulumi-pipelines/php-base:8.0.8 .
docker push registry.gitlab.com/felipe.girotti/gke-pulumi-pipelines/php-base:8.0.8

Create the registry a secret for Kubernetes

Because the container registry is private we need to create a secret in your cluster to make access to pull the image.

First, let's configure in the repository the new Deploy Token.
Goes to Settings -> Repository -> Deploy Token, create a new user with the permission read_registry.

Now we will store those configurations as variables for Pulumi.

cd infrastructure
pulumi config -s dev set --secret --path registry.password {YOUR_PASSWORD}
pulumi config -s dev set --path registry.username k8s
pulumi config -s dev set --path registry.url registry.gitlab.com

The file: infrastructure/k8s/greeting/config.ts

The secret file: infrastructure/k8s/greeting/regsecret.ts

The Application Pulumi Configuration Kubernetes

The new application "greeting" is a simple PHP Symfony application. We have some configurations per environment:

• autoscaling (configuration for our Horizontal Pod Autoscaler)
  - cpu
  - memory
• host (the hostname e.g greeting.pulumi-gke.com)
• replicas (configuration for the number of replicas)
  - min
  - max
• version (the container image tag)

The Pulumi.dev.yaml with the configuration of new application "greeting":

Pay attention to lines 10 to 26, this is the new configuration for greeting.

Now we have to configure the application for kubernetes, in the folder infrastructure/k8s/guestbook:

.
├── config.ts
├── deployment.ts
├── hpa.ts
├── index.ts
├── ingress.ts
├── namespace.ts
├── pod.ts
├── regsecret.ts
└── service.ts

Gitlab Pipeline

Gitlab offers a free tier pipeline that you are able to run since simple to complexity steps without install and maintain one Jenkins.

Let's build our first pipeline with the steps:

• build
  We build the docker images, Nginx, and PHP images and push for our private repository.
• test
  We doing all necessary tests, linters, static code analysis, unit tests, integration tests, end-to-end tests, security tests, etc…
• release
  We push the latest container images for our container registry, only after merge in the main (old master)branch
• deploy
  We push the changes using Pulumi, which could be changed in the application or at the infrastructure level, all changes will be reflected automaticly.

Create Service Account GCP

We run the infrastructure as code in our pipeline we need to create the Service Account with the permissions to create/edit/delete resources in our cloud provider GCP.

For this example I put the owner role, the best approach is the Principle of least privilege, that you only give access to the specific resources, you can check more details of the best practice on GCP https://cloud.google.com/blog/products/identity-security/dont-get-pwned-practicing-the-principle-of-least-privilege

export SA_NAME="gitlab-ci"
export PROJECT_NAME="YOUR_GCP_PROJECT_NAME"
gcloud iam service-accounts create gitlab-ci \
     --description="GitLab CI Pipelines" \
     --display-name=$SA_NAME
gcloud projects add-iam-policy-binding $PROJECT_NAME \
    --member="serviceAccount:gitlab-ci@$PROJECT_NAME.iam.gserviceaccount.com" \
    --role="roles/owner"
gcloud iam service-accounts keys create ~/sa-gitlab-private-key.json \
    --iam-account=$SA_NAME@$PROJECT_NAME.iam.gserviceaccount.com
cat ~/sa-gitlab-private-key.json | base64

Grab the output we will need to create variables

Pipeline Variables

Now we need to create two variables in our pipeline, one for GCP ServiceAccount another for Pulumi.

For pulumi access token you can create here: https://app.pulumi.com/{YOUR_USERNAME}/settings/tokens

Now in the Gitlab Repository -> Settings -> CI/CD -> Variables , create two variables:

• CI_GOOGLE_APPLICATION_CREDENTIALS (value of base64 encode of our ServiceAccount)
• PULUMI_ACCESS_TOKEN (the pulumi token)

GitLab Repository

The pipeline configuration file

And finally we are able to configure our pipeline, the file .gitlab-ci.yml

Now we run the pipeline for every Merge Request (Pull Request) for building the images and tests, release images and deploy.

In at build step we create images with the commit hash, that way we have the guarantee of each pipeline will build unique images, not override the existing ones.

On test we running the lint, static analysis code and unit tests.

On the release runs only when merge on the main branch, we just push the lasted tag for container registry repository.

And finally the deploy run only when merge on main branch, we are using the container image pulumi/pulumi that provides the pulumi, gcp and others cli components (you can check the details here). We authenticated on the GCP and set the new version of the greeting application and run pulumi up.

Conclusion

Using Pulumi as infrastructure as code combine with the GitLab with container registry and pipelines we automated everything, since create resources on your cloud provider and deploy applications.

You can check the code in the repository here: https://gitlab.com/felipe.girotti/gke-pulumi-pipelines