CI/CD AC/DC ABCDEFG

https://docs.gitlab.com/ee/ci/introduction/

CI/CD allowed us as a developer to focus more on our code rather than anything else basically. CI/CD also help us deliver our product more quicker to our customer while also maintaining the quality. In our project, we definitely use CI/CD methodology to help us create a better product for UKM Indonesia.

Continuous Integration

Many projects are built by a team, each of them push their work, multiple times a day. This flow have a chance to introduce a bug, which later in the development may cause other bugs along the line. If you are unlucky, the fix may cause those work after the initial bug to be wasted. Continuous integration help us by build and testing every work automatically, incrementally, halting every bug introduced.

Continuous Delivery

It is the ability to deliver new features, performance improvement, bug fixes, hot fixes, cold fixes, warm fixes, frozen fixes, boiling fixes, any kind of fixes to the customer more quicker and safer. Automation is key as it allowed these process to run following a predefined script which make the process have a lower chance of error since there is less/no human intervention.

How do we do it

Our partner which is UKM Indonesia want the product to be deployed to Amazon Web Service, specifically to Amplify Console. As part of this course’s guidelines, we are also required to deploy our code to Docker image registry which later going to be orchestrated at Fasilkom’s Docker engine using Docker Portainer.

AWS Amplify Console is a Git-based workflow for deploying and hosting a front-end full-stack serverless app. After a commit is pushed to a connected app on a specific pre-configured branch, the deployment will be triggered. Amplify Console can only connect to a 4 Git service which is Gitlab, Github, Bitbucker, and AWS Code Commit, but it cannot connect to other instance of Gitlab which our code is hosted on. So the workaround that we came up with is to setup a repository in Gitlab and then setup a repository mirroring. If your deployment does not have anything fancy you want to do with it, you can just continue on, but if you do, Amplify Console give you the ability to configure the build by adding the “amplify.yml” file to your repository. Here is an example of Amplify build script.

Amplify Console build script

Docker will containerize our Angular app so that we can easily deploy and run it anywhere. For our docker deployment, we will use built in Gitlab CI/CD. The deployment will be triggered because we have the “.gitlab-ci.yml” file, after any push, Gitlab can detect the file and will run those scripts. Currently, we have 2 stages. Testing stage for testing our code, though for now the only test is the default one generated by Angular. The second stage is the deployment to Fasilkom’s Docker registry.

In order to run our Docker deployment, we need 2 files which is the “Dockerfile” and the aforementioned “.gitlab-ci.yml” file. The “Dockerfile” serve as the recipe for building the Docker image. It tells the Docker how do we want our image to be build. Here are the 2 files and their brief explanation.

Here as you can see, we can divide this file into 2 part. The first part will compile our front-end files. Firstly it will change the working directory to “/app”, then it will copy the source code (excluding files that has been ignored in .dockerignore) into the directory that has just been made.It uses node image so that we can run npm command to compile our app.

The second part is suppose to build the final image using the nginx image as the http server to serve our app. Firstly, it will copy the compiled files from distribution folder into the nginx html folder. Then we can specify the port to open and the command to run the server.

As I’ve mentioned before, our CI/CD consist of 2 jobs. The first one is testing which I’m not going to elaborate in this post. The second one is deployment, here you can see there are 3 deployment jobs. Each of them triggered by different branch, master by master, staging by staging, and dev by other than master and staging which also triggered manually.

The job itself looks kinda complicated because those few lines above the actual script. Those are the result of finding the fix for some internal error at the Fasilkom’s image registry. To be honest, I don’t know most of them. The important lines are those inside the script. Consist of 3 commands which is pull, build, and push.

Docker pull will pull the latest image from registry to be used as the cache for the current build. Build will build. And the push the image. Also, the image are tagged properly.

Learning

Need to learn more stuff. DevOps is cool. (also I heard that it pays really well)

Thanks for reading.

Update

Elaboration on GitLab CI configuration. GitLab CI configuration generally consists of tasks. We can order these tasks into stages, tasks can run concurrently in the same stage or synchronously in multiple stages. In our configuration, we have two stages which is test and deploy.

Currently, there is only one task that ran in test stage which is the unit test. Later in the development, when we start making the e2e tests, we can create a new task for e2e testing which can run concurrently with the unit test.

Our deployment stage technically has three tasks, but each of them will run separately, depending on which branch the stage is triggered. I’ll explain later about this.

Common configs

• stage define the order of the job, basically it define when this job is supposed to run. In our unit test job, we define the job in the test stage, which will run before the deploy stage.
• image define what docker image the job should run in. This help us by entirely skipping the environment configuration for our job.
• script is the set of commands that will run in the job.
• before_script is the set of commands that will run before script

In human language

My attempt on explaining our CI flow for commit on staging branch.

There are two stages which is test and deploy.

Jobs that ran in test stage is unit test. This job will run in an environment named trion/ng-cli-karma which already has Node, Angular, and Karma which are required code to run the tests. Before (before_script) running the tests, download all extra code that we need (npm ci). After that, we do the test (ng test). The testing will spit out report, find part of the report that tells our coverage, then save the value. Done.

Jobs that run in deploy stage are development docker deploy, staging docker deploy, and production docker deploy. There will be only one job that run at a time. production only run only for commit in master and staging will only run only for commit in staging while development run on every commit except master and staging when manually triggered. The job run in an environment called docker:stable that contains the Docker application which we need. In order to access Fasilkom’s Docker registry where we deploy our application, we need the stuff inside variables and services. Quick googling told me that these two stuff will elevate the privilege of our job. Since Fasilkom’s CI runner does not support SSL, we need to replace all outgoing secured HTTP request with the non encrypted one in before_script. To make our script run faster, we can pull previous deployment to act as a cache. Next we will build our application using the --cache-from previously downloaded image. After the build is finished we can push our image.