Decomposing CI Jobs: GitLab CI & Docker

https://about.gitlab.com/product/continuous-integration/

GitLab has its own CI/CD tools with lots of useful features. It basically runs some script in a container defined for each job in the .gitlab-ci.yml

This comic has some other version, which the word “COMPILING!” is replaced with something like “DOCKER!” or “NPM INSTALL!” [https://xkcd.com/303/]

This is what our .gitlab-ci.yml looks like at first:

image: docker:stable
services:
  - docker:dind

build:
  script:
    - docker pull $DOCKER_IMAGE_NAME:latest || true
    - docker build --cache-from $DOCKER_SERVER_IMAGE:latest -t $DOCKER_SERVER_IMAGE:$CI_COMMIT_SHA -t $DOCKER_SERVER_IMAGE:latest .
    - docker run $DOCKER_SERVER_IMAGE sh -c "npm test"
    - docker push $DOCKER_SERVER_IMAGE:$CI_COMMIT_SHA
    - docker push $DOCKER_SERVER_IMAGE:latest

Let’s take a look at how these things work:

• the build job run inside a docker:stable container
• it first pulls previously built image (for caching purpose)
• built and tag it with latest and its commit SHA
• run the tests
• push to the registry

It runs successfully until we want to test some database functionality which needs a connection to a database server. Actually, GitLab has its own page of documentation describing how to tackle this.

But eventually, I realize that the container who wants to connect to the database is the one running inside the build job, while the postgres service is linked to thebuild job (not the container inside it). I’ve tried to use --network="host" argument but still can’t connect.

Here’s the question link about it (it only got 10 views for 5 days :’))

Accessing GitLab CI Service from A Container running Inside DinD

Well then, why not decomposing the jobs into build, test, and deploy? I think this is how the CI should be. With it, I can run the test script inside a job running the built image from build stage and link it to the postgres service.

tages:
  - build
  - test
  - deploy

build:
  image: docker:stable
  services:
    - docker:dind
  stage: build
  tags:
    - docker
  script:
    - docker pull $DOCKER_SERVER_IMAGE:latest || true
    - docker build --cache-from $DOCKER_SERVER_IMAGE:latest -t $DOCKER_SERVER_IMAGE:$CI_COMMIT_SHA .
    - docker push $DOCKER_SERVER_IMAGE:$CI_COMMIT_SHA

test:
  image: $DOCKER_SERVER_IMAGE:$CI_COMMIT_SHA
  services:
    - postgres:11.1
  stage: test
  variables:
    PGHOST: postgres
    PGPORT: 5432
    PGDATABASE: postgres
    PGUSER: postgres
    PGPASSWORD: postgres
  script:
    - npm test

deploy:
  image: docker:stable
  services:
    - docker:dind
  stage: deploy
  tags:
    - docker
  script:
    - docker pull $DOCKER_SERVER_IMAGE:$CI_COMMIT_SHA
    - docker tag $DOCKER_SERVER_IMAGE:$CI_COMMIT_SHA $DOCKER_SERVER_IMAGE:latest
    - docker push $DOCKER_SERVER_IMAGE:latest

Voila~ It works!

Bonus comic from xkcd which relatable on how I work in the team

Yeah, I got ‘lectured’ by the course lecturer because I was too busy taking care of the CI and not implementing any code yet for some first weeks. [https://xkcd.com/1319/]