Gitlab CI/CD using specific runners on AWS EC2

GitLab CI/CD can automatically build, test and deploy our applications. Runners are processes that pick up and execute CI/CD jobs for GitLab.

We can configure our gitlab CI/CD jobs to run on shared runners or specific runners. Specific runners are useful when repository have specific requirements which are not common with other shared runners setup on our gitlab instance.

Steps to configure CI/CD jobs to use specific runner on AWS EC2.

1. Add .gitlab-ci.yml to our repository root folder.

stages:          # List of stages for jobs, and their order of execution
  - build
  - test
  - deploy

build-job:       # This job runs in the build stage, which runs first.
  stage: build
  script:
    - echo "Compiling the code..."
    - echo "Compile complete."

unit-test-job:   # This job runs in the test stage.
  stage: test    # It only starts when the job in the build stage completes successfully.
  script:
    - echo "Running unit tests... This will take about 60 seconds."
    - sleep 60
    - echo "Code coverage is 90%"

lint-test-job:   # This job also runs in the test stage.
  stage: test    # It can run at the same time as unit-test-job (in parallel).
  script:
    - echo "Linting code... This will take about 10 seconds."
    - sleep 10
    - echo "No lint issues found."

deploy-job:      # This job runs in the deploy stage.
  stage: deploy  # It only runs when *both* jobs in the test stage complete successfully.
  environment: production
  script:
    - echo "Deploying application..."
    - echo "Application successfully deployed."

2. Install specific gitlab runner agent on AWS EC2 instance

Logon to our EC2 instance, switch as root user and execute following commands:
curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64
chmod +x /usr/local/bin/gitlab-runner
gitlab-runner install --user=root --working-directory=/gitlab-runner-data
gitlab-runner start

3. Register specific runner for a project

Goto repository settings -> CICD -> Runners -> Click Expand -> Specific runners -> copy repository-registration-token value
Now, logon to our EC2 instance, switch as root user and execute:
gitlab-runner register --url https://gitlab.com/ --registration-token <repository-registration-token>
Enter the GitLab instance URL (for example, https://gitlab.com/):
[https://gitlab.com/]: <press return>
Enter the registration token:
[<repository-registration-token>]: <press return>
Enter a description for the runner:
[ip-x-x-x-x]: ec2-build-box
Enter tags for the runner (comma-separated):
ec2-build-box
Enter optional maintenance note for the runner:
<press return>

Once the specific runner is registered for the project, it will be visible under: repository settings -> CICD -> Runners -> Click Expand -> Specific runners section.

4. Update .gitlab-ci.yml to run CI/CD pipeline jobs on specific runner

Now, we need to add tags section in our .gitlab-ci.yml file to instruct CI/CD jobs to run on our specific runner.

stages:          # List of stages for jobs, and their order of execution
  - build
  - test
  - deploy

build-job:       # This job runs in the build stage, which runs first.
  stage: build
  tags:
    - ec2-build-box
  script:
    - echo "Compiling the code..."
    - echo "Compile complete."

unit-test-job:   # This job runs in the test stage.
  stage: test    # It only starts when the job in the build stage completes successfully.
  tags:
    - ec2-build-box  
  script:
    - echo "Running unit tests... This will take about 60 seconds."
    - sleep 60
    - echo "Code coverage is 90%"

lint-test-job:   # This job also runs in the test stage.
  stage: test    # It can run at the same time as unit-test-job (in parallel).
  tags:
    - ec2-build-box
  script:
    - echo "Linting code... This will take about 10 seconds."
    - sleep 10
    - echo "No lint issues found."

deploy-job:      # This job runs in the deploy stage.
  stage: deploy  # It only runs when *both* jobs in the test stage complete successfully.
  environment: production
  tags:
    - ec2-build-box
  script:
    - echo "Deploying application..."
    - echo "Application successfully deployed."

Once all above steps are completed, CI/CD pipeline jobs should trigger on our specific runner on AWS EC2.