Rails 6 API Development and GitHub Actions CI with Docker

I often see the following cases for Rails CI on GitHub Actions.

• Set up Ruby on Host Runner
• MySQL (and Redis and more…) launched in service container
• Install dependencies (e.g. bundle install) and unit tests (e.g. rspec) directly on Host Runner

I want to run CI using Docker!. But I don’t see that case very often. So, I would like to introduce a case study of a project I was involved in.

Preconditions

• Ruby 2.7.1
• Rails 6.0.3 (API mode)
• MySQL 8.0.20
• Build and manage all development environment with Docker (Dockerfile / docker-compose.yml)
• Not using any multi-stage configurations in Dockerfile
• Not using any docker registries
• GitHub Actions have not officially supported “Docker Layer Caching(DLC)” yet (See: this issue)

TL;DR

• Minimize docker image size (e.g. Not contain COPY and ADD command in Dockerfile)
• Build CI environment same as local development environment
• Cache gems installed by bundle install in container with actions/cache and volume mount
• Cache docker image with docker image save, docker image load, actions/cache and volume mount
• Caching docker image is a little tricky on GitHub Actions ))
• I hope GitHub Actions will officially support the Docker Layer Caching. But it will need more time…

Dockerfile

is here (github gist)

Do not include app files in the image (Do not use “COPY” or “ADD” command)

We should not write some COPY or ADD commands in our Dockerfile if you want to minimize the docker image size.

Instead of CMD or ENTRYPOINT command, operations such as bundle install or rails s or others are executed in the container via docker-compose {run, exec}, and library and application files are mounted as volumes and copied to the container (not included in the image).

Wait for db container to start with ufoscout/docker-compose-wait

ARG ARG_COMPOSE_WAIT_VER=2.7.3
RUN curl -SL https://github.com/ufoscout/docker-compose-wait/releases/download/${ARG_COMPOSE_WAIT_VER}/wait -o /wait
RUN chmod +x /wait

ufoscout/docker-compose-wait is installed in the last 3 lines of Dockerfile.

It is a tool for waiting port listening with Rust. It’s intended to be used with docker-compose.yml.

How to wait for dependent middleware startup? Regarding, there are examples using netcat and dockerize, and writing shell scripts such as the official Postgres example. On the other hand, ufoscout/docker-compose-wait can handle the addition and deletion of middleware by adding a simple code to docker-compose.yml. And the operation is also reliable and easy to use.

docker-compose.yml

is here (github gist)

Defining multiple services by one Dockerfile

base: &base
    build:
      context: .
      dockerfile: ./Dockerfile
      cache_from:
        - rails6api-development-cache
      args:
        ARG_RUBY_VERSION: ${ARG_RUBY_VERSION:-2.7.1}
    image: rails6api-development:0.1.0

The “base” service does not perform any processing by command or entrypoint itself, and is defined as a service only for building.

As you can see from the fact that the alias is defined as “&base” , this is merged and used in the subsequent service (described later). Since the settings related to build are aggregated only in this “base” service, the settings in the build section will not appear in the services thereafter.

Notice that cache_from rails6api-development-cache is defined in this definition.

wait-middleware: &wait-middleware
    <<: *base
    environment:
      WAIT_HOSTS: db:3306
    depends_on:
      - db
    command: /wait

“wait-middleware” service definition is a service to wait for the start of “db” service using ufoscout/docker-compose-wait installed at the end of Dockerfile.

The “base” service defined earlier is merged, and the settings required by docker-compose-wait and the relationship with the “db” service are defined. If you want to run it independently, just run “docker-compose run”.

$ docker-compose run --rm wait-middleware

Creating network "rails6api_default" with the default driver
Creating rails6api_db_1    ... done
--------------------------------------------------------
 docker-compose-wait 2.7.3
---------------------------
Starting with configuration:
 - Hosts to be waiting for: [db:3306]
 - Timeout before failure: 30 seconds
 - TCP connection timeout before retry: 5 seconds
 - Sleeping time before checking for hosts availability: 0 seconds
 - Sleeping time once all hosts are available: 0 seconds
 - Sleeping time between retries: 1 seconds
--------------------------------------------------------
Checking availability of db:3306
Host db:3306 not yet available...
Host db:3306 is now available!
--------------------------------------------------------
docker-compose-wait - Everything's fine, the application can now start!

When you run it on a Mac, “db” starts up with almost no wait. At this speed, it’s unlikely that your application will run even though “db” isn’t running.

However, on GitHub Actions, it does not start at the same speed as the Mac, so it makes sense to wait for the port to be listened by “wait-middleware”.

backend: &backend
    <<: *base
    stdin_open: true
    tty: true
    volumes:
      - ./:/app:cached
      - ${GEMS_CACHE_DIR:-bundle-cache}:/bundle
      - rails-cache:/app/tmp/cache
    depends_on:
      - db

  console:
    <<: *backend
    ports:
      - 3333:3000
    command: /bin/bash

  server:
    <<: *backend
    ports:
      - 3333:3000
    command: bash -c "rm -f tmp/pids/server.pid && rails s -b 0.0.0.0"

volumes:
  mysql-data:
  bundle-cache:
  rails-cache:

“backend” is the service definition for executing prompt work after logging in to bash or executing rails s, and the definition part of volume.

Both “console” and “server” services merge the “backend” service definition. The volume defined as ${GEMS_CACHE_DIR:-bundle-cache}:/bundle in the volumes of this backend is synonymous with the directory of the bundle install destination. It is intended to “mount if the environment variable GEMS_CACHE_DIR is set, and if it is not set, mount it with a named volume named “bundle-cache”.

.env

MYSQL_ROOT_PASSWORD=root
MYSQL_ALLOW_EMPTY_PASSWORD=1
DB_HOST=db

MYSQL_FORWARDED_PORT=3806
MYSQL_FORWARDED_X_PORT=38060

The contents of .env are expanded for the environment variables of the “db” service.

.github/workflows/ci.yml

is here (github gist)

Speed up with BuildKit

env:
  DOCKER_BUILDKIT: 1
  COMPOSE_DOCKER_CLI_BUILD: 1

In Global environment variables, set the environment variables to enable BuildKit for docker-compose.

This Dockerfile does not include multi-stage definition, but enabling BuildKit will speed up the build time (around 12% reduction).

Jobs configuration

jobs:
  image-cache-or-build:

  test-app:
    needs: image-cache-or-build

  scan-image-by-trivy:
    needs: image-cache-or-build

First, be sure to restore the cache of the Docker image. Subsequent app testing and image vulnerability scanning should be performed using this restored image. App testing and image vulnerability scanning can be run in parallel.

Image cache restore and build

jobs:
  image-cache-or-build:
    strategy:
      matrix:
        ruby: ["2.7.1"]
        os: [ubuntu-18.04]
    runs-on: ${{ matrix.os }}
    env:
      ARG_RUBY_VERSION: ${{ matrix.ruby }}

    steps:
    - name: Check out code
      id: checkout
      uses: actions/checkout@v2

    - name: Cache docker image
      id: cache-docker-image
      uses: actions/cache@v1
      with:
        path: ${{ env.IMAGE_CACHE_DIR }}
        key: ${{ runner.os }}-${{ env.IMAGE_CACHE_KEY }}-${{ matrix.ruby }}-${{ hashFiles('Dockerfile') }}
        restore-keys: |
          ${{ runner.os }}-${{ env.IMAGE_CACHE_KEY }}-${{ matrix.ruby }}-

Cache restoration of Docker images is done with actions/cache which is the official action for cache processing.

Why is this example including ${{ hashFiles(‘Dockerfile’) }} in the cache key? This is to invalidate the cache when there is a change in the Dockerfile. If there is no COPY or ADD command in the Dockerfile, the image depends only on the contents of the Dockerfile.

Conversely, if the Dockerfile includes COPY and ADD processing, you can’t manage the cache with ${{ hashFiles(‘Dockerfile’) }}. This is because in this case, the generated image can be changed without changing the Dockerfile itself.

env:
  DOCKER_BUILDKIT: 1
  COMPOSE_DOCKER_CLI_BUILD: 1
  APP_IMAGE_TAG: rails6api-development:0.1.0
  APP_IMAGE_CACHE_TAG: rails6api-development-cache

# 略

    - name: Docker load
      id: docker-load
      if: steps.cache-docker-image.outputs.cache-hit == 'true'
      run: docker image load -i ${IMAGE_CACHE_DIR}/image.tar

    - name: Docker build
      id: docker-build
      run: docker-compose build --build-arg BUILDKIT_INLINE_CACHE=1 base

    - name: Docker tag and save
      id: docker-tag-save
      if: steps.cache-docker-image.outputs.cache-hit != 'true'
      run: mkdir -p ${IMAGE_CACHE_DIR}
        && docker image tag ${APP_IMAGE_TAG} ${APP_IMAGE_CACHE_TAG}
        && docker image save -o ${IMAGE_CACHE_DIR}/image.tar ${APP_IMAGE_CACHE_TAG}

In the above step, add the tag “rails6api-development-cache” to the built image and save it in tar, and save it in the actions/cache cache destination directory as “image.tar”.

When the cache DOES NOT hits…

1. A new image is built in the “docker-build” step
2. mkdir the ${IMAGE_CACHE_DIR} specified as the cache destination in actions/cache
3. Add a tag for image caching to the image of the build result
4. Save the image with the tag “rails6api-development-cache” added by docker image save. Specify the cache destination directory in actions/cache for this save destination (file name is “image.tar”)

When the cache hits…

1. In the “docker-load” step, image.tar restored from cache will be unpacked by docker image load
2. The image is built in the “docker-build” step, but this build is completed immediately because the image cache expanded in the “docker-load” step is fetched by “cache_from rails6api-development-cache”.
3. The “docker-tag-save” step is skipped because “if: steps.cache-docker-image.outputs.cache-hit !=’true’” is specified.

CI using the restored image

env:
  DOCKER_BUILDKIT: 1
  COMPOSE_DOCKER_CLI_BUILD: 1
  APP_IMAGE_TAG: rails6api-development:0.1.0
  APP_IMAGE_CACHE_TAG: rails6api-development-cache

# omission

jobs:

# omission

  test-app:
    needs: image-cache-or-build

    # omission

    - name: Cache docker image
      id: cache-docker-image
      uses: actions/cache@v1
      with:
        path: ${{ env.IMAGE_CACHE_DIR }}
        key: ${{ runner.os }}-${{ env.IMAGE_CACHE_KEY }}-${{ matrix.ruby }}-${{ hashFiles('Dockerfile') }}
        restore-keys: |
          ${{ runner.os }}-${{ env.IMAGE_CACHE_KEY }}-${{ matrix.ruby }}-

    - name: Docker load
      id: docker-load
      if: steps.cache-docker-image.outputs.cache-hit == 'true'
      run: docker image load -i ${IMAGE_CACHE_DIR}/image.tar

    - name: Docker compose build
      id: docker-build
      run: docker-compose build --build-arg BUILDKIT_INLINE_CACHE=1 base

When the job that builds and caches the image is completed, the “test-app” step that tests the application starts.

In the “docker-load” step, the image cache with the tag “rails6api-development-cache” is expanded, and in the “docker-build” step, this image cache is fetched with cache_from and the “base” service image is built.

Wait for db container to start with ufoscout/docker-compose-wait

- name: Wait middleware services
      id: wait-middleware
      run: docker-compose run --rm wait-middleware

    - name: Confirm docker-compose logs
      id: confirm-docker-compose-logs
      run: docker-compose logs db

Wait for db service startup using ufoscout/docker-compose-wait.

Starting with configuration:
 - Hosts to be waiting for: [db:3306]
 - Timeout before failure: 30 seconds 
 - TCP connection timeout before retry: 5 seconds 
 - Sleeping time before checking for hosts availability: 0 seconds
 - Sleeping time once all hosts are available: 0 seconds
 - Sleeping time between retries: 1 seconds
--------------------------------------------------------
Checking availability of db:3306
Host db:3306 not yet available...
Host db:3306 not yet available...
Host db:3306 not yet available...
Host db:3306 not yet available...
Host db:3306 not yet available...
Host db:3306 not yet available...
Host db:3306 not yet available...
Host db:3306 not yet available...
Host db:3306 not yet available...
Host db:3306 not yet available...
Host db:3306 not yet available...
Host db:3306 is now available!
--------------------------------------------------------
docker-compose-wait - Everything's fine, the application can now start!
--------------------------------------------------------

The above log is an execution example on the runner instance of GitHub Actions (sleep is executed every 1 second and waiting). It takes more than 10 seconds to listen to port 3306.

Use writable directory for gems caching

env:
      ARG_RUBY_VERSION: ${{ matrix.ruby }}
      GEMS_CACHE_DIR: /tmp/cache/bundle
      GEMS_CACHE_KEY: cache-gems

    # omission

    - name: Cache bundle gems
      id: cache-bundle-gems
      uses: actions/cache@v1
      with:
        path: ${{ env.GEMS_CACHE_DIR }}
        key: ${{ runner.os }}-${{ env.GEMS_CACHE_KEY }}-${{ matrix.ruby }}-${{ hashFiles('Gemfile.lock') }}
        restore-keys: |
          ${{ runner.os }}-${{ env.GEMS_CACHE_KEY }}-${{ matrix.ruby }}-

Caching and restoring dependent gems is also done by actions/cache.

The inclusion of ${{ hashFiles(‘Gemfile.lock’) }} in the cache key is intended to prevent a cache hit when a Gemfile.lock (Gemfile) is changed.

backend: &backend

    # omission

    volumes:
      - ./:/app:cached
      - ${GEMS_CACHE_DIR:-bundle-cache}:/bundle
      - rails-cache:/app/tmp/cache

# omission

volumes:
  mysql-data:
  bundle-cache:
  rails-cache:

What does ${GEMS_CACHE_DIR:-bundle-cache}:/bundle mean? If the environment variable GEMS_CACHE_DIR is set, specify the mount destination path in the environment variable. If not set, it will be mounted on the named valume.

In this example, bundle install is also executed inside the container. So if you want to cache the installed gems, you have to get the installation result from the volume mount.

If the installation result is extracted to a directory mounted with named volume, the real path will be /var/lib/docker/volumes/xxx. If I try to cache this real path with actions/cache, We get a “permission denied” error.

To avoid this error, set the environment variable GEMS_CACHE_DIR to any directory writable by non-root users (/tmp/cache/bundle in this example). This directory can be cached as is using actions/cache.

Run tests in the container

- name: Setup and Run test
      id: setup-and-run-test
      run: docker-compose run --rm console bash -c "bundle install && rails db:prepare && rspec"

Execute the processes in order of

1. bundle install
2. DB setup with db:prepare
3. app test (rspec is used in this example).

Instead of running each command directly on the runner instance, I run it via docker-compose run.

Usecases for development with these docker assets

Environment construction protocol for newcomer

# build
docker-compose build base

# setup
docker-compose run --rm console bash -c "bundle install && rails db:prepare && rails db:seed"

# run server
docker-compose up -d server

Start rails

docker-compose up -d server

Watch container’s log (like “tail -f”)

docker attach `docker-compose ps -q server`

Run spec

docker-compose exec server rspec SPEC_FILES

Connect rails console

docker-compose exec server rails c

Add, modify and apply migrations

# add
docker-compose exec server rails g migration MIGRATION_NAME

# apply
docker-compose exec server rails db:migrate