What is Docker Compose and why do you need it as a DevOps engineer?

In this tutorial, I will show you examples of why you need it if you work with Docker.

Docker Compose is a tool designed to solve several key problems that arise when managing multi-container Docker applications.

Here are some of the primary problems it solves:

Simplifying Configuration

Imagine you have a dedicated Linux server, and you run a backend application on it.

This is how the docker run command looks:

docker run -d --name dev-my-backend --hostname dev-my-backend -p 127.0.0.1:3001:80 --env-file /home/andrey/dev/my-backend.env --restart always registry.andrey.com/dev/my-backend:latest

• -d: Run the container in the background and print the container ID.
• --name dev-my-backend: Assigns the name dev-my-backend to the container.
• --hostname dev-my-backend: Sets the hostname of the container to dev-my-backend.
• -p 127.0.0.1:3001:80: Maps port 3001 on the localhost to port 80 in the container.
• --env-file /home/andrey/dev/my-backend.env: Sets environment variables from the specified file.
• --restart always: Configures the container to always restart.
• registry.andrey.com/dev/my-backend:latest: Specifies the image to use for the container.

As you can see, it’s not really user friendly to run long docker run commands. What if you have more parameters?

This is where Docker compose simplifying the configuration.
All you need to do is to create file docker-compose.yml and execute docker compose up -d command.

Here is the docker-compose.yml:

#docker-compose.yml
version: "3.8"
services:
  my-backend:
    container_name: dev-my-backend
    hostname: dev-my-backend
    image: registry.andrey.com/dev/my-backend:latest
    env_file:
      - /home/andrey/dev/my-backend.env
    ports:
      - "127.0.0.1:8001:80"
    restart: always

The command you need to execute in order to run docker (execute it from the same directory where the docker-compose.yml located):

docker compose up -d 

As you see all the parameters that was in command exist in this file. Now it’s readable and you can change any of values or add any keys easily.

Note the first line version: “3.8”. Read more about the compose file versions here — compose-versioning.
It’s a parameter defined by the Compose Specification for backward compatibility. It is informative only.

The second problem Docker compose solves is orchestration

Let’s say you need to add Redis for your backend service.

Here is the desired Redis configurations:

• -d: Run container in background and print container ID.
• --name dev-redis: Assigns the name dev-redis to the container.
• --hostname dev-redis: Sets the hostname of the container to dev-redis.
• -p 127.0.0.1:6379:6379: Maps port 6379 on the localhost to port 6379 in the container.
• --restart always: Configures the container to always restart.
• --memory 1024m: Limits the memory usage to 1024MB.
• --log-driver local: Sets the logging driver to local.
• --log-opt max-file=5: Sets the maximum number of log files that can be present.
• --log-opt max-size=100m: Sets the maximum size of each log file to 100MB.
• -v dev_redis_data:/data:rw: Mounts the volume dev_redis_data to /data in the container with read-write access.
• registry.andrey.com/dev/redis:latest: Specifies the image to use for the container.
• redis-server --appendonly yes --maxmemory 300mb: Sets the command to start the Redis server with specified configuration.

So the docker run command looks like this:

docker run -d --name dev-redis --hostname dev-redis -p 127.0.0.1:6379:6379 --restart always --memory 1024m --log-driver local --log-opt max-file=5 --log-opt max-size=100m -v dev_redis_data:/data:rw registry.andrey.com/dev/redis:latest redis-server --appendonly yes --maxmemory 300mb

Now you need to run backend service and Redis. To simplify the process you can add both services into one docker-compose.yml file.

version: "3.8"
services:
  my-backend:
    container_name: dev-my-backend
    hostname: dev-my-backend
    image: registry.andrey.com/dev/my-backend:latest
    env_file:
      - /home/andrey/dev/my-backend.env
    ports:
      - "127.0.0.1:3001:80"
    restart: always

  dev-redis:
    container_name: dev-redis
    hostname: dev-redis
    image: registry.andrey.com/dev/redis:latest
    volumes:
      - "dev_redis_data:/data:rw"
    command: redis-server --appendonly yes --maxmemory 300mb
    logging:
      driver: "local"
      options:
        max-file: "5"
        max-size: 100m
    ports:
      - "127.0.0.1:6379:6379"
    restart: always
    mem_limit: 1024MB

volumes:
  cc_redis_data:

Now when you will execute docker compose up -d command, both dev-my-backend and dev-redis containers will start.

In order to stop them, run docker compose down.

Now you can ask: Why do I need Docker compose for orchestration if I can use dedicated orchestration tools like Docker Swarm or Kubernetes?
Of course you can. However my personal opinion is — for simple problems, use simple solutions.

Let’s say I need to create some application on my local environment for test purposes only, or as a pilot, which consists of six services (frontend, backend, validation service, notification service, Redis and database).

I won’t raise a Kubernetes cluster for this, it will be a waste of time and resources. I can use couple of docker-compose.yml files instead.

In other hand, if you already have a running cluster in this environment, then maybe you should run your application directly on Kubernetes.

Here are some of the more advantages of Docker compose:

• Environment Standardization: It ensures consistency across different environments (development, testing, staging, production).
  You can use the same docker-compose.yml file across different environments, which reduces the risk of bugs caused by environment discrepancies.
• Convenient Scaling and Updating: docker-compose simplifies the process of scaling your application.
  You can easily scale a service to multiple instances and update configurations or versions of services without needing to manually stop and recreate each container.

Let’s say you did some change in the docker-compose file. In order to recreate the docker container, execute the following command:

docker-compose up -d --force-recreate

• Portability: A docker-compose.yml file can be shared among team members, and it works the same way on every machine that has Docker and Docker Compose installed.
  This portability ensures that your application runs identically on every developer’s workstation and in any staging or production environment.
• Resource Separation and Networking: docker-compose enables easy setup of separate networks, volumes, and other resources, which helps in isolating different components of an application for security, performance, or organizational reasons.

Networking in Compose — read here.
Volumes in Docker — read here.

• Debugging and Logging: Docker compose simplifies the process of collecting logs from multiple containers, making it easier to debug issues when they arise.

Docker compose logs — read here.

As you see, Docker compose significantly eases the management of Docker containers, especially in a development environment where applications consist of multiple interconnected containers.

If you liked my articles, join my newsletter, and you will receive weekly DevOps tutorials, articles, and tips every Saturday.

As a bonus, you will receive a free step-by-step DevOps CI/CD project, which you can use in your portfolio.

Subscribe here: https://junior-devops-hub.ck.page