Connecting Containers in Compose

How to Connect to a MySQL Docker Container Without Using a Static IP Address

Quinn Vissak
Nov 4 · 6 min read

TL;DR — Simplify your code with Docker Compose!

If you are dealing with more than one container in the same application, chances are they need to talk to each other, but have static IP addresses. The code for this looks terrible and requires at least a basic understanding of subnets. Static IP addresses also impede your ability to run more than one instance of your application on the same host.

To address this problem, Docker Compose (and vanilla Docker) by default uses an embedded DNS server to communicate within networks. It allows one to use the service name in the file as the domain in the connection details of your other containers. This amazing feature is not limited to MySQL either — it works with any containerized service.

One might think that using Docker Compose for only two services is overkill — especially if parallelization on the same host is not a requirement — but its simplicity will enhance your code quality, make it easier to scale, and simplify adding additional services later on.

Use Case

When teams grow, infrastructure should grow. At one point my team grew in size, but our infrastructure did not. We ran into a problem on one application where pull request validations were being run in a series, forming a bottleneck. When a pull request was made, it kicked off a continuous integration process on Jenkins. The job ran on a single virtual machine that contained all our software dependencies and used a single database instance in the cloud. As a result, when more than one pull request was made, jobs were queued so state inconsistencies in the database would not cause tests to fail. The test suite that executes on each pull request could not run in parallel; hence the bottleneck.

The application was already “dockerized” for local development without Docker Compose. Trying to run multiple instances of each container in the application simultaneously, with static IP addresses, resulted in the below error from the Docker daemon.

ERROR: Pool overlaps with other one on this address space

Even though a different Docker network is created for each new instance, the pool of IP addresses available is shared per host machine, not per network. One solution is to deploy a new virtual machine and database for each pull request. However, not only does this become costly on a larger team, it also drastically increases the length of time to get feedback on your code changes. Spinning up a Docker container with a lightweight base image containing software dependencies takes seconds, whereas waiting for a new virtual machine to be provisioned with the appropriate resources could take tens of minutes.

With Docker Compose, true network isolation can be easily achieved and an out-of-the-box Jenkins pipeline can be used without having to worry if two jobs are placed on the same machine to execute. The application stack can also be provisioned in seconds, and testing can begin executing immediately.

Tutorial: How to Connect Docker Containers using Docker Compose

1. Examine the Alternative

Connect to a MySQL Docker container from an Alpine Linux machine with a MySQL client using static IPs.

docker-compose.yml:

version: '3'services:    mysql:        environment:            MYSQL_DATABASE: mydb            MYSQL_ROOT_PASSWORD: qwerty        image: mysql:5.7        networks:            testing_net:                ipv4_address: 172.28.1.1    shell:        build:        context: .        networks:            testing_net:                ipv4_address: 172.28.1.2        stdin_open: true        tty: truenetworks:    testing_net:        ipam:            driver: default            config:                - subnet: 172.28.0.0/16

Dockerfile:

FROM alpine:3.7RUN apk add --no-cache mysql-client

In the example above, running will bring up both containers. However, running it again in a different directory or even with the flag in the same directory will yield the networking error previously discussed.

Log into the Docker container and connect to MySQL as usual using its IP address:

docker exec -it $(docker ps -q -f "name=$(echo "${PWD##*/}")_shell_1") shmysql --host=172.28.1.1 --port=3306 --user=root --password=qwerty

2. Simplify the Setup

Keep the same Alpine Dockerfile, modify the .

version: '3'services:    mysql:        environment:            MYSQL_DATABASE: mydb            MYSQL_ROOT_PASSWORD: qwerty        image: mysql:5.7        networks:            - testing_net    shell:        build:            context: .        networks:            - testing_net        stdin_open: true        tty: truenetworks:    testing_net:

Notice how the network is defined by only its name using the default parameters. Both containers are configured to use that network.

Now, bring the containers up.

docker-compose up -d

Log into the Linux machine.

docker exec -it $(docker ps -q -f "name=$(echo "${PWD##*/}")_shell_1") sh

Connect to the database using as the hostname (since it’s the name of our service in the ).

mysql --host=mysql --port=3306 --user=root --password=qwerty

You should see the following text:

Welcome to the MariaDB monitor.  Commands end with ; or \g.Your MySQL connection id is 2Server version: 5.7.26 MySQL Community Server (GPL)Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.MySQL [(none)]>

3. Profit

Under the hood, Docker is mapping the service name to the dynamically generated IP address for the new container. When using the service name, Docker automatically performs a DNS lookup within its network(s) to resolve the service name to an IP address in order to establish a connection.

If you tried to spin up another N instances, it would work fine since Docker would choose unallocated IPs in its address pool.

Next Steps

My team noticed immediate improvements in our continuous integration process upon making this change. Using Docker Compose standardized the setup, decreasing the learning curve for onboarding new hires. Permitting Jenkins to run separate instances on the machine in parallel eliminated the bottleneck in receiving pull request feedback. Now the software engineers on my team can develop new features on their local machines faster with a standardized process — on Jenkins, they are able to receive feedback in a timely manner and ultimately, everyone looks forward to expanding our use of Docker in production.

The documentation on networking in Docker Compose is excellent if you would like to dive deeper into the additional configurable options available.

I would highly recommend considering “dockerizing” the next project you work on, or refactoring your current project to use Docker Compose. Configuring a project so that local set up is refined to just will immediately show a return on investment. A short and standardized run command decreases the time it takes for new developers to onboard onto your team and welcomes other developers within your organization to make a contribution to your project. Think of it like a good first chapter of a book that invites readers to continue to the next.


DISCLOSURE STATEMENT: © 2019 Capital One. Opinions are those of the individual author. Unless noted otherwise in this post, Capital One is not affiliated with, nor endorsed by, any of the companies mentioned. All trademarks and other intellectual property used or displayed are property of their respective owners.

Capital One Tech

The low down on our high tech from the engineering experts at Capital One. Learn about the solutions, ideas and stories driving our tech transformation.

Thanks to Marika Gutzman

Quinn Vissak

Written by

Backend Senior Associate Software Engineer at Capital One — passionate about utilizing the best software practices to make systems and teams more efficient.

Capital One Tech

The low down on our high tech from the engineering experts at Capital One. Learn about the solutions, ideas and stories driving our tech transformation.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade