Automatic load-balancing for your docker-compose services !

I have already seen and heard about peoples getting a lot of troubles with load-balancing there docker-compose services.

They rely on the list of containers of the scaled service to edit HAProxy configuration. One classic way to do it is to register your docker service into a registry (let’s say consul). Then the load balancer can adapts its configuration based on HAProxy settings.

Architecture for registration of the service inside consul

Such an architecture have several drawbacks :

• You need a registry server
• You need to run a process in your load balanced services to handle interaction with registry
• You need a custom instance of HAProxy container (like this : https://github.com/hashicorp/consul-template)

All of this means devOps operation, and might be complicated to master.

Not satisfied about such a complicated situation, I lurked on the internet and found dockercloud/haproxy containers.

This modify instance of HAProxy listens on the docker socket :

• It discovers which groups it is linked with
• It discovers containers which are part of the service it is linked with
• It then inspect them to get the exposed port
• It finally edit the HAProxy configuration based on this information

To test it, we will first create a (dirty) web server image with this Dockerfile :

FROM monsantoco/min-jessie:latest
RUN apt-get update
RUN apt-get install -y apache2
RUN touch /var/log/apache2/error.log
EXPOSE 80
ENTRYPOINT /etc/init.d/apache2 start && tail -f /var/log/apache2/access.log

The important point is to have an EXPOSE directive for HAProxy to discover the port our servers are listening on.

Then we can build our image :

docker build --tag default_apache .

Now it is time to define our docker-compose architecture :

version: '2'
services:
  web:
    image: default_apache
  lb:
    image: dockercloud/haproxy
    links:
     - web
    ports:
     - '80:80'
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

We need to mount the docker socket in order for the HAProxy to inspect network settings. We define the link to the back-end.

And finally we can run our architecture :

docker-compose up

Scale it :

docker-compose scale web=3

In the log we can see HAProxy found our web instances :

lb_1   | backend default_service
lb_1   |   server automaticlb_web_1 automaticlb_web_1:80 check inter 2000 rise 2 fall 3
lb_1   |   server automaticlb_web_2 automaticlb_web_2:80 check inter 2000 rise 2 fall 3
lb_1   |   server automaticlb_web_3 automaticlb_web_3:80 check inter 2000 rise 2 fall 3

Then we requesting, we can simply see loadbalancing threw the logs :

web_1  | 172.25.0.3 - - [19/Nov/2016:08:32:47 +0000] "GET / HTTP/1.1" 200 485 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0"
web_2  | 172.25.0.3 - - [19/Nov/2016:08:32:47 +0000] "GET /logo.png HTTP/1.1" 200 13133 "http://127.0.0.1/" "Mozilla/5.0 (X11; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0"
web_3  | 172.25.0.3 - - [19/Nov/2016:08:32:48 +0000] "GET / HTTP/1.1" 200 485 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0"
web_1  | 172.25.0.3 - - [19/Nov/2016:08:32:48 +0000] "GET /logo.png HTTP/1.1" 200 13133 "http://127.0.0.1/" "Mozilla/5.0 (X11; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0"
web_3  | 172.25.0.3 - - [19/Nov/2016:08:32:49 +0000] "GET /logo.png HTTP/1.1" 200 13133 "http://127.0.0.1/" "Mozilla/5.0 (X11; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0"
web_1  | 172.25.0.3 - - [19/Nov/2016:08:32:49 +0000] "GET / HTTP/1.1" 200 485 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0"
web_2  | 172.25.0.3 - - [19/Nov/2016:08:32:49 +0000] "GET / HTTP/1.1" 200 485 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0"
web_2  | 172.25.0.3 - - [19/Nov/2016:08:32:49 +0000] "GET /logo.png HTTP/1.1" 200 13133 "http://127.0.0.1/" "Mozilla/5.0 (X11; Linux x86_64; rv:49.0) Gecko/201

Note : you can edit the HAProxy configuration (load balancing policy, etc…) threw environments variables defined in the docker-compose.yml .

Such a solution come however with some limitations :

• It presents some security issue as your load balancer have complete control on your docker installation. Having this container compromised is equivalent to have your whole infrastructure compromised.
• I did not tested it with swarm. I guess you might need a unified network, and to perform calls over https. It’s not impossible as docker-cloud is doing it, but might require more work.

However, even a is, this solution is interesting for scaling mono-core processes on a single machine (node.js I am thinking to you!).