Advanced Docker networking // Custom outgoing IP

In summary Docker is utilizing the iptables “nat” to resolve packets from and to its containers and “filter” for isolation purposes. Curious how I was I took a glance at the current state of my hosts iptables with the command iptables-save which led to the following output regarding the nat table:

*nat
:PREROUTING ACCEPT [2:139]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER — [0:0]
-A PREROUTING -m addrtype — dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype — dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
COMMIT

We can see that Docker introduced a custom chain named DOCKER and is redirecting specific packets from PREROUTING and OUTPUT to it.

The magic regarding outgoing traffic takes place with the instruction -A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE

Masquerade

Let’s break down the magic instruction -A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE

This rule

• belongs to the chain -A POSTROUTING which allows modifying routed packets.
• applies to packets coming from the subnet -s 172.17.0.0/16 that are not going to be sent via the interface ! -o docker0 where the latter is Dockers default bridge interface and the former is its IPv4 subnet.
• instructs to jump to -j MASQUERADE which assigns the corresponding IP of the outgoing interface to matching packets.

This rule is crucial in the sense that we want to change that behaviour to have it assign our desired secondary IP instead of automatically assign packets the default outgoing IP.

Let’s do it

To verify that everything is working out I’m using a docker container to query my current IP via http://www.myip.ch .

docker run --rm byrnedo/alpine-curl http://www.myip.ch

The service states that my current IP address is 78.31.xxx.xxx which is my hosts default ethernet IP.

Docker is creating the masquerading rule automatically so we need to disable this creation and handle the iptables part by our own. As I don’t like the idea of changing the behavior of Dockers default bridge docker0 I decided to create a new network named bridge-coi with the following command:

docker network create --attachable --opt ‘com.docker.network.bridge.name=bridge-coi’ --opt ‘com.docker.network.bridge.enable_ip_masquerade=false’ bridge-coi

The option name=bridge-coi specifies the name of the interface and the option enable_ip_masquerade=false disables the automatic creation of such MASQUERADE entries within iptables. (Source)

Okay so we got our network with masquerading disabled therefore we need to handle the iptables part by our own. So let’s add the crucial entry to the iptable nat and its chain POSTROUTING with the following command:

iptables -t nat -A POSTROUTING -s 172.18.0.0/16 ! -o bridge-coi -j SNAT --to-source 5.104.xxx.xxx

Note that -j MASQUERADE is doing an SNAT (source nat) under the hood though automatically choosing the --to-source ip based on the outgoing interface.

This looks a little bit different compared to the previous -j MASQUERADE rule. So let’s break it down again:

This rule

• belongs to the POSTROUTING chain of the nat table, so nothing changed.
• applies to packets coming from the subnet-s 172.18.0.0/16 that are not going to be sent via the interface ! -o bridge-coi where the latter is our new bridge interface and the former is its IPv4 subnet, which you can either specify yourself or check with command docker network inspect bridge-coi .
• instructs to jump to -j SNAT while giving the instruction to assign the source IP 5.104.xxx.xxx to all matching packets.