Docking a Docker Container — Part 3 : Networking

Milind Deore
Nov 7, 2016 · 7 min read

In the previous session, we saw namespace and cgroup creating execution environment for containers to run. The other important piece which stitch them together is networking. When you actually deploy your production application than it must be having multiple container but how would these multiple container talk to each other? What should be the IP architecture of the container framework.

How docker networking works?

By default docker creates three networks when you startup docker daemon, namely: bridge, none, host.

[root@docket ] docker network ls
NETWORK ID NAME DRIVER
8b169e0c38fd none null
0422d120f29c host host
19e614e5beec bridge bridge
[root@docker ] docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3673e651a804 ubuntu "/bin/bash" 16 hours ago Up 16 hours rcv
148edfe98193 ubuntu "/bin/bash" 16 hours ago Up 16 hours src
[root@docker ] docker network inspect bridge
[
{
"Name": "bridge",
"Id": "19e614e5beeca0bbb73907f37c3ccb81893349a91031264a3610e7a1c158c28d",
"Scope": "local",
"Driver": "bridge",
"IPAM": {
"Driver": "default",
"Config": [
{
"Subnet": "172.17.160.1/19",
"Gateway": "172.17.160.1"
}
]
},
"Containers": {
"148edfe981937f3f8b15b40e1088273aafaeb9bd57d03a3f102ba6248a9f6575": {
"EndpointID": "8c921a07b26255535f97045f9f7ce4a4d379f471c139d8d7837ed70d9d94a0b5",
"MacAddress": "02:42:ac:11:a0:02",
"IPv4Address": "172.17.160.2/19",
"IPv6Address": ""
},
"3673e651a804c872584a588d8e39d95cba7219d02541266a8bf4fe903696bb1c": {
"EndpointID": "249d1edf5f05304d89652cb10839d2bb731d964e32635f164829fd3702398065",
"MacAddress": "02:42:ac:11:a0:03",
"IPv4Address": "172.17.160.3/19",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.bridge.default_bridge": "true",
"com.docker.network.bridge.enable_icc": "true",
"com.docker.network.bridge.enable_ip_masquerade": "true",
"com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
"com.docker.network.bridge.name": "docker0",
"com.docker.network.driver.mtu": "1500"
}
}
]
[root@docker ] docker network inspect none
[
{
"Name": "none",
"Id": "8b169e0c38fdc80337f4e38ab1e89328fb79f260c501ffadad8ff1141e2384f0",
"Scope": "local",
"Driver": "null",
"IPAM": {
"Driver": "default",
"Config": []
},
"Containers": {},
"Options": {}
}
]
[root@docker ] docker network inspect host
[
{
"Name": "host",
"Id": "0422d120f29cd0a63952fdf5bc41dde7004bc34e4140458ea8ed90b1c60e4bde",
"Scope": "local",
"Driver": "host",
"IPAM": {
"Driver": "default",
"Config": []
},
"Containers": {},
"Options": {}
}
]

Looking at the above output, those two container are listed under network ‘bridge’ and this is the reason by default any container under bridge network is reachable (PINGable) via their IP addresses but not by their names ‘src’ and ‘rcv’. Containers can also use their names to communicate with each other but for that they need to use — link option (we will cover that in a while).

Docker bridge interfacing with containers

This figure shows the bridge interface that connects physical ethernet interfaces on the host to the docker container(s). Docker daemon has range of IP addresses from the same subnet where bridge interface falls and assign it to every new containers, essentially it uses NAT but based on IP addresses. NATing also isolates containers from communicating from external world, they can reach out to the external world only via bridge. By default all the containers within a docker daemon can talk to each other and this is done via docker daemon flag ‘ — icc=true’ (Inter Container Communication). There is another flag worth mentioning ‘ — iptables’, this overrides ‘ — icc’ functionality, it refrains docker daemon from making changes to iptables if — iptables=falue.

Now in the production environment you would definitely set proper addressing scheme to avoid interference and mostly likely private IP addressing. In this example we used class-B address for our gateway bridge and for all the NATed hosts within. By default docker setup private IP address based on RFC1918 but at times you may have to setup your own range of private IP scheme and this is when you have to change the default behaviour and setup bridge IP address, for more options:

[root@docker$ ] service docker stop 
[root@docker$ ] vim /etc/default/docker
DOCKER_OPTS="--bip=172.16.160.1/20"[root@docker$ ] service docker start
[root@docker$ ] ifconfig
docker0 Link encap:Ethernet HWaddr 02:42:64:0d:8b:b6
inet addr:172.16.160.1 Bcast:0.0.0.0 Mask:255.255.224.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth0 Link encap:Ethernet HWaddr 08:00:27:44:20:d8
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe44:20d8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:63 errors:0 dropped:0 overruns:0 frame:0
TX packets:112 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:16848 (16.8 KB) TX bytes:14347 (14.3 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:69 errors:0 dropped:0 overruns:0 frame:0
TX packets:69 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:7229 (7.2 KB) TX bytes:7229 (7.2 KB)

Now, if you start any container inside this daemon, it would automatically get IP address from the above IP address subnet range. Example:

root@9d51b764a3c1:/# ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:ac:11:a0:02
inet addr:172.16.160.2 Bcast:0.0.0.0 Mask:255.255.224.0
inet6 addr: fe80::42:acff:fe11:a002/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:10863 errors:0 dropped:0 overruns:0 frame:0
TX packets:10767 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:25248347 (25.2 MB) TX bytes:586961 (586.9 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

Again at times we want these containers to have a specific static IP addresses (of course from the same subnet range, but of our own choice rather than docker daemon assigning random addresses). There is a ways to do this:

  • Container Runtime: After running the container we can modify the configuration files that store network settings at /var/lib/docker/containers/<container ID>. But this may not be very useful as often such network setting are required to be given at startups because runtime is volatile and may vanish in the next startup. I could not find any other graceful way to assign specific IP address to the container at startup time.
[root@docker ] docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
708290c3743e centos "/bin/bash" 35 minutes ago Up 35 minutes silly_visvesvaraya
9d51b764a3c1 ubuntu "/bin/bash" 39 minutes ago Up 39 minutes small_jang
[root@docker ] ls -l /var/lib/docker/containers/9d51b764a3c1034585f61a2d8156702b8afbcf258f510d5c93bde91d82d7767d/
total 60
-rw------- 1 root root 36856 Nov 6 23:12 9d51b764a3c1034585f61a2d8156702b8afbcf258f510d5c93bde91d82d7767d-json.log
-rw-r--r-- 1 root root 2398 Nov 6 22:55 config.json
-rw-r--r-- 1 root root 720 Nov 6 22:55 hostconfig.json
-rw-r--r-- 1 root root 13 Nov 6 22:55 hostname
-rw-r--r-- 1 root root 176 Nov 6 22:55 hosts
drwxrwxrwt 2 root root 40 Nov 6 22:38 mqueue
-rw-r--r-- 1 root root 190 Nov 6 22:55 resolv.conf
-rw------- 1 root root 71 Nov 6 22:55 resolv.conf.hash
drwxrwxrwt 2 root root 40 Nov 6 22:55 shm

Linking Containers

This is the best way to interface containers in a docker host/daemon, this is also called ‘linking containers’. The most secure way of communication but with a downside that it works only between container to container communication and not for outside world. “In a way this is like blessing in disguise”.

[root@docker ] docker run —name=src -d <source container image>[root@docker ] docker run —name=rcvr —link=src:alias-src -d <receiver container image>[root@docker ] docker inspect rcvr
........
“Links”: [
“/src:/src/alias-src”
........
[root@docker ] docker inspect src | grep Links
“Links”: null,

This is because source container provides quite a few environment variables to the receiver container. It also adds an entry in the /etc/hosts file, as we can see below:

root@1c8b2b26e9fa:/#  env | grep ALIAS
ALIAS_SRC_PORT_80_TCP_ADDR=172.16.160.2
ALIAS_SRC_PORT_80_TCP_PROTO=tcp
ALIAS_SRC_PORT_80_TCP_PORT=80
ALIAS_SRC_PORT=tcp://172.16.160.2:80
ALIAS_SRC_NAME=/rcvr/alias-src
ALIAS_SRC_PORT_80_TCP=tcp://172.16.160.2:80
root@1c8b2b26e9fa:/# cat /etc/hosts
172.16.160.3 1c8b2b26e9fa
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.16.160.2 alias-rcvr

It is also important to mention that ‘src’ and ‘rcvr’ both can communicated with each other, which means it creates bidirectional channel. Also a source can connect to many receivers and similarly a receiver can link to as many sources.

Exposing Port To The External World

This is the most common way of communication between containers and the outside world, rather a much better way for external world to communicate with the containers, because here we just expose the port and not IP address. This also means any physical interface on the docker host can accept the incoming packets, because what iptable set the rule to is 0.0.0.0:80, let us take an example:

[root@docker ] vim Dockefile.........
EXPOSE 80
.........
[root@docker ] docker build -t="apache-img" .

Once the image is built with EXPOSEing port number in the application, run the image using ‘-p’ option and specify : {incoming port of docker host : docker container port}.

[root@docker ] docker run -d -p 5010:80 apache-img[root@docker ] docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9d51b764a3c1 apache-img "/bin/bash" 2 hours ago Up 2 hours 0.0.0.0:5010->80/tcp small_jang
[root@docker ] docker port small_jang
80/tcp -> 0.0.0.0:5010

This means all the packets coming from docker host interface on 0.0.0.0 (any interface) from port 5010 will enter port 80 of the docker container that has exposed port 80.

A specific interface/IP address can also be specific instead of 0.0.0.0 and this can be done using:

[root@docker ] docker run -d -p 192.168.56.5:5010:80 apache-img[root@docker ] docker port small_jang
80/tcp -> 192.168.56.5:5010

Where, lets say http://192.168.56.5:5010 request will land to apache server’s port 80 which is running within the container.

Before I say “Good Bye for Now!”

Default bridge will allow container-to-container communication using their IP address(which are dynamically assigned by daemon) this becomes difficult to know the IP of container and more difficult to program (restart will not guarantee that the same IP will be assigned back). In this case the best option is linking container i.e. by their name, where you know the name of the container before startup and easy to program. Therefore you may want to switch off the default IP address based communication isn’t it? This can be done by

--icc=false

After this you can link your containers and world would be much secure place to live.

Conclusion

There is no hard rule, but exposing ports most likely work best for communicating with the external world, whereas linking container is best suited for internal communication between containers. This also works as hidden LAN which is isolated from the external world and hence it is protected too.

Milind Deore

Written by

Senior Manager— Machine Learning @ FICO, Malaysia. https://tomdeore.wixsite.com/epoch

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade