Docker pentester series #1 : MACVLAN

Docker supports a network driver, called MACVLAN. It allows to bind a container directly to the network interface card of the docker host through a sub-interface, supporting VLAN’s as well as a flat underlay network. This is extremely interesting from a pen-testing point of view. It looks like the container is directly connected to the underlay network and network traffic is not intercepted by iptables, bridges, NAT rules, etc …

Configuring PROMISCUOUS mode

One thing to note is that MACVLAN requires one of the NIC’s from the docker host to be in PROMISCUOUS mode. Depending of the virtualization technology in use, this requires typically some configuration changes.

For the sake of this tutorial, I created two virtual machines using 
docker-machine on Vmware Fusion (Virtualbox seems more problematic, because it does not allow multiple MAC addresses per virtual machine)

$ docker-machine create -d vmwarefusion vm1 
...
$ docker-machine create -d vmwarefusion vm2
...

Once the systems are up and running, shut them down and add an additional network interface in bridge mode.

Hint: Do not change the existing adapter settings, because docker-machine requires them to manage the hosts.

  • Vmware Fusion will ask your credentials when promiscuous mode is requested by the vm.
  • Vmware ESXi requires promiscuous mode to be configured in the advanced section of a portgroup.
VMFusion

Restart the machines. Once rebooted, ssh into the machines from different terminals.

$ docker-machine ssh vm1

Verify that the adapter is added into the docker host and (optionally) is configured with an IP address from the underlay (if DHCP is available). 
In my example, this is eth1. (but it might as well be another adapter)

Hint: The adapter in promiscuous mode might/should have an unusual high number of RX packets

docker@vm1:~$ ifconfig
docker0 Link encap:Ethernet HWaddr 02:42:EB:91:1D:32
inet addr:172.17.0.1 Bcast:0.0.0.0 Mask:255.255.0.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth0 Link encap:Ethernet HWaddr 08:00:27:8B:A6:B5 
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe8b:a6b5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:679 errors:0 dropped:0 overruns:0 frame:0
TX packets:454 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:58194 (56.8 KiB) TX bytes:58792 (57.4 KiB)
eth1 Link encap:Ethernet HWaddr 08:00:27:14:7B:F1 
inet addr:172.20.10.3 Bcast:172.20.10.15 Mask:255.255.255.240
inet6 addr: fe80::a00:27ff:fe14:7bf1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:583325 errors:0 dropped:0 overruns:0 frame:0
TX packets:70 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:414564889 (395.3 MiB) TX bytes:6992 (6.8 KiB)
Interrupt:17 Base address:0xd060
eth2 Link encap:Ethernet HWaddr 08:00:27:84:15:DA 
inet addr:192.168.99.100 Bcast:192.168.99.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe84:15da/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:84 errors:0 dropped:0 overruns:0 frame:0
TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:26004 (25.3 KiB) TX bytes:2538 (2.4 KiB)
lo Link encap:Local Loopback 
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

Configuring the MACVLAN network

Once our adapters on vm1 and vm2 are configured in promiscuous mode, we can create the MACVLAN docker network. The networks are configured on a per host, so we need to take care of overlapping IP addresses between hosts and underlay (if shared).

Hint: You can use the --ip-range option to avoid overlaps

docker@vm1:~$ docker network create -d macvlan \
— subnet=192.168.123.0/24 \
— gateway=192.168.123.254 \
-o parent=eth1 mylocal_net
docker@vm2:~$ docker network create -d macvlan \
— subnet=192.168.123.0/24 \
— gateway=192.168.123.254 \
-o parent=eth1 mylocal_net

On both hosts, we should be able to run something like

docker@vm1:~$ docker container run -it --network mylocal_net \
--ip 192.168.123.98 busybox
docker@vm2:~$ docker container run -it --network mylocal_net \
--ip 192.168.123.99 busybox
/ # ping 192.168.123.98
PING 192.168.123.98 (192.168.123.98): 56 data bytes
64 bytes from 192.168.123.98: seq=0 ttl=64 time=1.017 ms
64 bytes from 192.168.123.98: seq=1 ttl=64 time=0.747 ms
^C
--- 192.168.123.98 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.747/0.882/1.017 ms

/ #

If your MACVLAN network shares the same IP subnet of the underlay, you should be able to ping all hosts on the network (Vmware Fusion and Esxi)

docker@vm2:~$ docker container run -it --network mylocal_net \
--ip 192.168.123.56 busybox
/ # ping www.radarhack.com
PING www.radarhack.com (138.197.37.254): 56 data bytes
64 bytes from 138.197.37.254: seq=0 ttl=52 time=109.799 ms
64 bytes from 138.197.37.254: seq=1 ttl=52 time=120.282 ms
^C
— - www.radarhack.com ping statistics — -
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 109.799/115.040/120.282 ms
/ #

Connecting the container to the docker0 network

It is possible to connect the container to other networks with the docker network connect command. In a separate terminal:

docker@vm2:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
587439c18799 busybox “sh” 2 minutes ago Up 2 minutes competent_euclid
docker@vm2:~$

docker@vm2:~$ docker network ls
NETWORK ID NAME DRIVER SCOPE
5f511f68a1b0 bridge bridge local
25352ffbf464 host host local
2262287cf4f8 mylocal_net macvlan local
2e6e20989a92 none null local
docker@vm2:~$ docker network connect bridge 587439c18799

Check inside the busybox container and verify the bridge adapter is added. You can now access containers in the default bridge. In the same way, you can connect to any network, including docker swarm attachable overlays.

/ # ifconfig 
eth0 Link encap:Ethernet HWaddr 02:42:C0:A8:7B:38
inet addr:192.168.123.56 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::42:c0ff:fea8:7b38/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:47 errors:0 dropped:0 overruns:0 frame:0
TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:10128 (9.8 KiB) TX bytes:1222 (1.1 KiB)
eth1 Link encap:Ethernet HWaddr 02:42:AC:11:00:02 
inet addr:172.17.0.2 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::42:acff:fe11:2/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:16 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1296 (1.2 KiB) TX bytes:648 (648.0 B)
lo Link encap:Local Loopback 
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
/ #

Running a webserver on the MACVLAN interface

docker@vm2:~$ docker run -d --network mylocal_net \
--ip 192.168.123.57 nginx
Unable to find image ‘nginx:latest’ locally
latest: Pulling from library/nginx
693502eb7dfb: Pull complete
6decb850d2bc: Pull complete
c3e19f087ed6: Pull complete
Digest: sha256:52a189e49c0c797cfc5cbfe578c68c225d160fb13a42954144b29af3fe4fe335
Status: Downloaded newer image for nginx:latest
6767fa925473c043f0311e1fe3ac467367281067608838394dbff9840090a13c
docker@vm2:~$

Please note we do not have to expose any ports. The webserver is available from any machine on the network.