Enabling docker remote API on OSX using vagrant

Recently I was playing with docker API on OSX and the setup was little complicated so I thought to put all steps together in one place. Just to summarise the default tcp port for docker is 2375 (unencrypted) and 2376(encrypted) communication over tcp(although we can choose any other port).

The setup on linux is fairly straight away using systemd and a lot of help is out there however on OSX there are issues to deal with.

For security reasons , on OSX the default tcp port is not exposed and listens to socket/var/run/docker.sock .

There are two ways using which you can consume docker api on your OSX. Easy and a complicated one.

  1. Easy way: Use socat container . This container can redirect the Docker API exposed on the unix domain socket in Linux to the port of our choice on OSX host. Most of the time you will need this if you want to just play with api. On your shell run -
>>   docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 127.0.0.1:2375:2375 bobrik/socat TCP-LISTEN:2375,fork UNIX-CONNECT:/var/run/docker.sock
>> export DOCKER_HOST=tcp://localhost:2375
>> curl localhost:2375/images/json | jq .
[
{
"Containers": -1,
"Created": 1536186012,
"Id": "sha256:cd6d8154f1e16e38493c3c2798977c5e142be5e5d41403ca89883840c6d51762",
"Labels": null,
"ParentId": "",
"RepoDigests": [
"ubuntu@sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378"
],
"RepoTags": [
"ubuntu:latest"
],
"SharedSize": -1,
"Size": 84117621,
"VirtualSize": 84117621
},
{
"Containers": -1,
"Created": 1432036646,
"Id": "sha256:e617a56c238ed06a0215366a122d19fab0b94b28c1413e2171bbe2f883686e6b",
"Labels": {},
"ParentId": "",
"RepoDigests": [
"bobrik/socat@sha256:afea7dbd06940b9979cec0f1b9e8ccb7111d6feb671687ba9281a57136c1564e"
],
"RepoTags": [
"bobrik/socat:latest"
],
"SharedSize": -1,
"Size": 6950494,
"VirtualSize": 6950494
}
]

2. Complicated one: This might be a general case, if you are experimenting more complicated setup where you need to test your setup for remote docker execution on coreos. We are using coreos here because of its minimalist nature and smaller size (you can choose your own image and tweak the below steps as required) . I am assuming you already know vagrant, docker and virtualbox(you need not be expert in any of them).

a. Download and install vagrant and Virtualbox

b. Clone the core-os vagrant repo using git clone https://github.com/coreos/coreos-vagrant

c. Install ct(config transpiler) on OSX using brew install coreos-ct . Config transpiler helps parsing the container linux configuration(friendly config specific to coreos) to json format(to be used by ignition plugin). Ignition plugin on other hand is like user-data in aws. It is used for partitioning disks, formatting partitions, writing files (regular files, systemd units, networkd units, etc.), and configuring users on first boot. Earlier coreos used coreos-cloudinit tool to install the contents of file named withuser-data. This is now deprecated, so we no longer edit the user-data file.

d. Move to the coreos-vagrant directory cd coreos-vagrant and install vagrant plugin install vagrant-ignition .

e. Change the line in Vagrantfile which looks like required_plugins = %w(vagrant-ignition) to look like required_plugins = %w() . Without this modification we will be stuck in an infinite loop of vagrant-ignition plugin installation. The change will look like

#required_plugins = %w(vagrant-ignition)
required_plugins = %w()

f. Edit the contents of cl.conf (human readable ignition config ) from the repo to look like

# installation of systemd services for docker 
# docker-tcp.socket service will enable API at endpoint 2375
# enable-docker-tcp.service is a onetime service required to start
# the docker.socket and docker-tcp.socket services.
systemd:
units:
- name: docker-tcp.socket
enable: true
contents: |
[Unit]
Description=Docker Socket for the API
[Socket]
ListenStream=2375
Service=docker.service
BindIPv6Only=both
[Install]
WantedBy=sockets.target
- name: enable-docker-tcp.service
contents: |
[Unit]
Description=Enable docker socket for the api
[Service]
ExecStartPre=/usr/bin/systemctl stop docker.socket
ExecStartPre=/usr/bin/systemctl stop docker-tcp.socket
ExecStartPre=/usr/bin/systemctl stop docker
ExecStart=/usr/bin/systemctl enable docker-tcp.socket
ExecStartPost=/usr/bin/systemctl start docker.socket
ExecStartPost=/usr/bin/systemctl start docker-tcp.socket
Type=oneshot
RemainAfterExit=true

g. Now we create the json ignition configuration by running commandct — platform=vagrant-virtualbox < cl.conf >config.ign .Using ct catches errors beforehand and is a good way to reduce common mistakes.

h. Create config.rb in the repo with following below contents

$expose_docker_tcp=2375

i. Run vagrant up . For first time ever if you run the command an image of the coreos will be downloaded which takes some time. From next time the same image is used to create vm and the creation time of vm reduces drastically. While the vm is created you can see logs like

==> core-01: Importing base box 'coreos-alpha'...
==> core-01: Configuring Ignition Config Drive
==> core-01: Matching MAC address for NAT networking...
==> core-01: Checking if box 'coreos-alpha' is up to date...
==> core-01: Setting the name of the VM: coreos-vagrant_core-01_1536592083135_11402
==> core-01: Clearing any previously set network interfaces...
==> core-01: Preparing network interfaces based on configuration...
core-01: Adapter 1: nat
core-01: Adapter 2: hostonly
==> core-01: Forwarding ports...
core-01: 2375 (guest) => 2375 (host) (adapter 1)
core-01: 22 (guest) => 2222 (host) (adapter 1)
==> core-01: Running 'pre-boot' VM customizations...
==> core-01: Booting VM...
==> core-01: Waiting for machine to boot. This may take a few minutes...
core-01: SSH address: 127.0.0.1:2222
core-01: SSH username: core
core-01: SSH auth method: private key
core-01: Warning: Connection reset. Retrying...
==> core-01: Machine booted and ready!
==> core-01: Setting hostname...
==> core-01: Configuring and enabling network interfaces...
==> core-01: Running provisioner: file...
==> core-01: Running provisioner: shell...
core-01: Running: inline script
==> core-01: Running provisioner: shell...
core-01: Running: inline script

j. Now time to ssh in to your vagrant and create a small image. Do vagrant ssh to ssh in your box

k. Run docker run -it hello-world

l. Now in any of your osx terminal, export the DOCKER_HOST and check results using curl command like below

>>   export DOCKER_HOST=tcp://localhost:2375
>> curl localhost:2375/images/json | jq .
[
{
"Containers": -1,
"Created": 1536348339,
"Id": "sha256:4ab4c602aa5eed5528a6620ff18a1dc4faef0e1ab3a5eddeddb410714478c67f",
"Labels": null,
"ParentId": "",
"RepoDigests": [
"hello-world@sha256:0add3ace90ecb4adbf7777e9aacf18357296e799f81cabc9fde470971e499788"
],
"RepoTags": [
"hello-world:latest"
],
"SharedSize": -1,
"Size": 1840,
"VirtualSize": 1840
}
]

m. All the docker commands from your osx terminal now refer to the docker server from virtualbox.

n. To troubleshoot you can always check the logs using journalctl — identifier=ignition or journalctl -xe .

o. Finally you can use vagrant halt to stop your vm or vagrant destroy -f to remove the vm. If you are removing the vm , all containers inside the vm will be removed. You can also use vagrant snapshot save and vagrant snapshot restore commands to save and restore your snapshots. For more details use vagrant list-commands .

Additionally I needed tmux in the coreos . To install the same I use the below script

Ref: