Developing a dockerized web app on Windows Subsystem for Linux (WSL)

It’s been a long time since I last used Windows for development. I mostly work with open source languages and tools, so when I started my professional career, around 10 years ago, I switched to Linux, after briefly using Windows for web development. Five years later, I got myself a Macbook Air and used that, along with my desktop PC which was still running Linux. Last summer, Microsoft released the Anniversary Update for Windows 10 and I was surprised that it also included the Ubuntu userspace (also known as Windows Subsystem for Linux). It’s also quite easy to install.

Last month, I decided to upgrade to a new laptop, and I switched back to PC. I was going to install Linux on it, anyway, but since it came with Windows 10 as well, I thought I would give WSL a try. I didn’t expect much, but I was pleasantly surprised. I could run Git, Vim and Ruby on Windows just as well as on Linux. That said, the Windows command line is terrible compared to any Linux terminal. I quickly switched to wsltty. It still doesn’t provide all the shortcuts and features of, say gnome-terminal, but it does come close. But I digress, this post isn’t about the Windows console, but WSL and Docker.

For the past few months, I have used Docker for all the projects I’m working on. It’s very easy to set up a development environment, regardless of the host operating system. Since my new laptop came with Windows 10 Pro, I could install Docker for Windows. If you’re fine with cmd.exe or Powershell, that’s all you need. However, I really wanted to use WSL, since I find it much easier to use zsh or bash. Git and Vim are available as Windows binaries, but I think it’s simpler to just install them via apt-get and use them from WSL. Not to mention, you don’t have to mess with (CRLF/LF) line endings this way.

While the WSL is pretty close to production-ready, it’s still in beta. Running simple command line programs just works, but setting up your whole development workflow is full of little incompatibilities and annoyances. You can probably search for each one of them and find a solution, but setting up everything took me a few days, so I thought it was a good idea to document the whole process.

The guide

First, you need to install WSL which is pretty easy, and adds a bash shortcut into your start menu. When you open it, you are greeted with the default bash prompt. Next, install Git and Vim, with apt-get:

sudo apt-get update
sudo apt-get install vim git

Installing Docker, takes a bit more work. There’s a guide for Ubuntu 14.04 on Docker’s website, which is the Ubuntu version that comes with WSL. I’ve summarized the relevant steps below:

sudo apt-get install apt-transport-https ca-certificates
sudo apt-key adv \
    --keyserver hkp://ha.pool.sks-keyservers.net:80 \
    --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" | sudo tee /etc/apt/sources.list.d/docker.list
sudo apt-get update
sudo apt-get install docker-engine

Now, Docker engine runs natively on Windows, so you only need the docker client. You don’t have to install any kernel packages and you can simply skip the service docker start command.

By this point, one would think that docker is ready to run, right? Well, it’s not that simple. Try running docker and this is what you get:

$ docker info
Cannot connect to the Docker daemon. Is the docker daemon running on this host?

Problem #1

With the Docker daemon running on the host OS (Windows), the Linux client can’t connect to the default Unix socket (it does not exist). The solution is pretty easy though, as documented in this useful reddit post. Just add a variable to your .bashrc:

export DOCKER_HOST=tcp://127.0.0.1:2375

And voila:

$ docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
...more...

Docker running fine! Now, just a git clone and we’re up and running! Not so fast…

Problem #2

The project I’m using for this example, mounts the code directory in the container. It also uses Docker compose to set up other services. Follow this guide to install compose before continuing.

This is the service definition in the docker-compose.yml file:

  web:
    depends_on:
      - postgres
    env_file: .env
    build: .
    command: bin/rails s -b '0.0.0.0'
    ports:
      - '3000:3000'
    volumes:
      - .:/opt/rails_app
      - bundler_cache:/usr/local/bundle

Pretty standard for a Rails app. Running docker-compose build builds the images as defined in the docker-compose.yml file. What happens if you want to launch the rails console? Let’s see:

$ docker-compose run --rm web rails c
/opt/rails_app/Gemfile not found

That’s unexpected! Digging further, we can see that the /opt/rails_app is completely empty, which means the volume is not mounted at all. Here’s the proof:

$ docker-compose run --rm web /bin/bash
root@0c6732a3742e:/opt/rails_app# ls -l
total 0

What’s wrong? Well, 2 things:

• In order to mount a host directory into a container, it must be shared via Windows file sharing. You can do that from Docker’s settings (see below).
• Docker for Windows expects Windows paths. If you simply git clone your project inside WSL, your working directory will be something like /home/my-user/my-project. It should be something like C:\Users\my-user\my-project.

The first issue is easy to resolve. Right-click the Docker icon and select “Settings”. Then, simply enable sharing for drive C and follow the prompts. Here’s a screenshot:

Enable sharing. Windows may prompt for your password.

The other issue is harder. WSL files are stored somewhere in your home directory (the path is C:\Users\win-user\AppData\Local\lxss). You could technically find a way to translate the WSL path into a Windows path. Then write a wrapper script that calls docker with the Windows path instead. If you use docker-compose, however, you need to edit your docker-compose.yml file. Well, that kind of defeats the purpose, doesn’t it? We want to use Docker to set up our development workstation regardless of its operating system.

After googling around, I found a satisfactory solution. If you look carefully at the screenshot above, you can see that you can use a forward slash for Windows paths with Docker. However, this doesn’t solve our problem, since Linux has no concept of drive letters, and paths start with a forward slash. In fact, WSL auto-mounts Windows drives under /mnt. For example, your C:\Users\win-user\Code\my-project path is accessible on WSL as /mnt/c/Users/win-user/Code/my-project. We’re getting closer! Well, it appears Docker can translate a path such as /c/Code/my-project to C:\Code\my-project. That’s pretty good! The only thing remaining is to have our C: drive appear on WSL as /c. How do we do this? Simply, by symlinking /mnt/c as /c:

sudo ln -s /mnt/c /c

Note: If you are running the latest April 2018 update there is a better way to do this. Scroll to the bottom of the article to read more.

And we’re done! The rest is the usual stuff:

cd /c/Users/win-user/Code
git clone git://..whatever..
docker-compose up

Well, there’s one more thing. Using the dot to denote the current working directory won’t work… You have to use $PWD, so edit the relevant docker-compose.yml line, and you’re truly done:

  web:
    volumes:
      - $PWD:/opt/rails_app

Update: Compatibility with Windows (PowerShell and cmd.exe)

While the solution above works on Linux, MacOS and WSL, it is not compatible with the native Windows terminals (PowerShell and cmd.exe). The reason is simple. There is no PWD environment variable. Both shells support environment variables, but neither sets PWD to the current working directory. If you need to keep your docker-compose.yml compatible with these shells, there’s an easy fix. You just have to provide a fallback value when the PWD variable is not set:

version: '2.1'
services:
  web:
    volumes:
      - ${PWD-.}:/opt/rails_app

This was added to Docker Compose v1.9 and needs version 2.1 of the configuration file format. That’s why I have also included the first line in the code snippet above. Also, keep in mind this is supported on Docker engine 1.12 and above.

Is it worth it?

Well, for me, yes. If you have to use Windows, you can keep your Linux, terminal-heavy workflow by using WSL. If you don’t use Docker, it’s much easier. But using Docker on WSL is feasible. There is only one minor issue with the solution above. Git is slower when the repo is located in a Windows-accessible path (/mnt/x/..). My prompt calls git to retrieve the current branch and other info and this results in a half-second delay before the prompt appears. That’s not a deal-breaker, however, and I can be just as productive on Windows, as I am on Linux.

git delay on WSL path vs on Windows path

That’s it! Hope you liked my guide. Do you use WSL and Docker? I’d love to hear your experience.

November 2017 — Fall Creators Update

It’s been many months since I’ve written this tutorial. During that time, Windows 10 has been updated twice, and with it, WSL has been updated and is now out of beta! What has changed? Not much. Here’s a list of the most relevant changes:

• File I/O is noticeably faster now so git operations are quick enough for large repos.
• The bundled Ubuntu version is now 16.04 and it’s distributed through the Windows Store. This means that the instructions to install Docker are a bit different.

You don’t have to do anything with your existing installation, but you’re encouraged to re-install WSL through the Store. The old WSL installation is deprecated and updates are only available through the Store.

April 2018 update —Configuration with /etc/wsl.conf

The latest update came out a bit later than expected, but has an exciting new feature. You can now configure WSL, using the new /etc/wsl.conf file. Among the things you can configure, is the directory where all fixed drives are mounted. This renders the previous symlink solution obsolete, so remove any symlinks, if you still have them. You have to create the file and add this content:

[automount]
root = /
options = "metadata,umask=22,fmask=11"

You need to restart Windows (or log out/log in again — haven’t tried) for this to take effect. All your fixed drives will appear directly in the root directory now. The options line enables Linux metadata under WSL, which means you can change the owner of files and directories (used to default to root before) and their permissions (the read-write-execute bits, which used to default to 0777). The WSL team has a more detailed post about this.

Another exciting addition is support for Unix sockets. This enables another approach to connecting the Linux Docker client to the Docker for Windows server, which is explained in detail in this post.