Developing a dockerized web app on Windows Subsystem for Linux (WSL)
It’s been a long time since I last used Windows for development. I mostly work with open source languages and tools, so when I started my professional career, around 10 years ago, I switched to Linux, after briefly using Windows for web development. Five years later, I got myself a Macbook Air and used that, along with my desktop PC which was still running Linux. Last summer, Microsoft released the Anniversary Update for Windows 10 and I was surprised that it also included the Ubuntu userspace (also known as Windows Subsystem for Linux). It’s also quite easy to install.
Last month, I decided to upgrade to a new laptop, and I switched back to PC. I was going to install Linux on it, anyway, but since it came with Windows 10 as well, I thought I would give WSL a try. I didn’t expect much, but I was pleasantly surprised. I could run Git, Vim and Ruby on Windows just as well as on Linux. That said, the Windows command line is terrible compared to any Linux terminal. I quickly switched to wsltty. It still doesn’t provide all the shortcuts and features of, say gnome-terminal, but it does come close. But I digress, this post isn’t about the Windows console, but WSL and Docker.
For the past few months, I have used Docker for all the projects I’m working on. It’s very easy to set up a development environment, regardless of the host operating system. Since my new laptop came with Windows 10 Pro, I could install Docker for Windows. If you’re fine with cmd.exe or Powershell, that’s all you need. However, I really wanted to use WSL, since I find it much easier to use zsh or bash. Git and Vim are available as Windows binaries, but I think it’s simpler to just install them via apt-get and use them from WSL. Not to mention, you don’t have to mess with (CRLF/LF) line endings this way.
While the WSL is pretty close to production-ready, it’s still in beta. Running simple command line programs just works, but setting up your whole development workflow is full of little incompatibilities and annoyances. You can probably search for each one of them and find a solution, but setting up everything took me a few days, so I thought it was a good idea to document the whole process.
First, you need to install WSL which is pretty easy, and adds a bash shortcut into your start menu. When you open it, you are greeted with the default bash prompt. Next, install Git and Vim, with apt-get:
sudo apt-get update
sudo apt-get install vim git
Installing Docker, takes a bit more work. There’s a guide for Ubuntu 14.04 on Docker’s website, which is the Ubuntu version that comes with WSL. I’ve summarized the relevant steps below:
sudo apt-get install apt-transport-https ca-certificates
sudo apt-key adv \
--keyserver hkp://ha.pool.sks-keyservers.net:80 \
echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" | sudo tee /etc/apt/sources.list.d/docker.list
sudo apt-get update
sudo apt-get install docker-engine
Now, Docker engine runs natively on Windows, so you only need the
docker client. You don’t have to install any kernel packages and you can simply skip the
service docker start command.
By this point, one would think that docker is ready to run, right? Well, it’s not that simple. Try running docker and this is what you get:
$ docker info
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
With the Docker daemon running on the host OS (Windows), the Linux client can’t connect to the default Unix socket (it does not exist). The solution is pretty easy though, as documented in this useful reddit post. Just add a variable to your
$ docker info
Docker running fine! Now, just a
git clone and we’re up and running! Not so fast…
The project I’m using for this example, mounts the code directory in the container. It also uses Docker compose to set up other services. Follow this guide to install compose before continuing.
This is the service definition in the
command: bin/rails s -b '0.0.0.0'
Pretty standard for a Rails app. Running
docker-compose build builds the images as defined in the
docker-compose.yml file. What happens if you want to launch the rails console? Let’s see:
$ docker-compose run --rm web rails c
/opt/rails_app/Gemfile not found
That’s unexpected! Digging further, we can see that the
/opt/rails_app is completely empty, which means the volume is not mounted at all. Here’s the proof:
$ docker-compose run --rm web /bin/bash
root@0c6732a3742e:/opt/rails_app# ls -l
What’s wrong? Well, 2 things:
- In order to mount a host directory into a container, it must be shared via Windows file sharing. You can do that from Docker’s settings (see below).
- Docker for Windows expects Windows paths. If you simply
git cloneyour project inside WSL, your working directory will be something like
/home/my-user/my-project. It should be something like
The first issue is easy to resolve. Right-click the Docker icon and select “Settings”. Then, simply enable sharing for drive C and follow the prompts. Here’s a screenshot:
The other issue is harder. WSL files are stored somewhere in your home directory (the path is
C:\Users\win-user\AppData\Local\lxss). You could technically find a way to translate the WSL path into a Windows path. Then write a wrapper script that calls
docker with the Windows path instead. If you use
docker-compose, however, you need to edit your
docker-compose.yml file. Well, that kind of defeats the purpose, doesn’t it? We want to use Docker to set up our development workstation regardless of its operating system.
After googling around, I found a satisfactory solution. If you look carefully at the screenshot above, you can see that you can use a forward slash for Windows paths with Docker. However, this doesn’t solve our problem, since Linux has no concept of drive letters, and paths start with a forward slash. In fact, WSL auto-mounts Windows drives under
/mnt. For example, your
C:\Users\win-user\Code\my-project path is accessible on WSL as
/mnt/c/Users/win-user/Code/my-project. We’re getting closer! Well, it appears Docker can translate a path such as
C:\Code\my-project. That’s pretty good! The only thing remaining is to have our
C: drive appear on WSL as
/c. How do we do this? Simply, by symlinking
sudo ln -s /mnt/c /c
Note: If you are running the latest April 2018 update there is a better way to do this. Scroll to the bottom of the article to read more.
And we’re done! The rest is the usual stuff:
git clone git://..whatever..
Well, there’s one more thing. Using the dot to denote the current working directory won’t work… You have to use
$PWD, so edit the relevant
docker-compose.yml line, and you’re truly done:
Update: Compatibility with Windows (PowerShell and cmd.exe)
While the solution above works on Linux, MacOS and WSL, it is not compatible with the native Windows terminals (PowerShell and cmd.exe). The reason is simple. There is no
PWD environment variable. Both shells support environment variables, but neither sets
PWD to the current working directory. If you need to keep your
docker-compose.yml compatible with these shells, there’s an easy fix. You just have to provide a fallback value when the
PWD variable is not set:
This was added to Docker Compose v1.9 and needs version 2.1 of the configuration file format. That’s why I have also included the first line in the code snippet above. Also, keep in mind this is supported on Docker engine 1.12 and above.
Is it worth it?
Well, for me, yes. If you have to use Windows, you can keep your Linux, terminal-heavy workflow by using WSL. If you don’t use Docker, it’s much easier. But using Docker on WSL is feasible. There is only one minor issue with the solution above. Git is slower when the repo is located in a Windows-accessible path (
/mnt/x/..). My prompt calls git to retrieve the current branch and other info and this results in a half-second delay before the prompt appears. That’s not a deal-breaker, however, and I can be just as productive on Windows, as I am on Linux.
That’s it! Hope you liked my guide. Do you use WSL and Docker? I’d love to hear your experience.
November 2017 — Fall Creators Update
It’s been many months since I’ve written this tutorial. During that time, Windows 10 has been updated twice, and with it, WSL has been updated and is now out of beta! What has changed? Not much. Here’s a list of the most relevant changes:
- File I/O is noticeably faster now so git operations are quick enough for large repos.
- The bundled Ubuntu version is now 16.04 and it’s distributed through the Windows Store. This means that the instructions to install Docker are a bit different.
You don’t have to do anything with your existing installation, but you’re encouraged to re-install WSL through the Store. The old WSL installation is deprecated and updates are only available through the Store.
April 2018 update —Configuration with /etc/wsl.conf
The latest update came out a bit later than expected, but has an exciting new feature. You can now configure WSL, using the new /etc/wsl.conf file. Among the things you can configure, is the directory where all fixed drives are mounted. This renders the previous symlink solution obsolete, so remove any symlinks, if you still have them. You have to create the file and add this content:
root = /
options = "metadata,umask=22,fmask=11"
You need to restart Windows (or log out/log in again — haven’t tried) for this to take effect. All your fixed drives will appear directly in the root directory now. The
options line enables Linux metadata under WSL, which means you can change the owner of files and directories (used to default to root before) and their permissions (the read-write-execute bits, which used to default to 0777). The WSL team has a more detailed post about this.
Another exciting addition is support for Unix sockets. This enables another approach to connecting the Linux Docker client to the Docker for Windows server, which is explained in detail in this post.