WSL 2 + Docker Edge- Tech Preview: Native Linux containers w/o emulation
Windows Subsystem for Linux gets new features and enhanced performance (Updated: 6/15/2020)
6/15/2020 Update: WSL 2 is no longer just for insiders, you can install WSL2 today with Windows 10, Version 2004, Build 19041.
3/13/2020 Update: Microsoft has announced that WSL 2 will be generally available in Windows 10, version 2004. Existing users on the Insider slow ring will be prompted to update their kernel, see this post for more information.
Introduction
It used to be complicated to run Linux applications in Windows, in the old days you were limited to tools like cygwin that provided limited functionality and could run some popular Linux utilities… later came virtualization tools like VirtualBox, VMWare Workstation and Microsoft Hyper-V which could emulate entire x86 systems, these tools allowed users to install different distributions of Linux, FreeBSD, and older versions of Windows.
Docker was released in March 2013, and introduced containers:
A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another. A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries, and settings.
They differ from virtual machines because they don’t virtualize hardware, they just use user and kernel space of the underlying operating system.
Docker has become incredibly popular because it is very useful for developing and testing new applications, especially when you are deploying those applications to cloud services like Amazon Web Services.
WSL (Windows Subsystem for Linux) has been available since Windows 10 1703. It is a Linux kernel compatibility layer for Windows. It allowed many Linux programs to run inside Windows by giving users access to Linux bash command-line applications. WSL allowed Windows users to install a full Linux distro like Ubuntu, openSUSE, Kali or Debian albeit with limitations… notably, there is no GPU support, so it can’t do GUI’s (e.g. Gnome, KDE, etc.) WSL 1 actually has no Linux kernel, so you can’t test drivers or Linux kernel extensions… you can’t run 32bit Linux applications… another issue was that Docker daemon could not run directly on WSL, instead, you had use workarounds to connect to a remote Docker daemon running through Docker for Windows.
WSL 2 was announced May 2019 and has been included in Windows Insider builds since version 18917. It is due for official release in the first half of 2020, but the preview versions are being rolled out on Insider builds which are available now. WSL 2 overcomes many issues by introducing a full Linux kernel built by Microsoft:
WSL 2 is a new version of the architecture that powers the Windows Subsystem for Linux to run ELF64 Linux binaries on Windows. This new architecture changes how these Linux binaries interact with Windows and your computer’s hardware, but still provides the same user experience as in WSL 1 (the current widely available version). Individual Linux distros can be run either as a WSL 1 distro, or as a WSL 2 distro, can be upgraded or downgraded at any time, and you can run WSL 1 and WSL 2 distros side by side. WSL 2 uses an entirely new architecture that uses a real Linux kernel.
WSL 2 dramatically improves Windows native Linux capabilities… and Microsoft report incredible performance improvements, with initial tests showing WSL 2 to be up to 20x faster than WSL 1. Over the last few months Microsoft added new features and with Windows 10 Insider Preview build 19018 or higher, you can finally install a version of Docker (Docker Desktop Edge 2.1.6.0 or a later) that allows native Linux containers and easily integrates into your chosen Linux distro.
This article will talk about some of the features, and will help explain how to get up and running with WSL 2, Windows Insider Preview and Docker Desktop Edge… we’ll also touch upon the new Windows Terminal and VS Code integration.
I’ve been using WSL 2 + Docker Edge for a few weeks without any serious issues. Installation had a few gotchas, notably when migrating from WSL 1>WSL 2, we’ll go over these here and hopefully the experience will improve before the proper release in 2020.
.. as of December 30th, 2019, all of this software is considered a technical preview, but if you need a Windows-based Linux development platform, you can get your hands on all of these features today.
Why WSL? / Who is this for?
WSL was designed for developers, especially web developers and people who work with open source projects. There are many Linux-first applications that are easy to use on Linux and Apple Macs because those systems use the Linux kernel and the Bash shell. WSL allows you to run Linux command-line tools.
WSL uses far less computational resources than a full virtual machine and you can access your Windows files… so effectively you can use Windows apps alongside Linux apps on the same workstation. WSL 2 allows you to use Docker natively.
My personal use case is that I had become familiar with using Bash at work on my Macbook Pro, but my desktop machine at home is a PC and I wanted to have a common developer experience. It is not nice to switch contexts and contend with the differences between commands, file systems, etc.
Docker is an essential tool for my development projects… using WSL 2 with Docker Edge, Windows Terminal and VS Code has given me a superior developer experience because I can also benefit from Windows applications.
WSL 2 is very exciting for machine learning development. There is currently no GPU support for WSL 2 but with the official Linux kernel and Docker support, it is now possible for Microsoft to add that capability in the future.
This is especially good for Windows because many Nvidia CUDA based machine learning applications run exclusively on bare metal x86 Linux systems, Mac still doesn’t have good GPU support, if Windows finally introduces GPU support for WSL, it could become a very powerful machine learning development platform.
Bash? / PowerShell? / Command Prompt?
If you’ve been reading about WSL, and are used to working with a single operating system, the terms ‘Bash’, ‘PowerShell’ and ‘Command Prompt’ may be confusing. They’re all different tools for issuing system commands, via a command-line interface (or CLI).
Before Windows, the most popular IBM PC compatible operating system was DOS, it had no graphical interface and relied on text commands, the familiar DOS command prompt was brought over to Windows and is still used today.
PowerShell was introduced with Windows 7, it is actually quite different from the command prompt. It uses a different type of command called cmdlets. It is used for many different system administration tasks, it can be used to update the Windows Registry and Windows Management Instrumentation. PowerShell is the default command system for Windows 10.
Linux and other Unix-like systems (like Apple Mac) use Bash. Bash is the shell, or command language interpreter, for the GNU operating system, it is favored by many open source projects, and often when reading documentation on a site like Github, command instructions will often be written in Bash.
Installation Prerequisites
WSL 2 is only available in Windows 10 builds 18917 or higher… as of 12/30/2019, these are only available via the Insider program. Today’s official OS build is 18363.535. You can find your OS build under ‘About your PC’ in system settings.
Docker Desktop Edge 2.1.6.0 + requires Windows 10 Pro or Enterprise Insider Preview build 19018 or higher to integrate with your WSL Linux distro. Windows Insider version 19041.1 is currently available on the slow ring, you don’t need a fast ring build to get WSL 2.
3/5/2020: Docker Desktop Edge 2.2.2.0 Windows 10 Home users can now use Docker Desktop through the experimental WSL 2 support.
Virtualization must be enabled in the computer bios, instructions for this vary between manufacturers but it’s usually a simple on/off listing in the BIOS.
You need to enable the WSL feature on Windows. For official WSL 2 instructions, refer to the Microsoft documentation. I have shared my installation notes below where I install WSL 1 first, and then upgrade Windows and migrate my distro from WSL 1 to WSL 2.
Installation Notes
I am providing notes from a clean installation, starting from Windows 10 18363.535. I’m going to start by installing WSL 1, installing an Ubuntu distribution. Then I’m going to install the Windows Insider slow build and upgrade my WSL 1 Ubuntu distro to WSL 2, finally, I will install Docker Desktop Edge and integrate that with the WSL 2 distro.
- To install WSL 1 you need to add an additional Windows optional feature, to do that open PowerShell as an administrator:
2. Enter the following command, hit return and restart when prompted:
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux
3. Once restarted, you can check the feature is installed via control panel> Turn Windows features on or off:
4. Next, you need to go to the Windows Store via the start menu and pick your Linux distro. We’re going to use Ubuntu, click Ubuntu then ‘Install’:
5. Once installed, the ‘Install’ button will change to ‘Launch’, click the ‘Launch’ button. You should be presented with an Ubuntu setup window that will ask for you some information like your UNIX username and password, follow onscreen prompts:
6. Now you have WSL1 + Ubuntu, and will be presented with the bash prompt, you can open this prompt by opening ‘Ubuntu’ in the start menu:
7. [Optional] I recommend taking a look at the new Windows Terminal, it has some amazing features and is very easy to open multiple panes with different prompts for bash, command prompt and PowerShell, you can install the preview via the Windows Store:
You can reach the Ubuntu prompt in Windows Terminal via the drop-down in the title bar:
8. That’s all you need to use WSL 1 + Ubuntu with Windows 10 Build 18363.535… a common question is how you access your hard drive:
Mount points for hard drives on the local machine are automatically created and provide easy access to the Windows filesystem.
/mnt/<drive letter>/
Example usage would be
cd /mnt/cto access c:\
So unfortunately Docker needs some workarounds to get it working with WSL 1… this article is about WSL 2 + Docker Edge, so we need to get on the Insider slow ring, as of today (Dec 30th 2019) that will install build 19041.1.
Special note/Disclaimer: Insider builds are being actively developed so they are not recommended for production or mission critical workstations. The slow ring has more stable builds than the fast ring but they are still a technical preview.
9. To install Windows Insider builds you need to register as a Windows Insider. Instructions can be found here. You have to open Windows Updates, link your account, pick slow or fast (I recommend slow, which is sufficient for WSL 2) then restart your machine to get the install rolling, you may also have to reopen the Windows Update to fix your diagnostic preferences:
10. You may have to trigger the Windows Update to get the build, note this can take a while to download and install:
11. Your machine will restart a few times during installation whilst it is ‘Working on updates’, once it’s up, you need to do a couple of things to enable WSL 2, you can see the official instructions here.
You need to open PowerShell as an admin again, and enter 2 commands to enable optional Windows features, this time it’s ‘Microsoft Windows Subsystem Linux’ and ‘Virtual Machine Platform’:
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestartdism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
12. Restart your machine.
13. the next step is to upgrade our distro to version 2, to see what version your distro is running, you need to open a command prompt and type the following command:
wsl -l -v
Note: my old Ubuntu distro is version 1, so we need to upgrade it to v 2.
14. To upgrade to v 2, open a command prompt as Administrator and run this command (replace <distro name> with the name from the -l -v command, in our case “Ubuntu”):
wsl --set-version <distro name> 2Here it will take a while to convert the distro-, some users have reported issues related to encrypted images on certain Windows setups, I have experienced it on one machine but not on the other. Virtual hard disk files must be uncompressed and deencrypted and must not be sparse. Fortunately, there are workarounds thanks to @squeaky-pl and @dmchurch on the official issue page.
If you get the error “The requested operation could not be completed due to virtual disk system limitation” you can do two steps to fix it, first enter these commands which will disable encryption and compression respectively:
fsutil behavior set disableencryption 1
fsutil behavior set disablecompression 1Restart your machine after issuing the commands.
You will then have to also go to the Ubuntu directory in your profile folder, you can go to:
%USERPROFILE%\AppData\Local\Packages\and look for a folder that starts with ‘CanonicalGroupLimited…’), right-click on “LocalState”, Properties, Advanced, deselect “Compress contents” (and “Encrypt contents” if it is checked). When it asks whether you want to apply to just this folder or to all sub-folders and files, you can say “just this folder”, because all you’re doing is clearing that “compress” flag.
You need to restart your machine again because the distro needs to start de-encrypted.
..after wsl --set-version from step 14 should work. Many thanks to @dmchurch for that fix, I needed to use it. Apparently, Windows has another issue where a past Windows Update enabled folder compression without asking and failed to decompress the image, that’s what causes this bug.
Once the conversion completes:
14. That’s WSL 2 installed!, if you don’t need Docker you can stop here, but I recommend giving it a whirl.
15. You can download Docker Desktop Edge here. Note this is a technical preview, but it has WSL 2 support and it is very easy to integrate with your WSL 2 Linux distro. Run ‘Docker Desktop Installer.exe’ to begin installation.
16. Once installed, go to the Docker dashboard and open settings via the cog icon in the navbar, or select ‘settings’ by right-clicking the docker icon in your Windows tray:
17. Check ‘Enable the experimental WSL 2 based engine’.
18. Go to ‘Resources’ in the settings sidebar and check your WSL 2 distro (Ubuntu):
19. That’s it!, you can now run docker commands from your Linux bash command-line in Windows. To learn more about what you can do with Docker containers check out the official site.
Gotchas with Gatsby/Vue/React webpack hot module reloading
There are some changes to the way the file system is virtualized in WSL 2. I didn’t realize it at first, but my WSL 1 projects were located on the Windows side of the file system, (under/mnt/c/Users/seand), and this has caused issues with features like hot module reloading in popular web frameworks like Vue, React and Gatsby. HMR didn’t work.
To alleviate the problem I had to move these projects into the Linux file system under under/home/sean , albeit frustrating, once I moved the projects, HMR started working again and the performance is much better, apparently there is less file emulation going on.
Pro tip: To reach your relative Windows folder from the WSL shell you can type explorer.exe . (don’t forget the dot).
Remarks on localhost : at the time of writing localhost doesn’t resolve to your WSL2 distro, you can however use the network IP address of your machine, if you’re having an issue accessing a server, try your network IP.
Docker Desktop Edge 2.2.2.0
Docker for WSL 2 now supports Kubernetes, offers VPN-friendly networking, provides an updated Docker daemon, and many new features.
Docker Desktop also leverages the dynamic memory allocation feature in WSL 2 to greatly improve the resource consumption. This means, Docker Desktop only uses the required amount of CPU and memory resources it needs, while enabling CPU and memory-intensive tasks such as building a container to run much faster.
Additionally, with WSL 2, the time required to start a Docker daemon after a cold start is significantly faster. It takes less than 2 seconds to start the Docker daemon when compared to tens of seconds in the current version of Docker Desktop.
Windows Terminal (Preview)
The Windows Terminal is a new, modern, fast, efficient, powerful, and productive terminal application for users of command-line tools and shells like Command Prompt, PowerShell, and WSL. Its main features include multiple tabs, Unicode and UTF-8 character support, a GPU accelerated text rendering engine, and custom themes, styles, and configurations.
VS Code Integration
Visual Studio Code can connect directly to your Linux subsystem using the Remote — WSL extension, type code . from your WSL 2 shell folder and VS code will open in Windows remotely connected to respective local Linux folder:
The Visual Studio Code Remote — WSL extension lets you use the Windows Subsystem for Linux (WSL) as your full-time development environment right from VS Code. You can develop in a Linux-based environment, use Linux-specific tool chains and utilities, and run and debug your Linux-based applications all from the comfort of Windows.
The extension runs commands and other extensions directly in WSL so you can edit files located in WSL or the mounted Windows filesystem (for example
/mnt/c) without worrying about pathing issues, binary compatibility, or other cross-OS challenges.
Conclusion + Hopes for the future
First impressions of WSL 2 and Docker Desktop are very good. This is the first time Windows has had a real Linux kernel. It would be great if they add GPU support so we can run Nvidia CUDA docker images. WSL 2 is really fast, on my i7 PC the performance is phenomenal and combined with remote VS Code and the new Windows Terminal, it is a wonderful development experience.
It’s never been easier to work with Bash and Linux applications in Windows, less workarounds, more official support. It’s great that Microsoft has fully embraced Linux and open source. I’m really looking forward to the full release in the first half of 2020.
