When We Brought Linux to Windows
My whole career at Microsoft has included creating tools for Linux developers. I joined Microsoft in August 2016 after graduating from the University of Virginia, having studied computer science and entrepreneurship. Most of my programming experience at the university was with C++, and the operating system I used almost exclusively was Linux.
My experience may not seem like an obvious fit for Microsoft, but at the time, Microsoft was undergoing a total shift in both engineering and culture to a place where all operating systems were important, including Linux.
My first job at Microsoft was on the SQL Server on Linux team — they specifically asked me to join them to bring my Linux experience to the team. It blew me away to hear that I could provide value to the team based on my experience, even though I was a new grad.
The idea that Microsoft would make a version of SQL Server on Linux had been an April Fools Day joke a few years before, but by 2016 it was real. I joined the team shortly after they had shipped the first version and I focused on improving the tooling for SQL Server — specifically for admins used to managing Linux servers and applications. Running SQL Server on Linux required making command line tools designed for the way that Linux ops normally work.
I also had a chance to design the first version of GUI tools for SQL Server for Linux, starting with a fork of Visual Studio Code that today is called Azure Data Studio, an Electron-based app that can work for all flavors of SQL Server, regardless of the operating system.
During my first year at Microsoft, working on SQL Server on Linux, I learned a lot, including an appreciation for program management, which combines technology with business-thinking.
In August of 2017, I joined the Windows Subsystem for Linux team as Program Manager. I had first heard about WSL when it was announced at the Microsoft Build event in 2016 as the “Bash on Ubuntu on Windows”. The video of the announcement on Channel9 went viral when first released, overwhelming many of the other stories coming from Build. WSL had a brief mention at Build by Kevin Gallo, maybe two minutes of the entire keynote, but the audience went wild and the press even more so. At one point the Channel9 team were afraid that the huge demand for the WSL video was actually a DDoS attack! Microsoft running Bash on Ubuntu inside Windows was an instant hit.
The team working on the Windows Console was the first one to identify the customer need for WSL. As they started doing customer development, they heard over and over again that people wanted something like Bash on Linux. Eventually, the team came to realize: why make something like Bash when you can just make Bash itself run on Windows?
Not that it was easy to do. Creating the Windows Subsystem for Linux was a combination of deep knowledge of Windows from a kernel team, along with a Microsoft Research technology called picoprocess. Funny enough, picoprocesses is also the technology that makes SQL Server on Linux possible! The picoprocess would host an unmodified user-mode implementation of Linux, and then the kernel team would implement shims to connect the Linux syscalls through to Windows. In other terms, WSL makes it possible to run binaries compiled for Linux on top of the Windows NT kernel, without re-compiling applications or using virtual machines.
What we didn’t write was a distribution of Linux — there are plenty of those around. Ubuntu was the first flavor of Linux available in WSL: we reached out to the folks at Canonical to see if they would be willing to help make WSL come true. They were enthusiastic about the idea, resulting in Ubuntu being available from the Windows Store. Which is a pretty funny sentence all by itself: there are flavors of Linux (at last count, six of them) available from the Windows Store. How many app stores have other operating systems in them?
The code we did write were the Linux-compatible kernel syscalls, interfacing Linux processes with the underlying Windows kernel. There are around 340 syscalls in Linux. The question was which of the syscalls to implement first: like all operating systems, new syscalls get added with new versions of the OS, but old calls never get removed to maintain backwards compatibility. There was an initial wave of syscall implementation and everything else was wrapped in “not implemented yet” events so that the WSL team could start to understand what syscalls people needed.
Answering the question of what syscalls to implement first meant connecting with people that would use it, and that was what the announcement and videos from Build were all about — getting people using WSL and providing feedback. The place to get WSL was Windows itself via the Windows Insider Program: anyone can be a Windows Insider and get the new WSL bits. You would think that it was just Windows enthusiasts in the Insiders program, but these days there are more than ten million subscribers, and they are interested in all sorts of things such as gaming, Bluetooth innovations, and WSL.
One of the groups interested in having Bash running in Windows were web developers building web apps that run on Linux servers. The entire build pipeline of their applications would often be a series of Bash commands. Also, if you are searching for help on building a web app, say on Stack Overflow, most of the sample code you find is going to run on Linux — and if you’re working on a Windows machine for development, that can be frustrating. Often times, the simplest solution for a web developer was to move to a Mac and macOS where that code just runs.
Within the first couple of weeks of WSL being in Windows, one enterprising WSL user managed to get XEyes as a GUI application running on WSL and X11. All XEyes does is draw a pair of cartoon eyes on the screen that follow the mouse pointer around, yet all social medias lit up when that demo worked!
There was a lot of discussion around how we wanted to collect feedback. The traditional mechanism is UserVoice, and there was a UserVoice site set up for WSL that has hundreds of ideas submitted and thousands of votes. The real question came with whether to use GitHub. Since one of the first audiences for WSL was web developers, GitHub made a lot of sense. But WSL was not an open source project — and putting a non-open-source project on GitHub seemed odd. We decided to meet the developers where they were and create a forum for issues, feedback, and discussions on GitHub. Since then, we have received thousands of issues over a range of topics related to Linux on Windows.
Thousands of people have contributed issues to the WSL GitHub repo and the WSL team reviews every one of those issues, discusses it through the issue comment system, and then decides what to do. If there is new code to write to implement a feature or fix an issue it gets written and added to the WSL project, which is included in the overall Windows build that gets distributed via the Windows Insider Program. The cycle time can be as quick as just a couple of weeks.
In the end, it is the rapid cycle time of people using WSL being able to discuss a feature they wish for or problem they have with the team via UserVoice or GitHub that then appears in the Insiders builds that makes all the difference — the community is part of the creation process for WSL.
When I came onboard the WSL team as program manager, I focused on getting WSL beyond beta. People’s complaints were around compatibility and performance. But from my point of view, if that’s what users are concerned with, then they are using your product seriously; you only care about performance when you’re running something big. So, while we still had work to do, it was for a good reason: people wanted to do more with WSL, and faster.
As the capabilities of WSL started to expand, we also took WSL to where the developers were, and not just those who have traditionally been in the Microsoft ecosystem. Going to events like PyCon and OSCON were amazing experiences: the developers attending those shows were surprised to see Microsoft folks there. And when we told them that they were running Linux on Microsoft developer tools, they were skeptical. I demoed SQL Server, WSL, and Visual Studio Code at these events.
I met their skepticism by letting them try it out for themselves. And when these developers started running their own commands, small scripts, and code snippets I always got a huge reaction: “Hold on, this is actually Linux. How did you do this? How did I not know about this?” And they often came to the same conclusion, that we had built something relevant to them, and that it is pretty cool.
We have even taken on the complaints about compatibility and performance with WSL and tackled that in a new architecture — WSL 2. It provides full compatibility through shipping a Linux kernel in Windows and 20x performance. Creating the foundation for WSL 2 and watching it go live at Build last May was a great experience. Today, WSL is out of beta and up to version 2. You can learn more on the announcement blog.
I also worked with other teams at Microsoft to enable WSL to be used with their products too: a big example of this was Visual Studio Code, which is the most popular text editor for JavaScript and Node.js. My exposure to Visual Studio Code began when we realized developers using it could benefit a lot from WSL. At first it wasn’t a huge amount of code, but the main goal there was to make it easier to debug Node.js code running in WSL. Developers could code on a Windows computer running WSL to target a Linux version of Node.js and debug it natively from Linux.
And when we got features like that working for Node.js, it led to an abundance of requests for similar support for C++, Python, and other languages. I was fascinated by this integration and found my passion there which led to my new role in creating tools for Linux developers. I’m now focusing on the Visual Studio Code experience for C++ developers with, of course, a focus on Linux. It was great to see Visual Studio Code Remote Development go live at PyCon this year where we released the WSL extension and my new team’s C++ extension support.
Despite my short time at Microsoft, I am thrilled by the breadth of tools I have helped build for Linux developers — from databases to operating systems to IDEs. I am excited to continue spreading the love for Linux and creating tools that awe developers around the world.
