An introduction to WSL 2

At its recent Build conference Microsoft introduced the second version of its Windows Subsystem for Linux. It’s a major reworking of the original WSL concept, moving away from translating Linux system calls to Windows to shipping a complete Linux kernel that runs alongside Windows’ own kernel.

The reasons for doing this are many, but the main one is simple: It’s impossible for an emulator that ships twice a year to keep up with the changes in the Linux kernel, changes that Linux binaries depend on. If Windows is to support developers building Linux apps for the cloud, then it needs to be more than consistent, it needs to be compatible.

Introducing WSL 2

Two major developments have shaped WSL 2. The first is Microsoft’s focus on using its Hyper-V virtualization tool as an isolation layer in Windows, turning it into a general-purpose hypervisor service. The second is the closely related collaboration with Docker, Linux Containers on Windows, known as LCOW.

LCOW is where it all starts, using Hyper-V to host a thin Linux kernel to run Docker containers with Linux apps on Windows servers. A lot of work went into making LCOW a seamless environment that could be accessed by Windows development tools (especially Visual Studio Code’s new remote development feature), running containers in a way that lets them still perform as if they were native Windows code.

That near-native performance came from the changes Microsoft made to Hyper-V to support its new role as a key security feature in Windows 10. It’s Hyper-V that hosts the new isolated log-in screen process or runs a Windows kernel that only has the features needed by the browser in Windows Defender Application Guard. It’s also the layer underneath the recently introduced Windows Sandbox, throwaway Windows instances that can be used for debugging or examining suspicious code.

The big change to Hyper-V and how it hosts code in these new scenarios is integration with the Windows scheduler. Instead of one process for the hypervisor and all its hosted code, Windows can now manage all the processes hosted by the hypervisor platform, letting them run freely, like standard Windows applications.

With that as a foundation, it isn’t hard to see how LCOW became the foundation of a new approach to WSL, hosting a full Linux kernel maintained by Microsoft and supporting the same Linux distributions as the old WSL. A preview release is in the 20H1 series of Windows Insider builds, available through the Fast Ring channel.

WSL 2 now has its own file system. Using the 9P protocol, WSL’s virtual hard disk storage (which uses the Linux-native ext4 file system) can be shared with Windows (and Windows with Linux using CIFS). That allows you to use Visual Studio Code’s remote development and debugging features, with a Windows UI but with code running on Linux, as part of a Linux toolchain.

There are a lot of changes between the two releases of WSL. For developers the most noticeable is that WSL 2 now has its own root file system. All of your Linux files now need to be stored in their own root, giving them a lot faster file access. Usefully Windows gets direct access to the new Linux file system, thanks to Windows’ 9P support. You can launch tools like the Windows Explorer or Visual Studio Code from the Linux shell, giving direct access to Linux files.

Getting started with WSL 2

You don’t need a new WSL Linux distribution to start using WSL 2. First, enable the new Virtual Machine Platform feature that comes with the 20H1 Windows Insider releases, either from PowerShell or the Windows Features control panel. Once it’s in place, use the same tools to enable WSL. This sets up both WSL 1 and WSL 2 support.

From the Windows Store download your chosen WSL Linux distribution. Once you’ve set it up, unpacked the distribution, and added a user account, you can convert it to work with WSL 2. Microsoft has added a new set of wsl commands to Windows to help manage your distributions. Start with wsl -l -v to get a list of installed distributions and their WSL versions. To convert from WSL 1 to WSL 2 use the command wsl –set-version <distroname> 2. To switch back, use the same command, changing the version number to 1.

Launching a Linux distribution under WSL 2 is the same as launching one under WSL 1; type its name in the command line or launch it from the start menu. It’ll be interesting to see how Microsoft integrates WSL 2 with its new open source Windows Terminal, currently available as a beta from the Windows Store.

Changes to WSL networking

The shift to a virtualization-based approach to using Linux on Windows has changed how you access Linux-based networking apps. Under WSL 1 everything was part of the same network; with WSL 2 the Linux environment gets its own dynamic IP address. In the initial release this meant that you had to use the VM’s IP address to connect to services.

Microsoft has said it’s working to simplify this and has begun to address it in the latest update to the initial WSL 2 release. This adds support for localhost-based networking from Windows to WSL, simplifying access to container-hosted applications and other Linux services. However, it’s still only one way, so you won’t be able to use X-based applications without using scripts to dynamically set DISPLAY variables.

Bring your own Linux kernel

Perhaps the biggest change in the latest release is the ability to use your own Linux kernels as an alternative to Microsoft’s. A kernel option in the .wslconfig file is used to set a path to a kernel that will be loaded at startup. This will allow you to use your own custom kernel, giving you the option of using WSL to develop embedded system applications as well as more familiar cloud-hosted code. There’s a lot of flexibility here for development and for testing. Code can be developed using a long-term support kernel and tested using, for example, the kernel being used in your cloud provider’s Kubernetes environment.

There is one important issue with this new approach that can affect existing development environments. Hyper-V is a very low-level tool, one that sits below Windows. Unless any other virtualization tools you’re using support the new Hyper-V API they won’t run on a system that is hosting WSL 2. For users who are unable to switch to a hypervisor that works with Hyper-V, such as the latest VirtualBox releases or Google’s Android emulator, Microsoft has said it won’t withdraw the original WSL. Rather, new WSL command line tools will allow you to manage which version a specific distribution release runs. With WSL 2 you can have, say, a copy of Pengwin Linux running on WSL 2 with an Ubuntu release running on WSL 1.

It’s been an interesting few weeks experimenting with WSL 2. Microsoft has made the process of switching between its Linux platforms easy, and with a full Linux kernel there’s no need to worry about compatibility. An added bonus is truly native Linux containers that don’t need an additional hypervisor layer. New features in Visual Studio Code work well here, extending your development environment down into the WSL virtual machine.

The only real problem? With WSL 2 being developed as part of the 20H1 Windows 10 release, we’re going to have to wait until May 2020 before we get to use it in production. But now we finally have a date for the year of Linux on the desktop.