Understanding Flatcar Container Linux

Microsoft isn’t only a Windows company. It now shepherds several Linux distributions alongside its own operating systems. It’s important to remember that these aren’t general-purpose Linuxes like Ubuntu or Red Hat. Instead, they’re targeted at specific use cases: SoNIC for cloud-scale network hardware, SphereOS for secured industrial Internet of Things hardware, and CBL-Mariner as a host for its Azure Stack edge systems and part of its Windows Subsystem for Linux.

A year ago, Microsoft bought German Linux vendor Kinvolk to help support container use on Azure, especially in cloud-native platforms like Kubernetes. Kinvolk is the home of the popular container-optimized Linux Flatcar, as well as a Linux-based secured Kubernetes platform, Lokomotive. Kinvolk had taken up the baton of producing a container-ready Linux release after Red Hat purchased CoreOS and merged its stack into Fedora, reducing its effectiveness for platforms like Kubernetes.

[ Also on InfoWorld: No one wants to manage Kubernetes anymore ]

Adding Kinvolk to Microsoft and Azure

In purchasing Kinvolk, Microsoft noted its importance to Azure and made a commitment to keeping the Flatcar community development project running, intending to learn from how the Kinvolk team works. It’s interesting to hear Microsoft say that it wants to learn how to do open source well, as it’s clear that the company is making a big transition to open source methods. It might never open source Windows or Office, but it’s using open source as a way to build and manage its newer tools and platforms.

It’s clear that Microsoft initially saw Flatcar as a replacement for CoreOS on Azure. A year before the acquisition it recommended that customers migrate to Flatcar before CoreOS’s May 2020 end-of-life deadline. Migration was a matter of changing your Azure deployment images in your Kubernetes or container repository. Alternatively, CoreOS systems could be switched to Flatcar using a simple script to download and run the update tools. This would allow existing nodes to continue running without a complete redeployment. With a new OS running you could then spend time building and testing new images before redeploying your container environment.

What is Flatcar Container Linux?

If you’ve not looked at Flatcar, it’s a minimal Linux distribution designed to run containers and nothing else. Flatcar builds on many of the underlying concepts in cloud-native application development, delivering an immutable infrastructure that is redeployed with each new build of your code. You don’t want your OS changing under your application, even if it is only hosting your application containers, so Flatcar is configured before deployment, with a read-only system partition. It can be configured to auto-update security patches, though you may prefer to make updates part of the build and deploy process. Kinvolk provides tools to manage the update process, managing what instances can be updated, when updates can run, and how frequently they should be applied.

Having a minimal Linux with a locked-down file system running in the cloud makes a lot of sense. Azure enforces separation between tenants running on the same hardware, but having an immutable container host reduces security risks considerably, keeping the attack surface small and ensuring that many classes of attack won’t run on your container host.

You can find Flatcar in the Azure Marketplace with versions for three different update channels: Stable, Beta, and Alpha. Most production systems should use the Stable channel, with Beta and Alpha recommended for test and development. You may prefer to run Beta and Alpha on your own internal systems directly from Kinvolk rather than from the Azure Marketplace.

Getting started with Flatcar on Azure

Kinvolk provides Azure CLI scripts for installing Flatcar. You can choose the version by specifying the SKU and version before constructing a URN for the image to be installed. This takes the format productname:channel:version. Kinvolk supports both gen 1 and gen 2 hypervisor images, with gen 1 the default. If you want to use gen 2, append -gen2 to the channel name in the SKU and the URN sections of the install scripts.

Alternatively, you can download your chosen Azure-ready Flatcar image directly, putting it in an Azure storage account. Kinvolk provides a script that requires a resource group name and a storage account. In practice, though, using Azure Marketplace is the best option as the Flatcar image is already in Azure and you won’t incur storage costs.

Although AKS (Azure Kubernetes Service) uses Ubuntu as its OS base, you can still take advantage of Flatcar on Azure using your own Kubernetes install or the Azure version of the Cluster API Provider to run your own managed Kubernetes instance. There are instructions for using AKS Engine on the Kinvolk site, and although it’s possible to use the standalone version of AKS, it’s now deprecated and no longer being updated.

Configuring and running Flatcar

Once you have installed a Flatcar image, you’ll need to configure it with a tool called Ignition that injects a configuration JSON file into the Flatcar userland prior to first boot. Once booted, Flatcar’s read-only filesystem means you can’t make changes. Configuration files are written using YAML, creating a Container Linux Config. This sets up the base configuration of a container host, defining what service containers it loads, how they’re run, and how a host responds to a container failure. For example, you can reload the most current container images at start-up, removing old versions, stopping cleanly on shutdown, and restarting failed containers after a set time.

Kinvolk provides a tool to transpile the CLC YAML into JSON for use with Ignition. Once the JSON is created, it can be inserted into Flatcar using the Azure CLI as custom data when you set up your container. The Azure CLI can add users and SSH keys to images if you need to log into them for development and debugging purposes. Production instances won’t need this, as it adds security risks.

The team recommends testing a Flatcar VM on a dev machine before moving into production. Although you can use Hyper-V, most of the documentation uses the open source QEMU, so you may prefer to test on a Linux system. This approach lets you use Kinvolk’s own test images, installing the Ignition JSON file on first boot. If you’re experimenting with different containers and configurations, simply delete the VM each time you make a change and use another copy of the Flatcar image.

Flatcar isn’t only for Azure. It runs on all the major public clouds, as well as on edge and private clouds. As the same configuration files work on all systems, Flatcar makes an interesting option for hybrid cloud and edge deployments where you can develop workloads once and deliver them to multiple targets with minimal changes. There’s also support for tools like Terraform, so you can make Flatcar part of an infrastructure-as-code platform, separating management of container hosts from applications.

Microsoft has learned a lot about managing open source acquisitions during the past few years. Like Deis Labs, Kinvolk remains a near-independent entity. This allows it to continue to support its open source community and build and run its own experiments. Meanwhile, Microsoft can use lessons from Flatcar as it expands its internal Linux offerings and adds more open source projects to its portfolio. At the same time, cloud-native application developers can keep using a familiar container host, with the peace of mind that comes from a committed backer with deep pockets.