Kubernetes meets WebAssembly in Microsoft’s Krustlet

Microsoft’s Deis Labs recently announced an intriguing direction for Kubernetes, one that offers a new, sandboxed environment for our code. Building on the increasingly popular WebAssembly, Krustlet is a tool for delivering Kubernetes-managed workloads to WebAssembly runtimes, with a v0.1.0 release now on GitHub.

The name Krustlet comes from how they’re built and what they’re intended to do. Krustlets are “Kubernetes RUST kubeLETs.” They take a fundamental Kubernetes component, the kubelet, and implement it in a new programming language and on a new platform. Kubelets, like much of Kubernetes, are written in Go; Krustlet has been developed in Mozilla’s type-safe and memory-safe Rust.

It’s interesting to note the choice of Rust. Microsoft has been investigating Rust as a way to reduce the risk of memory-based attacks on systems, using it as a systems programming language where, in the past, you’d have used C or a similar language. Rust is one of the main languages supported in WebAssembly, with both coming out of Mozilla-hosted research, making it a good candidate for applications that run on WebAssembly virtual machines.

From kubelet to Krustlet

A kubelet is a fundamental element of Kubernetes, sitting at the base of a node. Once running, a kubelet registers as a node with a Kubernetes controller, and then receives a PodSpec. This describes the containers that run on the node, and the kubelet manages those containers, ensuring they remain healthy. Without kubelets Kubernetes won’t run. Microsoft’s Virtual Kubelets are an essential part of its Azure Kubernetes implementation, giving Azure and Deis a lot of experience in designing and implementing alternatives to the default Kubernetes kubelet.

Krustlets are best thought of as an alternative kubelet implementation, but instead of working with container-hosted workloads, a Krustlet manages WebAssembly workloads. It uses the same mechanisms to get the workload details from controllers as a kubelet and uses those same tools to send back health and other management metrics. You can have both kubelets and Krustlets in the same Kubernetes infrastructure, with both container and WebAssembly workloads running and working together.

You don’t need new skills to work with Krustlets; they’re configured using YAML and managed with kubectl or your choice of graphical front end. All you need to do is build an appropriate configuration file that describes and deploys your WebAssembly applications as if they’re a standard Kubernetes pod.

Running Krustlet in Kubernetes

By building Krustlet in Rust, you can quickly deliver Kubernetes capabilities to WebAssembly and the under-development, stand-alone WebAssembly System Interface (WASI). Make WebAssembly a build target for Krustlet in the Rust compiler, and it’s ready to run in your browser. Application code will need a WASM compiler, with support for C in clang and .NET languages in Microsoft’s own Blazor.

The YAML for code that gets deployed to Krustlet is like any other pod YAML. The key difference is the architecture descriptor, “wasm32-wasi.” Krustlet will deploy and manage pods with this string. Beyond writing your application YAML, you only have to deploy your WebAssembly code to a container registry, ready for Krustlet to download and run.

Krustlet on tomorrow’s edge

Perhaps the interesting point comes when we start to think about Kubernetes as a management layer for our applications, one that can support scaling them out massively, far beyond the distributed code running on Azure or GCP or AWS. We need to start thinking in terms of the edge, with its myriad small servers operating as new hosts for our workloads, an edge that’s being brought into the hyperscale clouds with technologies such as Azure Edge Zones.

Then we need to think about smaller devices, right down to the Azure RTOS hardware I wrote about recently. How do we manage code running on them? There’s an answer buried in the Krustlet concept, one that goes a bit beyond the current WebAssembly implementation.

WebAssembly is only the start. Take away the browser and add a set of standard interfaces, and you’ve got WASI: a new general-purpose virtual machine that can take your WebAssembly-compiled binaries and run them without the overhead of the browser. It’s not a coincidence that many of the Mozilla team who defined WASI now work at cloud and edge providers, looking at how we can ship code that can run in hyper-converged and low-power edge hardware.

Working with Krustlet and WASI

What if we could go smaller still with WASI, and run it on devices like Azure Sphere, or even on Azure RTOS? Now we have a runtime with Krustlet that we can push code to as needed, using Kubernetes as a controller for the applications without needing the overhead of a container. There’s no need to manually update microcontrollers. Their firmware would be WASI on top of a networking stack, and application code could be delivered as needed.

Removing the dependency on containers is a big change from traditional Kubernetes operations, but we can keep those changes to a minimum by building on a trusted sandbox runtime like WebAssembly and WASI. It will be necessary to add new security scanning features to our repositories and build pipelines to ensure that the code we push out to devices hasn’t been compromised, perhaps building on the security tools being added to GitHub. Scanning code and dependencies will be easier with WebAssembly, as the code that comes out a WebAssembly compiler is designed to work on a relatively simple stack-based system with a minimal set of instructions.

That minimal instruction set is an advantage here, as code scanners in a build pipeline will be able to test and verify the WebAssembly code before it’s delivered to Krustlet at scale. Hopefully, any scanner will be an open source project, like both Kubernetes and WebAssembly, allowing anyone to build and deliver a Krustlet implementation without being locked into any of the big cloud providers and tied to their intellectual property.

It’s important to remember that both Krustlet and WASI are experimental projects, but there’s a lot of promise in a combination of the two technologies. Giving Kubernetes access to a second sandbox for code is a big step forward in delivering distributed applications that can run outside the data center, in a constrained edge. It’ll be interesting to see where Microsoft and Deis take Kubernetes next, and how the community will follow.