How Rust can replace C, with Python’s help

Proponents of Rust, the language engineered by Mozilla to give developers both speed and memory safety, are stumping for the language as a long-term replacement for C and C++. But replacing software written in these languages can be a difficult, long-term project.

One place where Rust could supplant C in the short term is in the traditionally C libraries used in other languages. Much of the Python ecosystem for statistics and machine learning is written in C, via modules that could be replaced or rewritten incrementally.

It isn’t difficult to expose Rust code to Python. A Rust library can expose a C ABI (application binary interface) to Python without too much work. Some Rust crates (as Rust packages are called) already expose Python bindings to make them useful in Python. But there is always opportunity for closer integrations between both languages.

To that end, a new spate of projects are making it easier to develop Rust libraries with convenient bindings to Python – and to deploy Python packages that have Rust binaries.

Rust-CPython

What it is: A set of bindings in Rust for the CPython runtime. This allows a Rust program to connect to CPython, use its ABI, run Python programs through it, and work with representations of Python objects in Rust itself.

Who it’s for: Rust programmers who want to hook into CPython and control it from the inside out. Note that because of the way Python objects are managed, Rust’s safeties don’t always apply to them, so you have to use an extra level of caution.

PyO3

What it is: For Rust developers, the PyO3 project provides a basic way to write Rust software with bindings to Python in both directions. A Rust program can interface with Python objects and the Python interpreter, and can expose Rust methods to a Python program in the same way a C module does.

PyO3 support for some Python features is still limited (exceptions, for instance), but the basics are there, and thanks to PyO3’s idiomatic use of Rust features, it is relatively easy to decorate existing Rust code to make it Python-aware.

Who it’s for: Those writing modules that work closely with the Python runtime, and need to interact directly with it.

Snaek

What it is: Another project in the early stages, Snaek lets developers create Rust libraries that are loaded dynamically into Python as needed, but don’t rely on being linked statically against Python’s runtime. Rust libraries that use Snaek don’t need to be written with Python specifically in mind, but simply expose C-compatible methods.

One potential disadvantage with Snaek is that it doesn’t use ctypes, the standard Python library for interfacing with C code. Instead, it uses cffi, an alternative library developed by the team that also produced PyPy. It isn’t hard to learn cffi, but any project that already uses ctypes will need to be reworked.

Who it’s for: Those who want to expose methods written in Rust to a Python script, or for Rust developers who don’t want or need to become familiar with Python.

The Cookiecutter template

What it is: A fourth project deals with one of the common mechanical issues that arise when bundling binary modules with a Python library. Cookiecutter creates Python projects from templates. One of the templates available – Cookiecutter PyPackage Rust Cross-Platform Publish – simplifies the process of bundling Rust binaries with a Python library.

“A very important goal of the project,” writes its maintainers, “is that it be able to produce a binary distribution (Wheel) which will not require the end user to actually compile the Rust code themselves.” Windows users are often frustrated by the lack of precompiled, Windows binaries of Python packages, so anything to alleviate that issue will be welcome.

Who it’s best for: Those creating many Python projects with Rust bindings, or those trying to distribute such projects to others.