What is platform engineering? Evolving devops

Platform engineering is a newer idea that aims to put the lessons of real-world devops into a more concrete, reproducible form. Devops is a powerful trend in software development because it helps to break down barriers in the software development life cycle. Platform engineering can benefit software organizations by improving quality, developer experience, and the customer’s time to value. By dealing with the specifics of technology and process, platform engineering is a concrete expression of devops and can help balance the conflicting needs for centralized control versus team autonomy.

Gartner identifies platform engineering as one of the key trends in development for 2023. Let’s take a look at what it is and how to take advantage of it.

Devops vs. platform engineering

Devops is devoted to seeing the software development life cycle in a holistic way to better understand and manage it. Devops asks us to treat development and operations as two parts of the same process. It aims to keep the people involved in building involved in deploying, thereby closing the feedback loop and improving the software development process as a whole.

Many believe that platform engineering is a practical evolution of the devops ideal. Platform engineering is the field of software engineering focused on providing centralized tools and support to manage the complexity of development life cycles. It’s an emerging concept, so the definition is somewhat amorphous and subject to change.

Although there is a sense in the industry that platform engineering is replacing devops, that is not really the case. As Matt Gowie, CEO of Masterpoint.io says:

People are screaming “devops is dead,” but I think in reality we’re just reframing how we actually implement devops and that new school of thought is nicely landing with the majority. The whole point of platform teams is that you’re enabling developers to do ops-related work but through abstractions or “golden paths,” as folks like to say. That increases [developer experience] because then developers don’t need to raise a ticket to get something done and they’re practicing “You build it, you run it.”

The changing role of the developer

“You build it, you run it” has come to be the slogan for devops. The idea is to ensure that those who have their hands on the levers and insight into what’s working are able to manage the process. The problem in real-world devops is how to implement the concept effectively. Only the most sophisticated and well-funded organizations are able to devote enough resources and attention to an independent devops team. As a result, the term devops often means tasking developers with both development and operations: a fundamental repurposing of what it means to be a developer.

This way of doing devops envisions the developer’s role crossing over the entire life cycle. Platform engineering is an evolution of devops that makes the focus on the developer as a cross-over role more explicit.

The rise of developer portals

The developer portal is the main mechanism and expression of platform engineering. Its main purpose is to gather together the developer’s tooling, documentation, and interactivity in one place. It is a kind of front end to the organization’s developer infrastructure.

Developer portals (aka internal developer platforms) have evolved out of several needs and trends. This primer on developer portals delineates these tools into three types: universal service catalog, API catalog tied to API gateway, and microservices catalog.

What’s inside the developer portal

APIs figure large in platform engineering because the uptake of microservices architecture has caused a great deal of increased complexity for modern software teams. Orchestrating microservices in a large organization can be very challenging. Just understanding what microservices are involved in a given use case can be difficult. A developer portal offers a unified view into the overall web of microservices.

Another aspect of the developer portal is offering a standard framework to combine the tools used by the organization. This enables teams to self-manage tool adoption, but in a way that is reproducible and traceable by the organization.

Finally, the developer portal can be used to host documentation in a kind of universal developer wiki. Full-text search is often a part of the portal, enabling global, cross-organizational access to information. It can also serve as an alerting and ticketing system. 

In general, the idea is to concentrate as much as possible about the development team’s work into the portal. That way, when a need arises, the portal is the first and most likely place to find the solution. Information and tooling gathers in the portal instead of in less formal conversations, where it could easily become part of unwritten tacit knowledge. 

Senior engineers and leadership can contribute guidance using a developer portal. It acts as a hub where the organization can gather and broadcast policy and best practices while impacting developer autonomy as little as possible.

Platform engineering entails building the internal developer portal and associated toolchains and workflows to go with it. The platform engineer is thus responsible for supporting developer autonomy while providing a central place for developers to interact with centralized support and leadership.

An example platform: Backstage.io

Backstage.io is an open-source developer platform from Spotify. It’s a useful model for understanding, in a more concrete way, what platform engineering is all about. Backstage is a web application that provides a pluggable and extensible framework for a variety of tools, APIs, and services used in development.

Backstage’s mission statement says “At Spotify, we’ve always believed in the speed and ingenuity that comes from having autonomous development teams. But as we learned firsthand, the faster you grow, the more fragmented and complex your software ecosystem becomes. And then everything slows down again.”

Backstage is a build-your-own solution, akin to Jenkins in the CI/CD world. As more players move into this space, we will see more software-as-a-service (SaaS) platforms with similar capabilities.

Backstage has four main features to achieve the goals of platform engineering:

• A software catalog for controlling tools like microservices, libraries, data pipelines, websites, and more from a central location.
• Software templates to define and enforce best practices and templates for new projects.
• TechDocs, a central, standardized repository for documentation (Backstage calls this a “docs like code” approach).
• Plugins that allow for extending the core platform’s functionality.

You can get a feel for Backstage by playing around with its live demo.

The software catalog

At the heart of Backstage and other developer portals is the software catalog. Here, APIs used to compose microservices, both internal and third party, can be automatically discovered, managed, and consumed, but in a consistent format. The APIs are modeled as a web of dependencies to make it easier to get a dashboard view of all the services in an organization, what teams own them, and how they interact.

In addition to APIs, a whole range of tools can be brought into the portal and orchestrated there, in relation to the APIs they support. CI/CD platforms like CircleCI and the code repositories like GitHub can be associated with the projects they relate to. The idea is to bring the organization’s technology landscape into the portal, making it accessible to both the teams using it in a self-service manner and the leaders tasked with managing their teams.

With the right configuration, a developer portal can provide a view of the status and health of a complex and sprawling microservices architecture, even when deployed on homogeneous stacks (like a blend of Kubernetes, serverless, and virtual machines) and across cloud providers.

The future of platform engineering

The core idea of platform engineering—a standardized way to centralize and manage the overall development lifecycle—is bound to continue growing in importance. It’s still nascent, but the practicality of platform engineering is helping to make the vision of devops a more attainable reality for more organizations.