Kubernetes and Application Platforms
Since my last post was copy/pasted from Twitter posts from 2019, I thought I’d follow with a perspective on a topic that has grown in awareness over the past 5 years: platform engineering.
Kubernetes is a portable, extensible, open source platform for managing containerized workloads and services.
An application platform is a cohesive software system that helps users develop, build, deploy, run, scale, secure, expose, monitor, and debug their applications without needing to manage the underlying infrastructure.
There’s a pretty sizable gap between those two descriptions.
In my post on the history of Kubernetes, I explained that we intended Kubernetes to be able to run a wide variety of workloads. I’ve described its core primitives of pods, workload controllers, and services as being IaaS-like in the project’s design documentation. We did not intend the project to become an application platform, as is still stated in the overview and in the more detailed project scope document, because the original goal was a compute platform more flexible than Google’s AppEngine and comparably flexible as Borg.
This turned out to be a reasonable decision, as many reported on the headwinds facing PaaS services over the years: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12. The application platform space has always been very fragmented, and many products struggled to achieve commercial success. A number of application platforms ported to Kubernetes, including RedHat Openshift and Cloud Foundry, and many new open-source platforms were built on Kubernetes.
Kubernetes also attracted a number PaaS users directly, for a variety of reasons, including cost, flexibility, and control. Some of these built their own platforms. By 2019 there were already many examples: Shopify, Spotify, Target, Cruise, AutoTrader, AirBnB, Conde Nast, Pinterest, Shopgun, John Lewis, Bloomberg, Adevinta, … In a way, this was an extension of the fragmentation of PaaS services and open-source projects.
Some amount of fragmentation is unavoidable. Simplicity and flexibility are opposing forces. Even a decade after the creation of Docker and Kubernetes, there are still programming-language-specific platforms. Workload type is also a common specialization axis: static/dynamic, stateless/stateful, request-driven / event-driven / streaming / batch, HPC, etc. Security/compliance posture carries usability tradeoffs, and can also be jurisdiction-specific and industry-specific. Monitoring and logging systems incur other tradeoffs, such as cost. CI/CD tools and workflows can be driven by company culture. The decisions that shape the developer experiences of platforms create combinatoric diversity.
While not an application platform itself, Kubernetes has been described as a platform for building platforms. But really Kubernetes was designed to make platforms possible to build atop it, not necessarily to make it easy.
For example, Kubernetes has some useful extension points, such as dynamic admission control, but no built-in way to manage all of the extensions. There have been some projects that have helped with this, such as helmfile, and some special-purpose package managers, such as Crossplane’s, but no cluster-level package manager has become ubiquitous. It will be interesting to see whether glasskube takes off, or whether another project or approach emerges.
So there’s definitely room to create platforms for building platforms on top of Kubernetes. Maybe the platform engineering and developer platform products will succeed at that, but the business challenges are bigger than the technical ones.
