A Brief History of Containers @ FC
From Mesos to Kubernetes, the story of the Funding Circle journey with containers…
Over the past decade containerisation has become the standard way to bundle an application, along with its dependencies, into a deployable and distributable package. From Docker, to the wide range of container build and runtime technologies available today, many options are available for creating, hosting, and running containers on a host OS, but what about actually running these containers in a production environment?
Container orchestration encompasses all the aspects of running containers in a highly available, secure, and maintainable way. In the last five years, Kubernetes has become synonymous with container orchestration, but before this, Kubernetes was not the answer (or at least not the only answer) to running containers.
This blog series explains how we at Funding Circle use Kubernetes. Later blogs in the series will perform deep dives into some of the tech we use to manage Kubernetes (node management, observability, networking, etc.), but first let’s explain the journey to where we are today.
The year was 2014…
We first started looking at how we would orchestrate containers back in 2014. After evaluating a number of options the real front-runner at the time for production grade container orchestration was the Mesosphere stack, including Marathon for container orchestration.
This was a great solution, it was open source, and it was the de facto industry standard at the time. Kubernetes was also very much in its infancy, having only been announced in 2014, and released as v1 in 2015. Even after it was released it still had a number of issues that hadn’t been solved. Other container orchestration solutions like Docker Swarm were also still in early stages, so we pushed ahead with Mesos and Marathon.
Since then we have run Mesos on AWS, self-managed using a mixture of services built around EC2.
So what’s changed?
Let’s fast forward to 2021 and the container orchestration world looks very different…
Kubernetes has exploded in popularity, and is now the standard across the industry. We will cover “Why Kubernetes?” later in this blog, but in the meantime Mesos did not enjoy the popularity growth of Kubernetes, and its popularity in proportion to the container orchestration market has dramatically reduced.
When surveyed by far the most popular solutions for container orchestration are all Kubernetes based platforms[1]. We have self managed Kubernetes, managed Kubernetes solutions, for example from AWS, Google, Microsoft, as well as more distribution-like solutions like OpenShift and Rancher that provide prebuilt stacks of software and tooling for Kubernetes. The common theme across all offerings is Kubernetes as the core solution.
At the same time, Mesos has struggled to keep community engagement. Recently, the Mesos project narrowly avoided being sent to the Apache “attic”[2] where Apache projects go when they are sunset. Although the project was able to gain enough support to avoid this, there is still little interest beyond organisations who have already got Mesos running and need continued support.
Why Kubernetes?
Organisations all across the world, from brand new startups to decades old large enterprises have adopted Kubernetes as the platform of choice for hosting applications. This has driven, and has been driven, by a huge community around the platform. If you need help, you can usually find an answer with a quick Google… from Stack Overflow, community Slack, samples, or open source.
This industry standard position means that skills across the industry can be utilised and eases hiring into teams. There is a very large ecosystem developing around the platform. The Kubernetes open source project is governed by the Linux Foundation and the CNCF, with supported platforms from AWS, Google, Microsoft, Red Hat to name just a few.
It’s not just the large cloud providers that build tools, there are thousands of supported and open source tools, free and paid for, to utilise and make management of Kubernetes easier. These tools, and the ecosystems from the cloud providers benefit Funding Circle in reduced management costs of the platform, making it easier to maintain, as well as implement new features and solutions.
From the point of view of our development teams, Kubernetes allows us to deliver value to our customers faster. Through easier provisioning of test environments, greater control of rolling out updates and many more features, development teams are able to iterate faster and with increased confidence.
Lastly, a wealth of new features helps to make our applications more reliable and reduce the overhead required to manually manage applications. Features such as auto-scaling applications based on metrics specific to that application, will automate much of the management of live applications, and make the platform more efficient.
To be continued…
The next blog in this series will detail how we use Kubernetes at Funding Circle, including the platform and tools we use to manage our Kubernetes ecosystem.