Serverless x K8S: Reflections and Proposing a Framework to Guide the Selection
The serverless computing model, introduced almost a decade ago, caused a stir in many organizations grappling with understanding its benefits and migrating workloads to it.
Choosing between serverless and other infrastructure options to host your applications can be complex. There are often diametrically opposed stances, with each side defending their chosen approach without necessarily considering the actual usages and trade-offs involved.
In this article, I will share a selection framework that can help in choosing the best option, or at least contextualize any discussions on the topic when developing new applications. With many different solutions available, I will focus on the selection between serverless (Lambda) and K8S.
Hosting Revisited
Over time, new technologies have continually revolutionized the ways in which network-based applications — independent of their style — are hosted, making them available to their users.
Let’s take a closer look at some of these options, highlighting their most interesting characteristics and exploring the potential motivations for switching between them.
Bare Metal
When opting for bare metal, our application(s) are installed on a physical server that we fully manage. If you need to scale it vertically, you can get a new or upgraded server; if you need to scale horizontally, you can get more servers and use some sort of load balancer or reverse proxy to distribute the traffic.
This is probably the most traditional approach, providing the greatest level of control as you have full access to tweak every knob available in the hardware or underlying software.
However, there are some challenges with this method, including the potential drift between the development and production environments, the need for over-provisioning when handling scaling issues, and the dilemma between having idle capacity versus non-isolation of applications in a shared setup.
Virtual Machines
Virtualization is not a new concept, its first usage dates back to the 1960s and 1970s with IBM mainframes.
The idea behind it is simple: you have a host operating system that runs in your bare metal, but you never actually interact with it. Instead, you install your operating system and applications on this virtual machine that has a subset of the resources — memory, CPU, persistent storage — of the host.
This allows you to address some of the limitations of the previous more traditional hosting option. Since it is virtualized, you could run the exact same virtual machine in the development environment. This means having full control over the OS, settings and libraries. Additionally, each virtual machine is isolated from the other, providing a higher degree of control to host different applications, one in each VM. This isolation ensures that if one application experiences issues, it doesn’t affect the functionality of others.
Scalability is also partially addressed, as you can increase a specific application’s virtual machine to add more CPU, memory and other resources. As long as you have unused resources within the same host or across separate hosts, you have this lever to pull.
However, in scenarios involving highly distributed/microservice systems, managing this setup can become complex. It requires a lot of effort and custom scripting to accommodate the dynamic and complex routing nature of many applications today.
Containers and Kubernetes
The idea behind containers is to take the best aspects of virtual machines and make them even better. A container, being a more lightweight type of virtualization, enables you to run multiple containers in the same host, including the development environment. When combined with Kubernetes (K8S), you can create complex topologies that can react — to an extent — and automatically regulate the availability of your applications.
All the aforementioned options can exist in a self-managed/self-hosted or, more recently, cloud-based manner. Within AWS that can translate to having EC2 for your VMs and EKS for your K8S setups as usual services.
EKS seems to check all the boxes, offering the benefits of virtualization, the lightweight nature of containers, support for complex topologies, and high scalability features. Is your application experiencing a surge in traffic? Launch more pods. The node can’t handle more pods? Spin more nodes.
However, configuring and maintaining a K8S (or EKS) cluster is not without its challenges and requires specific expertise that is not easy to master.
Serverless & Lambda
The newest option is the serverless compute model. In AWS, this translates to the lambda functions, which provide an ephemeral environment to execute your application’s code.
Serverless is known — or defined — by the capacity to scale to zero, which is associated with reductions in cost. For example, if your lambda is not being invoked, you do not pay, which differs from all the previous options as they assume some sort of always available, minimum capacity available, even if idle.
Besides the cost aspect, serverless proposes a much more simplified operation. In its simplest form you deploy the code you want to execute, connect it to an event source — the thing that will trigger the execution — and off you go.
This simplicity is beneficial when you do not need all the flexibility and fine-tuning that K8S brings, with its inherent higher setup/maintenance cost.
But serverless is not your get-out-of-jail-free card. Cold starts, provisioned concurrency, reserved concurrency, and lambda sizes are some concepts that will sooner or later be part of your vocabulary…
It Is the Best of Times, It Is the Worst of Times
As we have seen, even when discarding other options, there are many ways to host our applications. If we focus on the serverless versus K8S alone, we already face an interesting challenge when deciding between the two approaches.
Within the tech community, it is not uncommon to find those who have been using either one as their main (or only) go-to solution and categorically discrediting the other without giving it a fair chance.
This dogmatic approach is dangerous and usually leads to the “if all I have is a hammer, everything looks like a nail” scenario. You may end up missing out on the true benefits of the less-known approach or blindly adopting the new shiny thing irrespective of the hidden costs or relevant limitations it is associated with.
To handle situations like this, it is important to review your software architecture practices, be aware of common pitfalls, and foster the use of technology radar and architecture decision records (ADR).
So let’s see how we could tackle this particular problem!
Selection Framework
We all know context is king, so absolute statements such as “always use X” should be extremely rare. At the same time, we have to find a way to approach the selection of serverless x K8S (in our context) in a more predictable manner, shifting focus from potential personal preferences/bias solely based on previous experiences.
With this intention in mind, I propose a selection framework that presents a number of questions we should answer for any new project. Based on those answers, a score is generated that indicates the preferable solution.
As a starting point, consider the following criteria:
- Delivery Mechanism — Is the functionality of your application exposed as an API or via messaging? (events/commands)
API — 2 points
Messaging — 1 point
- Audience — Is the functionality of your application exposed directly to external customers or is it comprised of internal users?
External — 2 points
Internal — 1 point
- Latency — What is the required p95 for this application functionality?
P95 < 100 ms — 3 points
100 ms <= p95 < 500ms — 2 points
P95 >= 500 ms — 1 point
- Traffic Nature — Is it continuous or mostly sporadic?
Continuous (with occasional spikes) — 2 points
Sporadic — 1 point
If your score is >= 7, K8S is preferable
If your score is <= 5, Serverless is preferable
If your score is 6, both options are viable.
The closer you are to a score that indicates that either option is viable, the stronger the indication that you must have additional discussions.
Ultimately, you are likely looking at additional factors such as execution cost, what other services will be integrated, and the team’s familiarity with the technology, to ratify your choice and understand the trade-offs.
Moving Forward
As we have seen, hosting options have evolved and will continue to do so. Each option comes with its strengths and weaknesses, with new options trying to address different ways to make our applications available.
For many companies, a cloud transformation process took place, putting us in a position to choose the solution that best serves us: K8S or serverless.
As a “newcomer”, serverless can be met with a mix of excitement and apprehension. Support in the decision-making process is key to expediting those decisions and shifting focus towards the most relevant points. As part of this process, we recommend considering building a framework similar to the one I have shared. Make sure to tweak the criteria to fit your context and use it with any new project.
It is important to acknowledge that no framework is perfect, nor should you attempt to make it so.
Instead, focus on choosing the smallest number of questions that could expose the application’s non-functional requirements that are relevant to the decision-making process.
Design this framework to capture the gray areas, where additional discussions will be needed and likely dictate the selection to be made.
If you have adopted ADR as part of your process, add the score and decision to it for future reference. This will help you reassess when new requirements or technologies become available.
Good luck!
Editorial reviews by Catherine Heim & Sam-Nicolai Johnston.
Want to work with us? Click here to see all open positions at SSENSE!
