Serverless, Microservices, and Service Mesh — Oh My

Is Serverless eating the stack? Or is it just hype?

Serverless is eating the stack

Serverless is a huge trend right now. It’s easy to see why, with some claiming to see as much as 90% cost reduction compared with running VMs or physically-owned hardware.

In fact, the hype around Serverless seems to have reached a fever pitch, with some claiming that containers are deprecated and all future software will be built using Serverless.

Is the hype justified, or just hype?

In reality, Serverless is a subset of a more general-purpose computing trend towards auto-scaling, self-healing, and easily deployable microservices and simple provisioning of their dependencies (e.g., databases).

Rather than Serverless eating the stack, the principles that make Serverless popular are eating the stack.

In fact, we need to look no further than Kubernetes to see this in action.

First though, let’s hazard a definition of the often ambiguous term.

What is Serverless?

The term can be pretty broad, including almost any cloud hosting. For example, Wikipedia defines it this way: “a cloud-computing execution model in which the cloud provider runs the server, and dynamically manages the allocation of machine resources”. Serverless computing — Wikipedia

Whereas MartinFowler.com defines it more concretely: “Serverless architectures are application designs that incorporate third-party “Backend as a Service” (BaaS) services, and/or that include custom code run in managed, ephemeral containers on a “Functions as a Service” (FaaS) platform.” Serverless Architectures

In either case, there are several primary characteristics of Serverless computing:

Auto-scaling (utility computing)

Gone are the days of buying too much capacity and having it sit around; your application will scale as the demand increases.

Managed

You don’t have to worry about patching an OS, restarting a service, or handling a database upgrade.

Event-driven architecture (FaaS)

Code only runs in response to an event — no need for long-running containers or servers.

Easy provisioning of dependencies

Need a multi-AZ DB for your service? No need to file a ticket or talk to anyone. With a few clicks, you quickly have a datastore spun up, monitored, backed-up, and managed for you.

How Kubernetes narrows the gap

Many of the same benefits provided by today’s Serverless platforms have now been brought to the world of long-running microservices via Kubernetes.

Auto-scaling

Kubernetes can automatically increase or decrease the number of instances of your application running on the cluster. With Docker, a new instance can start in milliseconds.

Kubernetes can also automatically scale the number of nodes in your cluster as the demand increases or decreases.

Managed (self-healing)

Kubernetes constantly monitors your application and the cluster itself.

If an instance of your application goes down, Kubernetes will restart it.

If a node in the cluster goes down, Kubernetes will reschedule all the services from that node onto new nodes.

Easy provisioning of dependencies

Kubernetes has built-in facilities for requesting persistent backing storage that lives outside individual containers.

With the integration of service brokers through the Service Catalog API, Kubernetes provides seamless integration with backing services (e.g., databases, queues, etc.).

FaaS on Kubernetes

While Kubernetes brings the power of auto-scaling and self-healing to any containerized microservice, there are additional benefits and cost-savings that can be obtained by using a FaaS architecture.

FaaS architectures are event-driven and they are much closer to true utility computing, given that you are only charged for running functions.

Kubernetes actually supports this style of architecture as well, with a number of great projects already leveraging the power of Kubernetes to provide FaaS: OpenWhisk, Fission.io, Kubless, etc.

In fact, this is such an important use case that Google has started a project that deeply integrates Serverless with Kubernetes, called KNative. It provides a common approach to running Serverless functions, generating events, and wiring these together.

Running FaaS on Kubernetes actually makes a lot of sense, especially for the Enterprise. You get complete control over the underlying infrastructure and the underlying container image. You get complete control over the networking and connectivity. You also maintain more robust control over the runtime characteristics of your FaaS environment like cache warming.

Given that FaaS is really just another form of microservices, there can also be a significant advantage to managing service-to-service communication of both FaaS-style services and long-running microservices with the same approach — enter the Service Mesh.

The power of the mesh

As companies have begun decomposing large applications into smaller microservices (or even functions), a number of challenges have arisen.

Rob Whiteley, from NGINX, describes a Service Mesh this way:

“Think of a service mesh as a specialized Layer 7 network for microservices APIs. It offers authentication, authorization, security, and performance services to optimize the “east/west” traffic running between services. More importantly, it gives you a central point to apply these policies rather than having to code all of this directly into the business logic of your applications.” (Why You Should Care about a Service Mesh — The New Stack)

Both FaaS and long-running services have the same issues. Centralized policy enforcement, authentication, authorization, security, performance, and availability enhancements are needed by all services on the network.

Many of the FaaS offerings for Kubernetes already support integration with Istio, a popular Service Mesh (e.g., Enabling Istio on Fission, Serverless service mesh with Kubeless and Istio).

Conclusion

Almost a year ago, Mike Roberts (writing for MartinFowler.com) wrote:

“As we see the gap of management and scaling between Serverless FaaS and hosted containers narrow, the choice between them may just come down to style and type of application. For example, it may be that FaaS is seen as a better choice for an event-driven style with few event types per application component, and containers are seen as a better choice for synchronous-request–driven components with many entry points.” (Serverless Architectures, emphasis added).

Given the rapid adoption of Kubernetes over the last few years, coupled with the increased control and reduced vendor lock-in it provides, I would argue: In the future, Kubernetes will likely be the primary way enterprise companies deploy Serverless.

The choice between long-running containers vs. short-lived functions will come down to picking the right tool for the right job. But both will live in the same environment (Kubernetes), auto-scaling and self-healing as needed. Both will communicate over a shared Service Mesh that provides unified policy enforcement, observability, and enhanced network reliability. And the deployment and management of services (regardless of the type or granularity) will be done through the same process.

In Kubernetes, the industry has finally found a way to come together to solve some of the hardest (and most ubiquitous) problems of distributed computing—all while somehow avoiding vendor lock-in.

Now, that is a trend that will surely eat the stack and change the technology landscape for decades to come.