In context of the cloud, the term “serverless” generally means that the customer is no longer concerned with the nuts-and-bolts of server maintenance. Instead of worrying about uptime, scaling, security, etc, the customer focuses more on their own business.
There are several ways to deliver a serverless solution. A Platform as a Service is one such approach. I’ll focus on the Function as a Service (FaaS), a common approach to serverless architectures across cloud vendors. FaaS works because it’s opinionated. Each cloud vendor has their own offering with constraints (AWS Lambda, Azure Functions, Google Cloud Functions, to name a few). What the customer gets in return is true utility pricing and the promise of a scalable architecture built of stateless building blocks.
There’s promise. It feels like the early years of Internet with its notion of a global, scalable approach like HTTP. Then as now, successful architectures will require a different mindset than those used for monolithic apps. In my situation, I’m tilted towards stitching it all back together, given that I work in integration. Once that monolithic app is broken up into stateless functions, how is it best joined together into something that behaves like the original app?
There are several approaches to this problem. On the one side are directed, process-oriented solutions that define the states and transitions. On the other side are event-driven systems which feel more like choreography, where the application flow is effectively implicit.
The situation usually reveals the appropriate architecture (one wonders if the right solution won’t combine aspects of both), but there are some useful rules of thumb to consider which I’ll detail in turn.
Lessons from the Cloud | Not Everything is Serverless
In 2008 while working at a former employer, I was asked to investigate Google App Engine (Google’s early cloud offering). Other teams were simultaneously using AWS, so it was a good opportunity to understand the pros and cons of the opposing approaches to a public cloud. While AWS was essentially Infrastructure as a Service, App Engine was better seen as a Platform as a Service.
It was compelling early on to use Google’s infrastructure. You could get up and running in less than an hour. They had a database service based upon Big Table that allowed you to finally have a durable data store at Web scale, and importantly, you didn’t need to manage it. Even your application code (written in Python) was hosted. It was the first serverless platform I ever recall using and in certain embodiments would qualify as a FaaS.
And yet, looking back, it was too far ahead of its time. It was the right approach out of the gate but it left us with no fallback position. Sometimes you need access to private resources over VPN. Sometimes a relational database is the right tool for the job (not Big Table). And sometimes you just need to maintain state!
Ultimately, you need to make exceptions for real life use cases when building apps. And when all that’s being offered is a Platform as a Service (or a Function as a Service), there’s literally no escape valve — no way to just get things done when the opinionated service is just too limiting.
What’s different now is that Google, Amazon, Microsoft and others have mature clouds. Their robust offerings include FaaS as well as the stateful services and infrastructure necessary to support them and take a fall-back position as necessary. Customers are free to use FaaS where it makes sense, while at the same time they can work around it when it doesn’t.
FaaS is by no means the only answer, but it is becoming more so each day as technology limits and challenges are addressed.
Lessons from Ajax | Respect Stateless Principles
Just over 15 years ago, my colleagues and I filed for a patent using a pattern that came to be known as Ajax.
A method of…maintaining substantial state at the web browser regarding the data itself and the format for its presentation. In a preferred embodiment, the web browser need not include more than a script language interpreter, a parser for XML or other hierarchical tagged data, and an element for communication with the server.
Ajax is commonplace, but at the time it was unusual to design a Web application architecture that included state. I was told more than once, “Son, the Internet is stateless for a reason”. Eventually Ajax (and the improved usability it provided) did win the day, but along the way the conversations and heated debates regarding state left a lasting impression.
It would be easy to dismiss the other side as being wrong. In fact both sides were correct in this debate.
HTTP is a stateless protocol and Ajax works, because it never challenged that. Ajax overcomes the stateless limits of HTTP by using the browser as a semi-durable store. Using the Ajax approach, the browser loads just enough code to become a capable, stateful client. It then controls the interaction. The server, in this exchange, becomes the stateless HTTP endpoint it was intended to be while the browser is the semi-durable, stateful engine. Once the orchestration is done (i.e., once the user closes the Web page) it all goes away. HTTP remains the stateless protocol it was intended to be.
FaaS endpoints would do well to take an Ajax-like approach. Their stateless nature can be respected as long as the supporting architecture that surrounds them is appropriate. Just as HTTP benefited from the lightweight orchestration provided by Ajax, FaaS endpoints can benefit from a semi-durable orchestration layer. FaaS must remain stateless; like HTTP, that is the primary advantage.
As with Ajax, an orchestration layer can provide a type of semi-durable state. And when done, the vestiges of the orchestration should disappear. If done properly, it should feel like boiler-plate level work.
Lessons from REST | Specifications Matter
Having worked in enterprise software for years, it’s easy to get overwhelmed by specifications. They can feel anything but agile. It was actually a relief (and I would bet many colleagues would agree) when the enterprise started moving away from WSDL/SOAP and towards REST/JSON as the consumer space had already done. Over the years, XML technologies and the related specifications had become quite large. They were capable, but they were also cumbersome.
As is often the case, the pendulum swung reactively far — away from specifications. For several years there was no agreed-upon standard for defining RESTful APIs. But over time even strong proponents of simplicity started to acknowledge the importance of specifications (of API standards). There’s still a bit of hesitation to go all in, and the existing specifications themselves are still not as capable as WSDL. But the OpenAPI REST specification has emerged as the standard, with the backing of all major organizations.
With FaaS, there is no such common ground at this time across cloud vendors. FaaS is currently being augmented with HTTP standards and specs. Amazon, for example, allows developers to use their API Gateway product to define the public contract for a Lambda function. It’s a reasonable approach and leverages the OpenAPI specification described in the paragraph above. Google likewise piggybacks atop HTTP, since their cloud functions are authored using Express, a Web application and API framework.
Given the ubiquity of HTTP, It’s likely HTTP will continue to play an outsized role in defining message exchange patterns for FaaS for the foreseeable future. From an integration perspective the one thing we’ve learned is that it needs to be easy to author and maintain a specification. The closer the documentation is to the source, the easier it is to manage.
We’ve found that customers like familiarity most of all in their tools, so we always consider both standards (e.g., the OpenAPI spec) and the status quo (i.e., how are things currently) as solid candidates for how we design UIs. In our case we prompt the user for an HTTP specification. If that’s unavailable, we can inspect their code and parse the JS/JavaDocs. And finally, if that’s not present we ask for some sample JSON input and output documents (as shown here).
In the end, specs do provide agility by liberating producers and consumers alike. The predictability they provide will enable FaaS to flourish — just as has been the case with REST/JSON.
Lessons From EAI | Function Life Cycle Management
Understanding a function through its signature (e.g., inputs, outputs, and errors) is adequate in most situations. But when seen through the lens of enterprise application integration (EAI), there’s more to a function than just its runtime execution. It has a life cycle.
The integration product I work on is organized around these principles, so internally we take a very broad stroke when discussing “function calls”. We consider all the possible reasons one might make a call and route it appropriately. Here is an example (shortened for readability) that represents some common function life cycle events and how we categorize and route calls.
Most will be familiar with the call to execute. Here are some other reasons to invoke a function:
- get_features: get the static signature for the function (inputs, outputs, and errors). This is effectively its API
- get_activity_signature: get the dynamic signature given an authorization and a given configuration
- on_startup: allow the function to review the configuration and approve in advance of being called at runtime
- execute: make the actual runtime function call
- create_trigger: register a callback with the function, allowing it to make contact when a given event occurs
- delete_trigger: cancel a registered callback
As you consider the ways in which your function will be orchestrated, consider more than just its runtime execution to future-proof your efforts. EAI design principles are robust, and we find they apply very well when orchestrating stateless endpoints — FaaS included. The core pieces that make EAI relevant to FaaS are as follows:
- A data model that provides a uniform structure for describing data across all services.
- A connector or agent model that is pluggable and extensible.
- An independent modeling standard to define public APIs, internal data routing and conditional processing.
- A centralized engine that executes the rules and conditions.
What’s important about these qualities is that they’re managed by the orchestrating platform, not the endpoints being invoked. EAI is designed to not be intrusive or prescriptive to the services being connected. In terms of FaaS, it means that conditional routing and state management remain external to the function, being entirely managed by the integration engine. Importantly, FaaS remains stateless, independent of the rules that manipulate it.
Serverless architectures can benefit from a more nuanced approach to handling state. The lesson from Ajax was finding an approach that complements the stateless nature and the tools already in use. As a rule of thumb, don’t reorganize a monolithic application into stateless functions, only to connect them back together with the wrong approach.
Microsoft, Amazon and IBM already address the need for state to varying degrees for their FaaS offerings. Microsoft offers a concept known as Durable Functions, Amazon offers Step Functions, and IBM offers Composer.
In Microsoft’s case, the process orchestration is handled in code and treated like just another function call. In the background Azure Storage Queues and tables are used for persistence. There’s merit to the consistency of this approach: write your cloud functions and their orchestration rules in one place, in one language, on one platform, etc. The product is still in pre-release as of January 2018, and is limited to C#. But it does reveal the importance of state management tools when dealing with FaaS. As they describe in their blog,
“Because the state is managed by the Durable Functions runtime, you don’t have to implement your own status tracking mechanism.”
Amazon’s Step Functions has been around longer and takes a visual approach to orchestration, using process modeling metaphors. Developers describe their flow in JSON using finite state machine concepts. They then preview their work as a visual flow to help validate and visualize their models. The integrated logging and real-time reporting help to make understanding and testing the orchestration manageable.
IBM’s offering is the most recent and uses a Finite State Machine approach like Step Functions. It too has a means to visualize your transitions and states. As of this writing, Composer and its supporting tool (shell) are available as IBM Research previews.
From Finite State Machines, to Event Queues to API Gateways, there are numerous approaches to solving the challenge of serverless orchestration. And each approach shines when applied to the right use case.
Regardless of the approach chosen, acknowledging the need for such orchestration tools is what’s most important. It’s an increasingly stateless ecosystem, and HTTP really began this march. FaaS is just one of many stateless endpoint types to be orchestrated.
Consider the following example that runs an orchestration when an appointment needs to be scheduled.
The first step transforms the incoming text into JSON using FaaS (an AWS Lambda function). The output is then sent to Google Calendar to check for availability. If available, an external scheduling service is contacted over HTTP, and finally an SMS confirmation is sent using Twilio. If there is no availability, the system will try again until a slot is found.
All endpoints are stateless as intended, with no knowledge of the surrounding process. Whether a service like Google Calendar, a RESTful API, or FaaS, the future of the Internet is stateless (well, mostly). State will never fully go away, but it will definitely be better organized.
Originally published at www.intwixt.com on January 18, 2018. Updated to Include Reference to IBM Composer.