Serverless Mullet Architectures
Business in the front, party in the back. Bring on the mullets!
In residential construction, a mullet architecture is a house with a traditional front but with a radically different — often much more modern — backside where it faces the private yard.
Like the mullet haircut after which the architecture is named, it’s conventional business in the front — but a creative party in the back.
I find the mullet architecture metaphor useful in describing software designs that have a similar dichotomy. Amazon API Gateway launched support for serverless web sockets at the end of 2018, and using them with AWS Lambda functions is a great example of a software mullet architecture.
In this case, the “front yard” is a classic websocket — a long-lived, duplex TCP/IP socket between two systems established via HTTP.
Classic uses for websockets include enabling mobile devices and web browsers to communicate with backend systems and services in real time, and to enable those services to notify clients proactively — without requiring the CPU and network overhead of repeated polling by the client.
In the classic approach, the “server side” of the websocket is indeed a conventional server, such as an EC2 instance in the AWS cloud.
The serverless version of this websockets looks and works the same on the front — to the mobile device or web browser, nothing changes. But the “party in the back” of the mullet is no longer a server — now it’s a Lambda function.
To make this work, API Gateway both hosts the websocket protocol (just as it hosts the HTTP protocol for a REST API) and performs the data framing and dispatch. In a REST API call, the relationship between the call to the API and API Gateway’s call to Lambda (or other backend services) is synchronous and one-to-one.
Both of these assumptions get relaxed in a web socket, which offers independent, asynchronous communication in both directions. API Gateway handles this “impedance mismatch” — providing the long-lived endpoint to the websocket for its client, while handling Lambda invocations (and response callbacks — more on those later) on the backend.
Here’s a conceptual diagram of the relationships with its communication patterns:
When is a serverless mullet a good idea?
When (and why) is a serverless mullet architecture helpful? One simple answer: Anywhere you use a websocket today, you can now consider replacing it with a serverless backend.
Amazon’s documentation uses a chat relay server between mobile and/or web clients to illustrate one possible case where a serverless approach can replace a scenario that historically could only be accomplished with servers.
However, there are also interesting “server-to-server” (if you’ll forgive the expression) applications of this architectural pattern beyond long-lived client connections. I recently found myself needing to build a NAT puncher rendezvous service — essentially a simplified version of a STUN server.
You can read more about NAT punching here, but for the purposes of our discussion here, what matters is that I had the following requirements:
- I needed a small amount of configuration information from each of two different Lambda functions. Let’s call this info a “pairing key” — it can be represented by a short string. For discussion purposes, we’ll refer to the two callers as “left” and “right”. Note that the service is multi-tenanted, so there are potentially a lot of left/right pairs constantly coming and going, each using different pairing keys.
- I also needed a small amount of metadata that I can get from API Gateway about the connection itself (basically the source IP as it appears to API Gateway, after any NATting has taken place).
- I have to exchange the data from (2) between clients who provide the same pairing key in (1); that is, left gets right’s metadata and right gets left’s metadata. There’s a lightweight barrier synchronization here: (3) can’t happen until both left and right have shown up…but once they have shown up, the service has to perform (3) as quickly as possible.
The final requirement above is the reason a simple REST API backed by Lambda isn’t a great solution: It would require the first arriver to sit in a busy loop, continuously polling the database (Amazon DynamoDB in my case) waiting for the other side to show up.
Repeatedly querying DynamoDB would drive up costs and we’d be subject to maximum integration duration of an API call of 30 seconds. Using DynamoDB change streams doesn’t work here, either, as the Lambda they would invoke can’t “talk” to the Lambda instance created by invoking the API. It’s also tricky to use Step Functions — “left” and “right” are symmetric peers here, so neither one knows who should kick off a workflow.
Enter…The Mullet
So what can we do that’s better? Well, left and right aren’t mobile or web clients, they’re Lambdas — but they have a very “websockety” problem. They need to coordinate some data and event timing through an intermediary that can “see” both conversations and they benefit from a communication channel that can implicitly convey the state of the barrier synchronization required.
The protocol is simple and looks like this (shown with left as the first arrival):
Here we take full advantage of the mullet architecture:
- Clients arrive (and communicate) asynchronously with respect to one another, but we can also track the progression of the workflow and coordinate them from the “server” — here, a Lambda/Dynamo combo — that tracks the state of each pairing.
- API Gateway does most of the heavy lifting, including detecting the data frames in the websocket communication and turning them into Lambda invocations.
- API Gateway model validation verifies the syntax of incoming messages, so the Lambda code can assume they’re well formed, making the code even simpler.
The architecture is essentially the equivalent of a classic serverless “CRUD over API Gateway / Lambda / Dynamo” but with the added benefits of asynchronous, bidirectional communication and lightweight cross-call coordination.
One important piece of the puzzle is the async callback pathway. There’s an inherent communication asymmetry when we hook up a websocket to a Lambda.
Messages that flow from client to Lambda are easy to model — API Gateway turns them into the arguments to a Lambda invocation. If that Lambda wants to synchronously respond, that’s also easy — API Gateway turns its result into a websocket message and sends it back to the client after the Lambda completes.
But what about our barrier synchronization? In the sequence chart above, it has to happen asynchronously with respect to left’s conversation. To handle this, API Gateway creates a special HTTPS endpoint for each websocket. Calls to this URL get turned into websocket messages that are sent (asynchronously) back to the client.
In our example, the Lambda handling the conversation with right uses this special endpoint to unblock left when the pairing is complete. This represents more “expressive power” than normally exists when a client invokes a Lambda function.
Serverless Benefits
The serverless mullet architecture offers all the usual serverless advantages. In contrast to a serverful approach, such as running a (fleet of) STUN server(s), there are no EC2 instances to deploy, scale, log, manage, or monitor and fault tolerance and scalability comes built in.
Also unlike a server-based approach that would need a front end fleet to handle websocket communication, the code required to implement this approach is tiny — only a few hundred lines, most of which is boilerplate exception handling and error checking. Even the JSON syntax checking of the messages is handled automatically.
One caveat to this “all in on managed services” approach is that the configuration has a complexity of its own — unsurprisingly, as we’re asking services like API Gateway, Lambda, and Dynamo to do a lot of the heavy lifting for us.
For this project, my AWS CloudFormation template is over 500 lines (including comments), while the code, including all its error checking, is only 383 lines. Asingle data point, but illustrative of the fact that “configurating” the managed services to handle things like data frame syntax checking by exhibiting an embedded JSON Schema makes for some non trivial CloudFormation.
However, a little extra complexity in the config is well worth it to gain the operational operational benefits of letting AWS maintain and scale all that functionality!
Mullets all Around
Serverless continues to expand its “addressable market” as new capabilities and services join the party. Fully managed websockets backed by Lambda is a great step forward, but it’s far from the only example of mullet architectures.
Amazon AppSync, a managed GraphQL service, is another example. It offers a blend of synchronous and asynchronous JSON-based communication channels — and when backed by a Lambda instead of a SQL or NoSQL database, it offers another fantastic mullet architecture that makes it easy to build powerful capabilities with built-in query capabilities, all without the need for servers.
AWS and other cloud vendors continue to look for ways to make development easier, and hooking up serverless capabilities to conventional developer experiences continues to be a rich area for new innovation.
Business in the front, party in the back …
bring on the mullets!
