Orchestrating Serverless from Serverless
Using XState and Durable Objects with Cloudflare Workers
Serverless is an exciting field these days. There is a lot of competition from both big, medium and small players, and this makes it an interesting option in all kind of apps and architectures. But managing a lot of smaller functions in a coherent flow comes with some challenges. Here I present a PoC I did on how to implement the Orchestration pattern with a serverless function to manage the flow of other serverless functions. I’m using XState to manage the flow and Cloudflare Durable Objects to persist state between invocations. RabbitMQ is used as message broker.
Orchestration is often mentioned together with Choreography as one of the main architectural patterns for managing flow of microservices. Where Choreography is the designed flow of services picking up on other services gestures, Orchestration is the careful guiding of the flow between different services, creating a whole. Both are mentioned with Saga pattern, as a way to keep transactional integrity between services in a microservice architecture. I won’t go into details here, but assume some familiarity with the topic. If not, there is a lot out there to read. Myself, I like Burning Monk’s rule of thumb for when to choose Choreography or Orchestration (https://theburningmonk.com/2020/08/choreography-vs-orchestration-in-the-land-of-serverless/), and Kislay Verma’s writeup of orchestration and workflow pattern (https://kislayverma.com/software-architecture/architecture-pattern-orchestration-via-workflows/).
Orchestration can, of course, also be done using AWS Step functions, Azure Durable functions, or with 3rd party managed services like Camunda or Temporal.io. So there are options to a homegrown variant as I’m describing here, but where is the fun in that? Besides, those options can be very costly, or lock you into the major cloud platforms. And, not least, one clear result of doing this PoC, is that I suspect that using XState in this context, may not always be that much different from using one of the above alternatives (see for example https://aws.amazon.com/blogs/compute/building-a-serverless-distributed-application-using-a-saga-orchestration-pattern/).
Caveat
This is a Proof of Concept so I haven’t bothered with data validation, error handling etc that is needed for a robust implementation. Neither is the responses proper API-responses, but just the raw data.
Use case and setup
The hypothetical use case is the fruit based pizza shop Tropical Pizzas made famous in an earlier prototype of how to do easy micro-frontends 😉(https://itnext.io/prototyping-micro-frontends-d03397c5f770). Those micro-frontends was a huge success, so now the backend also needs to be modernized to keep up the increasing demands of the business. And that, of course, means serverless, message broking and all the hot stuff 😁
So you can order a pizza with pineapple, bananas, everything juicy and nice, and get it delivered to the safety of your home. The basic PoC architecture is something like the diagram above: An orchestrating service (Order) managing the ‘shopping cart’ and the related services (Kitchen, Delivery) that’s part of the flow of getting a great pizza experience. Also, some basic logging for observability. The client in the PoC is just using Postman or similar, Order log, Kitchen and Delivery are stubs just confirm and move the flow of things. So I will focus on Order and Message Broker.
The Order orchestrator consists of:
- A serverless function (Cloudflare Worker) that translates a request from the Client or the Message Broker into an event that may trigger a transition in the statemachine.
- A statemachine that acts on the event and may trigger actions and messages based on transitions and state changes.
- And a Durable Object that is used to persist state and workflowdata in between invocations
You create an pizza order by posting the first pizza to a basic endpoint (I use itty router):
router.post('/', auth, async (request, env) => {
const eventRequest = new Request('ADD_PIZZA', {
body: request.body,
method: request.method,
});
const orderId = env.ORDER.newUniqueId();
const order = env.ORDER.get(orderId);
const response = await order.fetch(eventRequest);
return response;
});Afterwards pizzas can be added to the order by referencing the created order-id:
router.post('/:id', auth, checkValidOrderId, async (request, env) => {
const orderId = env.ORDER.idFromString(request.params.id);
const order = env.ORDER.get(orderId);
const eventRequest = new Request('ADD_PIZZA', {
body: request.body,
method: 'POST',
});
const response = await order.fetch(eventRequest);
return response;
});env.ORDER is a binding to a Durable Object namespace that is configured to use an uploaded Javascript Class. From that I get a stub conforming to that class. So the route checks the requests, translates it into an event-request suitable for the statemachine, and forwards that to the Durable Object instance that also holds the statemachine for the order. See https://developers.cloudflare.com/workers/runtime-apis/durable-objects.
XState has a concept of interpreters that interpret and manages statemachines in form of a service that handles events, transitions, actions, etc. In the PoC, the interpreted statemachine is the orderService below, based on a custom interpreter I made that was more suitable in a serverless scenario than the standard one that comes with the library. So when the Durable Object gets the ADD_PIZZA event, it goes through something like this:
//Process request and get the event type
const type = new URL(request.url).pathname.substr(1);//Get data from event/request
const data = await request.json();//Process the transition and run associated actions based on the event type and -data
const newState = await orderService.send({ type, data });//Update statemachine context with some workflow data
const newContext = { ...newState.context, nextEvents: newState.nextEvents, currentState: newState.value };//Store it in the storage of the Durable Object
await storage.put(newContext);//Create response based on the updated context
const strContext = JSON.stringify(newContext)return new Response(strContext, {
status: 201,
headers: {
"content-type": "application/json;charset=UTF-8"
}
});
With statemachines it is useful to distinguish between discret states and continuous state. For example: With a traffic light, red, yellow, green will be the discrete states of the traffic light, while how many times the light switches during a day, could be part of the continuous state of the traffic light. In XState the continuous state is handled as the context of the statemachine, which always will be available for actions etc. So it is this context that is persisted in the Durable Object and holds the data for the pizza-order in question.
So the magic happens within the orderService that receives an event and returns a state with updated context according to the statemachine. The statemachine in the PoC can be visualized with the following statechart that is directly generated from the code:
The statechart has a state for orderPizza with different transitions, like ADD_PIZZA:
states: {
orderPizza: {
entry: ['updateStatus'],
meta: {
message: 'Oh, just ordering some pizza...',
},
on: {
ADD_PIZZA: {
target: 'orderPizza',
actions: ['addPizzaToItems', 'log'],
},
REMOVE_PIZZA: {
target: 'orderPizza',
actions: ['removePizzaFromItems', 'log'],
},
CONFIRM_PIZZA: {
target: 'bakePizza',
actions: ['log'],
},
},
},Actions can be invoked on entry (updateStatus) or as part of the transition (addPizzaToItems, log). The target of the ADD_PIZZA transition is the same state. Even if one doesn’t change state, I find it useful to handle changes as transitions instead of direct CRUD-like manipulations, since it lets me model and encapsulate the changes that belongs together and is relevant for the different states. addPizzaToItems will for example only be called when the state is orderPizza.
So the ADD_PIZZA event triggers the addPizzaToItems action:
addPizzaToItems: assign({
items: (context, event) => context.items.concat(event.data.pizza),
}),The assign-method is an XState action-creator that lets you manipulate the context. Here I just add the pizza to the items-array of the order.
When the customer is statisfied with her selection of pizzas, she will confirm that and trigger a CONFIRM_PIZZA event, and that again willl trigger a transition to the bakePizza state:
bakePizza: {
initial: 'sendingOrderToKitchen',
states: {
sendingOrderToKitchen: {
entry: ['updateStatus'],
meta: {
message: 'Sending the order for the pizza...',
},
invoke: {
id: 'sendOrderToKitchen',
src: 'orderPizzaFromKitchen',
onDone: {
target: 'waitingForPizzaToBeBaked',
actions: ['setSendOrderStatus', 'log']
},
onError: {
target: 'errorInKitchen',
actions: ['setErrorStatus', 'log'],
},
}
},
waitingForPizzaToBeBaked: {
entry: ['updateStatus'],
meta: {
message: 'Waiting for the pizza to be baked...',
},
},
errorInKitchen: {
entry: ['updateStatus'],
meta: {
message: 'Ooops! Something got burned...',
},
},
},
on: {
PIZZA_READY: {
target: 'deliverPizza',
actions: ['log'],
},
},
},
This state is more complex with three nested substates:
sendingOrderToKitchen: The default one for sending the order to the kitchen by invoking what XState defines as a invocable service. If the invoked sendOrderToKitchen is successfull, then the statemachine transitions to
waitingForPizzaToBeBaked : Where it just waits for the PIZZA_READY event. But if sendOrderToKitchen is not successfull, the statemachine transitions to
errorInKitchen: Which is an error state that can be starting point for error handling — which is not implemented as part of this PoC.
Message/event brokering
sendOrderToKitchen and the subsequent PIZZA_READY event is tied to the message brokering part of the PoC. Sometimes you will see the distinction between Orchestration and Choreography hinged on that the first depends on synchroneous calls, while the other depends on events/messaging. But more precisely the first will usually be based on a request/respons pattern (MAKE_PIZZA, PIZZA_READY) that can be implemented either synchroneously, using for example a REST-API ( I realize that a REST call strictly speaking is asynchroneous, but in this setting it would block the flow until it gets a response and act synchroneously), or asynchroneously as here, using message brokering. Benefit of doing this asynchronously is that it opens up for long running workflows and workflows with manual steps. But, of course, this can be mixed if that helps.
The message brokering in the PoC is based on RabbitMC (CloudAMPQ). As it is, the PoC is dependent on sending messages using the REST-API for RabbitMQ, and in the same way dependent on webhooks for receiving messages from RabbitMQ. In the PoC, the code for the method linked to the sendOrderToKitchen-service is defined like this:
const orderPizzaFromKitchen = async (context, event) => {
const mqEvent = createMQEvent(mqTypes.MAKE_PIZZA, context.orderId, context.items, true);
return await postEvent(mqEvent, env)
}The createMQEvent helper function creates a properly formated message based on data from the statemachine context (I have based that on https://cloudevents.io/). The MAKE_PIZZA type ensures it gets to the proper queue after being posted to the exchange-endpoint for my RabbitMQ instance on CloudAMPQ. A webhook for that queue will then delegate it to a suitable service (Kitchen in the PoC-diagram). If this goes OK, then the statemachine transitions to waitingForPizzaToBeBaked.
In the PoC, the Kitchen service is just a very simple serverless that lets me store the event in a Durable Object. It also provides an endpoint that lets me check if I have some orders in process, and then emits a PIZZA_READY event for the next order in line. I then have another webhook on my RabbitMQ instance for the PIZZA_READY queue that hits the event-endpoint on the Order orchestrator:
router.post('/event', mqCheck, async (request, env) => {
const message = await request.json();
const orderid = env.ORDER.idFromString(message.subject);
const order = env.ORDER.get(orderid);
const data = JSON.stringify(message.data); const eventRequest = new Request(`${message.type}`, {
body: data,
method: 'POST',
})
const response = await order.fetch(eventRequest);
return response;
});
This endpoints translates the event from RabbitMQ into something that can be interpreted by the statemachine on the Durable Object. With a PIZZA_READY event it will now transition into the deliverPizza state that is set up in the same way as bakePizza, and which will trigger a similar flow. It sends a request for DELIVER_PIZZA, and when it later receives PIZZA_DELIVERED, it transitions to done and the pizza order is completed.
Connecting with the client using a websocket
At any time of the workflow, a client can poll the order status from the Order orchestrator. In the PoC it just serializes the current context:
if (type === 'GET_ORDER') {
return new Response(JSON.stringify(currentContext), {
status: 201,
headers: {
"Content-Type": "application/json;charset=UTF-8",
}
});
}But since both Cloudflare Workers and Postman now supports websockets, I thought I try to add and test that as well. This means letting the Durable Object handle a special event that registers a socket on the Object and returns a socket client:
if (type.indexOf('websocket') !== -1) {//Request is for websocket
const [client, server] = Object.values(new WebSocketPair())
server.accept()
this.sockets.push(server);
return new Response(null, {
status: 101,
webSocket: client
})
}
So before returning a regular response, the Durable Object can then broadcast the same data to websocket clients:
//Create response based on the updated context
const strContext = JSON.stringify(newContext)//Broadcast state changes on registered sockets
this.sockets = this.sockets.filter(socket => {
try {
socket.send(strContext);
return true;
} catch (err) {
// Whoops, this connection is dead. Remove it from the list.
return false;
}
});return new Response(strContext, {
status: 201,
headers: {
"content-type": "application/json;charset=UTF-8"
}
});
This means that when the Message Broker hits the event-endpoint and trigger a transition/state change, this change will immediately be broadcasted to listening clients — which is very cool!
Concluding remarks
Using XState to orchestrate on serverless has mostly been a positive experience. It is not optimized for that, but still has a lot of relevant features and the setup was pretty straight forward. It has a flexible way to balance config and code, and you can easily get the statechart visualized. I had to make a custom interpreter to make it easier to use await/asynch with the actions and services, so I hope that can be part of future iterations of XState. But that was not a real problem neither, with the more ‘low level’ API of XState available.
I didn’t demonstrate a Saga pattern here, but using XState to handle a Saga should not be difficult. At least not more difficult than using the alternatives. A statechart can be become very complex fast, but this is by nature and not necessarily due to implementation details.
Having to be dependent on REST-API and Webhooks for message brokering may exclude some use cases based on performance. It might be possible to set up something using websockets instead, so that could be a future experiment. But having a ‘native’ way to integrate serverless with messaging and queues is/will be a huge selling point for the platforms that can offer that.
