Don’t Confuse Events With Responses — Microservice Design

“The result of an Event Storming session” (video game “Control”)

A health enthusiast died from the overdose of carrot juice. And you can kill your application design with the excessive use of Event-Driven Architecture in less than 5 minutes. Especially if you use the Event Storming technique to model your application, as you may think that there is nothing else but events and commands.

Let’s Model

Modeling an online shop is a typical example that is easy to understand. We are not going to model everything but focus on a simplified order placement flow. We will use the Event Storming to describe the workflow and don’t worry if you are not familiar, it’s quite simple — “commands” convey an intent to perform an operation which may succeed or fail, and “events” state the fact of an operation result (either succeeded or failed). Commands are executed as a reaction to events.

Simplified workflow of order placement.

Then we put boundaries for commands and events to constitute (micro-) services, which will play a crucial role in understanding the problem.

Service boundaries for modeled commands and events.

Now it must be a straightforward implementation of one-to-one mapping of depicted components to the code — we have three commands and three events. Wrong.

Naïve Implementation

For the clarity’s sake, the following code examples are written in a pseudo-language and the irrelevant code is omitted with the ellipsis (…).

The Order Service:

HandleCommand(PlaceOrderCommand command) {
  ...
  PublishEvent(new OrderPlacedEvent(...));
}

The Payment Service:

HandleEvent(OrderPlacedEvent event) {
  ...
  SendCommand(new MakePaymentCommand(...));
}
HandleCommand(MakePaymentCommand command) {
  ...
  PublishEvent(new PaymentSucceededEvent(...));
}

The Shipment Service:

HandleEvent(PaymentSucceededEvent event) {
  ...
  SendCommand(new ShipGoodsCommand(...));
}
HandleCommand(ShipGoodsCommand command) {
  ...
  PublishEvent(new GoodsShippedEvent(...));
}

This implementation follows the choreography model, where every service reacts to events without sending a direct command to another service. You can think of it as if services were humans saying: “I’m done with my part! Whoever knows what to do next, please proceed.” So far so good.

Now, let’s amend the workflow to represent reality better. Goods may not be available due to limited quantity and the concurrent nature of purchase requests. In this case, we want to void the payment transaction.

Added “out-of-stock” scenario.

Following the same pattern as shown before, we need to change the code in two services — shipment and payment.

The Shipment Service:

HandleCommand(ShipGoodsCommand command) {
  ...
  if (outOfStock) {
    PublishEvent(new GoodsOutOfStockEvent(...));
  } else {
    PublishEvent(new GoodsShippedEvent(...));
  }
}

The Payment Service:

HandleEvent(GoodsOutOfStockEvent event) {
  ...
  SendCommand(new VoidPaymentCommand(...));
}
HandleCommand(VoidPaymentCommand command) {
  ...
  PublishEvent(new PaymentVoidedEvent(...));
}

Congratulations! We just made the workflow hard to understand, but what even worse, we introduced cyclic dependencies! And this example doesn’t include all possible scenarios, like a payment failure.

Problem Explained

From the code demonstrated above, we can build a component diagram — contracts, services, and their dependencies. Each service implements its contract, however, in addition, services reference each other’s contracts.

Circular service dependencies.

It means that these services subscribe to events of each other, which creates a strong logical dependency between them. This situation is worse than having a monolith composed of these two services. The challenges of service dependencies are explained in “You Get Microservice Decoupling Wrong!”.

Now let’s put together all the pseudo-code above in one listing. Imagine that you look at this code for the first time, can you tell what it does?

HandleCommand(PlaceOrderCommand command) {
  ...
  PublishEvent(new OrderPlacedEvent(...));
}
HandleEvent(OrderPlacedEvent event) {
  ...
  SendCommand(new MakePaymentCommand(...));
}
HandleCommand(MakePaymentCommand command) {
  ...
  PublishEvent(new PaymentSucceededEvent(...));
}
HandleEvent(GoodsOutOfStockEvent event) {
  ...
  SendCommand(new VoidPaymentCommand(...));
}
HandleCommand(VoidPaymentCommand command) {
  ...
  PublishEvent(new PaymentVoidedEvent(...));
}
HandleEvent(PaymentSucceededEvent event) {
  ...
  SendCommand(new ShipGoodsCommand(...));
}
HandleCommand(ShipGoodsCommand command) {
  ...
  if (outOfStock) {
    PublishEvent(new GoodsOutOfStockEvent(...));
  } else {
    PublishEvent(new GoodsShippedEvent(...));
  }
}

Even knowing the flow, looking understanding this code is much harder than looking at a diagram. Keep in mind that this code does not represent the full flow and is spread across multiple services. Making changes tends to be extremely difficult with this approach.

When-Then

If we forget about multiple services for a moment, how would you simulate the order placement flow in a console app? Perhaps like this:

PlaceOrder(...)
{
  paymentService.MakePayment(...);
  try
  {
    shipmentService.ShipGoods(...);
  }
  catch (OutOfStock)
  {
    paymentService.VoidPayment(...);
  }
}

The try-catch block can be replaced with the error code check for the programming languages that don’t support exceptions.

This code represents exactly the same flow but uses the orchestration model, where the orchestrator controls and directs the execution of the workflow. It is much more readable, easy to understand, and no circular dependencies.

The main difference in this concise form is the absence of the Observer Pattern — no events, no publishers, no subscribers, merely function calls. Yet, you can read the code using the “When-Then” words: “When an order is placed, then make a payment. When payment is succeeded (no exception; no error), then ship goods.” And so on.

Responses vs. Events

The events are replaced with results of function calls in the concise orchestration example. Unlike events, a result of a function call is not “published” to the entire program, but directed to the caller instead. A sub-function does not “know” what a super function calls it — the caller information is stored in stack memory.

Now let’s get back to a multi-service application. The orchestration example can stay as is if we call services with the synchronous communication (REST/gRPC). However, for mission-critical applications, the synchronous calls are replaced with asynchronous ones with the help of a message broker (like RabbitMQ). This is where things become ugly again — you have to decompose a single function into several small command and response handlers because there is no programming language or a framework (except D-ASYNC) that guarantees a stateful method execution. In this case, similarly to a regular method call, a command is represented as a message on a queue with a reply address — a service that invoked a command. A command result becomes a response — a message put back on the caller’s queue.

Commands and Responses.

A response is a special case of an event, which delivered to only one subscriber — the invoker of a command.

Regarding service dependencies, the Order Service depends on the Shipment Service only. The response is sent to the caller — a service — which routing info (like a name) is stored in the command’s message. And the contract (both ShipGoodsCommand and ShipGoodsResponse) is defined in one place — on the Shipment Service side.

Responses + Events

There is no rule saying that you need to choose between responses or events, orchestration or choreography. You can combine both at the same time, which can be very useful. Here is a code and a diagram example:

HandleCommand(ShipGoodsCommand command) {
  ...
  SendResponse(new ShipGoodsResponse(...));
  PublishEvent(new GoodsShippedEvent(...));
}

Responses and Events.

With such an amendment, the SMS Notification Service depends on the Shipment Service, and not the other way around. Otherwise, the Shipment Service has to send a “Notify Customer” command to the SMS Notification Service, which creates a whole list of the app design problems. I.e. events are not bad and crucial for many scenarios. In fact, you can build an entire app based on cloud platform events, where you don’t have control over built-in infrastructure services.

Things to Remember

1. Don’t overuse events, look at the whole system and decide where to invert service dependencies with events instead of commands+responses.
2. The choreography model can easily lead to circular dependencies and convoluted workflows.
3. Choreography and orchestration models (events and commands + responses) are not mutually exclusive. Use both where applicable.
4. Don’t confuse events with responses, especially when modeling with Event Storming.

Effortless workflows with dasync.io