Async systems API — Event-driven architecture
And a few ways to adapt it to sync interfaces
Event-driven architecture is getting popularity last years and many questions arise. One of the biggest questions we had when we started working with this architecture was how we can communicate async systems with other systems or with frontends.
Allow other systems to use our message brokers cluster was excluded as a solution mainly for two reasons: it could be insecure and the system clients would be coupled to the tech selected to use as message broker.
The solution is use HTTP endpoints as the system external interface. The HTTP APIs can be secured with a bunch of protocols and tools developed over the last decades and HTTP is also a standardized communication option. But this APIs cannot be like the sync systems ones, it is important bear this in mind:
- The API should contains the async nature of the system. This means, for instance, that in tasks that are started and then updated with the state of the task, the API should have two endpoints: one to start the task and other to fetch the state of the task.
- It should exist an async way to call the calling system. For instance, in the case above, when the task is over we have to call the system which starts the task. We can do this with webhooks: in the start task request, the external system gives our system a URL to be called when the task is over.
The points above allow external systems to be async as well and to capitalize the performance of our async system design.
When the calling system is a browser
One of the external systems could be a frontend. In this case, the calls are made by the browser. The problem with this is that browser cannot be called, so webhooks cannot be used.
To solve this problem, and taking advantage of the Backends for Frontends pattern, we could transform from using webhooks to using websockets in the BFF service. The async API is called by the BFF with the webhook URL. Messages are sent through the websocket each time the BFF webhook is called.
Adapters — When our interface cannot change
But everything is not a greenfield in software engineering and sometimes there are interfaces that we cannot change. Even more, there are interfaces endpoints that we cannot split in two.
Imagine the previous example: an endpoint to start the task, an endpoint to get the status of the task and a webhook to be called when the task is over. Imagine that one of the providers calls an endpoint with some request to start the task and that it expects the outcome of the task in the response of the call. What if the provider cannot be changed?
It is a good practice to expose the system according to the bounded context, with the data and the format of the system domain. If this exposed API has to be transformed to be consumed by a provider, an adapter service does the work.
But the problem is not only a problem of different formats or parameters names, but a problem of different paradigms. The adapter becomes a bridge. The bridge must be developed with reactive programming in order to avoid blocking so many threads. When a request reach the bridge, it performs the following actions:
- It generates an ID, it creates a Future and it saves the (key, Future) in a map.
- It starts the task by calling the system. In the request, a webhook URL is added. The webhook URL points to a bridge endpoint with the previously generated ID as param (path or query param).
- The future is returned with a timeout set. The response will be sent back when the Future will be resolved.
- In the webhook endpoint: the Future is get from the map using the ID given as param. Then, the Future is resolved with the body of the webhook request.
- Important to notice: the IP used in the webhook URL is the private IP of the bridge. If there is more than one bridge, it is important to use the IP of the bridge itself and not the IP of a service balancer.
