Synchronise an async REST Api.

People often ask me “How do you do, on an aync architecture (like kafka), to “synchronise” a REST api ?”

So there are 3 possible ways (all can be cumulative, to work together) :

  • tell to your client the location where he can find the answer of his request, when it will be available
  • Have a webhook where you are responsible to push the information to the client.
  • Synchronise your api, in order that the client never know that behind it is asynchronous.

The 2 first are easy, so let’s see it quickly.

Send the location to your client.

So For you client request (create user for example) :

POST /api/users HTTP/1.1
Host: localhost:8000
Content-Type: application/json

{
"username": "Felipe",
"age" : 100
}

Your api should answer :

HTTP/1.1 202 Accepted
...
Location: /users/52f6cf53-d90e-4fa5-8a73-45c433f2c0a0

So of course when you create your event that will trigger the creation of the user, you should create a correlationID guid (that can become the user_id).

On golang the code is easy :

w.Header().Set("Location", fmt.Sprintf("/users/%v", id)) w.WriteHeader(http.StatusAccepted)

And your client will be responsible to synchronise his data.

Having a webhook

I will not dig to much on this point, just explain the idea, because the goods practices of a webhook, is too huge to explain quickly.

So if your api already answer 202 Accepted, you can also create a webhook.

The difficult part will be to be consistence in the answer. But the idea is when we have the response, push it to your client (of course you will need :

  • security
  • log everything
  • an admin section where the client could subscribe to your webhook, and verify the logs, and send tests data, to develop his endpoint
  • A good documentation

But you got the point that instead of be responsible to search the data, we push it to him. He will continue to be responsible to synchronise his data.

Synchronise your endpoint.

The idea that we will develop, is that your endpoint will listen for the answer of the event, and sent the data to the client synchronously (and if it is too long answer 202 + Location ;) )

For our example if we take the following flow (in orange are the events) :

The blue box (front-end) is responsible to trigger an event CreateUser::Requested and then wait listening for 2 possible events :

  • CreateUser::UserCreated (once front-end received it he can continue with hist stuffs and then answer a 200 status)
  • CreateUser::UserNotCreated (if front-end received it, he can eventually (depending how you handle your degraded mode) answer 500 status)

But there are a 3rd possibility : if for some reasons, we never get one of the answers event. On this case don’t panic (:D), you just have to implement a timeout process and follow the first flow and answer :

HTTP/1.1 202 Accepted
...
Location: /users/52f6cf53-d90e-4fa5-8a73-45c433f2c0a0

And when our issue will be solve, the system will process our CreateUser::Requested event, And the client will be able to have the information or going to the endpoint /users/52f6cf53-d90e-4fa5–8a73–45c433f2c0a0 or receiving the information, if you have implemented the webhook.

Conclusion

The best is to do, all the 3 techniques all together.

  • Try to be synchonous
  • If it is too long answer 202 + Location
  • Have a webhook