How to optimise your front-end by reducing the chattiness to APIs
Rewriting Reg/Auth with the help of a Management API
About a year ago, the Apolitical engineering teams began rewriting the user registration and authorisation system on the platform. Or, as we internally called it, the “Reg/Auth re-architecture” project.
Apolitical is a global social learning network for government. By connecting public servants with knowledge, skills and connections, we aim to help them build a better world for citizens. There are around 200,000 people on Apolitical, and we only allow public servants to create accounts. This volume and focus mean that we need a user-friendly yet robust account creation and verification system.
The platform is designed to approve user registrations from trusted government domains automatically. Plus, it allows for manual vetting of registrations that don’t fall under the category.
So, why did we undertake this project?
Firstly, from a business perspective, we wanted to reduce the drop-off rate in the Reg/Auth process as well as the time spent by the customer support team helping users to complete it. Secondly, from a technical perspective, we wanted to improve the implementation of some critical aspects of the Reg/Auth lifecycle and the way data was stored and consumed.
TL;DR: A common problem is making multiple API calls that need to be sequenced and assembled on the front-end. If you are facing that, then, you can consider implementing a Management API to push the front-end logic to the back-end. With this simple but powerful concept, you will be able to optimise the processing with server-side parallelism, embrace observability, and reduce networking costs. And more importantly, make your users happier!
What a Management API did for us
When we started on this project, our microservice architecture meant that we had a very complex and technical jigsaw puzzle to solve. We wanted to improve the implementation of some critical aspects, but perhaps that’s a little too broad.
So, although there are many exciting aspects of this project, the focus here will be on exploring the benefits of a Management API: an API to manage other APIs.
As the Netflix API team highlighted in their blog, one of the motivations for implementing a Management API was to reduce the chatty nature of the client/server communication, which was slowing down the performance and making it harder to diagnose incidents.
To give an example, we can explore one of the first things that happen when a user signs up for the Apolitical platform. The user would input the email address in the front-end, and then some business logic would verify that the email address is valid. For the sake of simplicity, the following diagram illustrates a possible implementation of this validation process:
In Figure 1 shown above, there are three API calls, and each of them is made to different APIs: API 1, API 2, and API 3. It’s important to note that the sequence of the API calls is fixed. This means we need first to call API 1, then API 2, and lastly API 3. In addition to this sequence, the final validation logic gets executed in the front-end and will depend on the result of all of the API responses.
For the case of an email address validation, you may be wondering, why isn’t a single API doing all of that? Unfortunately, the complete answer to that question is outside of the scope of this blog post. But, as mentioned before, our platform implements a complex process with auto-approval and manual vetting of registrations. And, on top of that, our platform relies on some third-party services, such as Auth0 and HubSpot, and internal databases. Therefore, for our microservice architecture, it makes more sense to keep a good separation of concerns between APIs to have consistency in the way each API talks with each third-party service or database.
Now, the suggested improvement for the previous implementation would require collapsing those three requests into a single request optimised for the specific use case:
That way, the front-end pays the price of the networking latency only once and makes better use of the more powerful hardware of the back-end to compute the validation logic. Also, because the Management API is running in the same network as the other APIs, each call should be more efficient than if it was executed from the front-end. In this case, we are assuming that the API calls are HTTP-based but the Management API would allow the use of alternative, more reliable protocols, such as messaging or remote procedure invocations.
Another big advantage of moving the validation logic to the Management API is that implementing logging and monitoring on the back-end side is more robust than tracing events on the front-end. It also allows the implementation of retries, and other more advanced concepts, such as circuit breaking, that you may be able to get out-of-the-box with the use of a modern service mesh.
What we learnt
Despite not having the chance to flesh out the whole spectrum of the Reg/Auth re-architecture project, I can say that the project was a success!
The customer support team has stopped receiving as many user requests as before. And not only that, for most of the incidents that have happened since then, the engineering teams have been able to get full traces of where the issues were for 90% of the incidents. There’s a newfound confidence in the platform.
From a technical perspective, the incorporation of the Management API into our microservice architecture has helped us significantly improve the performance of our front-end. More importantly, it has brought a powerful layer of control over the most complex parts of our business logic across the platform.
