How we implemented the Incognia API Gateway
At Incognia, our product engineering team takes advantage of a microservice-based architecture and therefore develops dozens of APIs that collectively compose our Incognia Platform. Every time someone accesses our platform, what they see is the result of numerous requests directed to those different APIs, each responsible for a specific piece of information.
For this to be feasible, we implemented an architectural pattern called API Gateway, and in this article, we are going to talk a little bit about this pattern and our journey to achieve the ideal API Gateway solution to our case.
API Gateway
This architectural pattern consists of a component that acts as a single entryway to your services: every single request passes through it and then is routed to the correct service.
In the context of microservices, an API gateway provides all of your services with a common set of features, such as authentication and observability, while abstracting away details that don’t need to be visible to your clients, such as the location of each service.
Our first API Gateway implementation
Back in 2019, we decided to do our first API Gateway implementation with two components that were widely used and that our team had some prior experience with AWS’ Application Load Balancer (ALB) and Nginx.
ALB was valuable for providing both authentication and TLS termination. We were able to implement authentication simply by using ALB’s Open ID Connect capabilities, as presented in AWS re:invent 2018 by Netflix.
Nginx was required to proxy requests to our frontend and backend servers (that were spread across different Kubernetes clusters) and to make request header transformations (remove, add, and edit headers to requests).
There were many problems with this approach, and some of them were deeply related to Nginx operational processes, like observability and integration, which will be explained in more detail in the next sections. Furthermore, in our configuration, Nginx presented some annoying IP cache problems.
Besides that, ALB’s authentication also presented some operational and security issues, like ALB session cookies being stateless, making it difficult to revoke a session, and logout not invalidating the user’s session (cookie still valid after logout).
Given these pain points, we decided to spend some time researching the best alternatives for delivering a secure and flexible authentication approach that had a more high-performant and maintainable proxy solution.
Improving our first solution
Let’s take a look at how we replaced ALB and Nginx and how it affected our system.
Replacing Nginx
As we searched through popular proxies in the market (HAProxy, Envoy, Traefik) to replace Nginx, Envoy caught our attention. We liked that it is not owned by any commercial entity and at the time, other services were using it with great results, so leveraging it again would enable us to build more expertise on it.
Let’s take a look at how Envoy helped us solve some of the pain points Nginx presented.
Observability
Envoy exposes thousands of metrics in Prometheus format on the /stats/prometheus endpoint. It was a huge difference from our default Nginx monitoring, which exposed only 7 metrics.
By using Envoy, we were able to monitor each request and response properties (latency, status_code, etc) per each service, something crucial to an API Gateway’s observability.
Envoy also provides an Admin API with valuable endpoints. By calling /logging?level=debug we were able to get more log information about proxied requests and responses and this was vital to investigating some route to service mapping errors we had during the implementation of this new solution.
Integration
Envoy’s configuration can be either static or dynamic, the latter by using gRPC/REST-based APIs that provide configurations, collectively known as xDS. As we have complex proxy settings and an ephemeral environment (Kubernetes), the dynamic configuration would be very handy and also a huge integration improvement from our default static Nginx configuration.
Additionally, by using dynamic configuration, we would be able to avoid unpleasant things like hot restart or disruption to traffic when updating Envoy’s configuration. That said, the best alternative for us at the time was to use open source solutions that could help us get started developing with Envoy’s xDS.
That’s where we found Gloo. Gloo is a “control plane” for Envoy (which is a data plane) or simply a component that provides us plug and play xDS APIs and Custom Resource Definitions (CRDs) for our Kubernetes environment, allowing us to define and update Envoy’s configuration in a very simple and more readable manner.
Gloo vs Ambassador
Before deciding to use Gloo, we had to decide between Gloo and Ambassador, another famous control plane for Envoy and Kubernetes. We chose Gloo because it is more secure given that the control plane is separated from the data plane.
Replacing ALB
Envoy provides a filter called External Authorization. For each request, Envoy consults this filter to check if the request is authorized, and if it’s not, it is denied with a 403 (Forbidden) response.
It’s possible to configure this filter for authentication purposes. By using this functionality, we were able to ditch our old ALB authentication solution and develop our own external authorization filter.
This gave us more control over users’ sessions, the security problems we had with ALB (stateless session cookies and logout cookies not being invalidated) were no longer a problem. Furthermore, as authentication configuration became much more flexible, we were able to get rid of some of ALB’s limitations, including the inability to maintain a single session across subdomains and to individually choose which route to authenticate.
Nevertheless, our current API Gateway still has an ALB but it is for the purpose of enabling TLS termination. In the future, we plan to pass this responsibility on to Envoy as well.
Conclusion
The need for an API Gateway to make our microservice architecture viable led us to start implementing it with AWS’ Application Load Balancer and Nginx, which did the job for a long time but presented many problems and limitations. By using Envoy and Gloo as an API Gateway, we were able to implement a solution that made our system more robust and secure. We also managed to increase observability and improve our integration process, as Gloo made it easy to use Envoy’s xDS, ideal for an API Gateway placed on a Kubernetes ephemeral environment.
