Tide’s API Gateway Journey

In this article I will share my experience analysing a number of different API gateways solutions, for use at Tide.

I will go over the rationale of why an API gateway is an indispensable component of any micro-service architecture, the challenges that come with using an API gateway and finally, I will cover a comparison of three different API gateway products.

The need for an independent API gateway

The pros and cons of micro-service vs monolithic architectures is beyond the scope of this article. What I would say however, is that micro-service architecture has gained a lot of traction in recent years. More and more companies are starting to realise that in order to scale their enterprises, they need to decompose their existing monoliths in smaller, independently deployable services.

This migration from monolith to micro-services does not happen overnight, it takes time to decompose the monolith into meaningful services. When transitioning from a monolithic to micro-service architecture, one of the biggest quick wins is to introduce an API gateway. Otherwise, you can end up with a tangled mess of requests between clients and micro-services.

Spaghetti calls to micro-services

There are a number benefits to API gateways, other than simplified client-server communication:

• Security: With an API gateway there is a single entry point into the system. The gateway can also manage authentication and authorisation concerns. This results in a reduced attack surface area for hackers to target.
• API Composition: Clients might often need to join data from 2 or more responses. API gateways can perform this aggregation, closer to the services, which in turn simplifies client code further and also reduces the number of round trips clients have to make.
• Shared Concerns: It is not only auth concerns that can be handled in the gateway. Data analytics, request/response logging and caching can all be centralised in the same manner.
• Decoupled Clients and Micro-services: Clients and micro-services are loosely coupled when an API gateway is introduced. This means that both client and service code can develop at an independent rate. In cases where code changes overlap, the gateway can transform requests and responses to mediate integration risk.
• Enables Decomposition of Monoliths: A gateway can provide more flexibility when implementing the strangler pattern, and can make the transition towards a fully micro-service based architecture much quicker

Aside from these architectural improvements there are some other business benefits to using an API gateway. For example, rate limiting and quotas can be tied to billing customers, based on their usage of services.

What can happen when migrating from a Monolithic to Micro-service architecture

During Tide’s transition path from monolith to micro-services we introduced an API gateway as part of the monolith in order to quickly demonstrate value to the business of a micro-service architecture. This gateway enabled external clients to interact with micro-services and lent to the strangulation of the monolith.

Temporary gateway solution

Why this is not enough

Although introducing a gateway in the monolith did provide some immediate benefit, it is not a viable long term solution for a number of reasons:

• Gateway security model is coupled to the monolith’s security model: Security concerns are strongly coupled. This is because the gateway will need to support the existing authentication and authorisation mechanisms in the monolith.
• Gateway is unlikely to be able support full feature set of an independent API gateway: Commercial and open source gateway product offerings have been around for a while and they support extensive feature sets. It would be difficult (and ill-advised) to try to replicate all those features from scratch.
• Cannot deploy new API routes independently of other code changes: With any company large enough, code changes become difficult to coordinate. It becomes likely that new deployments will contain a mixture of code merges from multiple teams. If the gateway is part of the monolith then it suffers from the same issues as the rest of the monolith code base.
• Unlikely to be able deploy a new API route without deploying the monolith: In order to avoid deploying the monolith, the gateway would need to be able to support adding new routes at runtime. Although commercial gateways do support this feature, again it would not be easy to code it from scratch.
• Cannot scale the gateway independently of the monolith: With an increase in traffic, the number of monolith instances would have to increase regardless of the request destination. For example, if there was increased demand for payments and there was an independent payment service, there would be no need to scale the monolith other than to support the additional load on the gateway.
• Cannot optimise the gateway independently of the monolith: Different services use resources differently. For example, some require additional compute resources and some require additional memory capacity. By coupling the gateway to the monolith we cannot performance tune either to the best of their ability.

As you can imagine we want to minimise the length of time we are using this interim solution so we decided to conduct a series of PoCs, working with different API gateway products, with the aim of picking one for eventual production release.

API gateway PoCs

Comparison criteria

• Open source vs proprietary: Open source technologies lend themselves to extension. Proprietary solutions come with enterprise support. There is also the grey space where the core offering is open source but certain features are provided in the enterprise solution.
• SaaS vs cloud hosted: Do we want to host the gateway ourselves? Can the provider host the gateway? Is there a significant difference in cost?
• Complexity: There is deployment complexity e.g. number of nodes or number of different databases. There is also code complexity e.g. is the code written in a known language with known frameworks?
• Feature set: Does the gateway support basic features e.g. rate limiting, circuit breakers or payload transformation. Are more rich features such as a developer portal supported?
• Community support: Is there an open source following for the underlying language/frameworks or the product itself?

PoC choice rationale

There are over twenty reputable gateway products available in the market to choose from. We only had a limited time frame to conduct our PoC analysis so we narrowed our efforts to analysing three gateway products — Spring Cloud Gateway, Tyk API Gateway and Gravitee API Gateway. We decided to choose these products for reasons disclosed below, but also all three are open source, can be self hosted and low in complexity.

Spring Cloud Gateway

• Closely aligned to our existing code base: Expertise in-house with Java and Spring Boot applications
• Configuration through code: All changes to the gateway would have to pass peer review
• High degree of control: Routes, predicates and filters can be extended easily

Tyk API Gateway

• Proven enterprise gateway: Already in use by other financial providers such as CapitalOne (https://tyk.io/why-tyk/case-studies/)
• Extensive documentation: Very well documented API platform
• Extensible plugin capability: Can create gRPC plugins which run in separate container/VM from gateway

Gravitee API Gateway

• Built in Java: Uses known frameworks such as Spring Core and Vertx
• Extensible plugin capability: Can create custom plugins which run in the same process as the gateway
• Compelling Identity and Access Management (IAM) product: Potential to consolidate tool providers

Spring Cloud Gateway

Pros

• Can reuse our existing libraries: A number of Tide libraries and configurations can be reused as they are also Spring based
• Gateway routing configuration enforced through code: One concern at Tide is proper version control of API definitions. Enforcing definition updates through code minimises chances of error. Enforcing all API definition changes through code would also work well from CI/CD perspective
• Easier to stay up-to-date with dependencies

Cons

• WebFlux orientated: Tide engineers have a large amount of experience with Spring MVC but much less in WebFlux — this framework is completely different as the underlying architecture is a Reactor architecture
• Poor performance: The framework seems to put too much workload on the event loop threads. This goes against a Reactor architecture. Even with changing the event loop thread count-Dreactor.netty.ioWorkerCount and ensuring that there was no blocking code, using blockhound, the performance did not change significantly

Tyk API Gateway

Pros

• Battle hardened: Tyk has shown in realistic bench marking tests to be capable of handling up to 4000 requests per second (https://tyk.io/why-tyk/benchmark/)
• Tyk Developer Portal: The developer portal is best of breed among open source API gateways.
• gRPC plugins: These allow gateway plugin resources to be independently managed of the gateway

Cons

• Written in Golang: There is very little experience of Golang at Tide
• gRPC plugins: Using gRPC on AWS has some major challenges. Without full HTTP/2 support on AWS Application Load Balancer, you have to spin up and manage your own load balancers. Neither NLB and ELB are viable alternatives either. See this article for more detail
• No request/response log obfuscation capability: Financial and GDPR regulations require companies to make reasonable attempts to prevent data leakage etc.

Gravitee API Gateway

Pros

• Complimentary IAM product: Built on top of their gateway product and would facilitate a smoother transition to a decoupled auth service
• Well understood technology stack: Gateway built in Java with core framework being Vertx. Data-store is flexible — can use MongoDB or MySQL
• Embedded plugins: Modularized plugins which can be embedded in the gateway on deployment allow removal of redundant plugins and reduced latency when compared to external gRPC plugins

Cons

• Not a well established product: Fewer customers than other providers such as Kong or AWS API Gateway. Whilst this is not a problem in itself, it does suggest adopting Gravitee API Gateway would pose a higher risk
• Basic user interface: The management user interface is not intuitive and design needs to be enhanced. However, in their v3 release there is a marked improvement in user experience

Performance

Having applied relevant customisation to each gateway as required basic performance tests were conducted against potential user journey’s of our members in the Tide app. For each performance test the following bench marking conditions were applied:

1. They all had the same memory — 2 GB
2. They all had the same CPU spec — 1 vCPU
3. There were all single instance
4. All tests run from home network through company VPN
5. All tests run out of hours against our test environment
6. All tests ran 10x at each stage — average results shown in results table
7. No caching of responses

Results

PoC performance results

As you can see we ran these tests at VERY low volumes of concurrent users. We just wanted to get a feel of the throughput that could be achieved in each gateway.

From these results you can see that the Spring Cloud Gateway performs worse at every different level of concurrent users. As the number of concurrent users increases the performance of Tyk and Gravitee API gateways converges. Gravitee is expected to perform worse at very volumes of requests when compared to Tyk because the first request made to a Java web application is substantially slower than the average response time. This warm-up period can usually be attributed to lazy class loading and just-in-time compilation.

Next steps

Based on the experience of working with each product and their relative performance, Tide has decided to begin the process of productionising its Gravitee API Gateway PoC.

Why?

• It is the only gateway that meets all existing and new functionality requirements
• It is built in a language, with frameworks, employees already know
• The custom plugin mechanism is flexible and in a future release Gravitee will also support remote plugins
• It meets performance criteria
• We intend to consolidate tool providers and use Gravitee for both API and IAM