Handling API requests at scale using Meesho’s in-house Edge Proxy service
Both the app and web interface of Meesho is powered by multiple backend services — interdependent on one another and running 24x7 throughout the year. With millions of entrepreneurs transacting on Meesho every day, we have developed more scalable backend services to handle the complex features within the app. And Introducing a proxy layer in this ecosystem has provided a way for developers to easily access all the downstream backend services via the custom edge proxy APIs.
What is edge proxy?
Edge Proxy layer allows API calls from Meesho app to be authenticated and routed to the relevant backend microservices with minimal latency and zero coding effort.
At Meesho, the edge proxy layer consists of a collection of filters that help redirecting incoming requests based on its payload, as well as validating the responses before sending it back to the application.
Why do we need edge proxy?
- Serves as a common layer, where developers could proxy API calls to their respective backend services without writing any code.
- A single authentication and throttling layer for all app side API requests, which is scalable if more backend services are being introduced in the future.
- Edge proxy is developed with a circuit breaker that ensures fault tolerance in all services that are proxied.
Edge proxy explained with a use case
In our old architecture, the storefront service acted as an aggregator to route most of the API calls to downstream Meesho services. It also created data-transfer objects to serve to the app. For instance, whenever a user writes a product review on the app, the incoming API request has to go through the storefront to reach the backend review service. Developers had to write a good chunk of code into both the storefront and the review service to take the feature live. This indirectly slowed down our development timeline due to the additional boilerplate code efforts.
With the edge proxy layer in place, the API calls from the app to the backend review service can be redirected as proxy requests. Once we create a proxy for the ratings and review APIs, developers can further use this backend service directly in their projects without writing any boilerplate code.
Edge proxy filters allow us to authenticate and throttle (rate limit) all incoming API requests and validate responses as well. In addition to that, if the backend service is not equipped with a circuit breaker, we could utilize the one present within the edge proxy setup to make sure that the respective API gateway is fault-tolerant. Let’s now further take a look inside this edge proxy service and its associated filters.
Edge Proxy Architecture & Filters
Meesho’s edge proxy is developed as a Netty web container using Spring Cloud Gateway framework for processing the inbound/outbound API calls.
- Authentication Filter — The authentication filter supports multiple strategies to authenticate incoming API requests. For example, we use JWT-based authentication for users who are using Meesho without signing up. If the authentication fails at some point, the edge proxy will return the relevant error code.
- Service Security Filter — This filter allows us to add security credentials for the backend service that is being proxied. The authentication headers consists of a key-value pair of the respective credentials which will be passed along with the API request header to the backend services.
- Circuit Breaker — Developed using the Resilience4j open-source library, this filter ensures fault tolerance to all proxied APIs. The respective service owners will be able to set the request threshold limit for their APIs and the configuration contains a fallback mechanism that redirects failed requests back as 5xx error codes.
- Server Error Filter — Acts as an outbound filter and helps in masking the error code in the response payload.
- Metrics Filter — Used to gather all the requests and response metrics and push it into the respective monitoring platform such as Prometheus for tracking purposes. Micrometer metrics registry is a standard library that provides the necessary interface to segregate the metrics.
All the proxied API requests and responses use an HTTP client to transfer information between the other downstream backend services.
Advantages of having Edge Proxy
- Developers can now easily proxy their service API paths to communicate with their respective backend services, instead of solely depending on the storefront service.
- Edge proxy ensures that the downstream services are resilient by wrapping all API calls within a circuit breaker.
- After introducing edge proxy, we noticed a decrease in latency of the API calls, which improved the overall response time of the major app features.
Service Metrics
Gradually we are migrating most of the proxy calls from storefront to edge proxy. As we’re scaling the infrastructure, the current edge proxy setup is tested for handling up to ~ 3.2k QPS per EC2 instance (with a typical cluster setup of 2–5 nodes each and set to autoscale).
Edge proxy architecture was designed as a proactive measure and a long-term scalable solution to improve the app side experience for our Meesho entrepreneurs. Also, adhering to our Speed over Perfection motto, edge proxy helps our engineers develop and ship new features much faster. With millions of entrepreneurs growing their business with Meesho on a daily basis, edge proxy is one of the many projects that will help us achieve 10x growth in 2021.
This blog has been co-authored by Abhishek Pasi and Ashok Lingadurai.
How Meesho’s dedicated A/B Infra enables us to experiment and scale new features
Optimising elasticsearch cluster for reads
Interested in joining the Meesho rocketship? We are hiring!
