Inventory Fetch Engine at redBus using throttling and actors.

Being an E-commerce marketplace the first problem facing is how to aggregate inventory(i.e buses in our case) and cache it. We at redBus fetch inventory from API providers(sellers), store it, and forward it to users(buyers) when they search. More details about the search platform here.

More dynamic the inventory or more frequent the inventory info changes problem becomes harder and harder. As an organization keeps growing importance of a stable inventory fetch engine becomes a necessity. Because more and more API providers(i.e sellers) come on board.

redBus, the largest bus ticket selling platform in the world, sells millions of bus tickets in a month. redBus manages millions of inventory/buses and many bus operators(travels) sell tickets on the redBus platform. redBus operates in six countries (India, Singapore, Malaysia, Indonesia, Peru, and Colombia). There are hundreds of API Providers integrated with redBus.

We at redBus come up with a modular/decoupled, actor(process) based system and fuse-based QoS to counter the problem.

Following is the underlying abstract system architecture.

Inventory Fetch Engine Architecture

Pre-fetch Engine

• The pre-fetch engine will analyze historic inventory data, sales data, and some days of future inventory data available.
• It will then assign simple ranks 0 or 1, 0 being high priority and 1 being least priority. Data with ranking will be stored in Ranking Database(i.e. RDBMS).
• Ranking logic will be based on the potential of sales, availability of inventory across past and future days.

Processing Engine

• The processing engine will read Config DB and Ranking DB. It will schedule(spawn) an actor based on the configuration of the API providers.
• We create a different process for each API Providers and store Pids for the same i.e API 1, API 2 ... API N. Each process is linked to the processing engine, in case of any failure processing engine can restart/pause/resume the process.
• The configuration contains the API provider’s API hit rate(number of requests per minute), time window, latency threshold, etc.
• Many API providers allocate a fixed time window to fetch inventory, post that time window fetch must stop.
• Engine restricts process API N run time beyond the specified time window.
• High rank(rank 0) data is refreshed for more business days and low rank(rank 1) data is refreshed for fewer business days. Low-rank data gets migrated to high rank based on changes in historic events by the pre-fetch engine.
• The processing engine maintains the QoS based on the performance of API provider. In case of low performance from the API provider, requests will be throttled. More details about QoS are followed in the next topic.
• The processing engine stores the data into Inventory DB(not mentioned in architecture). It will be used to power the user(buyer) search.
• Below is a sample for few API Providers.

API hits to different API providers

Quality of Service(QoS)

• The toughest challenge faced was how to maintain QoS in case there is latency or API outage at API providers during fetch(pull).
• We at Redbus implemented a throttling mechanism using the concept of fuse(aka circuit breaker).
• Each API provider has been pre-configured to certain conditions. Each API provider will have pre-configured max latency per request, T. Once X number of API hits getting latency of more than T in Y mins, the fuse will be blown and the circuit breaks.
• API hits to API provider will stop. One process will keep checking for response time if it comes under a configured threshold latency of T, the fuse will be reset and the circuit will be restored.
• API hits to API provider will resume from where it stopped.
• QoS is more important for API Providers as if we don’t stop API calls when there is latency or outage, the API Providers system can crash or go down and take time to recover. In any such API outage case, OTA loses the opportunity to make sales during API outage.
• Check the below sample from one of the API providers, API call stops at 11:26 and resumes at 12:04 once API latency is back to normal.

QoS for API provider

Benefits

• High availability of API from API providers.
• Increased Freshness of Inventory(bus).
• High availability of inventory(bus) across the future dates of journey.

References:

• J. Armstrong and R. Virding, Erlang — an experimental telephony programming language, in XIII International Switching Symposium, 1990, pp. 4348. doi: 10.1109/ISS.1990.765711. Ericsson AB. (1999). Erlang/OTP, [Online]. Available: http://www.erlang.org (visited on 07/23/2018).
• [Online]. Available https://erlang.org/doc/man/gen_server.html
• [Online]. Available https://github.com/jlouis/fuse