Scaling a huge API

Juan Manuel Formoso
2 min readFeb 19, 2015

--

When our API was born some years ago, we only exposed a few methods. Just enough to mimic our shopping flow for hotels and flights.

This, added to the fact that our internal APIs where very different among themselves, led us to the decision of creating an API layer above the internal ones, and that one team was going to be in charge of maintaining it.

API v1 architecture

This version received all public requests, transformed parameters into something App(i) could understand, called it, transformed the output into a semi standard, and wrote the response.

You can see how huge it became after a few years here http://api.despegar.com/v1/

As the number of services grew (because we wanted more products and support applications to be available) this architecture didn't scale. The team that was in charge of the public API slowly went crazier and crazier.

Capitalising on our existing virtual services infrastructure, we decided to distribute responsibility of the public API (as it was already distributed for the internal ones). We had the technology!

That's when version 3 was born (version 2 is a story for another time)

API v3 architecture

All public traffic is routed to the internal application directly, and it can handle by itself whether or not the request is valid and how to best serve it.

The initial version is also there for your browsing pleasure at https://api.despegar.com/v3/

This obviously needs an API standard that everybody is required to adhere to, a centralised service that provides all these apps with the list of valid api keys, which ones are banned (for misuse or simply because they exceeded their allotted quota), and some sort of automated process to validate your API for public consumption.

This is developed and enforced by the same crazy people that worked in our previous version of the API — everybody was already scared of them from before so it was the most natural thing to do.

--

--

Juan Manuel Formoso

Geek with a passion for technology, science fiction, and gaming.