Scaling Airbnb’s Payment Platform
By Angela Zhu & Karen Kim, Airbnb Payments Engineering
When people think about Airbnb, they usually think of beautiful travel destinations, unique homes, and living like a local. Most people don’t think of Airbnb as a Payments company. However, similar to the manner in which PayPal was critical to eBay’s success as the first global online marketplace, payments at Airbnb is critical to Airbnb’s growth as a global platform for travel.
In this post we’ll explore the origins of Payments at Airbnb and the reasons why we’ve built such a large in-house payments operation. We’ll also take a look at some of the technology that has helped Airbnb scale to 191 countries, over 70 currencies, and a network of over two dozen payment routes.
Early Days
In 2008, when Airbnb started out, guests and hosts used the platform to find each other and not much else. Many of the trust features that Airbnb provides today, such as real identities, messaging between guests and hosts, and professional photography didn’t exist. Payments were handled offline directly between guest and host. As you can imagine, this didn’t always go smoothly.
Case in point: in early 2008, co-founder Brian Chesky traveled to Austin to attend SXSW, and naturally stayed in an Airbnb. When he arrived at his listing, Brian didn’t have any cash on him but he assured his host that he’d stop by an ATM that day while attending the event. However, in the excitement of SXSW he forgot about his plan, and returned home empty handed. The next morning, his host asked about the money and Brian had to again promise (sheepishly) to withdraw cash during the day. That evening though, it had once again slipped his mind. Things were starting to get pretty awkward between the two of them.
While Brian did eventually manage to get him the money, he realized there had to be a better way. Back in San Francisco at Airbnb HQ (aka the living room of his apartment), Brian declared that Airbnb had to start handling payments itself. This would eliminate the inconvenience and awkwardness of an in-person cash exchange and would also allow Airbnb to support both the guest and the host in case anything went wrong. With that, the beginning of Airbnb’s payments infrastructure was born.
Going Global
As the sole engineer at the time, Brian’s technical co-founder, Nate, was tasked with making payments happen on the platform. At the time, there weren’t many off-the-shelf solutions for handling bi-directional transfers, so he implemented the most straightforward solution available: guests paid via PayPal and hosts received a paper check in the mail.
The platform grew, and users began asking for an alternative to PayPal so Nate reached out to processors to set up partnerships that would enable Airbnb to accept credit cards. Additionally, Brian got tired of writing checks by hand, so Nate added support for automated bank transfers.
As usage spread outside the US, the limitations of this system became clear. Guests in Europe looking to book listings in the US would be charged in dollars and ended up paying an additional conversion fee. In some countries such as Germany, credit cards weren’t commonplace, and German guests couldn’t pay with their most trusted payment method. For hosts outside the US, receiving US dollars in a PayPal account wasn’t always convenient (or even possible), and US bank transfers didn’t work. This led the founders to their next big insight around payments — to be a truly global travel company, they had to build a global payments operation.
A good test came in the form of Brazil. Payment support in Brazil is extremely fragmented due to the presence of numerous competing payment networks, and most Brazilian credit cards do not work internationally due to government regulations. Many companies opt to skip support for local cards in the name of simplicity, but with our goal of creating belonging on the platform this definitely wasn’t an option. We wanted to make sure to add support for local credit cards as part of our expansion in Brazil.
In addition, cash-based mechanisms are very popular for handling online payments in Brazil (reference: How Airbnb Made Its Payments System More Accessible To Brazilians For The Rio Olympics). In order to support cash-based payment, we ended up introducing a new payment method in the form of Boletos. In this model, a guest can opt to book a reservation using a Boleto, which is an invoice for the amount owed to Airbnb. The guest can take the Boleto to their local bank or convenience store and pay the amount owed in cash or from his/her bank account directly. Once payment is confirmed by Airbnb, the reservation proceeds from a pending payment status into a confirmed state. Introducing Boletos to our platform not only presented us with an opportunity to support new users in Brazil, but also allowed us to develop a general-purpose pattern for asynchronous payments and booking which is now leveraged to enable new payment methods in many other countries.
As we expanded internationally, local needs like this kept coming up and our early infrastructure wasn’t always well-suited to the task of adding new processors and flows. Next, we’ll take a look at the history of our payments infrastructure and how it has evolved over time to meet a growing set of business needs.
> rails new airbedandbreakfast
In the beginning, payment integrations were introduced in an ad hoc fashion with little thought for code reuse. Since there was no real framework for integrating processors, each new integration had to be built from scratch which was extremely time consuming. Additionally, each integration added to the complexity of the system and introduced the potential for new bugs and issues.
Because we were using Rails, we relied on simple ActiveRecord models to access the database and its data; thus our core payment objects were mutable by default. Due to the way our low-level data model was exposed, we lacked a clear mechanism to track these changes from within our application and came to rely on DB-level triggers on our core payment tables to capture CREATE/ UPDATE/ DELETE actions in separate audit trail tables. Downstream, we had to reverse-engineer these audit entries to infer what happened in the production system and use elaborate SQL queries to process this data and generate the various financial reports that we needed.
This rudimentary system worked well for a while, but over time the money moving through our platform grew dramatically. The first real growing pains we encountered came from our reporting pipeline. Over time, it became increasingly painful to maintain those sprawling SQL queries, and the sheer volume of records began pushing the limits of our reporting database. New reporting requirements and product changes became increasingly expensive to support, and the overall runtime of our reporting process grew to many hours.
Payments, V2
Let’s review the biggest challenges we were facing at this point:
- The integration of new payment processors was time-consuming and continued to increase code complexity.
- The lack of clean interfaces around billing and payments made it difficult to introduce support for new products and features.
- Financial reporting was reverse-engineered from audit trails, which were error-prone and limited our ability to change the schema of our core DB tables.
To address these challenges, we introduced three new standalone systems, which represent the core components of a next-generation payments platform.
Billing Interface API
Historically, Airbnb Payments has handled the basic use case very well. The guest books and pays, the reservation is confirmed, and the host is paid out — everyone is happy. Over time, as we started adding more complexity to the transactions, these simple assumptions broke down. For example, when a reservation was altered, recalculating the price, fees, and taxes quickly became overwhelmingly complicated. We had difficulty associating additional financial actions with a reservation, such as resolution payments, security deposits, professional photography costs, and coupons. The addition of taxes, such as VAT in Europe, further complicated things.
Our new billing interface solves for all of these issues by taking each Airbnb product and treating each item (reservation nights, fees, taxes, etc) as discrete items. It supports a unified checkout flow, removes product details, and replaces them with abstract data models and attributes, such as pricing and taxes. By abstracting the interface for processing each product, we can easily integrate new products on our platform.
The checkout flow for various products becomes very similar: first we check product inventory, then we take the buyer through a standard checkout flow, and finally, we execute a fulfillment process. The final checkout result is represented using our normalized data model with elements such as fees, taxes, and payables all broken out. This data model serves as the source of truth for any subsequent changes to the purchase. Behind the scenes, the billing interface generates a series of platform events in a consistent schema which will be distributed to all interested consumers via our Kafka message bus.
This new billing interface allows us to easily create, edit, and remove products on the Airbnb platform. We’re optimistic that with this investment in our infrastructure, we’ll continue to increase trust in the Airbnb platform by increasing the transparency of each component of the transaction.
Payments Gateway & Transaction Patterns
In order to address the idiosyncrasies of cash payments, we developed clear reservation statuses as well as a timer so both parties knew who needs to take action and how much time was remaining for that stage. We also implemented a general-purpose pattern for asynchronous payments and bookings, which is now being leveraged to enable new payment methods in many other countries. We designed our payments gateway to provide a unified API that supports bi-directional transactions across all our processors. All payment transactions pass through this API layer and are recorded using an append-only data model. This system produces payment events with a separate well-defined schema, which are again distributed via Kafka. By using this unified API, it allows us to reuse payment models as we add new local payment and payout methods around the world.
By investing in our infrastructure to handle different payment transaction patterns, we can increase global support for many different payment methods. Because transactional behaviors vary greatly based on regions, this will considerably help our mission to enable both our guests and hosts to exchange money with ease.
Financial Data Pipeline
Our financial data pipeline system has been the biggest evolution in our payments ecosystem. In the last year, we built our second generation financial data processing system. It is a Spark + Scala based financial reporting application. It is built around the concept of discrete financial events with well-defined schemas that map to the booking, payment, and supplementary events that occur on our platform. This serves as the complete replacement for our SQL-based reporting system. This new system has proven to be easier to debug, maintain, and extend than the legacy system, and is designed to scale horizontally.
The financial system tracks financial accounts (such as receivables, payables, revenues, and taxes), and how funds are transferred in between these accounts. It does not depend on product details such as how reservations block a calendar, how users are notified about activity, or any other business logic. The financial pipeline receives platform events from the Billing Interface and payment events from the Payment Gateway, then converts these events into accounting information. We then construct unified reporting logic based on discrete events and filter it into different accounts, such as receivables, liabilities, revenue, etc. The financial data pipeline is also constructed around the principle of double-entry bookkeeping, which means money must be added somewhere and removed from somewhere else. In other words, money can’t be created out of thin air.
This new financial data pipeline now produces all of the data required for financial reporting to regulators, local governments, and even Airbnb’s investors. In a future blog post, we’ll dive a little deeper into this topic.
Looking to the Future
With these services complete, we have a solid foundation for our next-generation payments processing system. Looking further out, we can begin to build on top of this foundation by adding sophistication to our stack, leveraging new data flows, employing machine learning techniques, and overall, making the system more flexible and intelligent. Some of the things we’re planning:
- For many payment methods such as credit cards, we often have multiple choices of which combination of gateway, processor, and acquirer to use. These decisions can affect transaction risk, acceptance rates, and processing costs. By leveraging machine learning and consuming signals from both our production system and our processors, we can make dynamic decisions about how to route a transaction, optimizing for cost, acceptance, or speed.
- By introducing a stored value system behind our payment gateway, we can begin allowing users to maintain balances in the system. This could enable features like letting hosts spend their earnings on trips, and provide a unified system for tracking gift credit, referral credit, and more.
- We can also move away from batch-based financial report generation to a streaming ingestion model that allows us to provide accurate financials in near-real-time and catch issues and anomalies as they occur.
Payments at Airbnb has come a long way since that first awkward interaction between Brian and his host. As Airbnb has expanded around the world to 191 countries, the decision made back then to own payments and keep it as a core part of the platform experience continues to be the right one. Without that ownership, we wouldn’t be able to add the trust, belonging, and legitimacy that is required for a marketplace like Airbnb to work for so many millions of people.
If this sounds like an interesting challenge to you — and you’ve made it all the way to the end of this post — we’re hiring engineers, data scientists, and counselors.
Thanks to Brian Wey, Ian Logan, Lou Kosak, John Terenzio, Jiang-Ming Yang, Sharda Caro, and Marissa Coughlin for reading drafts and edits!
