Case study of how airbnb used aws to grow and prosper

About Airbnb

Airbnb is the company that revolutionized the way we travel.

By being a community marketplace, they connect home owners and vagabonds for the sole purpose of renting unique and quirky flats, villas, treehouses and caves all around the world. Offering unique holiday spaces with the possibility of socialising, Airbnb are now outperforming hotels in many major cities across the globe.

They are currently offering property rentals in more than 25,000 cities in 192 countries - and they continue to grow every single day.

Airbnb is a classic story of how a few people with a great idea can disrupt an entire industry. Launched in 2008, over 80 million guests have stayed on Airbnb in over 2 million homes in over 190 countries. They recently opened 4,000 homes in Cuba to travelers around the globe. The company was also an early adopter of AWS.

In the guest post below, Airbnb Engineering Manager Kevin Rice talks about how AWS was an important part of the company’s startup days, and how it stays that way today.

— Jeff;

How Did Airbnb Start?

In 2007, flatmates Joe Gebbia and Brian Chesky found themselves completely broke.

To pay their rent, they tried to come up with wild suggestions for earning money. “Hey”, Chesky suddenly said, “There’s a design conference here in San Francisco next week, all the hotels are fully booked. How about we buy an air mattress?”

It was a crazy idea. But they bought three.

To accompany their new and seemingly strange business venture, they sat up a website, airbedandbreakfast.com, and - surprisingly enough - their floor was soon filled with sleeping strangers, charged $80 each per night. “As we were waving these people goodbye”, Chesky recalls, “Joe and I looked at each other and thought there’s got to be a bigger idea here”.

And indeed there was. Today, Airbnb has completely changed the way we travel. It has over 10 million nights booked, and it can certainly cater to all tastes and requirements. Can they tempt you with a cozy cottage in the outskirts of Montreal? Or would you rather prefer to live out your Downton Abbey-dreams in a historic mansion in the Surrey countryside?

Whatever you want, chances are that Airbnb can provide it. Currently listed are private islands, idyllic treehouses, romantic caves, and - for the more adventurous souls - houseboats.

New properties are listed every single day.

The Challenge

A year after Airbnb launched, the company decided to migrate nearly all of its cloud computing functions to Amazon Web Services (AWS) because of service administration challenges experienced with its original provider. Nathan Blecharczyk, Co-founder & CTO of Airbnb says, “Initially, the appeal of AWS was the ease of managing and customizing the stack. It was great to be able to ramp up more servers without having to contact anyone and without having minimum usage commitments. As our company continued to grow, so did our reliance on the AWS cloud and now, we’ve adopted almost all of the features AWS provides. AWS is the easy answer for any Internet business that wants to scale to the next level.”

Lets make it more clear by starting again with the problems in detail

Airbnb grew faster than Chesky and Gebbia could ever have predicted.

Pretty soon their website was struggling under the constant stream of clients that wanted to utilise their services. As Airbnb started to plan out their new business venture, Airbnb Experiences, they realized that something needed to change.

They simply couldn’t scale quickly enough.

AS a result, the company decided to migrate nearly all of their functions to Amazon Web Services (AWS). CTO of Airbnb, Nathan Blecharczyk, explains that “the appeal of AWS was the ease of managing and customising the stack. (..) As our company continued to grow, so did our reliance on the AWS cloud and we’ve adopted almost all of the features AWS provides”.

In retrospect, Blecharczyk wish they would have migrated earlier: “AWS is the easy answer for any internet business that wants to scale to the next level.”

Automatic Scaling proved out to be a boon

The migration to AWS needed to happen quickly - the fast-growing Airbnb did not want its community of users locked out of their marketplace for too long. In the end, the effortless and easy migration only resulted in 15 minutes of downtime.

After moving away from MySQL as their original datastore, Airbnb is now using RDS. The service, short for Relational Database Service, is a web service offered by Amazon in which you can run a scalable MySQL setup in the cloud - all while hiding the administrative overhead and eliminating the faff that usually comes with maintaining it. Additionally, their focus on safety means that Amazon at all times will have a dozen background tasks handling all things related to payments and analytics - without slowing down the site.

The new system eliminates the effects of long-running queries on the frontend performance, and it seems to perform significantly better than MySQL. Perhaps more importantly, Tobi Knaup, an Airbnb engineer reveals that “We are now well prepared for our future growth”.

The Amazon Benefits At a Glance

Airbnb benefited greatly from migrating to Amazon Web Services. The flexibility and responsiveness of AWS is helping to prepare the company for future growth, and Tobi Knaup reveals that "Amazon Web Services listens to customers’ needs. If the feature does not yet exist, it probably will in a matter of months”.

So what were the main benefits of the migration to Amazon?

• Automatic Scaling: Airbnb can now automatically scale based on the needs of their ever-increasing customer base.
• Running MySQL in the Cloud: This allowed Airbnb to reduce their management of MySQL and it had a positive impact on frontend performance
• Focus on Safety: AWS has a strong focus on the security of its cloud, and the migration enabled Airbnb to ease their own workload without compromising security levels.
• Saving Resources: With AWS, the engineers of Airbnb can now free up their time and focus on what really matters - innovation.

How Airbnb Used Amazon Web Services to Conquer the Rental Market

The six-year-old startup Airbnb has essentially changed the way people rent their homes with its peer-to-peer lodging network, allowing hosts to rent their homes to creeps and non-creeps alike the world over. It seems like such a simple concept -- rent your home to a stranger! -- but it carries with it a tremendous set of technical challenges around payments, data collection, and around global availability.

But the startup has managed to grow, recently closing a $450 million round of funding, and its user numbers have spiked through the roof, with over 500,000 listings across 192 countries. At GigaOM Structure this week, I spoke to Airbnb VP of Engineering Mike Curtis about how Amazon Web Services helped them get settled in at the top -- and why they're never checking out of Amazon's public cloud.

The reason why Airbnb runs on the public cloud in the first place is pretty simple: If they don’t have teams worrying about making sure servers are up and running, they can spend more time working on the things that matter to Airbnb’s business. Besides, having formed in 2008, it had no legacy IT to migrate.

"We concentrate more of our energy to solving problems unique to Airbnb," Curtis said on stage at the conference.

Airbnb's engineering team is just over 100 people, Curtis says, and since none of them are on server or operations duty, they can self-organize around different tasks, with strategy set by product management and tactics left to the team. For contrast, Curtis says that during his pre-Airbnb stint at Yahoo, two thirds of the team was working on infrastructure and operations (and not working on product development) at any given time.

"It was a different environment," Curtis says.

These days, Airbnb runs just over a thousand EC2 instances at any given time, with instances dying and getting spun up again at any given time. Airbnb's technology stack includes Hadoop for data processing, Ruby on Rails as its application framework, the new and upcoming Apache Mesos for cluster management, and, crucially, MySQL running on the Amazon Relative Database Service (RDS) for its database services.

I say "crucially" because this is both the key to Airbnb's scalability and its Achilles Heel, at least when it comes to portability. Whether the company likes it or not, its deep investment in the Amazon Web Services ecosystem with EC2 and RDS means that it's parked there indefinitely given the potential switching costs involved. That wasn't always for the better, as Curtis acknowledges.

"There was a time when Amazon wasn't as reliable as it is now," he says.

But over time, Amazon Web Services got more reliable and more robust in its quality of service. Moreover, because Airbnb is a flagship customer for Amazon Web Services, it has more direct representation in the cloud's headquarters, resulting in better transparency and more insight into service performance.

At the same time, Curtis wants to make it clear that Amazon Web Services isn't for everyone, which is an important point as more and more businesses migrate applications into the cloud.