The six-year-old startup Airbnb has essentially changed the way people rent their homes with its peer-to-peer lodging network, allowing hosts to rent their homes to creeps and non-creeps alike the world over. It seems like such a simple concept -- rent your home to a stranger! -- but it carries with it a tremendous set of technical challenges around payments, data collection, and around global availability.
But the startup has managed to grow, recently closing a $450 million round of funding, and its user numbers have spiked through the roof, with over 500,000 listings across 192 countries. At GigaOM Structure this week, I spoke to Airbnb VP of Engineering Mike Curtis about how Amazon Web Services helped them get settled in at the top -- and why they're never checking out of Amazon's public cloud.
The reason why Airbnb runs on the public cloud in the first place is pretty simple: If they don't have teams worrying about making sure servers are up and running, they can spend more time working on the things that matter to Airbnb's business. Besides, having formed in 2008, it had no legacy IT to migrate.
"We concentrate more of our energy to solving problems unique to Airbnb," Curtis said on stage at the conference.
Airbnb's engineering team is just over 100 people, Curtis says, and since none of them are on server or operations duty, they can self-organize around different tasks, with strategy set by product management and tactics left to the team. For contrast, Curtis says that during his pre-Airbnb stint at Yahoo, two thirds of the team was working on infrastructure and operations (and not working on product development) at any given time.
"It was a different environment," Curtis says.
These days, Airbnb runs just over a thousand EC2 instances at any given time, with instances dying and getting spun up again at any given time. Airbnb's technology stack includes Hadoop for data processing, Ruby on Rails as its application framework, the new and upcoming Apache Mesos for cluster management, and, crucially, MySQL running on the Amazon Relative Database Service (RDS) for its database services.
I say "crucially" because this is both the key to Airbnb's scalability and its Achilles Heel, at least when it comes to portability. Whether the company likes it or not, its deep investment in the Amazon Web Services ecosystem with EC2 and RDS means that it's parked there indefinitely given the potential switching costs involved. That wasn't always for the better, as Curtis acknowledges.
"There was a time when Amazon wasn't as reliable as it is now," he says.
But over time, Amazon Web Services got more reliable and more robust in its quality of service. Moreover, because Airbnb is a flagship customer for Amazon Web Services, it has more direct representation in the cloud's headquarters, resulting in better transparency and more insight into service performance.
At the same time, Curtis wants to make it clear that Amazon Web Services isn't for everyone, which is an important point as more and more businesses migrate applications into the cloud.
When AWS doesn't work
For starters, AWS just isn't cost-effective if you're not running workloads that rapidly scale up and down, Curtis says, though the march towards ever-cheaper cloud resources brings it closer to that reality.
The second main consideration, in Curtis's estimation, is all the marketing spin around public cloud that makes it sound like the only point of difference between providers is price. In fact, it really comes down to lots of little details.
As an example, Amazon RDS only recently started making it easier to get MySQL logs that Airbnb developers had been dying for. Airbnb engineered around the lack, and came up with stopgaps and workarounds that sort of addressed the problems that the logs would clearly lay out for them, but actually having access was a revelation. But "easy access to logs" is a tiny little feature that would never make it into marketing materials.
"In that level of detail is where the gotchas will be," Curtis says.
To address that, Curtis recommends trying real workloads on different cloud providers to make sure it will do what you need it to. "Actually try to get it up and running," Curtis says.
That kind of try-before-you-buy wasn't exactly a luxury afforded by the early days of Airbnb, when the options for web-scale cloud hosting were "Amazon Web Services" and "laughably small providers who aren't Amazon Web Services," but there's never been more choice in the cloud and it's up to the CIO's office to do their due diligence.
In the meanwhile, try to enjoy the poetic consistency of the fact that Airbnb, which built its business on home rentals, is a loyal customer of Amazon Web Services, which built its business on computing rentals.