Airpal: a Web UI for PrestoDB

AirbnbEng
AirbnbEng
Mar 5, 2015 · 3 min read

By James Mayfield

Image for post
Image for post

Airbnb is pleased to announce the launch of Airpal, a web-based query execution tool that leverages Facebook’s PrestoDB to facilitate data analysis.

People who spend time using SQL for exploration and investigation know that the workflow is not always smooth. Remembering how a query was written, copying and pasting from the command line, and running multiple terminal windows can slow down analysis and be frustrating. Additionally, when diverse teams are using SQL for analytics, the learning curve can be steep for beginners, so good UI tools can help drive adoption and promote knowledge sharing.

At Airbnb, we launched Airpal internally about a year ago and now more than 1/3 of all employees have issued a query through the tool. This is an astounding statistic and it shows how integral Presto is to our company’s data infrastructure.

We currently hold about one and a half petabytes of data as Hive managed tables in HDFS, and the relatively small data size of our important “core_data” tables allows us to use Presto as the default query engine for analysis. When running ad hoc queries and iterating on the steps of an analysis, Presto is much snappier and more responsive than traditional map reduce jobs. The biggest benefit to adding Presto to our infrastructure stack, though, is that we don’t have to add additional complexity to allow “interactive” querying. Because we are querying against our one, central Hive warehouse, we can keep a “single source of truth” with no large scale copies to a separate storage/query layer. Additionally, the fact that we don’t need change data storage type from RC format to see the speed improvements, makes Presto a great choice for our infrastructure.

We are excited to share this tool with the open source community and we hope that it can provide similar utility for others.

Image for post
Image for post

Key features of Airpal

Keeping with the spirit of Presto, we have tried to make it simple to install Airpal by providing a local storage option for people who would like to test it out without any overhead or cost. For more detailed information, visit the GitHub page here: https://github.com/airbnb/airpal

A few of the notable technology features in Airpal are:

  1. uses Dropwizard as an easy way to provide a REST service in Java
  2. employs SSE (Server Sent Events) to push messages from the server to the client
  3. the front end JavaScript uses react.js

Finally, we would be remiss if we did not mention the awesome direction that Facebook provided as the original developers of Hive and the pioneers of building UI tools to facilitate easy access to big data. We stood on the shoulders of giants to make this tool and we appreciate the influence and input that the data infrastructure and data tools teams at Facebook were able to provide.

If you’re interested in helping build a world class suite of data tools, check out this open job position: https://www.airbnb.com/jobs/departments/position/48112

Image for post
Image for post

Check out all of our open source projects over at airbnb.io and follow us on Twitter: @AirbnbEng + @AirbnbData

Originally published at nerds.airbnb.com on March 5, 2015.

Airbnb Engineering & Data Science

Creative engineers and data scientists building a world…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store