Chronos: A Replacement for Cron

The Backstory

At Airbnb, we heavily rely on data analysis to build great products. Our data-pipeline consists of many technologies such as Hadoop, MySQL, Amazon Redshift and S3. Our software engineers and analysts use a mix of Cascading, Cascalog, Hive and Pig for interfacing with Hadoop. We have scripts that export tables from a vast number of databases into S3 and we use various ETL (extract transform and load) processes to turn blobs of bytes into meaningful information. Many of these transformations consist of multiple steps and some tables are composed of a myriad of data-sources and joins.

Chronos UI

Chronos comes with a UI which can be used to add, delete, list, modify and run jobs. It can also show a graph of job dependencies. These screenshots should give you a good idea of what Chronos can do.

Sample-chronos-ui

Check it out on Github

Over the past weeks, we have open-sourced Chronos, you can check it out on our github page: https://mesos.github.io/chronos/

Check out all of our open source projects over at airbnb.io and follow us on Twitter: @AirbnbEng + @AirbnbData

--

--

Creative engineers and data scientists building a world where you can belong anywhere. http://airbnb.io

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
AirbnbEng

Creative engineers and data scientists building a world where you can belong anywhere. http://airbnb.io