Nehil JainYear-end review — 2019It is too late for the traditional “Year-end review” article but it is still worth reviewing my 2019 for my own sake. I was browsing Julia…Mar 13, 2020Mar 13, 2020
Nehil JainBuilding a data pipeline to handle >50M events/day (with a startup budget) — Part II. LoadingIn the previous post, I discussed considerations for a good data pipeline solution and the collection part of the event data. Here, I will…Nov 19, 2019Nov 19, 2019
Nehil JaininSuper.comBuilding a data pipeline to handle >50M events/day (with a startup budget) — Part 1Building the foundations of the data stack for a startup can be overwhelming. It requires a mixture of system architecture and software…Apr 12, 2019Apr 12, 2019
Nehil JaininSuper.comAirflow Part 2: Lessons learnedAt SnapTravel we use Apache Airflow to orchestrate our batch processes. It is one of the key systems we depend on for keeping track of…Jun 17, 20189Jun 17, 20189
Nehil JainHope is not a Strategy, may be Terraform is?As I was thinking “Oh Shoot!! Did I just delete the production database? 😨”, the CEO walked in to enquire why were all his searches…Jan 8, 2018Jan 8, 2018
Nehil JainLessons learnt while Airflow-ingVendor/Solution Selection ProblemApr 13, 20172Apr 13, 20172
Nehil Jaininathletigen-engineeringAWS Lambda: A New Generation of Cloud Compute ServicesWe have been building an internal tool to manage and troubleshoot our product logistics pipeline. We embarked on this task with a small…Oct 14, 2016Oct 14, 2016