Spark deserves a better IDE

Jan 29, 2020 · 4 min read

Authors: Raj Bains, Maciej Szpakowski

Spark has become the default Data Engineering platform in the cloud. With Databricks, AWS EMR, AWS Glue, Azure Databricks, Google Dataproc and Cloudera, one can rely on Spark being ubiquitously available.

As we work with Enterprises to move legacy ETL to Spark, we’ve been focusing on building the right replacement. We find that the current interfaces…