Data Mechanics
Published in

Data Mechanics

Free Hosted Spark UI & Spark History Server

How to install our free, open-source, Spark UI & Spark History Server on top of any Spark platform

Spark UI Screenshot. Image by Author.

How Can I Use it?

  1. Create an account on Data Mechanics Delight
This is the dashboard that you’ll see once logged in. Image by Author.

How Does It Work? Is It Secure?

  1. An open-source Spark agent which runs inside your Spark applications. This agent will stream non-sensitive Spark event logs from your Spark application to our backend.
  2. A closed-source backend consisting of a real-time logs ingestion pipeline, storage services, a web application, and an authentication layer to make this secure.
Architecture of Data Mechanics Delight.

What’s Next?

Design prototype for the next releases of Data Mechanics Delight. Image by Author.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Jean Yves

Co-Founder @Data Mechanics, The Cloud-Native Spark Platform Senior Product Manager @ Spot.io — Building Ocean for Spark Former software eng @Databricks.