Open in app
Sign inGet started
Trending
Databricks is Now Free for Learners — No Cloud Account Required!

Databricks is Now Free for Learners — No Cloud Account Required!

In a major win for the data learning community, Databricks has just made its platform freely available for learners — no cloud account or…
Go to the profile of THE BRICK LEARNING
THE BRICK LEARNING
Databricks Quietly Prepares Platform for Real-Time Data Apps — Starting with Databricks Apps and…

Databricks Quietly Prepares Platform for Real-Time Data Apps — Starting with Databricks Apps and…

In the age of intelligent applications, the next wave of digital transformation is being fueled by real-time Data Apps — systems that…
Go to the profile of THE BRICK LEARNING
THE BRICK LEARNING
Fail Fast or Quarantine? Two Data Quality Patterns Every Spark Engineer Should Know

Fail Fast or Quarantine? Two Data Quality Patterns Every Spark Engineer Should Know

Learn when to fail fast or quarantine bad data in Spark pipelines.
Go to the profile of Marcel Kennert
Marcel Kennert
Latest
Databricks Data Engineering Preparation Question 16: Fixing Python Conditionals in Data Engineering…

Databricks Data Engineering Preparation Question 16: Fixing Python Conditionals in Data Engineering…

A Case of Invalid Syntax
Go to the profile of THE BRICK LEARNING
THE BRICK LEARNING
Databricks Data Engineering Preparation Question 15: How Long Is Too Long?

Databricks Data Engineering Preparation Question 15: How Long Is Too Long?

Understanding the Default Retention Period of VACUUM in Delta Lake
Go to the profile of THE BRICK LEARNING
THE BRICK LEARNING
Databricks Data Engineering Preparation Question 14: Creating a Table in Databricks Using JDBC

Databricks Data Engineering Preparation Question 14: Creating a Table in Databricks Using JDBC

Fill in the blank to create a table in Databricks using data from a PostgreSQL database.
Go to the profile of THE BRICK LEARNING
THE BRICK LEARNING
Databricks Data Engineering Preparation Question 13: Mastering CDC in Delta Live Tables with APPLY…

Databricks Data Engineering Preparation Question 13: Mastering CDC in Delta Live Tables with APPLY…

A data engineer is designing a Delta Live Tables (DLT) pipeline. The source system generates files containing changes captured in the…
Go to the profile of THE BRICK LEARNING
THE BRICK LEARNING
Databricks Data Engineering Preparation Question 12: How to Query a Delta Table in PySpark?

Databricks Data Engineering Preparation Question 12: How to Query a Delta Table in PySpark?

The Right Way to Access Spark SQL Tables
Go to the profile of THE BRICK LEARNING
THE BRICK LEARNING
Databricks Data Engineering Preparation Question 5: What Best Describes Auto Loader in Databricks?

Databricks Data Engineering Preparation Question 5: What Best Describes Auto Loader in Databricks?

As you dive deeper into the Databricks Data Engineering ecosystem, one component you’ll frequently encounter is Auto Loader — a powerful…
Go to the profile of THE BRICK LEARNING
THE BRICK LEARNING
Databricks Data Engineering Preparation Question 6: How Does Auto Loader Load Data Incrementally in…

Databricks Data Engineering Preparation Question 6: How Does Auto Loader Load Data Incrementally in…

When working with streaming or near-real-time data ingestion pipelines in Databricks, a common question that comes up during interviews or…
Go to the profile of THE BRICK LEARNING
THE BRICK LEARNING
Understanding Spark’s Catalyst Optimizer: Demystifying Query Optimization

Understanding Spark’s Catalyst Optimizer: Demystifying Query Optimization

Non-members can access the full article through this Link.
Go to the profile of Ritam Mukherjee
Ritam Mukherjee
From Data Assumptions to Data Quality Assurance: Why we need Testing, Cleaning, and Monitoring

From Data Assumptions to Data Quality Assurance: Why we need Testing, Cleaning, and Monitoring

Introduction
Go to the profile of Naidu Rongali - Senior Big Data and ML Engineer
Naidu Rongali - Senior Big Data and ML Engineer
11 High Performance Query Engines I Wish I Knew About Sooner

11 High Performance Query Engines I Wish I Knew About Sooner

How I stopped waiting and started analyzing Victor Oketch Sabare · 6 min read · Jun 14, 2025
Go to the profile of Victor Oketch Sabare
Victor Oketch Sabare
About Towards Data EngineeringLatest StoriesArchiveAbout MediumTermsPrivacyTeams