Whispering Data

All the latest tools and trends in data engineering.

Spark joy in your data warehouse by following these data modeling best practices!

Gain the confidence to mess with your data without making a mess of your data.

If you’re like me, you bought your first cryptocurrency in the past year or so, right when it stopped going up in price and making random…

Take your data lake game to new heights with these two architecture improvements.

In October, it was announced that the Pandas API was being integrated with Spark. This is particularly exciting news for a Pandas-baby like…

Let’s put the bagel to use by querying branched lakeFS data from Trino’s distributed engine.

Already familiar with versioning code with git? A look at how it works to version data using the same abstractions.

Databricks has come a long way since growing out of a Berkeley Lab in 2013 with an open-source distributed computing framework called…

Use docker compose to create local replicas of a modern data stack with one command.

Second in a series about Hive Metastore. In the last post, Einat covered its history, the problems that it solves — and questioned whether…

A majority of data architectures still feature Hive Metastore. Why has it survived and what can finally replace it in the future?

Useful datasets are created through a process that involves several predictable steps. Learn how to organize your workflows in this…

Learn how to integrate lakeFS hooks to validate data on commits.