Wahid AtouiLake FS and Unity Catalog to Enhance Lakehouse Data Quality & Workflows ProductivityI worked recently on a data project with a main goal to reduce the lead time of data features implemented on top of the lakehouse…Dec 5, 2023Dec 5, 2023
Wahid AtouiHive & Unity Catalog External Tables CleanerThe goal of this article is to walk through a quick routine to help you drop tables which don’t have data files in the storage layer…Sep 19, 2023Sep 19, 2023
Wahid AtouiDatabricks’ VS Code Extension and Databricks Connect V2 for Unit TestingIf you ever worked on building data platforms around the databricks ecosystem, you may have noticed the lack of a native coding…Aug 31, 20234Aug 31, 20234
Wahid AtouiSync Delta Tables Stored in DBFS (Managed or External) To Unity CatalogTo seemlessly migrate from hive metastore to unity catalog, Databricks comes with a synchronization process which can be ran in one shot…May 5, 20231May 5, 20231
Wahid AtouiHow to Dynamically Add an Identity Column To Existing Delta TablesAt the beginning of this year, I worked on a data project on which we had to build a lakehouse based on delta table and designed as a star…Oct 12, 2022Oct 12, 2022
Wahid AtouiDBT & Delta Share for Interoperable Multi Cloud Data ArchitectureYear after year data engineering tools are growing everywhere like mushrooms. Certainly this is a prolific situation for data engineering…Aug 23, 2022Aug 23, 2022
Wahid AtouiDelta Table Updating Patterns For WriteOnce -ReadMany LakehouseToday I am sharing with you about two loading patterns that helped us getting the head out of the water a couple of months ago when I was…Jul 26, 2022Jul 26, 2022
Wahid AtouiAzure Synapse — How to use Delta Sharing ?A couple days ago I wrote an article about how to setup Delta Sharing on Azure [here is the link], as I promised i will continue digging…Dec 27, 2021Dec 27, 2021
Wahid AtouiinDev GeniusDelta Sharing on AzureThis year has been a heck of a year! Data wise, it was the year that saw new data engineering/processing/storage features spring up…Dec 21, 20211Dec 21, 20211
Wahid AtouiUsing Azure Analysis Services With Azure Synapse ServerlessAs far as I am aware, there are 3 ways to connect a data lakehouse as source for tabular cubes (deployed in Azure Analysis Services):Oct 14, 2021Oct 14, 2021