PinnedViral PatelFix data issues in existing delta tables without reloading data from source.Fixes a data issue in a Databricks Delta table by performing an update on selected columns using a specified SQL function.8 min read·Mar 2, 2024----
PinnedViral PatelApply Encryption to PII Fields of Delta TablesWhen it comes to the PII (Personally Identifiable Information) or Commercially Sensitive data, It is necessary requirement of most of the…6 min read·Jan 17, 2024--3--3
PinnedViral PatelDatabricks Delta Table StatusIn this blog, I will walk you through quick POC I have implemented to track the status of the the Delta table using delta table history. I…5 min read·Jan 25, 2024--4--4
PinnedViral PatelMetadata Driven Unity Catalog PermissionsMetadata driven permission for UC Securables6 min read·Jan 15, 2024----
Viral PatelDatabricks Sql Warehouse ServerelessVertical Scaling Databricks Sql Warehouse10 min read·6 days ago----
Viral PatelMigrate Hive metastore (HMS) Schema to Unity CatalogWelcome to the Migration Story of HMS to Unity Catalog.4 min read·6 days ago--1--1
Viral PatelDelta Lake Database/Schema Maintenance Part-2Optimize Performance and Cost of Delta Lake at Scale8 min read·Apr 6, 2024----
Viral PatelFlatten Json data/file in to PySpark Dataframe using Python function.In this blog post, I will walk you through how you can flatten complex json or xml file using python function and spark dataframe.4 min read·Feb 18, 2024--1--1
Viral PatelDelta Lake Database/Schema MaintenanceIn this blog post, I am going to walk you through the journey of delta table maintenance. The process I have impalement as POC and It has…8 min read·Feb 12, 2024----
Viral PatelExport your Delta Table DDL Statements in few minutes.In this blog, I will walk you through how quickly you can export you delta table definitions with the help of show create statements and…5 min read·Feb 9, 2024----
