Miles ColeDecoding Delta Lake Compatibility Between Fabric and DatabricksWith Microsoft going all in on Delta Lake, the landscape data architects deeply integrated with the Microsoft stack is undergoing a…Mar 22Mar 22
Miles ColeCluster Configuration Secrets for Spark: Unlocking Parallel Processing PowerSomething I’ve always found challenging in PaaS Spark platforms, such as Databricks and Microsoft Fabric, is efficiently leveraging compute…Feb 19Feb 19
Miles ColeRLS in Databricks Unity Catalog and Power BIUnity Catalog introduces many new concepts in Databricks, particularly around security and governance. One significantly improved security…Jan 26Jan 26
Miles ColeQuerying Databases in Apache Spark: Pandas vs. Spark API vs. Pandas-on-SparkApache Spark offers tremendous capability, regardless of the implementation-be it Microsoft Fabric or Databricks. However, with vast…Jan 24Jan 24
Miles ColeInstall Data Software Like a ProTL;DR For developers, Chocolatey is an essential tool to address the challenges of installing and managing software.Jan 18Jan 18
Miles ColeLinked Service Property MagicEver wished you could add dynamic content, parameterize, or reference a Key Vault secret value for Linked Service properties that only…Nov 1, 2023Nov 1, 2023
Miles ColeEnhancing Your Lakehouse with Unity CatalogSince its introduction, Unity Catalog has been creating significant buzz in the data community. But what exactly is it, and why is enabling…Oct 28, 2023Oct 28, 2023
Miles ColeDelta Table Maintenance 101So you’ve figured out how to write data into Delta format, congratulations! You’ve joined the Delta Lake club and are enabled for all the…Oct 9, 2023Oct 9, 2023
Miles ColeAutomate Everything — Azure Synapse / SQL User Provisioning via Azure DevOpsData engineers and architects will often spend many hundreds of hours building a complex and fully automated solution for delivering data…Oct 9, 2023Oct 9, 2023
Miles ColeSynapse Optimization Series: Table DistributionsProper use of table distributions in Synapse Dedicated Sql Pools is easily the #1 shortcoming in Synapse implementations.Jan 20, 20231Jan 20, 20231