Abhinav PrakashDatabricks: A comprehensive optimization guideI have been using Databricks for ETL workloads for 4 years now. In these 4 years, I have come across optimization techniques in bits and…Feb 2Feb 2
Abhinav PrakashDelta Lake vs. ParquetIf Delta lake tables also use Parquet files to store data, how are they different (and better) than vanilla Parquet tables?Jan 244Jan 244
Abhinav PrakashSix point checklist for Spark job optimizationI have been scouring the internet to try and understand the best ways to optimize a spark job. Here, I am summarizing my findings. This…Mar 22, 2023Mar 22, 2023
Abhinav PrakashData stores over the yearsLets discuss the different data storage architectures over the years to understand which is the best one for you. Also, why is it best?Mar 6, 2023Mar 6, 2023