Nethaji KamalapuramScenario based Data Engineering QuestionsQ: In your recent project, it’s intriguing to hear about your experience managing data from diverse sources like NoSQL, RDBMS, and Kafka…Sep 15Sep 15
Nethaji KamalapuraminDev GeniusWhy Data Vault 2.0 is the Future of Data WarehousingData Vault, a data modeling methodology designed to provide a flexible, scalable, and auditable approach to data warehousing, has evolved…Jul 20Jul 20
Nethaji KamalapuraminDev GeniusSpark Dynamic Resource AllocationSpark Scheduling: Allocation StrategiesMay 16May 16
Nethaji KamalapuraminDev GeniusSpark Data Caching, Repartition, Coalesce, Data Frame HintsLet's deep dive into the data caching concept in spark, why it's needed, different caching techniques and parameters, what is best fit for…May 15May 15
Nethaji KamalapuramSpark Dynamic Partition PruningDynamic Partition Pruning for Optimized JoinsMay 15May 15
Nethaji KamalapuramAdaptive Query execution — spark 3.0This article explores Apache Spark 3.0’s Adaptive Query Execution (AQE) feature and its benefits for optimizing query performance.May 13May 13
Nethaji KamalapuramSpark Memory allocation & ManagementLets talk about how memory allocation works for spark driver and executors. Understanding driver and executor memory allocation is crucial…May 131May 131