Solon DasinTowards Data EngineeringAzure Storage Account : The NuancesAzure Storage Account is a cloud storage solution provided by Microsoft Azure. It offers a scalable and durable platform to store various…Jun 6Jun 6
Solon DasinTowards Data EngineeringMost Challenging Database Questions for 2024In today’s digital era, databases are crucial for managing and utilizing data effectively. They organize data for easy retrieval, enhance…May 23May 23
Solon DasinTowards Data EngineeringMost Asked Questions on Data Pipeline DesignThese are the most asked questions :Apr 151Apr 151
Solon DasinTowards Data EngineeringMost Asked Data Modeling Questions in 2024Medium Level QuestionsApr 13Apr 13
Solon DasinData Engineer ThingsMemory Management in Apache SparkApache Spark’s performance advantage over MapReduce is greatest in the use-cases involving repeated computations. Much of this performance…Apr 11Apr 11
Solon DasinTowards Data EngineeringAdvanced File Formats and Compression TechniquesThere are several file formats that we use for data processing and data storage. In this detailed blogpost we will learn about all the…Apr 6Apr 6
Solon DasinTowards DevQuery Plans in Spark Internals — Performance TuningSpark Internals have 4 kinds of Query Plans. Lets discuss about them in detail below:Apr 41Apr 41
Solon DasinTowards DevSpark Internals for groupBy().count()Spark’s groupBy operation is a fundamental wide transformation that allows you to group data in a distributed manner based on the values…Mar 18Mar 18
Solon DasinTowards DevHow is Generative AI and open AI models helping Data Engineers ?OpenAI provides tools and models that can be leveraged by data engineers to enhance various aspects of their work. Here are a few ways in…Jan 24Jan 24