Debayan KarMicrosoft Fabric, Microservices, and Data Engineering: A Comprehensive OverviewMicrosoft Fabric, officially Azure Service Fabric, stands as a powerful platform designed by Microsoft to simplify the development and…Feb 10Feb 10
Debayan KarAre you following these Best Practices for Optimizing Big Data Workflows with Delta Lake ?IntroductionOct 28, 2023Oct 28, 2023
Debayan KarThe right way to do Retention & Vacuuming in DatabricksDelta Lake is a data lake technology built on top of Apache Spark that provides ACID transactions and other advanced features to manage big…Apr 26, 20232Apr 26, 20232
Debayan KarAre you Optimizing Vacuum Retention with Zorder in PySpark on Databricks for Improved Performance…In the field of data science, data analysis and processing are very important. The most commonly used tool for data analysis and processing…Apr 14, 20231Apr 14, 20231
Debayan KarWhy Upsert in Databricks using Pyspark is a bad idea, or is it ?Databricks Delta is a powerful data storage layer that enables efficient data management, processing, and querying in data lakes. One of…Apr 10, 20234Apr 10, 20234
Debayan KarThe easiest way to get Information Schema details of DatabricksDatabricks is a cloud-based big data processing platform that offers a variety of tools and services for data engineers, data scientists…Mar 14, 20232Mar 14, 20232
Debayan KarETL vs ELT- the factsIn the field of data engineering, ETL and ELT are two commonly used acronyms that describe the process of moving and transforming data from…Feb 13, 2023Feb 13, 2023
Debayan KarHow to take the first steps in Exploratory Data Analytics (EDA) using R,Python & SQLExploratory Data Analysis (EDA) is an important step in any data analysis process, and it involves the examination of data to uncover…Feb 13, 2023Feb 13, 2023
Debayan KarA Beginner’s Guide to Regular ExpressionsRegular expressions, also known as regex, are sequences of characters that define a search pattern. They are used to match and manipulate…Feb 12, 2023Feb 12, 2023
Debayan KarDiscover the Hidden Gem of Purulia: Green Villa Eco ResortPurulia, located in the western part of West Bengal, is often referred to as a hidden gem, with its stunning landscapes, rich cultural…Feb 12, 20231Feb 12, 20231