Homepage
Open in app
Sign in
Get started
Towards Data Engineering
Navigating the Path to Data Engineering Excellence
About
Follow
Trending
Why Parquet is the Best File Format for Big Data
Why Parquet is the Best File Format for Big Data
When working with big data, you want to store your data in a way that makes it easy to handle, quick to process, and doesn’t take up too…
Vishal Barvaliya
Aug 31
Mastering SQL Recursive CTEs: Key Concepts and Interview Questions
Mastering SQL Recursive CTEs: Key Concepts and Interview Questions
Unlock the Power of Recursive CTEs in SQL: Learn Key Concepts, Hierarchical Data Solutions, and Top Interview Questions to Boost Your…
Pritam Deb
Sep 6
Latest
Apple Pay Data Engineering Interview Question.
Apple Pay Data Engineering Interview Question.
Let us Solve It Together and learn Data Engineering Better.
Brahma, The Data Engineer.
Sep 19
Comprehensive Guide to Hadoop FileSystem API in Spark: Copy, Delete, and List Files
Comprehensive Guide to Hadoop FileSystem API in Spark: Copy, Delete, and List Files
Explore Hadoop FileSystem API functions in Spark for efficiently copy, delete, and list files and directories for optimized data management
Pritam Deb
Sep 19
Create Data Lakehouse Using Spark+Iceberg+Nessie+Dremio
Create Data Lakehouse Using Spark+Iceberg+Nessie+Dremio
Hi everyone, in this article I will talk about the concept of data lakehouse and develop example using Spark+Iceberg+Nessie+Dremio+Minio
Burak Uğur
Sep 19
3 Essential Skills Every Data Engineer Should Adopt for Career Growth.
3 Essential Skills Every Data Engineer Should Adopt for Career Growth.
Must Know these skills Sooner than too late.
Kamireddy Mahendra
Sep 18
Inside a Netflix Data Engineering Interview.
Inside a Netflix Data Engineering Interview.
A Real time Media — OTT Domain Use Case Question and How to Solve It Together.
Brahma, The Data Engineer.
Sep 18
Mastering SQL Self-Joins: Common Interview Questions and Solutions — Part 2
Mastering SQL Self-Joins: Common Interview Questions and Solutions — Part 2
Deep Dive into SQL Self-Joins: Explore Advanced Questions, Practical Solutions, and Enhance Your Interview Preparation
Pritam Deb
Sep 5
Mastering SQL Self Joins: Common Interview Questions and Solutions (Part 1)
Mastering SQL Self Joins: Common Interview Questions and Solutions (Part 1)
In SQL, a self-join is a powerful technique that allows you to join a table with itself. This is particularly useful when dealing with…
Pritam Deb
Sep 3
Optimizing Spark: Strategies for Scalable and Efficient Data Pipelines
Optimizing Spark: Strategies for Scalable and Efficient Data Pipelines
Optimize Spark Performance: Tuning Garbage Collection, Memory Management, and Tungsten Execution for Efficient Data Processing
Pritam Deb
Aug 23
What to Do If There’s No Time Zone Information in Your Data: A Guide for Data Engineers
What to Do If There’s No Time Zone Information in Your Data: A Guide for Data Engineers
Time zones can be a headache for data engineers.
Aleh Belausau
Sep 16
How to Trigger Airflow DAG Using REST API
How to Trigger Airflow DAG Using REST API
Sometimes you want to programmatically trigger an airflow DAG. Airflow allows you to enable this functionality when needed. In this…
Obinna Onyema
Nov 10, 2023
About Towards Data Engineering
Latest Stories
Archive
About Medium
Terms
Privacy
Teams