Homepage
Open in app
Sign in
Get started
Making Sense of Data & Helping Others Grow: Tips, Advice, and Stories from the Front Lines of Data Engineering
MOST READ
WRITE FOR US
FEATURED ARTICLE
LinkedIn
Follow
5 Key Mindset Shifts to Succeed in Data Engineering
5 Key Mindset Shifts to Succeed in Data Engineering
Forget the Roadmaps You’ve Seen — Create Your Own Path in Data Engineering
Tim Webster
Sep 6
Modern Data Engineering in the LLM Era
Modern Data Engineering in the LLM Era
How Large Language Models are Shaping the Future of Data Engineering
Pavan Emani
Sep 6
From Zero to Data Engineer: What I’d Do Differently Today
From Zero to Data Engineer: What I’d Do Differently Today
Strategies for Fast-Tracking Your Data Engineering Career
Pavan Emani
Sep 3
Stop using Lambdas to trigger AWS Glue workflows based on S3 Events!
Stop using Lambdas to trigger AWS Glue workflows based on S3 Events!
How to use Eventbridge to trigger Glue jobs
Pavan Emani
Sep 2
Important SQL Topics for Data Engineers
Important SQL Topics for Data Engineers
A Comprehensive Beginner’s Guide to Mastering SQL
Nnamdi Samuel
Aug 31
Traits Of A Great Data Engineer
Traits Of A Great Data Engineer
What Makes Top Performers Stand Out in Data Engineering
Santosh Joshi
Aug 30
SQL: You Can’t Be a Data Engineer Without It (Non-Negotiable)
SQL: You Can’t Be a Data Engineer Without It (Non-Negotiable)
Why SQL is Essential for Your Career
Tim Webster
Aug 29
Can You Crack This SQL Query in 30 Minutes?
Can You Crack This SQL Query in 30 Minutes?
Put your SQL skills to the test with this engaging, time-bound challenge.
Santosh Joshi
Aug 29
Best Orchestration Tools Besides Apache Airflow
Best Orchestration Tools Besides Apache Airflow
Orchestration Tools You May Not Have Heard of
Nnamdi Samuel
Aug 28
SQL Query Optimization Tips I Regret Not Knowing Earlier
SQL Query Optimization Tips I Regret Not Knowing Earlier
15 Tricks to Level-Up Your Data Game
Nnamdi Samuel
Aug 26
Control S3 Prefix (Folder) level access using AWS Lake Formation Credential Vending
Control S3 Prefix (Folder) level access using AWS Lake Formation Cr...
AWS Lake Formation simplifies Data Governance and Access control on cloud by allowing organizations to centrally manage and govern data…
Pavan Emani
Aug 25
Mastering SQL: How to Find and Remove Duplicate Rows Efficiently
Mastering SQL: How to Find and Remove Duplicate Rows Efficiently
Optimize your SQL skills with these strategies for managing and removing duplicate data in large databases.
RAKESH CHANDA
Aug 24
7 Surprising Habits That Make Data Engineers Unstoppable
7 Surprising Habits That Make Data Engineers Unstoppable
How to Thrive in Data Engineering
Tim Webster
Aug 23
Bing New Search — End-to-End Azure Data Engineering Project using Microsoft Fabric.
Bing New Search — End-to-End Azure Data Engineering Project using M...
This project uses Bing News API within Azure to create an end-to-end Data Engineering project using Microsoft Fabric to determine the…
Musili Adebayo
Aug 20
Data Warehouse vs Data Lake vs Data Lakehouse | The Quick Guide
Data Warehouse vs Data Lake vs Data Lakehouse | The Quick Guide
Key differences to know
Humberto López-Zavaleta
Aug 16
Stop Trying to Be a Data Hero (Here’s Why)
Stop Trying to Be a Data Hero (Here’s Why)
Pitfalls of Data Engineering Heroics
Tim Webster
Aug 15
Understanding Apache Spark: A Deep Dive into Big Data Processing
Understanding Apache Spark: A Deep Dive into Big Data Processing
Learn why Spark is the go-to choice for data engineers and data scientists.
Rui Carvalho
Aug 12
How Data Engineers Win (Discipline and Fundamentals)
How Data Engineers Win (Discipline and Fundamentals)
The Real Things Every Data Engineer Should Care About
Tim Webster
Aug 8
How to Clean Up a CSV File
How to Clean Up a CSV File
Complete simple tasks faster
Violeta Rakonjac
Aug 8
Advanced Data Engineering Interview Questions-Part 2
Advanced Data Engineering Interview Questions-Part 2
Advanced Data Engineering Interview Questions
Arpita Mishra
Aug 2
What They Don’t Tell You About Being a Data Engineer
What They Don’t Tell You About Being a Data Engineer
What No One Prepares You For
Tim Webster
Aug 1
Microsoft Fabric End-to-End Project — with Shorcut, Data Pipeline, DataFlow, and Regular File…
Microsoft Fabric End-to-End Project — with Shorcut, Data Pipeline,...
The aim of this project is Microsoft Fabric Project is to:
Musili Adebayo
Jul 30
5 Lessons from 5 Data Teams (Lessons for Data Engineers)
5 Lessons from 5 Data Teams (Lessons for Data Engineers)
Common Sense Data Engineering Lessons Across Teams
Tim Webster
Jul 26
Learn with me: Using .env file in python
Learn with me: Using .env file in python
Hello everyone,
Violeta Rakonjac
Jul 25
Steady as She Flows: Rate Limiting in Apache Beam Pipelines
Steady as She Flows: Rate Limiting in Apache Beam Pipelines
A Data Engineer’s Toolkit for Controlled Streaming
Josi Aranda
Jul 21
About Art of Data Engineering
Latest Stories
Archive
About Medium
Terms
Privacy
Teams