Homepage
Open in app
Sign in
Get started
Making Sense of Data & Helping Others Grow: Tips, Advice, and Stories from the Front Lines of Data Engineering
MOST READ
WRITE FOR US
FEATURED ARTICLE
LinkedIn
Follow
Should my data engineering title be revoked if I don’t know Python?
Should my data engineering title be revoked if I don’t know Python?
A self-discovery into life, liberty, and the pursuit of data.
Monica Miller
May 29
4 Stupidly Simple Strategies for Your Data Engineering Career
4 Stupidly Simple Strategies for Your Data Engineering Career
Learn, Try, Stay, and Prepare for Anything
Tim Webster
May 27
Understanding Reverse ETL to Increase Operational Effectiveness
Understanding Reverse ETL to Increase Operational Effectiveness
Rui Carvalho
May 16
Why you should change your CSV export method on Databricks
Why you should change your CSV export method on Databricks
How to efficiently export spark dataframe to CSV
ANGE KOUAME
May 14
17 Non-Code Lessons Every Data Engineer Should Remember (Especially Me)
17 Non-Code Lessons Every Data Engineer Should Remember (Especially...
No-Cost Secrets to Thriving as a Data Engineer
Tim Webster
May 10
We Were Miscalculating Our ML Features Until We Changed One Thing
We Were Miscalculating Our ML Features Until We Changed One Thing
And now it has become a habit!
Akash Mehta
May 4
Why you should consider using transform on Spark
Why you should consider using transform on Spark
How to efficiently process data and write cleaner code.
ANGE KOUAME
May 2
Hard-Learned Lessons from a Data Engineer (Mistakes Included)
Hard-Learned Lessons from a Data Engineer (Mistakes Included)
Lessons Learned on the Front Lines
Tim Webster
Apr 19
Understanding Batch and Stream Processing.
Understanding Batch and Stream Processing.
A Guide to Processing Data
Musili Adebayo
Apr 16
Organizing S3 as a Data Lake: Insights from My Latest Project
Organizing S3 as a Data Lake: Insights from My Latest Project
Efficient Folder-based Organization for Data Management and Lifecycle on Amazon S3
Lorena Gongang
Apr 9
Unleashing the Power of Data Quality with Collibra : Introduction
Unleashing the Power of Data Quality with Collibra : Introduction
Metadata is the new data.
Hana Rumbak
Apr 8
Implementing a Scalable Data Pipeline for Healthcare Claims Analysis.
Implementing a Scalable Data Pipeline for Healthcare Claims Analysis.
Leveraging PySpark for Efficient Risk Assessment in Big Data Environments.
Brahmareddy, The Data Engineer.
Apr 8
How Data Engineering Became Fun Again (Escaping the Rut)
How Data Engineering Became Fun Again (Escaping the Rut)
Why I Stopped Chasing Titles and Started Solving Problems
Tim Webster
Apr 4
The 9 Rules That Keep Me Honest and Focused in Data Engineering
The 9 Rules That Keep Me Honest and Focused in Data Engineering
The Framework of Focus For Data Folks
Tim Webster
Mar 25
The Overlooked Data Engineering Tactics of 2024
The Overlooked Data Engineering Tactics of 2024
Strategies for Long-Term Success
Tim Webster
Mar 18
Mastering Data Transformation: Essential Insights for Data Engineers
Mastering Data Transformation: Essential Insights for Data Engineers
Dive into the critical role of data transformation in the data engineering process. Explore the challenges, strategies, and best practices…
Rui Carvalho
Mar 18
What to expect in the first 10 days as a Data Engineer
What to expect in the first 10 days as a Data Engineer
Starting your Data Engineer Journey
Violeta Rakonjac
Mar 13
How To Become a Data Engineer P5: Mettle
How To Become a Data Engineer P5: Mettle
If you’ve been following this series, we’ve spoken about what mindset we need in order to get started, the overall fundamentals and skills…
MikeDoesEverything
Mar 12
How to Ruin a Data Engineers Day (in 20 Easy Steps)
How to Ruin a Data Engineers Day (in 20 Easy Steps)
A Step-by-Step Guide to Derailing a Data Engineer’s Workflow
Tim Webster
Mar 11
How I automate my Glue Job using AWS SNS and AWS Lambda
How I automate my Glue Job using AWS SNS and AWS Lambda
A user-friendly approach to scheduling AWS Glue jobs effortlessly
Lorena Gongang
Mar 6
My journey to my first job as a Data Engineer
My journey to my first job as a Data Engineer
First story of many
Violeta Rakonjac
Mar 4
My Advice for Aspiring Data Engineers (If You’re New to This)
My Advice for Aspiring Data Engineers (If You’re New to This)
5 Essential Lessons for New Data Engineers
Tim Webster
Mar 3
Where are the Jobs for Analytics Engineers?
Where are the Jobs for Analytics Engineers?
A simple strategy to find the perfect position for you
Fumiaki Kobayashi
Feb 28
To the Data Engineer: You Don’t Need to Master It All
To the Data Engineer: You Don’t Need to Master It All
Why Being a Data Engineer Doesn’t Mean Knowing Everything
Tim Webster
Feb 21
A Data Engineer’s Journey Through Curiosity
A Data Engineer’s Journey Through Curiosity
Ways I Fuel My Curiosity in Data Engineering
Tim Webster
Feb 16
About Art of Data Engineering
Latest Stories
Archive
About Medium
Terms
Privacy
Teams