Homepage
Open in app
Sign in
Get started
Making Sense of Data & Helping Others Grow: Tips, Advice, and Stories from the Front Lines of Data Engineering
MOST READ
WRITE FOR US
FEATURED ARTICLE
LinkedIn
Follow
How to Clean Up a CSV File
How to Clean Up a CSV File
Complete simple tasks faster
Violeta Rakonjac
Aug 8
What They Don’t Tell You About Being a Data Engineer
What They Don’t Tell You About Being a Data Engineer
What No One Prepares You For
Tim Webster
Aug 1
Microsoft Fabric End-to-End Project — with Shorcut, Data Pipeline, DataFlow, and Regular File…
Microsoft Fabric End-to-End Project — with Shorcut, Data Pipeline,...
The aim of this project is Microsoft Fabric Project is to:
Musili Adebayo
Jul 30
5 Lessons from 5 Data Teams (Lessons for Data Engineers)
5 Lessons from 5 Data Teams (Lessons for Data Engineers)
Common Sense Data Engineering Lessons Across Teams
Tim Webster
Jul 26
Learn with me: Using .env file in python
Learn with me: Using .env file in python
Hello everyone,
Violeta Rakonjac
Jul 25
Steady as She Flows: Rate Limiting in Apache Beam Pipelines
Steady as She Flows: Rate Limiting in Apache Beam Pipelines
A Data Engineer’s Toolkit for Controlled Streaming
Josi Aranda
Jul 21
Detecting Risks in Claims Data Using Databricks and PySpark.
Detecting Risks in Claims Data Using Databricks and PySpark.
Databricks and PySpark for Efficient Claims Risk Management in healthcare claims.
Brahmareddy, The Data Engineer.
Jul 21
Data Architecture Principles Every Data Engineer Should Know
Data Architecture Principles Every Data Engineer Should Know
Discover the key principles of data architecture critical for any data engineer.
Rui Carvalho
Jul 18
The Reality of Mistakes in Data Engineering (Tips from the Trenches)
The Reality of Mistakes in Data Engineering (Tips from the Trenches)
How to Avoid the What-the-Hell-Have-I-Just-Done Moments
Tim Webster
Jul 18
How to Fix an Error in the DBT Command with the Empty Flag Due to _TABLE_SUFFIX or _PARTITIONTIME
How to Fix an Error in the DBT Command with the Empty Flag Due to _...
Utilize dbt variables to fix the problem
Fumiaki Kobayashi
Jul 18
Data Transfer Metrics: Bandwidth, Throughput and beyond
Data Transfer Metrics: Bandwidth, Throughput and beyond
A Data Engineer’s Guide to Understanding and Leveraging Data Transfer Metrics
Santosh Joshi
Jul 13
A Data Engineer’s Biggest Hurdle (Figuring Out What People Want)
A Data Engineer’s Biggest Hurdle (Figuring Out What People Want)
Bring Real Value by Asking the Right Questions
Tim Webster
Jul 10
11 Useful BigQuery Tricks (for Newbies)
11 Useful BigQuery Tricks (for Newbies)
Tricks Every New BigQuery User Should Know
Tim Webster
Jul 1
SQL Aggregation: Rollup Explained
SQL Aggregation: Rollup Explained
Understanding SQL Rollup for Enhanced Data Aggregation
Santosh Joshi
Jun 30
SQL Aggregation: Grouping Sets and Cube Explained
SQL Aggregation: Grouping Sets and Cube Explained
Comprehensive Guide to SQL Aggregation: Exploring Grouping Sets and Cube
Santosh Joshi
Jun 30
How Idempotent Data Pipelines Prevent Data Consistency Issues
How Idempotent Data Pipelines Prevent Data Consistency Issues
Discover the concept of idempotent data pipelines in maintaining data consistency in your database. Learn practical strategies to build…
Rui Carvalho
Jun 28
Why Data Engineers Must Balance AI Use with Hands-On Problem Solving
Why Data Engineers Must Balance AI Use with Hands-On Problem Solving
Embracing AI Without Losing Critical Skills
Tim Webster
Jun 20
Exploring the Latest Features in Databricks: A Comprehensive Guide
Exploring the Latest Features in Databricks: A Comprehensive Guide
Databricks Latest Features Explained.
Brahmareddy, The Data Engineer.
Jun 17
Why Data Engineers Can’t Afford to Wing It
Why Data Engineers Can’t Afford to Wing It
Why Preparation Is Everything
Tim Webster
Jun 11
Should my data engineering title be revoked if I don’t know Python?
Should my data engineering title be revoked if I don’t know Python?
A self-discovery into life, liberty, and the pursuit of data.
Monica Miller
May 29
4 Stupidly Simple Strategies for Your Data Engineering Career
4 Stupidly Simple Strategies for Your Data Engineering Career
Learn, Try, Stay, and Prepare for Anything
Tim Webster
May 27
Understanding Reverse ETL to Increase Operational Effectiveness
Understanding Reverse ETL to Increase Operational Effectiveness
Rui Carvalho
May 16
Why you should change your CSV export method on Databricks
Why you should change your CSV export method on Databricks
How to efficiently export spark dataframe to CSV
ANGE KOUAME
May 14
17 Non-Code Lessons Every Data Engineer Should Remember (Especially Me)
17 Non-Code Lessons Every Data Engineer Should Remember (Especially...
No-Cost Secrets to Thriving as a Data Engineer
Tim Webster
May 10
About Art of Data Engineering
Latest Stories
Archive
About Medium
Terms
Privacy
Teams