Member-only story

These 6 Data Engineering Shortcuts Will Burn You In Year 1

For new data engineers, it’s tempting to cut corners; avoid common development shortcuts to work toward advancement.

Zach Quinn
Pipeline: Your Data Engineering Resource
9 min readJan 27, 2025

--

I received one of the most accurate and ominous pieces of advice during year 1, week 1 into my role as an entry-level data engineer.

Noticing my visible nervousness as I struggled to ingest a deluge of information on everything from basic BigQuery features to the intricacies of Airflow production DAGs, my then-senior engineer said: “You don’t have to know this all now. You might be good at this in a year.” Thanks to a slow fall (in which I could refine and master job-critical skills) and sheer will, I felt most “up to speed” in the next 3–6 months.

But then, just as I felt like I was a productive member of the team (no longer on “new guy” probation), I felt myself stumble a bit. Luckily, there was no singular, catastrophic inciting incident. Just a series of small blunders and slip ups, mostly noticed by me, that would delay my process or, in worst-case scenarios, the projected release dates of my builds.

Where I had lacked confidence before, most of my issues stemmed from being a bit overconfident in a job I really had only been doing for a few months. This overconfidence combined with an ambition to excel led me to cut corners so I could take on and complete more work.

--

--

Pipeline: Your Data Engineering Resource
Pipeline: Your Data Engineering Resource

Published in Pipeline: Your Data Engineering Resource

Your one-stop-shop to learn data engineering fundamentals, absorb career advice and get inspired by creative data-driven projects — all with the goal of helping you gain the proficiency and confidence to land your first job.

Zach Quinn
Zach Quinn

Written by Zach Quinn

Journalist—>Sr. Data Engineer; new stories weekly.

No responses yet