Forget Your Sprint — When Data Engineering Tasks Must Be Completed In The Same Day

When data engineers must meet an 8-hour, same-day deadline and how to get it done with minimal panic.

Currently job searching? Give yourself an edge by developing a personal project using my free 5-page project ideation guide.

When I have school-related nightmares, I don’t dream of high school. I shudder, bolt upright and my eyes snap open when I remember the specific terror of missing an assignment for broadcast reporting class in journalism school. And unlike 99% of academic courses, my journalism classes didn’t assign homework. Instead, work was to begin and end within a class period.

In reporting, this end of day deadline is known as a day turn.

It’s incredibly common and, if missed, unquestionably devastating, because in a news cycle hungry for information even incomplete pieces get published–and definitely noticed.

Needless to say, transitioning from day-of assignments to a two-week Agile cycle provided a bit more breathing room and allowed me to ease the pace at which I digest, develop and release data pipelines.

For the record, I’ve still written (and humble bragged) about developing pipelines in a short amount of time.