3 Lines Of Python To Create Timezone-Aware Pipelines

Building time-sensitive data pipelines is a challenge; luckily, there is a way to build localized pipelines with 3 lines of Python.

Currently job searching? Give yourself an edge by developing a personal project using my free 5-page project ideation guide.

From 2014 to 2017 I lived in Phoenix, Arizona. Despite cities like Phoenix promoting cultural offerings and boasting one of the U.S.’s largest metro areas, Arizona is known for one thing.

It is one of the two US states (and the only contiguous state) that doesn’t follow Daylight Saving Time.

This makes Arizona and its time zone (Mountain Standard Time) a terrible place to farm but a great place to build data pipelines.

Standardizing datetime values in data pipelines remains a nuanced challenge. Handling time zones in Python is such a challenge that I once attended a 45-minute lecture on this problem at Pycon 23 a state away from Arizona in Salt Lake City, Utah.

The problem of time zone standardization has inspired the creation of timezone-aware packages like pendulum, which can be used to write datetime-based schedules in Airflow.

While field conversions are straightforward, passing a time zone naive value to an API can be difficult.