4 Self-Created Data Tools That Saved Me 100 Hours Of Work In 2024

Data engineering projects and Python automations that have refined skills and (slightly) improved my quality of life.

Given the volume of articles I’ve written about technology, programming and data science, it might shock you to learn I’ve spent most of my life actively avoiding STEM fields. I finally realized why programming, out of any discipline, finally “clicked” — because I could use programming concepts to build tools that were concrete and, honestly, downright useful.

Before jumping into my “show and tell” below, I want to highlight that, as a data engineer, I’m a backend developer. I create scripts with an emphasis on autonomy and durability.

This is to say that I haven’t built front-ends (aside from dashboards, if you count that), for the tools I find useful. If all goes well, the output is simply new rows in BigQuery.

But, as you’ll see, this is critical when solving data-oriented problems.

As I’ve reached my early mid-career (I just completed my first full year in a senior role), I’m not necessarily at a point where I build side projects to complete a hypothetical portfolio. I now have 3+ years of work experience and a library of data engineering pieces to demonstrate my ability and knowledge of the field.

So, when I work on something outside of work, I focus my time on builds I’ll use…