When Isn’t Automation Worth It?

A simple formula to calculate ROI on development time

Currently job searching? Give yourself an edge by developing a personal project using my free 5-page project ideation guide.

My favorite teammate is perpetually silent and, despite never completing a ticket, produces damn good code. Like all legends, they have a one-word name: Bro. As much as I’d like to stretch this exercise in personification to a groan-inducing conclusion, I’ll save you and I the trouble.

Bro is not a person; it’s a GitHub repository.

Created by one of my peers, Bro’s sole purpose is to store code that simply makes life easier (like a true bro would).

Typically this comes down to one goal: Automation.

Bro contains scripts to automate schema generation, complete backfills and any other “menial” data engineering labor. Bro has truly done me a solid on many occasions. Despite not being deployed to production or even being included in a formal sprint, the scripts within Bro have required many hours of development, testing and tweaking.

This iteration is worth it because the backfill script alone has saved the team at least 40 hours of work.

But the ROI (return on investment) of development time isn’t always this high. And, especially for newer data engineers, setting out to automate a particular task may require an amount of development time that is disproportionate to the…