Why Saying “I Don’t Know” Is Essential for Data Engineers

The Dangers of Pretending to Know Everything in Data Engineering

Photo by Jyotirmoy Gupta on Unsplash

Here are three words that will serve you well in Data Engineering:

“I don’t know.”

We can’t know everything. We just can’t, it’s not possible.

The sooner you come to peace with this fact, the better.

Here’s the problem

The data industry shows no signs of slowing down. This is just the way it is. Just when you think you’ve grasped one thing, you can expect something to come along and change the game completely.

Every other day, some new whatchamacallit, thingamabob, or doodah gets released, designed to make the world of Data Engineering and data in general easier. When this happens, it’s up to you to add it to the ever-growing list of things to have a vague, if not semi-good, understanding of.

You may change jobs — your new company may not use dbt or Airflow, but instead, use Prefect and Databricks. Sooner or later, you will run into a situation where you don’t have the answers.

During my transition into Data Engineering, I faced countless days of hopelessness staring at the sheer amount of things to learn or problems to fix…