The technology world is an interesting market where people doing the same work can be labeled and treated differently. ‘Data Engineering’ has become a buzzword in the last 2–3 years, but it is not at all a new requirement. But, definitely, it has become a new trend in the post-covid world.
#date Engineering # Data Engineer
If we go back 12–15 years, we still had a community of data engineers known as ETL developers who were using fancy tools in the market to process the data, and move the data from Point A to Point B. If we go back even further, we will see the same, but maybe using a different process or custom coding.
We need to understand that data engineering is just a skill. We all have definitely created some sort of an excel document to calculate our share after a group tour with our friends or calculate how much we would be saving at the end year based on our income and expenses. In data engineering terms, we extracted the data from some sources (maybe we just entered it manually only), just added a few transformations on top of it (e.g. formula / derived fields, etc), and then used it. Congratulations, you had become a data engineer from that day.
We don’t need to use fancy tools, or techniques to be data engineers. A techie who is writing a simple SQL stored procedure using some tool can be equally qualified as someone using jinjasql, data frames, or using any different solution.
Then what matters?
End of the day, the first thing that matters is to understand what needs to be done. People who understand the ‘what’ part the best, will become the most successful data engineers in long term. ‘How’ comes later, it will go through continuous transitions as the technologies evolve. Being relevant to the latest ‘How’ trends would help, but without ‘what’ …..it means nothing.
So use your time to understand ‘ what’ and use your curious mind to learn and think more about ‘how’.
That’s all for today. See you later with more shortly.
“Keep learning and Keep growing” !!
