TDS Archive

An archive of data science, data analytics, data engineering, machine learning, and artificial intelligence writing from the former Towards Data Science Medium publication.

Member-only story

Stop Using The Term “Data Engineer”, There’s Something Better

mehdio
TDS Archive
Published in
4 min readNov 25, 2021

--

Git fork at Standup — Image by the Author

The Data Engineer scope has never been so large and confusing. Some new roles definitions have been popping up to focus on specific areas, but they are still underused. If you look at Data Engineer job offers, you may get confused as the content and job itself may differ from company to company. In this article, we will remind ourselves what the close cousins of data engineer roles are to be able to spot a misnomer.

📕 Data Engineer aka Database Administrator (DBA)

This role has been fading out as most analytics use cases are based on OLAP systems. Some companies are still heavily relying on OLTP databases (e.g. MySQL, Postgres) for their analytics. They are many reasons for this :

  • They don't have any pub/sub system to fetch data in real-time and have low-latency query requirements (milliseconds response time)
  • They still haven't invested in a data platform and just relying on existing software engineers to do analytical jobs
  • They may need a DBA to maintain their production database: performance tuning, monitoring, migrations, backups, etc.

--

--

TDS Archive
TDS Archive

Published in TDS Archive

An archive of data science, data analytics, data engineering, machine learning, and artificial intelligence writing from the former Towards Data Science Medium publication.

mehdio
mehdio

Written by mehdio

✍️ Data Engineering, Tech Career & Code | 🎥 https://www.youtube.com/@mehdio

Responses (4)