The Rise of the Data Engineer
Maxime Beauchemin

Note that I’m not arguing that people don’t need abstractions, GUIs, or that everyone should write data pipelines as code, but I’m arguing that it’s the job of data engineers to create the actual abstractions that their peers need, the inputs (GUI / config file / code) and outputs (dashboards, tabular datasets, csvs) of these “data engineering frameworks or services” should be defined on a case by case basis. These abstractions may be very different in different environments, and should be informed by data engineers finding repeating patterns in the work that they do and the services they provide: automating their own processes.

Show your support

Clapping shows how much you appreciated Maxime Beauchemin’s story.