The End of ETL As We Know It

If you’re as sick of this three-letter phrase as I am, you’ll be happy to know there is another way.

Paul Singman
Jan 20 · 4 min read
Image for post
Image for post

Take a Look Around You…

Replacing ETL with Intentional Data Transfer

Image for post
Image for post
Example IDT architecture on AWS with a real-time Lambda consumer + durable storage to S3 | Image by author

1. IDT Forces Upfront Agreement on a Data Model Contract

{
"event_name": "transaction",
"user_id": 12345,
"event_action": "purchase",
"action_object": "gift_card",
"event_timestamp: "2021-01-02T03:04:05+01:00",
...
}

2. IDT Removes Data Processing Latencies

Taking The First Steps

Whispering Data

Softly sharing the best kept analytics and productivity secrets

Sign up for Whispering Data

By Whispering Data

Whispering Data is a publication for all the data & productivity secrets you wish you knew years ago! Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Paul Singman

Written by

ML Engineering Lead at Equinox. Whisperer of data and productivity wisdom. Standing on the shoulders of giants.

Whispering Data

Whispering Data is a Medium publication for all the data & productivity secrets you wish you knew years ago!

Paul Singman

Written by

ML Engineering Lead at Equinox. Whisperer of data and productivity wisdom. Standing on the shoulders of giants.

Whispering Data

Whispering Data is a Medium publication for all the data & productivity secrets you wish you knew years ago!

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store