Write Your First SQL ETL Pipeline (Part I)

You’ve written enough queries; it’s time to move to production.

At Some Point You Stop Querying Data And Do This Instead

The ETL “extract” step in real life. Photo by Neil Mark Thomas on Unsplash.

Despite being a 40-year-old method for communicating with databases, the debate surrounding SQL’s status as a programming language persists.

However, regardless of your stance on whether SQL is a programming language or not, as you advance in learning SQL, you will inevitably move from writing simple (then complex) queries to creating full scripts.

Depending on your organization, your script may be treated like just another saved query or, possibly, be converted into a view.

If you’re working at Google, you will treat your SQL script like full-on code.

Regardless, if you apply your SQL skills to a more infrastructure-oriented discipline like data engineering, you will be using SQL to not only query data sources, but also to actively create them.

If you stick around, in the next few minutes we’ll examine how to apply your SQL knowledge to creating end-to-end pipelines without ever having to leave your SQL environment.

E-T-What?