The Prefect Blog
Published in

The Prefect Blog

How to Build a Modular Data Stack — Data Platform with Prefect, dbt and Snowflake

Build a data platform with Snowflake & dbt, and use Prefect to observe and coordinate your data stack

Modern Data Stack vs. Hadoop Data Stack — for a full meme, visit @moderndatastack on Twitter

Part 1: defining the problem

Part 2: the end outcome of this tutorial series

Part 3: implementing the building blocks

Part 4: scheduling, data ingestion, backfilling, and getting started with local dataflow development

Part 5: failure handling with alerts & retries, dbt transformations & separate tasks per dbt node

· Failure handling: alert notifications & retries
Slack alerts on failure
Automated retries
Manual retries from the UI
· Simple dbt transformation flow
Parametrization
Alerts on failure in dbt tests

· Parsing the dbt manifest
Orchestrating dbt from manifest
Slack alerts and troubleshooting of dbt runs

Part 7: Parent flow tying all components together

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store