The Prefect Blog
Published in

The Prefect Blog

Modular Data Stack — Build a Data Platform with Prefect, dbt and Snowflake (Part 7)

Coordinate dataflow across multiple domains and teams, and orchestrate your data platform with a parent flow in Prefect

Prefect as a bot coordinating work across multiple domains and teams in a dataflow universe and orchestrating your data platform with a parent flow
Table of contents· Orchestrating data platform with subflows
Pros of subflows for data platform orchestration
Cons of subflows (and how to overcome those)
TL;DR of subflows
· Orchestrating data platform with deployments
How to run deployments
🤖 Parent flow running deployments
Pros of running deployments
Cons of running deployments
TL;DR of running deployments
· The key takeaway about both approaches
· Fun fact
· Next steps

Orchestrating data platform with subflows

Pros of subflows for data platform orchestration

Cons of subflows (and how to overcome those)

TL;DR of subflows

Orchestrating data platform with deployments

How to run deployments

🤖 Parent flow running deployments

python flows/orchestration/run_deployments/parent.py
prefect deployment run parent/local-process
The parent flow on the right has more runs because we added the marketing flows as part of the same parent flow

Pros of running deployments

Cons of running deployments

TL;DR of running deployments

The key takeaway about both approaches

Fun fact

Next steps

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store