The Prefect Blog
Published in

The Prefect Blog

Modular Data Stack — Build a Data Platform with Prefect, dbt and Snowflake (Part 4)

Scheduling, data ingestion, and backfilling implemented with modular building blocks and simple deployment patterns

Modular Data Stack — as colorful and diverse as your teams, projects, and the capabilities of your tools.
Table of contents· 🤖 Creating a flow run from deployment
· 🤖 Inspecting parent and child flow runs from the UI
· 🤖 Scheduling a deployment
Scheduling is decoupled from execution
🤖 Scheduling from the UI
🤖 Scheduling from CLI
· Local development of data platform workflows
🤖 Getting started with data platform development
🤖 Coordinate Python with Prefect
· Ingestion flow
What is data ingestion
Example: on a conceptual level
🤖 Running the ingestion flow locally
· Backfills
🤖 Backfilling locally from code IDE
🤖 Backfilling from a terminal
🤖 Backfilling from the UI
· Adding ingestion logic to a custom Prefect Block
Prefect Block capabilities
Using a custom Prefect Block for data ingestion
Using the same Prefect Block for ML use cases
Benefits of Prefect Blocks
· Next steps

🤖 Creating a flow run from deployment

prefect agent start -q default
prefect deployment run jaffle-shop-ingest-transform/local-process

🤖 Inspecting parent and child flow runs from the UI

🤖 Scheduling a deployment

Scheduling is decoupled from execution

🤖 Scheduling from the UI

🤖 Scheduling from CLI

Local development of data platform workflows

🤖 Getting started with data platform development

🤖 Coordinate Python with Prefect

Ingestion flow

What is data ingestion

Example: on a conceptual level

🤖 Running the ingestion flow locally

python flows/ingestion/ingest_jaffle_shop.py

Backfills

🤖 Backfilling locally from code IDE

🤖 Backfilling from a terminal

🤖 Backfilling from the UI

Adding ingestion logic to a custom Prefect Block

Prefect Block capabilities

Using a custom Prefect Block for data ingestion

Using the same Prefect Block for ML use cases

Benefits of Prefect Blocks

Next steps

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store