It’s Time for a dbt Data-Review Revolution

Data Engineers and non-technical stakeholders can review data together

Dave Flynn
In the Pipeline
3 min read3 days ago

--

Data teams, are you ready for collaborative PR review? I don’t mean a few lifeless comments back and forth in the PR thread, I mean true, interactive, collaborative, data review.

dbt data project pull requests need a special review process — It’s time for a data review revolution!

Recce Cloud provides a platform for data engineers and non-technical stakeholders to review data-change together

Stop fixing in prod

Data is too important to fix in prod. Unless you like getting “the call” and working under pressure, but your heart can’t take 130/80 forever.

It’s time to stop:

  • writing SQL and pushing directly to prod;
  • creating unclear PR comments without data checks;
  • clicking merge and hoping for the best.

By validating data together, you’ll have trust in your data.

Collaborate on accurate data

Take down the walls that separate data engineers and data stakeholders.

Data Engineers and Stakeholders working together, mass hysteria!

PR review used to be a static process, a one way street. Here’s the SQL, here’s some random checks (if you’re lucky), merge it if you dare.

We say, “no more!”.

Analysts want data, engineers want stability, let’s work together!

We’re talkin’ data

Sure, the standard PR comment thread works for software, but we’re talking about data. We’re special (don’t tell anyone), so PR review for data needs a special review process.

Reading ‘Files changed’ and checking the SQL code only tells part of the story. It’s the data we care about, show us the data.

  • How did it change?
  • What does it mean?
  • Is that right?

For data that matters, these questions matter.

Build better data (and data team)

Better data is what we want, and by building a better process for data PR review, you’ll build a lot more than that:

  • Build relationships through collaboration by working together on data checks and validations
  • Build trust in your data and your team by visualizing and interpreting change with stakeholders
  • Build confidence for downstream use by providing a solid foundation of data

Get Started

Recce Cloud is in early access, and brings all of this goodness to your PR review process for dbt data projects.

  • Create and share data validation checks with your team
  • Manage multiple PRs and continue validating where you left off
  • Automate data validation checks with fine tuned control
  • Block merging until data checks have been reviewed
  • Reproducible data validation environments

…all while working as a team.

Oh, and open-source Recce is available right now.

--

--

Dave Flynn
In the Pipeline

Dave is a developer advocate for DataRecce.io — the data modeling validation and PR review toolkit for dbt data projects