TDS Archive

An archive of data science, data analytics, data engineering, machine learning, and artificial intelligence writing from the former Towards Data Science Medium publication.

Member-only story

Open-Source Data Observability with Elementary — From Zero to Hero (Part 2)

6 min readSep 10, 2024

--

Photo by Caspar Camille Rubin on Unsplash

In the previous part, we have set up Elementary in our dbt repository and hopefully also run it on our production. In this part, we will go more in detail and examine the available tests in Elementary with examples and explain which tests are more suitable for which kind of data scenarios.

Here is the first part if you missed it:

Opensource Data Observability with Elementary – From Zero to Hero (Part 1)

While running the report we saw a “Test Configuration” Tab available only in Elementary Cloud. This is a convenient UI section of the report in the cloud but we can also create test configurations in the OSS version of the Elementary in .yaml files. It is similar to setting up native dbt tests and follows a similar dbt native hierarchy, where smaller and more specific configurations override higher ones.

What are those tests you can set up? Elementary groups them under 3 main categories: Schema tests, Anomaly tests, and Python tests. So let’s go through them and understand how they are working one by one:

Schema Tests :

--

--

TDS Archive
TDS Archive

Published in TDS Archive

An archive of data science, data analytics, data engineering, machine learning, and artificial intelligence writing from the former Towards Data Science Medium publication.

Sezin Sezgin-Rummelsberger
Sezin Sezgin-Rummelsberger

Written by Sezin Sezgin-Rummelsberger

Data Engineer | Writing articles on Data & Analytics Engineering | MSc | https://de.linkedin.com/in/sezinsezgin

Responses (2)