Netflix has 223 million subscribers in countries all around the world watching over 200 million hours of Netflix each day. Assuming an hour of Netflix HD content is three GB of data, then Netflix delivers more than 6 petabytes of data to customers daily. So how do they maintain the…

This is your one-stop shop for all things data observability. Learn what data observability is, who it’s for and who uses it, how it benefits organizations, how it works, key terms, use cases, and best practices. What is data observability? Definition Data observability refers to the ability of an organization to see and understand the…

The term “data quality” can be vague. It’s abstract to the point where it might seem less urgent. Programming a new feature or putting out a sudden customer success fire? Probably WAY more important than “data quality”, right? But that’s not the experience of engineers, data scientists, executives, and product…

While there’s widespread consensus that data quality is very important, it’s still a relatively new field. Even when organizations prioritize their data pipelines, they are likely newcomers to a lot of the terms and context surrounding data quality. That’s why we’ve compiled this dictionary based on common terms we encounter…

So you’re considering a data observability platform, but want to make sure you’re choosing the best one for your business. We’ve got some tips to help you run a thorough evaluation process. We wrote this article after running hundreds of POCs (proofs of concept) — some great, and some painful…

Test-Driven Development (TDD) is a discipline wherein automated tests are written before implementing production code. Kent Beck, the founder of this core Agile practice, put it poetically when he wrote [you] can find in TDD a way to do well by doing good. TDD’s relevance follows a predictable cycle. Peaks…

Every growing company eventually has to ask itself the existential question, “how do we maintain data quality?” You can’t ignore this question. It will just rear its head at a later date, with potentially catastrophic consequences. If you tackle the question in good time, you’ll reap wonderful rewards: meaningful analysis…

Chad Sanderson is the head of the data platform at Convoy, Inc. In this episode of The Observatory, he speaks with Kyle Kirwan, CEO and co-founder of Bigeye, on hub-and-spoke data teams, the resurgence of 1990s data initiatives, and resolving long-term technical debt. Read on for a lightly edited…

What is data observability? Analogous to observability in software engineering, data observability refers to the practice of instrumenting your data systems to give a comprehensive view of what is going on in each component of your data stack at any given time. Convincing your organization that you need a data observability solution Building a data observability practice in your organization often requires upfront investments…