Data Science

Ten Must Ask Questions for Data Engineering Projects

Think of this as a checklist of the questions that you need to ask before giving an estimate of when you will deliver, or when you will start design. Andyou definitely need to ask these questions before coding.

Developing Reproducible Data Science Projects

Just like any scientific field, data science is built on…

Data Quality for Data Engineers

If you manage and deploy data pipelines, how do you ensure they are working? Do you test that data is going through? Do you monitor uptime? Do you even have tests? If so, what exactly do you test?

Cultivate good working relationships with data consumers

The relationship between Data Engineering and the data consumers is always complex, be they data scientists, BI or one of a multitude of…

Read in the data in a Pandas DataFrame like an expert

Pandas DataFrame is widely popular in a typical…

Why Data Science Teams Need Generalists, Not Specialists

In The Wealth of Nations, Adam Smith demonstrates how the division of labor is the chief source of productivity gains using the vivid example of a pin factory assembly line: “One person draws out the wire, another straights it, a…

Preventing the Data Lake Abyss

How to build trust in your data stores

When to Be Cautious About Sharing Data

When I was fresh out of business school, I was taught that silos are bad and toxic to…