How data paranoia is good for you

And actually potential to make your life better

Alex Harlan
Sep 4, 2018 · 2 min read

You’ve gathered all your data, perhaps you’ve spent hours cleaning it, and when you finally start analyzing it you see something that doesn’t look right. Being a good analyst you digger deeper, pulling on that thread until you realize… this was not the data you were looking for.

Maybe you thought you had data for some demographic when you really only had a subset of the data. Maybe you realized that you’ve made a wrong assumption about the data you are working with.

But what if you didn’t? What if all the data intuition you’ve cultivated fails you? You could end up making recommendations on the basis of faulty assumptions or bad data.

All the sophisticated analysis in the world won’t save you if you don’t truly know your data. This is where a healthy level of paranoia can save you.

When I say data paranoia I’m not talking about how you should be worried about companies collecting data on you, or paranoid that statistics are being used to lie and manipulate people.

I am telling you to be paranoid about your data itself. Paranoid about the data’s quality, paranoid about its completeness, and paranoid about your understanding of it. A little bit of paranoia can save you mistakes down the road.

Sure, maybe paranoia conjures up the wrong image. Maybe you prefer skeptic. I don’t, and I’ll tell you why.

A skeptic, to me, is someone who looks at the analysis after the fact and asks how we are sure our conclusions are valid. They wait for someone else to provide solid proof. Data analysts can save time by adopting paranoia that we have missed some critical understanding of our data. Approaching problems like this has the added bonus of benefiting your entire analysis process when you truly understand the data you are working with.

I’m curious. Tell us about times your analysis could have benefited from a healthy dose of data paranoia and what went wrong.

Alex Harlan

Written by

A used to be math major, doing things with data. Data Analyst @ FairTradeUSA alexforrest.github.io

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade