How data paranoia is good for you
And actually potential to make your life better
You’ve gathered all your data, perhaps you’ve spent hours cleaning it, and when you finally start analyzing it you see something that doesn’t look right. Being a good analyst you digger deeper, pulling on that thread until you realize… this was not the data you were looking for.
Maybe you thought you had data for some demographic when you really only had a subset of the data. Maybe you realized that you’ve made a wrong assumption about the data you are working with.
But what if you didn’t? What if all the data intuition you’ve cultivated fails you? You could end up making recommendations on the basis of faulty assumptions or bad data.
All the sophisticated analysis in the world won’t save you if you don’t truly know your data. This is where a healthy level of paranoia can save you.
When I say data paranoia I’m not talking about how you should be worried about companies collecting data on you, or paranoid that statistics are being used to lie and manipulate people.
I am telling you to be paranoid about your data itself. Paranoid about the data’s quality, paranoid about its completeness, and paranoid about your understanding of it. A little bit of paranoia can save you mistakes down the road.
Sure, maybe paranoia conjures up the wrong image. Maybe you prefer skeptic. I don’t, and I’ll tell you why.
A skeptic, to me, is someone who looks at the analysis after the fact and asks how we are sure our conclusions are valid. They wait for someone else to provide solid proof. Data analysts can save time by adopting paranoia that we have missed some critical understanding of our data. Approaching problems like this has the added bonus of benefiting your entire analysis process when you truly understand the data you are working with.
I’m curious. Tell us about times your analysis could have benefited from a healthy dose of data paranoia and what went wrong.
