The Data Sniff Test
This didn’t pass my sniff test because it just seemed extraordinary unlikely that Instagram’s terms of service changes outraged such a massive percentage of users in such a short period of time. So much so that these users left the service en mass.
If anything, the chart looks more like a data error. As if somehow AppData become unable to capture a portion of Instagram usage starting on the 24th.
Yet this story went on for hours. Sending FB stock down 3% on the open:
“This data is inaccurate. We continue to see strong and steady growth in both registered and active users of Instagram.”
It took hours for this all to get straightened out, but even within the first minutes of so many outlets running the story, it didn’t smell right and ran against gut instinct. Is the average person’s affection for Instagram trumped by terms of service changes? Most of the people I know using Instagram don’t even know about the kerfluffle.
False precision and the reliance on data in the absence of back of the envelope math or observational logic almost always results in mistakes. If the numbers defy logic, just because they are numbers, does not allow them to trump that logic. And often, back of the envelope calculations, accompanied by logic are far more accurate and actionable.