Dissecting Data #8: “Misleading Metrics and Dodgy Data”

Step right up to the grand stage of data analysis, where misleading metrics and dodgy data often perform their cunning acts of illusion. In this circus of statistics, numbers can be more deceptive than a magician pulling rabbits out of a hat. Let’s take a comical peek behind the curtain and see how data can be contorted into saying almost anything.

The Art of Data Distortion

Like a funhouse mirror, data can be bent and twisted to reflect what we want to see, not necessarily what’s there.

• Cherry-Picking Cherries: Ever seen data that only highlights the good stuff while sweeping the rest under the carpet? It’s like taking a selfie from just the right angle — flattering, but not the full picture. This can be seen for example in organisations where you might only be focusing on the positive feedback and comments from your customers and ignoring the less-than-favourable ones
• Correlation Circus: Remember, correlation does not imply causation. Just because ice cream sales and shark attacks both go up in the summer doesn’t mean ice cream causes shark attacks (or does it?). Another example is let’s say a company observes that as their social media advertising spend increases, so do their sales. They may conclude that the advertising directly causes the sales surge, ignoring other factors like seasonal demand or overall market growth.

Avoiding the Data Trap

To avoid falling into the pit of misleading metrics:

• Context is King: Always look at data in context. It’s like reading a detective novel — you won’t solve the mystery by reading just one page. Look, a business might see a rise in its stock price and interpret it as a sign of its success, ignoring broader market trends or economic stimuli that lifted the entire sector. Without understanding the wider economic context, the company’s analysis is incomplete, much like trying to understand a complex plot by only glancing at a single scene.
• Question the Source: Scrutinize where your data is coming from. Sometimes it’s less of a crystal ball and more of a murky, magic 8-ball. So let’s just imagine acompany might rely on performance data provided by a software vendor to evaluate the tool’s impact on productivity. However, this data could be selectively curated to highlight successes and downplay any shortcomings or issues(I’m sure this never happens 🤣). Without independent verification or cross-checking with external data, decisions based on this information can be misleading

The Honest Data Detective

In a world of data trickery, be the honest detective seeking the truth.

• Seek Multiple Perspectives: Look at data from different angles. It’s like getting a second (or third) opinion before making a big decision. So let’s say your company is considering entering a new market, relying solely on initial market research that shows high demand for your product 🧐just be sure not to overlook potential challenges such as local competition, regulatory hurdles, or cultural differences.
• Transparency Triumphs: Be clear about how and why you’re using certain data. Honesty is the best policy, especially when it comes to numbers. Now your company has decided to use customer purchase history data to personalise marketing efforts. You don’t keep this strategy opaque, instead, you openly communicate to customers how their data is being used and the benefits they can expect, such as tailored discounts and product recommendations.

The Takeaway

In the grand theatre of data analysis, not everything is as it seems. Misleading metrics and dodgy data can lead us astray, but with a keen eye for detail and a commitment to transparency, we can unmask these illusions and reveal the true story behind the numbers.