Hypodermic needle” (CC BY 2.0) by aldenchadwick. HIV doesn’t cause hepatitis C. Both can be caused separately by using needles.

How to Combat Cargo Cults

Mike Zawitkowski
Aug 28, 2017 · 6 min read

This is part two of a series. Part one covered the basics of the cargo cult phenomenon. In this second and final part we cover what to do to counteract them.

After being diagnosed with having contracted HIV, a patient will often learn they have hepatitis too. This leads some to the conclusion that HIV may have caused the hepatitis infection. Every month there is an average of two dozen web searches in the US for combined information about “HIV” and “hep.”

Google Trends on US-based searches for “hiv hep c” since 2004.

For those trying to understand why they contracted HIV and then hepatitis, it’s not because HIV made them more susceptible to other diseases. According to this page on the aids.gov website:

“these viruses are transmitted in some of the same ways HIV is transmitted — through injection drug use and condomless sex…”

The aids.gov website goes on to explain that liver disease related to hepatitis B (HBV) and hepatitis C (HCV) viruses has become the leading cause of non-AIDS-related death among people living with HIV. This may explain why the trend of these searches has been slowly increasing over time.

Correlation” by xkcd

In both cases—cargo cults and HIV/hepatitis comorbidity—we have two events. One event is followed by the other event. Event A happened, and then event B happened soon after. This leads some to jump to a false conclusion about a causal relationship. “Event A must be causing event B, right?” The truth is that both event A and B are being caused by a third event. Correlation does not imply causation.

This happens in businesses too. You try to do a day-parting analysis to see if a site or service is more popular on weekends versus weekdays, in an attempt to apply more ad spend or other investments to the days and times when it will do the most good and yield the highest ROI for the company. Too often there is not enough data to make a conclusive decision. Sure, the raw count of visits and revenue on weekends is higher than on weekdays, but not significantly higher, you explain. You can’t reject the null hypothesis, which is that there are no day-parting behavior patterns to exploit.

Null Hypothesis” by xkcd

If event A happens first, and then event B happens, it doesn’t mean one caused the other. There could be no relationship between them. There could also be a third element C that causes A and B to happen. This is a very difficult concept for humans to grasp. Human tendencies like survivorship bias exacerbate this.

Survivorship Bias” by xkcd.com, used with permission

Here are a couple of tips and tricks to consider when trying to overcome cargo cults in business and life:

Go with your gut; throw out the data. Do not use faulty data to support or reject what is ultimately a decision 100% based on intuition. Blatantly state and declare “There is no data that conclusively supports or rejects this. I am doing it in the absence of data. This decision has not/cannot be evaluated by anyone.” Kind of like the statements the FDA writes on bottles of pills of herbal remedies.

Publish your process, warts and all. There are instances where even in scientific academia foundational research papers were written on experiments that were not replicated or checked for errors. They were published and taken at face value, and lacked sufficient information about why the finding may have been completely invalid. It was a long time before some of those mistakes were corrected, because nobody thought to check! This sort of thing happens all the time.

Fortunately we have mechanisms for correcting these mistakes. and part of that mechanism is full disclosure of everything you did or think that had an impact on the accuracy of the result. If you provide enough information to allow others to try and repeat, critique, and reproduce your work, everybody wins. Hiding or omitting the parts of your process that led to a result may save your ego and get you some funding in the short term, but history is long and will uncover your error in the long term.

Unscientific” by xkcd

Don’t invest in the unknown. In the dayparting example, I remember once having this argument about ad spend at a company. If you looked at the raw numbers, it seemed like weekends were cheaper and more profitable. More new registrations and customers that generated revenues were joining on Sunday than any other day. Applying a simple statistical test flipped the story in the data. It showed that the difference between Sundays and non-Sundays could be attributed to random chance.

Betting the total ad spend on that one day was a very bad idea, and there were many reasons why. Unfortunately I lost that fight or gave up or simply was ineffective in stating my case. It is an argument I regret losing. If I had managed to keep spend the way that it was, it would have allowed us to prove or disprove whether that day-parting strategy had any merit. Instead it took longer to collect enough observations to compare Sundays with other days.

It turned out that it was the wrong decision. The Sunday phenomenon was a seasonal effect. It was caused by other organizations impacting both our ad spend AND the weekend traffic to our products and services. Just like HIV and airplanes, B wasn’t caused by A. Both were independent events that were being caused by a third event.

Looking back, that was a lot of money that was poured into Sundays. In an organization like the one I worked for, there is no shortage of ideas of where to put extra budget in order to improve things. It would have been much better to stick with the plan, or take a scientific approach with the ad budget, or even stop spending it on ads and put that same money toward some other initiative that we could have been more confident would yield a positive result.

Get an outside perspective. One of the reasons consulting companies like my firm Acorn Analytics are successful is because of the value of a second opinion. I recall Malcolm Gladwell once saying, “to a worm in horseradish, the world is horseradish.” When you are immersed in the day-to-day flavor of your organization’s culture, it’s hard to see the big picture.

Good consultants work to improve lives for a great number of organizations and individuals. This variety of experiences includes problems that transcend industries and geographies. He or she understands that choosing the right approach to solving a problem depends on the stage of development of the organization. A company needs to know what it shares in common with other companies, but also what’s unique about its situation that sets it apart.

Epilogue

I’ll leave you with a quote from the book, one that won’t spoil it, because Moore’s book is a fun read. At one point in the book, the ghost/spiritual leader of the Shark People tells the protagonist this:

“Islands are like, you know, incubators. You got to start things and let em grow. Isolate ’em. That’s why all your loony-toon cult guys have to get their people out in the boonies somewhere where no one can talk any sense into ’em.”

The sad truth of the matter is that Moore’s character Vincent wasn’t entirely right about this. As Feynman figured out, we don’t have to go out to the boonies to find cult-like behavior. It’s all around us, every day.

Acorn Analytics Blog

Articles from Acorn Analytics Inc. (https://www.acornanalytics.com)

)

Mike Zawitkowski

Written by

Principal at Acorn Analytics. Visit https://www.acornanalytics.com for more info.

Acorn Analytics Blog

Articles from Acorn Analytics Inc. (https://www.acornanalytics.com)

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade