Finding Analytic Success — part 1
Hint — Look at the intersection of Excellence & Return
Around the globe, organizations struggle to find and keep analytic success. Analytic units are structured and restructured in a never ending attempt to create a true “Center of Excellence” and a consistent “Return on Investment”. These are buzzwords that I often hesitate to bring up in conversation because I am always inclined to cringe when others do. Their meaning has become jaded and lost in the corporate world.
Across the land, executive staff actively ignore analytics as unreliable, a lesser priority, or not affordable. Worse still other executives actively weaponize analytics. Tortured data is manipulated in mad schemes for glory and power. Data is powerful in the right hands — but neither of these scenarios forward that cause. They are far more of a negative feedback loop.
So how does an organization combat this trend? How does one overcome jaded executives, endless reorganization, analytic arms races, and bastardized processes “fixed” to reinforce the broken stories of broken leaders?
For starters, take a deep breadth. This is science after all, or at least it should be. Science has had the right answer for decades (although many studies show they have lost their way, too — but we can rant about that another day).
Also recognize you are not alone. People like Mark Cuban and Billy Beane — All Stars of meaningful analytics — have admitted to similar challenges. I appreciated their candor. These are men who would be excused for pretending to know it all.
Science Is A Discipline
Many people really don’t know science at all. They wrongfully believe it is a body of knowledge. They build/hire analytic teams to create knowledge in their organization. This is all WRONG.
Science is a process. It is a discipline. It is a method. And if you organize it poorly it will cease to be… science. When Matt Damon uttered those memorable lines in The Martian, real scientists smiled. We KNEW — that his character did NOT. Science isn’t what you know — it is how you can come to know it. When a real scientist doesn’t know — they turn to science.
If your organization wants real analytic success — you need to science the shit out of it.
Stop With The PhD’s
Again, science is a process — not a pedigree. Or at least not an irrelevant one. With all due respect to my PhD holding colleagues, that doesn’t equate to relevant experience in the business world. Universities may do well teaching a process, but to employ it successfully you need to know the space.
This also means that small businesses and start-ups can’t just pull scientific minds from the Fortune 500 either. As I have stated many times, analytic systems are feedback systems that drive accountability. The process you need to design to create this is different across industries, enterprises, and most definitely from university to business.
Excellence — Embrace The Process
The concept of a “Center of Excellence” or less fortunately named “Center of Competence” is the newest “fad”. It isn’t — or shouldn’t be. Unfortunately, they suffer from all the baggage of earlier buzzwords.
So what does the scientific process say about Centers of Excellence? They should be a mechanism for Peer Review. The scientific process demands repeatable outcomes and long ago scientists took to Peer Review as a means of enforcing it. Unfortunately, feedback and incentive structures have damaged those structures in academia and will utterly crush them inside your organization. So has science failed us? Not at all.
Peer Review is a process designed to work in academia. Corporations and other start-ups should considering using Expert Third Party Review or Oversight. The process is similar. Analysis created by internal teams is reviewed by an outside expert. It is a process utilized in other disciplines. Lawyers, as an example, often utilize this practice.
The process has several distinct advantages. First incentives are aligned. Internal review is liable to be bias by competition over resources and priority. Politics will govern outcomes rather than more important concepts like logical consistency, statistical significance, repeat-ability, and other more meaningful reviews. Further, while internal review might create conflict, external review spurs greater care and integrity. Analysts who are aware their work will be vetted are more apt to guard the quality of their work. They are also better armed to counter the demands of leaders looking to “sell” their ideas.
Return — Economic Efficiency
Often executives rebuff this model on economic grounds. They argue that “expert” services are too pricey. But the reality is far different, most analytic projects take somewhere between 2 1/2 days and 2 1/2 months. Back of the envelope, this puts the cost of each project somewhere between $2k and $100k (as measured by time and hourly rate). To review a project of that sort would add no more than 10% to the cost and potentially much less.
Further, few companies have the scale necessary to support more than a handful of projects at any given time. Meaning, that even if you thought you could hire expert talent more cheaply on a full-time basis — you probably don’t have enough work.
Finally, analysts that know their work will be reviewed as far less likely to allow fruitless projects to linger. Unaudited analysts have a bad habit of favoring self-preservation over the value to your company. It is only human. No one wants to admit the thing they are being paid for is a waste of their time (and your money), but they are far more likely to if they believe the truth will come out regardless.
That is all the time we have in this article. In part 2, we will discuss further opportunities for improving analytic excellence and providing a real return on your investment. For now, consider the Third Party Oversight model — it is a clear extension of the scientific process and one that I have seen reap huge benefits.
About the author — George Earl is an analytic executive with more than two decades of experience in analytics and business intelligence. Learn more here.
