Convoluted Analytics — Part 3
No, It Is NOT That Simple
It is simple. I have heard it so many times. To amateur analysts, it is always so simple. Which only makes it so much more frustrating when all that ‘simple’ goes so horribly wrong.
You have heard it before. The data is messy, the quality is bad, the numbers don’t match, the models don’t work — the list goes on. Often it is someone’s fault… and then someone else’s. Soon it is realized that ‘we don’t have the right people’. Which might be true? But in the end it was the process that failed… it seems that process assumed it was all so simple and of course, it wasn’t… In the world of analytics, it NEVER is.
The Days Of Simple Analytics Likely Past Before The Industrial Revolution. The World Is Now Full Of Complex Systems, With Complex And Convoluted Processes.
Ecosystems, economic systems, or solar systems — the universe is full of systems. They consist of many simple actors — but in concert there is nothing simple about any of them. Feedback loops, positive and negative, make a mockery of amateur analysts the world over. That is before any data is considered.
Data really isn’t dirty… that is an over simplification. Surprised? It is complicated. Or maybe you still thought it would be simple?
Cleaning data is one of the most basic tasks in analytics, but it is also the first task that should teach you things are often more complicated than they seem. Data is typically not dirty because of sloppiness or incompetence. It is dirty because it is an effort to shoehorn complicated information into an simple construct — a table.
Of Course, No Business Ever Fit In One Table
Fitting a business’ data into a table is no simple act. It ALWAYS requires a fairly complicated table structure. This is the heart of a data warehouse, a seemingly simple construct that is often cursed in many of the world’s most successful businesses. As you might suspect, building one properly is not so simple. Getting it right however, will make future steps far simpler.
Complex data plus complex systems is actually still too simple. Surprised? I hope not. All of this complexity is conquered in the name of helping organizations (with complex politics) to take actions (with complex conseqences). Can any of this ever approach simple any longer?
If all of this seems too convoluted, you may not be ready for the world of analytics, yet. The fact is — analytics tends to be pretty damn convoluted. Understanding the complexity of each of its many layers is just one of many examples.
Thanks for reading. Stay safe. More importantly, stay informed. For more on Convoluted Analytics consider:
For a chance to further your experience and analytic education:
