How I Learned to Stop Worrying and Love the Error
The world is a crazy place for an AI to order. Let’s not try — instead compute like it’s 1999.
For those of you who can’t remember (or never cared), the year 2000 brought to light an existential problem for society. There were a ton of 2-digit years floating around in code bases that would cause the world to explode because computers weren’t smart enough to know if it was 2000 or 1900 or Whatever-Year-Ends-With-00.
The frightening result?
Some slot machines at a racetrack in Delaware stopped working. Until they got updated.
Ends up the world works fine with a little imprecision. That’s the opposite of precision and not always a bad thing. It’s also a type of error. Also not a bad thing.
Why?
Because letting machines run with some freedom creates machine learning (ML) magic.
From error to error, one discovers the entire truth
-Sigmund Freud, possibly maybe predicting ML
(Aside: There is a place for certainty. Neurosurgery. Stoichiometry. Cooking raw meat. Hating bad traffic. This is not about any of those).
In a world where 64 bit operating systems are standard, why is a company like Nvidida doubling down on hardware with 16 bit floating point architecture?
Here’s why — more calculations are better than good calculations. Trust numerical methods to eventually hone in on correct answers when real-world complexity prevents analytical solutions. (These don’t need to be actually numerical — Natural Language Processing is solved in a similar way.)
Nvidia is quietly pushing the hardware envelope of machine learning by optimizing for teraflops over precision. This is not regressive computing quality, but a clear strategy to give us tools required for crunching immense volumes of data in our machine learning future.
Unlike Y2K, real world ML excels with low precision.
GPU architecture (Nvidia’s historical power base) is great for ML, but having the foresight to shift expertise from pure graphics to algorithms is commendable. </fanboy begin> ML needs champions in every corner to live up to its potential, and I’m excited to see where Nvidia takes us </fanboy end>.
Now the creativity. Error in individual calculations gives us the freedom to uncover hidden or novel outcomes. This also prevents us from getting stuck in a less than desirable outcome.
Like ants. Watching one forage is an exercise in frustration — the food is right there. But as a colony, an overall optimized solution emerges that exceeds mathematical models. There’s more — operational strategies develop to overcome complexities such as negotiating traffic conflict, overflow capacity and repeatability.
Hive intelligence can be mimicked by repeatedly repeating repetitions (see what I did there?) or structurally developing a node network that allows multiple inputs and transforms simultaneously.
And there is the benefit of a neural network. A single agent might (or even should) act in a non-optimized way, but as a whole the network will trend toward a desirable outcome.
How does this work?
Imagine a problem with two potential solutions (like climbing to the highest peak of your solution space). How can a single algorithm investigate both?
By giving individual trials or agents the potential to go either way. Over repetitions, individual trials will accumulate outcomes at both optimized solutions.
There must be error in the system for one algorithm input to result in multiple solution outputs. If your model is using real life data, this will be baked in from training. Embrace it. You’re not getting stuck on an inferior peak.
Real world solution topographies could mean you are better able to complete customer segregation, investigate novel NLP trends you weren’t training for, find ant-like solution strategies applicable to other problems, monetize your metrics in new ways, et cetera ad infinitum….
In other words — Data Magic.
(Aside 2: Just be ready on the other end to make sense of the outcomes your AI finds. As always, the hard part of data science is to make sure you’re really finding what you think you’re finding.)
Don’t fight the little errors in your algorithm. Properly used AIs should excel with imperfect knowledge. The real world always operates with imperfect knowledge — AIs uncover the buried data truths. That’s why they’re so powerful in application.
