The Black Box “Problem”
I am a bit tired of the term “black-box” when referring to deep learning-based networks. It has become a way for people to pre-judge and not have an honest conversation about the advances the AI community is making.
Look — I get it — we truly do not understand what every little neuron is, in fact, learning or why wildly overparameterizing your problem seems to give us performance comparable or better than humans.
But, at the same time, the mystery that is embedded within these weights should spark something inside of you. A spark that would lead to an inquiry into the interesting set of patterns that we can try to tease out from our models.
In fact, data scientists and researchers are starting to lay the ground work for this suite of tools that, I believe, will allow us to truly look into the so-called black-box models. It’s just a matter of time until they are scaled and put into production for everyday workers.
In the meantime, we can poke and prod our models — look inside at the individual layers, check to see the output as we change the input, investigate the distribution of probabilities that come out, and a whole host other emerging methods.
The limit is your imagination.
Don’t be fooled when people label a whole field of study as magic, not understandable or unexplainable.
Thanks for reading.
