Can you tell how processing in neural networks truly works? Apart from gesturing to a network’s weights and elementary operations, are we able to say how it classifies an image as a cat or a dog, or how it chooses one Go move over another?
For neural networks, there is no doubt that the understanding we currently have about their properties after learning is more shallow than the understanding that we have about the code used to train it: the rules for its development and learning.
What is the ABC of Neural Networks?
This research work discusses the issues of complexity as applied to neural networks and other artificial systems. Even with full system observation, total description of all the involved functions, the researchers have trouble producing a meaningful “understanding”. From this background, they ask, what we mean when we talk about understanding neural computation.
They argue that the brain’s generative process is not that unlike that of neural networks since it obtains information from a world which it stores as a distributed pattern of weight changes, a pattern that is remarkably hard to wrap one’s head around.
Neural network scientists try to understand their networks by studying the sensitivity of outputs to changes in the system to ask what matters. They ask which stimuli can fool a system. They visualize network elements. By removing units, they analyze what happens if a system is perturbed. Still, no one who is familiar with these approaches would say that they offer a good understanding of models like AlexNet, AlphaGo, or GPT2.
Potential Uses and Effects
Current neural approaches do not provide a path to a meaningful understanding of the computations done by the systems they build. The approaches remain far from offering practitioners enough understanding to be useful to improve networks for task performance.
This paper concludes that neuroscience should focus on understanding development and learning for the moment. And, should be informed by approaches deployed to understand brains and instead of asking how the brain works, we should maybe ask how it learns to work.
Read more: https://arxiv.org/abs/1907.06374v1
Thanks for reading. Please comment, share and remember to subscribe to our weekly newsletter for the most recent and interesting research papers! You can also follow me on Twitter and LinkedIn. Remember to 👏 if you enjoyed this article. Cheers!