Image for post
Image for post

What Does it Mean to Understand a Neural Network?

Can you tell how processing in neural networks truly works? Apart from gesturing to a network’s weights and elementary operations, are we able to say how it classifies an image as a cat or a dog, or how it chooses one Go move over another?

For neural networks, there is no doubt that the understanding we currently have about their properties after learning is more shallow than the understanding that we have about the code used to train it: the rules for its development and learning.

What is the ABC of Neural Networks?

This research work discusses the issues of complexity as applied to neural networks and other artificial systems. Even with full system observation, total description of all the involved functions, the researchers have trouble producing a meaningful “understanding”. From this background, they ask, what we mean when we talk about understanding neural computation.

They argue that the brain’s generative process is not that unlike that of neural networks since it obtains information from a world which it stores as a distributed pattern of weight changes, a pattern that is remarkably hard to wrap one’s head around.

Neural network scientists try to understand their networks by studying the sensitivity of outputs to changes in the system to ask what matters. They ask which stimuli can fool a system. They visualize network elements. By removing units, they analyze what happens if a system is perturbed. Still, no one who is familiar with these approaches would say that they offer a good understanding of models like AlexNet, AlphaGo, or GPT2.

Potential Uses and Effects

Current neural approaches do not provide a path to a meaningful understanding of the computations done by the systems they build. The approaches remain far from offering practitioners enough understanding to be useful to improve networks for task performance.

This paper concludes that neuroscience should focus on understanding development and learning for the moment. And, should be informed by approaches deployed to understand brains and instead of asking how the brain works, we should maybe ask how it learns to work.

Read more: https://arxiv.org/abs/1907.06374v1

Thanks for reading. Please comment, share and remember to subscribe to our weekly newsletter for the most recent and interesting research papers! You can also follow me on Twitter and LinkedIn. Remember to 👏 if you enjoyed this article. Cheers!

Christopher Dossman

Written by

Deep Learning Engineer, Teacher, and Entrepreneur

AI³ | Theory, Practice, Business

The AI revolution is here! Navigate the ever changing industry with our thoughtfully written articles whether your a researcher, engineer, or entrepreneur

Christopher Dossman

Written by

Deep Learning Engineer, Teacher, and Entrepreneur

AI³ | Theory, Practice, Business

The AI revolution is here! Navigate the ever changing industry with our thoughtfully written articles whether your a researcher, engineer, or entrepreneur

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store