The Black Box of AI. How our understanding of biology created unpredictable AI systems.

As a biophysicist I am convinced that the future happens at the boundaries of science and technology. There is so much untapped potential in this powerful combination. I am very lucky to have my office at the Wyss Zurich where we are mixing robotics and biology to create the most amazing solutions for mankind. I work on an Augmented Reality Robot that teaches children about science & programming! Check it out: Project Rosie

Neurons through a portion of mouse brain. (Cred. The MouseLight Project)

In today’s blog I want to show you our how deep understanding of biological system yielded spectacular advances in AI research and how by ultimately leaving the constrains of biochemistry and human-processing power spawned a new world of mysteries, challenges and discoveries.

AI seems to be all the buzz right now — I will attempt to bring it into a context so that also none techies can understand the potential. Be it good or bad. It will be a short blog in which I will compare biological systems with technological systems and how awesome the future might become.

As a biophysicist I got confronted with AI very early on in my studies — it was in a lecture about particle detectors. Particle detectors are complex multistationary machines that are able to detect the results of two particles colliding with each other — the result is the so-called shower or spray of particles. Each collision pair will have a specific shower pattern. You might have heard of the works from particle detectors during the studies that discovered the Higgs boson, the particle that is the linchpin to physicists’ explanation of how all other fundamental particles get their mass.

LHC’s Compact Muon Solenoid detector.

To make sense of the torrent of data that is given by such massive particle showers AI was used starting in the 1980s. Its machine-learning algorithms can tell the difference by sniffing out correlations among the multiple variables that describe the showers. Such complex algorithms, which are deep neural networks are spurring innovation across all of science. They can not only be used to detect proton showers but also to analyze the mood of crowds (Cambridge Analytica comes to mind), elegantly combining the human genome to find upstream phenomena leading to diseases or efficiently synthesizing new chemical compounds by understanding the products better.

However, as utopian this all seems there is an ever-increasing problem on the sky — much like the big black cloudy shadow thingy that tormented Will Byers there is an actual big black box that torments us, scientist when talking AI.

As a scientist, you are not primarily interested in the results, but you are interested in the understanding and the story these results provide. You need to be able to understand the arch of the experiment, meaning the input — the modifications — the output. This narrative is lost using the newest and most powerful neural networks. Why? They have become more complex than our own brain.

Powerful machine-learning algorithms have always been loosely modeled after a humans brain. But the mechanics of the models have become so intertwined that they have become mysterious. We have created something that is more powerful than our own thinking — it got difficult predicting the outcomes.

Simplified neural network for image recognition. Rosie the robot is used as an example.

A simple example of a neural network is shown above. A neural network, such as the one in the picture, is taught to perform image recognition. It is made out of layers or triggers, or “neurons”. The neurons fire when given data that cross certain thresholds, and pass that information to a new layer and a new layer and a new layer. At the end of the cascade, it will generate an output. Now the big problem to us scientists, we literally do not know how this complex net of neurons will produce an output. The decision making tree has become too complex as to be fully understood for our very own neural network in our brains.

To somewhat fight against the trend of black box science some smart individuals have come up with an exciting way of bringing light into the dark. The method used is called counter-factual probing.

An “old-fashioned” way of counter-factual probing is to vary the inputs to the AI system in a combinatorics manner to see which changes affect the output, and how. Take a neural network that, for example, uses restaurant reviews as input and outputs the positive ones be marking them with a yellow star. Now the smart AI detective would take a yellow-starred review and subtly change or delete words. Now run them through the black box once more to see whether it still is yellow starred. On the basis of thousands such boring and tedious tests the detective can identify the words –or parts of an image or molecular structure, or any other futuristic piece of data — most important in the AI’s original judgment. In our specific case, the review lost its yellow-starred as soon as the words: raw fish, curry or “there is no cheese in this dish” appeared.

You might think — much like I did at first — that being a detective sure sounds boring and tedious. Columbo did not have the time to probe thousands of different people to see the subtle changes in their decision-making process. Heck think about all the “… Just one more thing.” scenes we would need to go through before the end of the episode.

How can you not like this guy?

So in recent months’ people have come up with something that doesn’t require testing the network a thousand times over. The trick, instead of varying the input randomly, introducing blank references and transition it step-by-step toward the example being tested is more efficient. 
Running each step through the network, it is possible to watch the jumps from layer to layer with certainty, and from that trajectory, features can be inferred that are important to a prediction. A fast, lean and elegant approach to trying to solve a black box of decision-making processes.

With that I am finishing up this little excursion into the worlds of AI, neural networks and the similarity to our own biological system.

For me as a biophysicist, it is fascinating to see how we as humans are pushing the boundaries in tech by first understanding and then applying fundamental biological processes.

However, electricity and clever ingenuity allow for much more potent systems than simple, wet, chemical environments subjected to time and space.

Just one more thing!

Check out Project Rosie — if you are facinated about Augmented Reality in education please say hi at: selim at

Children and Florian visualising sciences by playing with Rosie. (AR for education)