Transferring Entropy In Deep Learning Systems

Neural Spiking process in continuous time systems

The state of information flow of neural networks in distributed complex systems has gathered broad interest in the recent times. The transfer entropy can quantify the information flow among multiple variables with time-series activities in discrete time. Neural models can explain this phenomenon.

The Artificial intelligence can improve human life, but cannot replace humans. Machines can read million essays on their own without human intervention. However, machines will not be able to replace humans. There are several machine learning algorithms that are applied in every walk of our lives with supervised and unsupervised learning. In the recent times, Google built DeepMind artificial intelligence with deep learning algorithms. DeepMind artificial intelligence was able to defeat the human in the AlphaGo game, which is particularly a complex board game built some thousands of years ago. Despite the fact that, artificial intelligence defeated person, it cannot make decisions in everyday life. The normal process to invent and manufacturing the drugs is a complicated process. However, MIT applies deep learning to invent and manufacture new drugs by leveraging big data to find the compounds through bioinformatics, chemistry, and deep learning algorithms. There have been several theorems to back up the proposal that artificial intelligence will be able to make decisions, though there has not been any such technology that exists to date. A computer program is written with algorithms has a particular objective in finding the patterns in the data and it cannot function beyond that objective.

The speed and quality are associated with each other. It is important to make quality decisions along with the accelerated computation to assess the quality of the decision outcome. There are different applications built for healthcare that are installed on DeepMind artificial intelligence in collaboration with NHS by Google. However, they cannot compete in AlphaGo game or invent new chemical compounds. The artificial intelligence is purpose-built for a particular objective and cannot perform a task beyond that objective. Currently, researchers are trying to see if artificial intelligence can make decisions for the coverage of the insurance policy or appoint artificial intelligence as a judge or doctor. Currently European Union is in the process of formulating a new legislation for the robot laws. The reason why European Union is examining the morality of artificial intelligence is that, if a decision was taken wrongly by artificial intelligence as a judge, the human would not be able to ask the artificial intelligence why? Because, there is no loopback mechanism from a computer program for all the questions due to the involvement of cognitive abilities. IBM introduced cognitive computing through IBM Watson, though the name suggests cognitive computing for the commercial distribution, internally IBM Watson applies supervised, semi-supervised, and unsupervised learning algorithms to perform the tasks or apply statistical algorithms and methods. A human has cognitive ability to think and reason the events. However, artificial intelligence does not have the cognitive ability to think or to make the decisions or respond with the reasons made behind such decisions. The big data is definitely the reason for making such decisions. Artificial intelligence can show the big data for making decisions based on the algorithms built. US is currently reviewing the ethical laws for robots, because, the big data used should be without bias. If a self-driving vehicle breaks down built with artificial intelligence and deep learning algorithms, a human may never know why the vehicle broke down. The artificial intelligence may not be able to respond to such questions, till the deep learning computer is analyzed through diagnostics to determine the problem by a human.
When the artificial intelligence system becomes a complicated machine with abundant input of the big data and processing on billions of transistors of the chips performing computations to arrive at the output of the decision leveraging artificial neural networks, this process becomes complicated, when it cannot be explained, an artificial intelligence expert won’t be able to reverse the process to explain how the big data was used to arrive at this decision. For example when a human brain computes with 86 billion neurons it may be possible to measure the computations of the neurons. In 2012, Google ran deep learning training on 16,000 processors with one billion individual neurons. However, the computations are not an answer that can be used to determine the decision. That is because; there are billions of paths that the neural networks went through to come to a conclusion to make a decision. Those paths cannot be reverse engineered by just observing or measuring the neural computations either in a complex machine or in a human. The big data will be colossal to perform clustering on the neural networks and billions of paths the neurons went over to come to the conclusion. For this reason, one cannot mathematically dissect the machine decision to find the neural network paths it went through to arrive at that conclusion. It’s like un-baking the baked cake. It’s like trying to reverse the maximum entropy in quantum mechanics leveraging second law of thermodynamics. Reversing entropy can lead to reversal of time, which is not possible due to second law of thermodynamics. Despite the fact that, all neural computations are pure mathematics, the reverse engineering does not have a particular path in the neurons to go back, as the path is pretty complex, similar to entropy in the universe. At present, humans have not built any reverse-engineering machine that can perform such process. Even such machines are built, the legislation is not clear, how does law compensate a person that is affected by an algorithm. Who takes the responsibility to compensate that person? Unless such machines are built that can reverse-engineer the decision making process of artificial intelligence and artificial intelligence has achieved cognitive function beyond being a purpose-built machine, it is safe to say that, artificial intelligence cannot be used as a tool for decision-making process with 100% accuracy.


Pearson, J. (2016). When AI Goes Wrong, We Won’t Be Able to Ask It Why. Retrieved November 28, 2016, from

Spinney, R. E., Prokopenko, M., & Lizier, J. T. (2016). Transfer entropy in continuous time, with applications to jump and neural spikingprocesses. Retrieved January 16, 2017, from