Speculation: Information, Computation, Consciousness and Theory of Everything in full circle.

In my quest to work on AI, I have been very interested in understanding the fundamental nature of computation. Neural networks as we know are black boxes and while we know that they “compute”, what really goes inside a network remains a mystery.

In theory of computation, we define specific type of computation by a language which is a set of string from built from some alphabet. All of primitive/analytical models of computing (automata, pushdown automata, and Turing machines ) that accept the language, can essentially tell apart (by accepting or rejecting) strings that belong to the language from those that don’t. In essence, these models have reduced all types of computations to answering “yes” or “no” / “accept” or “reject” / “on” or “off” / “1” or “0”. Let me explain how.

Here is the clever insight. This is very analogous to mathematical definition of a function, where function is a relation ( a predicate with two variables). It is a cross product of a set of elements in domain and range, assigning truth value to each tuple. Now the colloquial way to think of computation is usually a black box that takes in an element from domain and spits out an element from range. Clearly a function “computes”. So the mathematical definition of function as a relation, is an alternate model / form of computation. In other words they are equivalent. This also relates computation with information in terms of bits (truth or false).

In theory of computation, we are not concerned about the efficiency of the computation. That is, it doesn’t matter if the computation takes infinite time, as long as it terminates at some point. This is apparent in the “relation” definition of function, where one can assume given such a relation, one can iterate over all (albeit infinite sometimes) pairs of tuples (from domain, and range) and be able to perform any computation from this function. In this view, the tools of mathematics such analytical form of function (like y = log(x)) is just for efficiency purposes, but does not change the nature of computation.

Given that in order to compute, one can iterate over infinite tuples, it seems plausible (perhaps for likes of Einstein) to come up with a thought experiment involving elementary particles to define equivalence between work and computation. To best of my knowledge, I didn’t know of any such equivalence. So I googled!

It seems that there is a 2010 paper which defines equivalence between information and energy. This is encouraging even though “information” is not quite the same as “computation”. There is increased hype around “information” being the fundamental element in the universe.

One of the arguments about the equivalence of information to energy is described by Maxwell’s demon. In this experiment, the demon is able to open and close slits between two gas chambers. The demon (because she is a demon) is assumed to know the state (velocity, position, etc) of all particles in the gas on both sides of the chamber. Now in principle, the demon can use this information to separate out all particles of high energy to one side and low energy on the other, creating a temperature gradient. And we already know that such a system has energy (think steam engines). Did the demon create energy out of nothing? The entropy must increase, and we can attribute this to the computation that goes inside demon’s head.

Some proponents of this idea (information being the primitive entity) cite that the maximum amount of information that a blackhole can have is bounded. I don’t understand the theory behind this, but they are able to connect entropy of a black hole to the area of its event horizon and this limiting the amount of information that can be encapsulated within it, or something to that effect.

All elementary particles seem to have two different values of spin (call it truth values or bits). And when they interact, there is a predefined manner in which the spin of the resulting system turns out. So in some sense, these elementary particles are doing logical operation at microscopic scale. No one has actually seen fundamental particles, and they are really just abstractions that help to visualise ultimately what computations take place by the laws of the universe. Therefore, my view is that all the theories that we have are different perspective/abstractions to conceptualise the underlying computation within the universe. Classical Mechanics did it, and then GR, then Quantum Mechanics and QFT. I wouldn’t be surprised if we get another theory, perhaps better theory which encapsulates computation that is captured by all previous theory and more (theory of everything). And computation is nothing but manipulation of information.

Now time for my wild speculation. According to our best quantum theory, the effect of “observation” by a conscious being seems to affect how the universe renders itself. Could we think of consciousness in terms of information and find a connection between the two? Let’s try.

For double split experiment, when no observation is made, the information (as defined by Shannon) is high due to uncertainty, there is equal probability of electron passing through any of the slits. However, the act of observation immediately drops the information content to zero. After observation, there is no uncertainty about the location of the electron. It is as if the information transferred from the system to the mind of the conscious observer. So could we come up with a theory where we argue that information in the universe remains conserved, and it only gets transferred.

Continuing further with this idea, what really is observation? No matter how the conscious being observed (deterministic or probabilistically), the observer must do “computation” to understand what really just happened. Thus, observation is nothing but computation? If this were true, then we can think of computation as a medium of information transfer within the universe. Since the observer is really part of the universe, there is nothing special about being conscious.

I say there is nothing special about being consciousness because in above argument, the special status of being conscious in quantum mechanics is due to the ability of the conscious being too “compute”. The universe would behave in the same way (collapse wave function, etc) if there was a modern computer which could observe the experiment and then display “Left” or “Right”. I think this is important point to dwell on for a bit. A modern computer would suffice as an observer in a double split experiment.

I think I am more convinced about this last part because inarguably we believe humans are conscious, and we know for a fact that humans evolved over thousands of years from even simpler beings to basic amino acids which were clearly not “conscious” in the mystical sense associated with it.

I think it is worth exploring the connection between computation as a means to transfer of information between connected systems of the universe. Entropy on the other hand is the side effect of any act of computation (think again of Maxwell’s demon).

We know that the entropy of the universe increases, but above hypothesis possibly lets us understand that it is the underlying “computation” that is necessary ingredient. We typically think of entropy in terms of statistical likelihoods. An ordered state is exponentially less likely than any other random unordered state. Even in this view, for the system to get from one state to another, some form of computation must take place.

I know there is one glaring loophole in this idea of computation as medium of information transfer. In we agree with Shannon’s definition of information as a measurement of uncertainty, then it is not clear what information transfer really means mathematically. In double split experiment, I argued that the information content dropped down to zero by act of observation (aka computation), but how do we quantify that this information is transferred to the observer? The observer was uncertain about the location of electron until observation. So it seems that the drop in information content (in Shannon’s sense) of the observed system is equal to the information gained (in colloquial english sense) by the observer. May be that is what defines a conscious being after all.

When two conscious beings interact, they exchange information. A teacher conveys information to a student which uses neural computation to decrease uncertainty about the concept learnt.

Going full circle then, we can define computation as a process of reducing uncertainty of the system. This fits well with the mathematical definition of the function as well. Before computation, we knew about what elements there are in domain and range sets (which can hypothetically include everything) and there were combinatorially many possibilities (each element is set of domain could map to any element of range). The act of computation helped reduce uncertainty. For example, for a parabola (u=4x²) on a real plane, before computation, x=1 could be associated with any of infinite values of y. However, computation deterministically reduced the uncertainty to zero.

My knowledge of physics, world or anything else for that matter, is very limited compared to great thinkers of the past and present and I am also aware that my speculation is more likely to be wrong than it is right, yet I find it an interesting take on things. Let me know what you think, I would love to be shown flaws in my thinking or my outright leaps of faith.