Quantum Computers, Neural Computers, and the future of 0s and 1s.

Norman Di Palo
Jul 22 · 13 min read

Discussing the pervasiveness of computers would be, without doubt, unnecessary. In the era of the Internet of Things (did it ever really arrive?) the mantra is everything computes. The unprecedented evolution of processors and computers in all of their forms, from desktop all-in-ones to Raspberry Pi-like microcontrollers, was somehow predicted by the Moore law. But in recent times, this law is approaching its physical limit. The length of silicon circuits is approaching the point were the quantum laws start interfering with the behavior of electrons.

New fields of computer science and computer engineering are starting to emerge. In the tech and pop culture new terms are appearing with increasing frequency: quantum computing and neural computing. How do these fields fill some of the gaps in the ever-evolving field of computer science? In this article, we will explore what these buzz terms really refer to, and how they can impact the future of our digital lives.

Quantum Computing: a new playfield.

Quantum computers play by changing the rules of the game. The rules, in this case, are the laws of physics. If classical computers, made of voltage potentials and currents flowing through uncountable transistors are reaching their theoretical limit, one way of escaping this limitation is by changing the playfield entirely.

Quantum physics was first developed during the 20th century. Physicists empirically saw how, when studying matter at an atomic and sub-atomic scale, the rules of classical mechanics didn’t hold anymore. A global change of paradigm was needed in order to explain the vastly bizarre behavior of particles, atoms, and the fundamental, indescribably tiny bricks of our universe. While describing the laws of quantum physics is vastly beyond the scope of this article, it is needed to say how its findings still bother the minds of theoretical physicists, as they did with Einstein, who famously disagreed with several theories proposed by colleagues of his time.

A quantum computer from IBM. The actual quantum processor is the tiny black square at the very lower tip of the machine, not larger than a biscuit. The machinery is used to keep it at a precise physical state, at temperatures close to the absolute zero. Taken from https://www.wired.com/story/wired-guide-to-quantum-computing/, pic by Amy Lombard.

But while we are not interested in understanding the underlying workings of quantum physics, we need to understand how this branch of physics offers a new way of building a computer, able to tackle certain problems vastly faster than the silicon-based counterparts. First of all, what does quantum mean? Quantum refers to the property of these systems of possessing only certain defined and discrete states or amounts of energy. An electron orbiting around an atom, for example, can only be in a finite number of orbits. If it receives an amount of energy that is enough to make it change orbit, it will jump to the neighboring one, but there are no intermediate orbits or states. As such, a photon might be in one of two polarized states, and a subatomic particle might have one of two spin directions.

But here comes one of quantum physics superpowers: superposition. Superposition is the ability of quantum objects of being in many states at the same time. This property has generated skepticism in every physicist who encountered the subject, but there have been experiments that proved this counter-intuitive property of subatomic matter. This behavior is at the base of the advantage of using quantum computers instead of classical ones. [1]

When a quantum object is measured, it collapses into one of those possible states, losing this property. While this may simply seem the effect of our lack of knowledge of its real state until we measure it, experiments show that in fact the quantum object switches from a superposition state to a single state.

Furthermore, one of the most remarkable aspects of quantum mechanics is that its laws predict certain effects that work in a nonlocal manner. Locality is the intuitive principle that states that objects can interact with objects “close” to them. Two particles can be connected or “entangled” in such a way that an action performed on one of them can have an immediate effect on the other particle light-years away, apparently violating the speed-of-light limit. This “spooky action at a distance,” to use Einstein’s colorful expression, was one of the most shocking discoveries of quantum mechanics. [1]

Niels Bohr, one of the fathers of quantum physics. He won the Nobel Prize in 1922. Illustration by the author.

We introduced, at the bare minimum, the essential properties that an object needs to have to be part of the quantum world. We can use electrons, photons, or subatomic particles as the physical medium to store information. Similarly to the classical computer science world, we design quantum computers to store information in blocks of two states, like bits, that we call qubits. The main and fundamental difference with normal bits is that qubits, as described, can be in both states simultaneously, with different probabilities assigned to each state. The field of quantum algorithms hence employs the superposition property in a fundamental way. While normal computers, represented as state machines, can be in a single state at a time, “ […] quantum computers can be in several states simultaneously. One might think of this as massive parallelism [quote from (1)]”. The general pipeline of quantum computation is the following: an algorithm starts by initializing the quantum computer in a well-defined single position. It then places it in a superposition of many states. From there, it manipulates the qubits in a specified way, keeping them in a superposition of many states. Finally, the qubits are measured, collapsing into a single state, like normal bits. Hence, not all the possible paths can be measured, but instead one gets a single answer, based on the probability of the various qubits of collapsing into that configuration. The result you measure, in a sense, is random, since it can change each time based on the underlying probability distribution of possible states. As Scott Aaronson wrote, “the goal in quantum computing is to choreograph a computation so that the amplitudes leading to wrong answers cancel each other out, while the amplitudes leading to right answers reinforce.

A byte is formed by 8 bits and can represent 256 different states. In normal computers, only one of these states can be active at any instant in time. Qubits, on the other hand, have for each possible state a probability, defined as a complex number (that is made of two real numbers). While we can visualize a bit as a switch being open or closed, we mathematically describe a qubit as a vector, pointing somewhere on a sphere. This describes, simply speaking, the state of a qubit with its probabilities of collapsing, when measured, into one of the two possible states, 0 or 1. Quantum gates, that are the unitary operations of a quantum circuit, act on qubits by “moving” this vector. Hence, a qubyte will have a complex number for each of these 256 possible states. To simulate it on a normal computer, we would need to store and manipulate 256*2 floating-point numbers for an 8 qubit computer. For a 64 qubit computer, we would need 18,446,744,073,709,551,616 complex numbers. This is a glimpse of the real power of quantum computers: this exponen-
tial growth was one of the reasons researchers started giving thought to the notion of quantum computing. [1]

There are several fields in which quantum algorithms can surpass any possible classical implementation: Grover’s Algorithm shows that a quantum algorithm can find a particular entry in an unordered array in O(sqrt(N)) steps, compared to the worst-case O(N) of classical computers. This can give an unprecedented speed boost to information retrieval in large, unordered arrays. Shor’s Algorithm describes how a quantum computer can factor large integers in primes in polynomial time, much faster than classical algorithms. Furthermore, quantum computers can excel at simulating quantum objects, unsurprisingly. When computing and predicting the interaction between molecules, tremendous computing power is needed since, as we saw, quantum objects of the atomic world tend to have exponentially many possible states at the same time. Trying to simulate this behavior with classical chips is inefficient. Quantum computer may emulate the superposition of such objects by putting their qubits into similar superpositions, hence exploiting the extreme parallelism that superposition can have.

One of the fundamental equations of quantum physics, Heisenberg Uncertainty Principle.

It is necessary, though, to conclude with an important remark: quantum computers =/= better computers. Not everything quantum is better than classical algorithms. We described some scenarios in which quantum computer can be vastly superior to their counterparts (a property called quantum supremacy), but this does not apply to all aspects of computer science. Classical architectures are perfectly fine for a vast series of problems, that over the years have been tackled efficiently with sequential, silicon-based processors. Hence, don’t expect your next smartphone to be quantum. Quantum computers are extremely difficult to build and maintain, as they operate at conditions at the edge of what is physically possible, and their advantages over classical computers have been proved in few fields. It could be possible to see wider adoption of such machines in research labs, were their superpowers may allow faster discovery of drugs, if the research will be able to produce satisfyingly performant machines. Companies are starting to use them to predict the behavior of atoms and ions in batteries, designing more efficient energy storage systems for our insatiable devices and to accelerate the adoption of electric cars.

We discussed how the quantum world can lead to a major change in the hardware of computers, down to the physics of its (qu)bits. But at the same time, another profound breakthrough is taking shape in the world of 0s and 1s, that has the potential to reshape how software is designed and implemented: neural computers.

Neural Computers: plastic, self-modifying machines.

Algorithms rule the world. This motto appears countless times in journals, blogs and books, that give us a glimpse of how all of our digital lives are controlled by a predefined series of instructions running on remote servers, one next to the other, packed into vast rooms constantly kept at a set temperature by loud fans and air conditioning systems. Not only that: the impact of computers into our physical, real life is unmatched, from the routes our buses take, to the products we buy and the people we meet.

In a few decades, computer science and algorithms have taken over the world. An algorithm is nothing more than a series of instructions explaining how to do a task, how to go from an input to the desired output, how to crunch data in order to obtain other data. Countless scientists, engineers and hobbyists have spent their lives designing, evaluating and implementing algorithms for all sorts of things. To store data in a computer, we encode it into data structures: a list of the songs on your phone, a graph describing your friends and their friends, and more. Designing efficient algorithms and data structures to make sense of the wild mess of information in the real world has kept brilliant minds busy for over a century. But there are some kinds of objects that are just too difficult to describe and analyze with traditional methods. Things such as images, videos, speech, unstructured text. In recent years, though, a new technique gave computers new superpowers: machine learning.

Machine learning brought an impressive change of paradigm in computer science, and many fields, like computer vision, machine translation, games, etc. were profoundly transformed by these techniques. Neural networks have shown to be able to learn surprisingly complex patterns, like generating human voices and faces, captioning pictures and even discovering exoplanets.

The compass of AI research points at the construction of a human-like artificial brain, guiding scientists and engineers towards the creation of intelligent algorithms that are able to learn, comprehend and ultimately help humanity to solve some of its most stubborn problems. The question is: can a neural network design the algorithms of tomorrow?

At the simplest level, a computer needs two things in order to execute algorithms: a processor which can execute instructions, and a memory, where the processor can read and write data. Computers then read instruction sequentially from memory, written by a programmer, and execute them writing outputs in other parts of memory. Can a computer learn to write programs by itself, deciding how to manage memory, what to read and what to write, in a language that could be obscure and incomprehensible to humans, but possibly even more efficient? To answer this question, in recent years researchers have designed a computer that behaves like one giant, trainable neural network: a Neural Turing Machine.

Alex Graves, one of the principal investigators of NTMs. Illustration by the author.

Neural Turing Machines (NTM) and their successors, Differentiable Neural Computers (DNC) are a relatively new architecture that pairs an external working memory to a neural network, that can hence read and write data in memory, using it to solve complex tasks. It behaves similarly to a Turing Machine, but all of its operations are differentiable, and hence can be trained with widely used techniques like supervised learning: this means that NTMs can observe examples of certain algorithms provided by a human, and learn by themselves how to replicate this behavior on unseen data, possibly even in a more efficient way. It is composed by a controller network, that acts similarly to a processor: receiving inputs, deciding what to write and read in and from memory, and computing an output, and a memory, which can be manipulated by the controller and stores data unless it is erased.

There are some tasks in which organizing data in a structured way and processing it with algorithms seems dauntingly difficult. Think of, for example, the task of question-answering from text. While describing every character as a series of bytes is straight forward, encoding all the semantic dependencies between words, sentences and concepts is nearly impossible. How can one write down a series of rules to find an answer to any possible question in any possible text? What is easy to do, though, is providing a series of examples. Given a text and a series of questions regarding the content, it is trivial to provide answers as text. Given lots of these examples, NTMs can then learn an internal representation of the text and of the questions, given their ability to write down in memory what they need and reading it at a later stage in order to answer questions. As shown by a Nature paper, DNCs are able to learn how to represent graphs in their internal memory, understanding the relations between data and objects.

Another interesting aspect of their memory is the way it can be accessed. Computers can access memory using addresses: a series of numbers describing exactly which slot of memory you want to read from, and the numerical content of that slot is given to the processor. In this way, you have no way of understanding what is stored in memory if you don’t have the exact addresses and know of the read the bits written at that address. Human memory, on the other hand, works as a content-addressable memory: you just need to think to the concept of “dogs” to retrieve images of dogs, and “home” to remember the rooms of your apartment. You don’t specify a region of the brain and see what comes out. NTMs are able to use memory in a similar way: they can learn an internal representation of data, beyond what humans could design, and store it in memory. Then, they can decide what to read based not on the address, but on the similarity between what they are looking for (say, dogs) and what is stored and encoded in memory (say, a neural representation of pictures of dogs).

Immagine correlata

Even though they possess the ability of manipulating enormous amount of numerical data, performing millions of operations in order to see if a picture contains a cat or a dog, neural networks have a hard time learning to do basic math by example: operations like the sum of two given numbers, their multiplication, the square root of an input. Researchers have recently designed and proposed a differentiable and trainable counterpart to the Arithmetic Logic Unit of processors, the part responsible for math: the Neural Arithmetic Logic Unit. This architecture is explicitly designed to perform basic mathematical operations on its inputs, learning by example the concepts of summation, division and more, and showed unprecedented results in various tasks. This is another potential building block to assemble differentiable computer that can be trained, end-to-end, to perform cognitively difficult tasks.

Feed-forward neural networks process data into one pass, hence without storing anything in memory except for internal activation: they receive an input and they process it, layer by layer, generate an output. Recent research, though, shows how residual feed-forward architectures implicitly learn to manipulate data iteratively in order to arrive at a final answer. Furthermore, recurrent neural networks are inherently able to store intermediate steps of computation and process data indefinitely long, until they believe they’ve obtained the final desired output. Nevertheless, to solve complex, unstructured tasks, these techniques are often not enough. This is what mainly drove research in Neural Turing Machines.

In the future, more and more aspects of our lives, both digital and physical, may be impacted by quantum and neural computers […]

Depiction of a DNC’s neural activity. Top left: inputs in red and targets in blue. The task consists of storing the red patterns and copying them after the interruption bit. Mid and lower left: activity of the network’s memory. Taken from https://en.wikipedia.org/wiki/Differentiable_neural_computer.

Conclusion: the future of 0s and 1s.

We are approaching a new era of algorithms, in which designers implement extremely plastic, differentiable models that can learn algorithms by themselves, receiving tons of examples and finally tackling tasks which have resisted numerous attempts of being solved with traditional approaches.

Neural computers, similarly to quantum computers, are still in their infancy. Will they replace software as we know it? Again, no. Neural computers, similarly to what we said for quantum computers, are well suited for a particular set of problems: problems in which organizing and structuring data is difficult, and providing a set of rules in fundamentally harder than providing examples at scale. That said, neural networks are becoming more and more pervasive in the tech industry, and are gradually shifting the role of the programmer and how we think about software. In the future, more and more aspects of our lives, both digital and physical, may be impacted by quantum and neural computers in such a pervasive way that, as we do now, we won’t even realize.


Bibliography:

  • Yanofsky N., Mannucci M., Quantum Computing for Computer Scientists, Cambridge University Press
  • Graves A. et al., Neural Turing Machines, https://arxiv.org/pdf/1410.5401.pdf
  • Trask A. et al., Neural Arithmetic Logic Units, https://arxiv.org/pdf/1808.00508.pdf
  • Jaeger H., Deep Neural Reasoning, Nature 538, pages 467–468
  • Graves A. et al., Hybrid computing using a neural network with dynamic external memory, Nature 538, pages 471–476

Norman Di Palo

Written by

deep learning x robots. twitter: @normandipalo

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade