Soft Machines

Binary symbols of I Ching: a philosophical taxonomy of the universe. source

Neural networks are constructed of very simple elements, where packets travel back and forward between those nodes. While reading the brilliant introductions and examples by Andrej Karpathy on the methods of how networks can learn, I realised that the power of these methods and networks are lying in their multi directional aspect of communication. Neural networks are constructed of very simple building blocks (neurons), that are connected with each other, organised in different layers. However, the information that is travelling through them is travelling both forward (computed values) and backward (results, or gradients, propagated back through the system)!

A single neuron (perceptron) is passing inputs (x1,y1) forward, and passing weights (w1, w2) backwards. b is the so-called bias, representing the threshold to determine whether or not a neuron is activated by the inputs. This single neuron can be calculated with the following formula: w1*x1 + w2*x2 > b. source

This tiny fact that the neurons are firing outputs of calculated values but also receiving previous results (weights, gradients) elevates the whole process from triggering simple computing gates into a level of a meaningful conversation. Instead of trying to make the system better (aka learn something) by trying to populate & evaluate random data, the system balances itself with the individual help of each neuron, and the computation is tuning its accuracy based on the results coming out of itself.

In a historical context, random probabilities and chance operations are used in a wide range from calculus to creativity, think of the methods of ancient permutation techniques on language, i-ching, early automated calculations for sacred purposes by Ramon Llull, analytical engines by Gottfried Wilhelm Leibniz, contemporary art techniques from the twentieth century (William S. Burroughs, musical & visual pieces of John Cage, etc).

At the moment I see those methods as snapshots within time: noise, possibilities, computational imagination can be introduced in a system through them, they can be used very creatively in all kind of processes. However, when time, prediction, progress is taking part in the process, they become extremely slow to solve problems. Think of cryptography, steganography, the famous brute force method: if the previous states do not affect the next state while solving a problem, the time that we will spend on it is a long linear journey, with respect to the number of elements we use to solve the problem. And this is where learning, self-balancing, the conversational nature of neural networks become handy.

There is an excellent introduction that helps understanding this “bi-directional” nature of neural networks made by Karpathy, using ConvNetJS. A neural network is trying to guess an image, based on it’s pixel values, but with only a few neurons.

“But wait”, you say, “the machine can simply memorize all the pixels, all the mapping of the entire image… what is the purpose of it?”. Of course, our computer can store images (connected arrays of pixels), that is crucial for navigating in a digital audiovisual landscape. However, this neural network has far fewer weights than the image had pixels, and so the network has to do something interesting — learn to put together some simple shapes, approximate the original image, while taking fewer parameters (numbers) to do it — a kind of compression.

A brush painting & a calligraphy, artists unknown. (left: represented by neural networks, right: original) source , source

The images used for this experiment are all chinese calligraphies, since human creativity and intention becomes very clear and focused through the gestures of hand, accidental constellations of the ink confronted with the white canvas. The networks used here are just stacked fully-connected layers (constructed of 20 neurons each) with a final dense layer of output dimensionality 3. See the animations above, of what the output looked like at the end of each training epoch.

Apart of the interesting aspect to see the process of the network in action,I think the results might remind us to some lossy compression techniques. Even if these networks are operating on images, their inner representation can be misleading and confused with some creativity in the process. However, it is important to note, that these experiments are just demonstrating the bidirectionality of the networks, it should remain clear, that human visual culture (based on physical waves of light) is really far from the invisible nature of machine-machine visual culture.