Tech and Art walked into a bar
Artists tell you a story. Sometimes this is just a fleeting emotion captured on the canvas like the passionate kiss immortalised by Gustav Klimt. Sometimes, it is a window into the artist’s mind itself, like Munch’s Scream or van Gogh’s Starry Night. With careful brush strokes and selective colour choices, the artist is able to talk to you, the viewer.
In the new digital age, gentle brush strokes are replaced by mouse clicks or writing code and the meaning of creativity is redefined. The new age digital artist thinks not just about the painting itself but also about the process used to create the digital art. The design of the algorithm involved in creating the code art (also known as procedural art) becomes just as important as the picture itself.
“ We don’t make mistakes, just happy little accidents” ~ Bob Ross.
Algorithmic art reflects Bob Ross’s artistic attitude more so. It is an exploration of where an evolutionary algorithm is going to lead the painter as much as the painter controlling the parameters. Generative art involves as much randomness as careful manipulation by the artist. These happy accidents in algorithmic art could also help us gain a deeper understanding of the world around us and the algorithms we choose.
A Sunday afternoon experimenting with Deep Dream Generator resulted in not what I would quite consider visually appealing art but some interesting insights into neural networks. Deep Dream is an experiment designed by Alexander Mordvintsev which enhances the patterns that a convolutional neural network(CNN) detects in an image. It does so by forwarding an image through the network, then calculating the gradient of the image with respect to the activations of a particular layer. The image is then modified to increase these activations, enhancing the patterns seen by the network, and resulting in a dream-like image.
Understanding what the neural network is “learning” is one of the main challenges in the field of AI now. What we do know is that each layer progressively extracts higher and higher-level features of the image, until the final layer essentially makes a decision on what the image shows. For example, the first layer maybe looks for edges or corners. Intermediate layers interpret the basic features to look for overall shapes or components, like a door or a leaf. The final few layers assemble those into complete interpretations — these neurons activate in response to very complex things such as entire buildings or trees. This knowledge could be used to manipulate generative art using the Deep Dream Generator.
Starting from a black and white griffin, I attempted to generate something psychedelic by going deeper at each level by slowly increasing the activation layer used. I was hoping that using lower layers initially would result in highlighting edges and contours of the griffin and later stages of the layer activations would then create the psychedelic images. See here for examples of this. However, the algorithm definitely has a ‘mind’ of its own and below are the resulting ‘Dreams’ for different levels of activations. Generative art is not a carefully planned and predictable process. Instead the resulting paintings are happy accidents resulting from the artist’s initial design.
Layer 3 was off to a good start and started filling in the image of the griffin. However, very quickly the network started perceiving the eyes of various reptiles and amphibians and every layer accentuated these patterns. Once eyes were perceived in layer 8, a feedback loop caused the network to recognise the eyes even more strongly on the next pass and so forth, until more and more eyes cropped up all over the griffin.
What I find most interesting about this experiment is that it can be leveraged for understanding or ‘explaining’ what our network has learnt. The network was trained mostly on images of animals, so naturally it tends to interpret shapes as animals. This technique gives us a qualitative sense of the level of abstraction that a particular layer has achieved in its understanding of images. Another visualisation that would help us understand what the network has learnt can be created by turning the network upside down and asking it to enhance an input image in such a way as to elicit a particular interpretation. Thus, Deep Dream is not just a tool for digital artists but help data scientists understand and visualise how neural networks are able to carry out difficult classification tasks, improve network architecture, and check what the network has learned during training.
“Geometry is the foundation of all painting.” ~ Albrecht Durer
Using machine learning algorithms is rather new in the field of generative art and traditional algorithms take advantage of fractals for geometric art. Among the most widely recognized fractals forms in mathematics is the Mandelbrot Set. The Mandelbrot set is the “most complex object on the complex number plane” but arises from a simple formula, commonly expressed as Z = Z² + C.
Extending the Mandelbrot Set to a 3D space renders the fractal more fully and allows us to appreciate these fractals in all their complexity. The 3D fractal called Mandelbulb is an archetypal fractal form, embodying principles of deterministic chaos. There are plenty of online tools for playing with fractals and using Fractal Lab, I was able to play around with these fascinating fractals.
Generative art at the intersection of algorithms, mathematics and art is not just about beautiful visuals. It is not just about an artist pushing the boundaries of creativity. It is a bridge to a deeper understanding of abstract levels of knowledge, be it nature, mathematics or code. I found that it is easy to be a digital artist in one afternoon (am I though? :P), thanks to the convenience of digital tools. But understanding the complexity in the code and the math is an important part of appreciating digital art. And if we are building a future where machines are closely aligned with humankind, understanding tech through art would cultivate the steps towards that future.
