Augmented Imagination: Machine Learning Art as Automatism
In the periphery of the landscape that is the current boom of machine learning (ML) sits a playground for tech-savvy creatives who re-appropriate the technology for their means.
In one corner, there are designers focussing on applying the strengths of neural networks to their field. They dream up new “intelligent”, generative tools that, for example, help them analyze data or produce a thousand variations of a design to select the best [1].
In another part of this playground are artists interested in the new medium’s own expressiveness: “[ML] is becoming a tool, just like painting”, artist Trevor Paglen told me at the opening of his recent exhibition [2]. Paglen and other artists like Mario Klingemann and Sascha Pohflepp have recently produced interesting visual work — let’s call it machine learning art (MLA) — that prompted the thoughts behind this essay.
Right: Sascha Pohflepp. Spacewalk: Carnivores 3, Generation 320.
These images are similar in their their visceral evocative qualities, their organic “brushstrokes” and textures, their surreal creatures and objects. They are usually made to depict familiar objects and draw from visual material of this world, but have something alien to them that fascinates me.
The technology to make these works wasn’t invented for or by artists. The scientists and engineers around our playground use these images as well (but tend to discard them after). They use them to better understand how their neural network works — or doesn’t. In other words, the images — as debugging tool or artwork — often tell us more about the machine itself and the people who made it rather than about the depicted subject matter. [6]
MLA is interesting because of the network’s tendency to mis-represent or represent in alien ways. The glitches in the system make the fascination. As researchers are working on making the technology more accurate and explicable, its associative potential decreases. Any sufficiently advanced MLA system loses its magic, so to speak.
Machine Learning Art and Surrealism
The works remind me of the paintings and drawings of Surrealist artist Max Ernst. Surrealists like Ernst sought to resolve the — in their opinion — previously contradictory conditions of dream and reality [3]. They borrowed techniques from psychoanalysis to stimulate their art, believing that “the creativity that came from deep within a person’s subconscious could be more powerful and authentic than any product of conscious thought” [4].
Right: Max Ernst. The Angel of the home or the Triumph of Surrealism. 1937. Paris, France
I will not proclaim Paglen, Klingemann, or their computers as contemporary Surrealists. Rather, I want to compare MLA with Surrealist technique to propose a different way of thinking about machine learning as a creative tool. Imagine looking over to the corner of the playground where the designers are playing. I want to explore the space in-between.
What do Surrealist Automatism and Machine Learning Art have in common?
MLA does not adhere the Surrealist movement, but its techniques. The Surrealist artist’s toolbox features a whole range of “automatisms” — techniques for making art that poke the subconscious, so to speak. In physiology, the term automatism describes automatic bodily movements. We usually don’t consciously control our breathing, for example, and that is what automatisms are supposed to achieve for making art [9]: “Pure psychic automatism … the dictation of thought in the absence of all control exercised by reason and outside all moral or aesthetic concerns” [3]
There are many automatism techniques. Let us look at one that every kindergartener knows: Frottage. Frottage is a technique developed (not invented) by Max Ernst in 1925. This technique picks up textures from structured surfaces by placing a sheet of paper on top and rubbing over it with a pencil. [5]
Frottage starts with an object (fig. 1). By transferring the object’s textures to a canvas or piece of paper, the artist creates a two-dimensional abstraction of the object. Often, multiple frottages are combined on a single sheet. The textures form evocative images. They prompt connections to other, often unrelated objects, places, or creatures that the artists responds to by refining the image to bring out more the desired subjects.
MLA often follows a similar process (fig. 2). Leaving aside technical details, this process can be generalized as follows: It starts with a dataset of thousands or tens of thousands of objects (or digital representations thereof). Through training, the artificial neural network creates a high-dimensional abstraction of the object’s features: the model. A model can be made to visualize what it “sees”. It shows us textures and shapes somewhat characteristic to the object in ways that often look similar to frottage.
I would argue that both processes create images with similar evocative qualities or potential. But there are, of course, also differences between the two. One is that neural networks inherently strive for representation whereas frottage and Surrealism in general seek to dissolve it.
Another striking difference is that frottage is generally used to launch an artistic process whereas MLA usually is not. And this, finally, brings me to the point of this essay.
Machine Learning as a tool for imaginaries
Max Ernst published his experiments with frottage in a 1926 publication titled ‘Histoire Naturelle’. The drawings in the series aren’t just rubbings that leave the interpretation to the viewer. We see fantastic landscapes and creatures — rendered with the precision of scientific illustration — inspired by what Ernst saw in a piece of wood. [8] The frottage was the start of a process rather than the end of one.
As I have shown earlier, in current, early MLA the images mostly stand for themselves as artworks. The tool itself is the center of attention.[7] In a way, they stop where things get interesting.
What if we employed this means of making images neither as practical debugging nor as art in itself? But rather as an art and design tool for mind bending — like Surrealist frottage; one that caters to the subconscious, the associative, the imaginary rather than rationale?
This essay is part of my graduate research at The New School. Thanks to my advisor Fei Liu for her invaluable feedback.
References
[1] Hebron, Patrick. “Rethinking Design Tools in the Age of Machine Learning.” Medium.com. April 26, 2017. Accessed December 03, 2017. https://medium.com/artists-and-machine-intelligence/rethinking-design-tools-in-the-age-of-machine-learning-369f3f07ab6c.
[2] Paglen, Trevor. “A Study of Invisible Images.” Metro Pictures. September 8, 2017. Accessed December 03, 2017. http://www.metropictures.com/exhibitions/trevor-paglen4/press-release.
[3] Breton, André. “Manifesto of surrealism.” Manifestoes of Surrealism 15. 1924.
[4] MoMA. “MoMA | Surrealism.” Museum of Modern Art. Accessed December 03, 2017. https://www.moma.org/learn/moma_learning/themes/surrealism.
[5] Tate. “Frottage — Art Term.” Tate. Accessed December 03, 2017. http://www.tate.org.uk/art/art-terms/f/frottage.
[6] Paglen, Trevor. “Invisible Images (Your Pictures Are Looking at You).” The New Inquiry. October 02, 2017. Accessed December 03, 2017. https://thenewinquiry.com/invisible-images-your-pictures-are-looking-at-you/.
[7] EyeEm. “Photography through the Eyes of a Machine — EyeEm Blog.” EyeEmƒ. September 08, 2017. Accessed December 03, 2017. https://www.eyeem.com/blog/mario-klingemann-ai-art/.
[8] MoMA. “Natural History (Histoire naturelle) | MoMA.” The Museum of Modern Art. Accessed December 03, 2017. https://www.moma.org/collection/works/portfolios/10056?locale=en.
[9] Tate. “Automatism — Art Term.” Tate. Accessed December 06, 2017. http://www.tate.org.uk/art/art-terms/a/automatism.
