AI Art and the Portrait de Edmond Belamy
In October 2018 an artwork generated by an AI program made history when it was sold at auction for $432,000. It had been estimated that it would fetch between $7,000 and $10,000. The painting was called Portrait of Edmond de Belamy and it was created by an art collective known as Obvious.
Unlike today, when most AI for producing artwork is based on Diffusion models and trained on billions of images, this artwork used a Generative Adversarial Network model (GAN) and was based on a comparatively small sample size of only 15,000 portraits painted between the 14th and 20th centuries.
In 2018 GAN AI models were state of the art. They are called adversarial because they involve two sides. One side generates random images; the other side has learned from inputs how to judge these images and determine which best align with the input.
At the time, the following was said about the piece:
The piece sold at Christie’s is part of a new wave of AI art created via machine learning. Paris-based artist Hugo Caselles-Dupré, Pierre Fautrel, and Gauthier Vernier fed thousands of portraits into an algorithm, “teaching” it the aesthetics of past examples of portraiture. The algorithm then created “Portrait de Edmond Belamy.
Source: The Conversation, “When the line between machine and artist becomes blurred”, Oct 2018
What is interesting is that this news went largely unnoticed. The simple pronouncement that this was a “new wave of AI art” did not start the raging debate that is being carried on today. AI art is not new, but what has changed is the quality of what is being generated and the widespread scale of adoption.
It is now 2023. What if I prompted one of today’s Diffusion models with only the words “Portrait de Edmond Belamy”? Keep in mind that Edmond Belamy is a fiction, a figment of an AI model’s imagination. Of course, neither Midjourney nor any of the other AI art generative models understand this. To the modern Diffusion AI models, this is simply an image like any other image.
After the auction in 2018 the image labeled “Edmond de Belamy” was all over the news web sites. So unless it was curated out it would exist in the database known as LAION-5B that was used to train most, if not all of the current Diffusion models. The 5B refers to the fact that there are well over 5 billion images in this database.
Here is the result of the prompt which was simply “ Portrait of Edmond de Belamy” — Midjourney routinely generates 4 images for any prompt.
What is interesting is that Midjourney could have left the face deformed, as it was in 2018. Instead, the AI model imagined an Edmond Belamy as he might have been, had he been alive. It is clear that the quality of the output has gone beyond anyone’s expectations in less than five years.
As for how widespread the adoption is, at the time of writing it has been estimated that on four services alone — Midjourney, Stable Diffusion, Artbreeder, and DALL-E-2 we (humans) are generating more than 20 million images every day.
This scale of adoption and the quality of the images generated have led to understandable panic amongst some artists. The lawsuits have already started.
The argument for the lawsuit is in part about three C’s: The artists had not consented, they were not compensated for their involvement, and their influence was not credited when AI images were produced using their work.
Some artists want to be able to remove their works from any database used to train AI models, but the algorithms behind today’s AI tools were trained on over 5 billion images and this number will grow. Unless you are a very influential artist, removing any work from the database is not going to make any difference.
Moreover most artists routinely are influenced by many others whether they acknowledge it or not, but no artist provides financial compensation to another for the results of that influence. So what has this to do with machine learning? We are talking about artists and influences, surely that is a uniquely human endeavour?
To answer this question we must first ask: is a machine as part of its learning process influenced in the same way a human artist is? To understand this we need to understand how a machine learns to create an image, and the similarities with and differences between, the way in which a human does.
When I asked ChatGPT how a child learns how to draw a cat, this was the response:
When I asked ChatGPT about the similarities and differences with a machine learning how to draw a cat. ChatGPT’s response is below:
The bottom line is that the process of learning is nearly identical, except that as humans we can bring real creativity to the process and our purpose for creating a drawing can be driven by motivations like self-expression, while machines have the advantage of speed and scale.
If AI was simply scraping and then reproducing images either whole or in part then the matter would be straight forward. By scraping I mean the automated means by which images can be retrieved and retained (copied) from websites. Instead, AI is using a canvas consisting of pixels, points of lights on a screen, to create images that no one has seen before, images that do not exist in the training data. And the AI model is the using the patterns and relationships it has learned about objects in its traing data to create those images. If the machine-learning model is trained on a large enough dataset of images of cats, it can then generate original images of cats over and over again.
We mentioned earlier that Diffusion models overtook the GAN models as the preferred way of creating images. According to the paper Diffusion Models: A Comprehensive Survey of Methods and Applications (Oct 2022):
Diffusion models have emerged as a powerful new family of deep generative models with record breaking performance in many applications, including image synthesis, video generation and molecule design.
That said, how do they work? Diffusion models learn by doing. Imagine seeing a picture of a cat, then erasing it, and then drawing the cat from scratch. Diffusion models do something similar when learning how to generate images. Diffusion models perturb data by adding noise to the image then they reverse this process to generate a new image from the noise. In the example below, note that the image of the dog generated is not the same as the image that was used as input for training.
The debate about the use of any art form, visual or otherwise, for the training of AI models is an important one to have. The answers will not be simple, but the generative AI capabilities are here to stay. What is important to understand is that this is not copying work, it is learning from it, much as any artist would.
The examples in history ranging from the introduction of photography in the 19th century, the widespread adoption of smart phones for photography, the introduction of calculators and then personal computers in the classroom are all being cited as examples of how we as humans adapt to this kind of change. The only certainty is that in the coming years, AI is going to permeate the technology we use for just about everything.
What has changed is this: Creating art that no one has seen before, influenced by the works of others, is no longer a uniquely human ability. However, creating art that in some way or another transcends what has been seen before likely will always be the work of a human artist.