How AI reimages emotions
Could AI transform in images concepts that are even hard to explain with words?
The world of AI-generated art exploded within a few months, with the publication of DALL-E, MidJourney, and Stable diffusion. In a short time, with the use of simple textual prompts, it is possible to obtain incredible images. In fact, you just choose the subject and style and the AI does the rest.
This all started a short time ago with the release of CLIP from OpenAI. The first step was that it was possible to use neural networks to associate text and images (such as text captioning). For example, by providing an image the neural network was able to return a caption. CLIP worked by providing a match between a text description and an image.
Of course, one could also do the opposite, using a description as input and finding the representation with an image. From there came the idea of using a prompt search in the representation space of CLIP and then associating a GAN (generative adversarial network) to generate an image.
In previous experiments, I have used DALL-E to generate images using haikus as text, or descriptions of the little prince with stable diffusion. In another experiment, I tested whether artificial intelligence could represent words that are difficult to translate and the concepts themselves are fuzzy.
Today I decided to test whether AI could understand and try to represent concepts as human as emotions. I added in the description, the Oxford Dictionary definition that shows that the very definition of an emotion is a complex operation. The same words can hardly clearly define emotions such as “happiness” or “sadness.” Can AI succeed with an image?
Some time ago I read about Dream by Wombo. It is not described in the literature what the model behind it is, one enters the text (limited to 200 characters) selects a style among several available, and after about twenty seconds gets the image. In contrast to DALL-E and stable diffusion, the result is much less defined, there are far fewer concrete elements (people or objects). the images are much more ethereal and abstract as if they were timeless aesthetic compositions. that’s why I decided to use it because I think it can better render a nuanced and complex concept like emotions.
Happiness
The Oxford dictionary's definition of happiness is not very clear (nor will the others be). Endless words have been written about happiness and its pursuit, just as philosophy questions whether happiness is something universal or whether everyone can be happy in his or her own way. Interestingly, Wikipedia reports that there is also a World Happiness Report.
One thing is for sure, I think we all know when we are or someone is happy.
the state of feeling or showing pleasure
Sadness
The Oxford dictionary definition is clearly of little help:
the feeling of being sadin the same way, the sad definition simply describes being sad with the opposite of happiness.
unhappy or showing unhappiness“All happy families resemble one another, each unhappy family is unhappy in its own way.” The incipit of Anna Karenina is among my favorite incipits. Indeed, one can be sad for infinite reasons, and there are so many nuances that art and literature always find new ways to describe this feeling.
Anger
Among the strongest feelings, even at the level of physical effects. it is interesting how the Oxford dictionary ties it to the concept of injustice (perceived or real).
Perhaps the first image captures the meaning, a kind of exhibition that crushes everything. Or perhaps it is a case of pareidolia.
the strong feeling that you have when something has happened that you think is bad and unfair
Hope
The dictionary definition, in my opinion, does not capture how strong hope is (which as Sandman says in the eponymous series nothing is stronger than hope).
Similarly, I am not happy with the AI result; these images I would not tie these to hope.
to want something to happen and think that it is possible
Sorrow
According to the dictionary, it almost seems that sorrow is just a reinforced version of being sad. I don’t know, what you think but the third image comes close to the concept of sorrow, something that grips a powerful shadow.
a feeling of being very sad because something very bad has happened
Surprise
Again, this does not seem very representative of artificial intelligence. Indeed, it is difficult to represent the feeling of surprise, either in words or pictures.
piece of news, etc. that is unexpected or that happens suddenly
Guilt
Although it is difficult to interpret them as the representation of guilt, they are still interesting images that show an oppressive and strong feeling.
the unhappy feelings caused by knowing or thinking that you have done something wrong
Love
I find it interesting, how even the AI seems to represent more than one entity linked in the image. After all, love is always defined as a projection toward someone or something.
strong feeling of liking and caring for somebody/something, especially a member of your family or a friend
Confusion
It seems that even AI itself has difficulty defining what confusion is.
a state of not being certain about what is happening, what you should do, what something means, etc.
Calmness
On the one hand, these images convey calm to me, but it is difficult to say that I would associate them with the very concept of calm.
the quality of not being excited, nervous or upset
Nostalgia
Nostalgia itself is also a difficult concept to describe, perhaps there is a vague reference to it in these images but it is hard to tell.
a sad feeling mixed with pleasure when you think of happy times in the past
Conclusions
I used three different styles for each image so that there are more shades for the same word. I gave the model as textual input only the word (for example, happiness or sadness) and no other elements. I tried not to add any other elements beyond the word and selected the styles that seemed most appropriate.
In general, the task itself was not easy. On the one hand, Dream is much faster than other methods I have tried (especially stable diffusion when I use the version on my computer). On the other hand, I find the possibility of being able to choose only from a limited number of styles (40 possibilities) severely limiting. Only some of the images in my opinion come close or give the impression that they can represent emotion. Some, because of their representation and because they are so abstract, are very special. In the future, one could try other models (i.e. DALL-E, stable diffusion), as each model has its own peculiarities.
What do you guys think? Which images do you think are the aptest? Which images did you like the most? let me know in the comments.
If you have found it interesting:
You can look for my other articles, you can also subscribe to get notified when I publish articles, and you can also connect or reach me on LinkedIn. Thanks for your support!
Here is the link to my GitHub repository, where I am planning to collect code and many resources related to machine learning, artificial intelligence, and more.
Or feel free to check out some of my other articles on Medium:
