How I turned Google Photos into an art critic
Any one who has used Google Photos in the past year will know that how well the image recognition technology works. The feature, which can identify all the picture of trees, ocean and streets that you’ve taken, is the basis of Google’s move towards artificial intelligence, perhaps the defining technological advancement of the 21st century.
The cool little feature piqued my curiosity. If the image software could identify objects and even certain facial characteristics, what could it do with thousands of different works of art? Well, the results were nothing short of fantastic and intriguing.
It turns out, when you have a trove of ~40,000 unique paintings ranging from ‘Cupid-stabs-Zeus’ neoclassicism to ‘my-face-is-over-here’ cubism, you can begin to see patterns in how the image recognition software thinks. And not just objectively (as in, this = that), but also subjectively (abstract painting = love/beauty/sadness/hope).
Below, I’ve posted the search results of various search queries. The results are based on the “interpretation” of the paintings by Google’s neural networks, the same program that transformed your family pictures into acid trips.
So this is what happens when you search a basic object into the archive. The results are almost exclusively paintings featuring, well, boats.
“Boat”
But the program is a little smarter than you might think. Here is what happens when you search “canoe”
“Canoe”
You get the picture, right? But what happens when you include abstract concept and not just objects. This is where Google Photos become surreal, for lack of a better word.
“Sadness”
“Happiness”
“Laughter”
“Lost”
An interesting thing that the Google Photos engine does is hone in on the primary theme of the painting. The woman in white could easily be interpreted as a background figure, but Google Photos manages to recognize that her emotion is one of the key aspects of this work.
“Daydream”
“Reluctant”
“Beautiful”
But one of the most interesting things I noticed with Google Photos is that, with a wide enough image set, you can begin to understand that the program has its own opinions about what’s beautiful.
Since the archive consisted of mostly pre 21st century material, male models were unfortunately scarce. While the majority of the images were female, some men did appear under ‘beautiful’:
Another really intriguing thing that I found is that Google Photos seems to be able to grasp symbolism in paintings, perhaps even better than humans. When I searched ‘death’, I mostly got neoclassical paintings depicting Greek Gods and Christ meeting their demise. But a particular work intrigued me the most:
“Death”
There is no direct suggestion of death in the painting, save for an ominous black drape suspended within the ropes of the ships. Without associating the item with death, I would have thought it inconspicuous. But Google Photos seems to know something I don’t.
“Loss”
“Despair”
“Hope”
I imagine that I’ll find plenty more astute observations by Google Photos, but this image interpretation proves that artificial intelligence will have a massive impact on our lives. What will happen, for example, when our phones or CCTV knows that we are sad? And what are companies like Facebook and Google doing with the billions of images they own right now?
