Art via Algorithm
An introduction of image classification based off of Artistic Style by Leon A. Gatys, Alexander S Ecker, Matthias Bethge
Which of these images are computer generated and which is human-made?
Both of these two interpretations of van Gogh’s The Starry Night and Munch’s Der Schrei were both made with an algorithm.
This post is going to be the laymans version on how art or images of any kind are artifically composed in a system made through algorithms that are designed for image recognition and classification.
I’ve heard these buzz words but how does it work?
Those paint by numbers pages you did as a kid are similar to image classification. If you were painting a picture of a forest in this manner you would be told which blue-green paint the Blue Spruce tree needs to be. The apple tree right next to it would be closer to a hunter green shade. Even though trees give a sea of greens each one has it’s own name and distinct color. Each tree having a color assignment is also how a computer is able to interpret it. Every picture you see on your screen is created by pixels. These pixels are being assigned a color code known as an RGB value which instructs it to show up a certain color. The correlating pixel just follows the assignment and acts accordingly.
Now lets imagine as if we used the CSI type computers to flatten one image where under a microscope you can see stacked into cake like layers. The bottom layer of cake is funfetti- many different colors similar to our paint by numbers. The next layer is about the content of the image minus the specific color values. Are there many houses? Many trees? Next layer is style. This is like the texture vibe of the painting. Is it comprised of big swishy brush strokes or many tiny dots? These process these layers continue to be refined in are listed in the next steps.
Step 1: Convolutional Neural Network- this is the vehicle where the initial images are placed. It’s how we’ll be able to computationaly do what we need to. From locating the specific RGB values to finding what the overall style is.
Step 2 is Content Reconstruction- where all the layers are pieced out and matched for similarities.
Step 3 is Style Reconstructions. In addition to the original images uploaded, the results of all of the layers processed are combined for the final unveiling.
In all I found the sentiment of image classification fairly simple to understand as I had started learning the process when I was in Kindergarten. It will be fascinating to see how paint by numbers can be elevated to prepare children specifically for machine learning foused jobs.
Resources: