Art Detective: AI for Art Style Recognition
Being both a computer science student and an artist, I love testing the ways in which I can combine my two interests. In doing so, I decided to design an AI-powered application called Art Detective. Art Detective is an art style recognition application that uses visual recognition technology and takes user-inputted images of artworks, and responds by telling the user which art movement/style the inputted piece is recognized as belonging to. The app targets artists, art lovers, and even those who are curious about art. If an artist is curious about which art style their own work most aligns with, they can use this app to find out. If somebody likes a piece of art, they might use this app to identify the art style or movement that this piece reflects so that they can further explore the style and discover more art work that they love.
AI refers to technologies that take on human intelligence. The AI feature that this application takes on is its ability to not only look at an artwork (visual recognition), but also to be able to examine its visual characteristics and reach a conclusion about its style and even the art movement it is a part of simply based off of such visual cues. This requires learning and analysis, a feature of human intelligence. This is a good AI feature as it seeks to take some of the more thoughtful, analytical and intuitive parts of human intelligence and apply them in order to recognize some of the more nuanced details of visual art.
The most similar application to this concept is found in Justin Johnson’s application of the paper A Neural Algorithm of Artistic Style (Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge). We looked at this concept in our Design with AI class, as this is the project that combines one artwork’s style with another image to produce a version of said image in the style of the artwork. I think that a pro of this application is the fact that the computer is capable of understanding the style of a particular artwork well enough to create a new image out of it. Its ability to recognize one work’s particular style speaks to AI’s ability to potentially identify the more broad style or movement to which a piece belongs. Justin Johnson’s work is also fun, which I think is definitely a pro. It is entertaining to see another image in the style of a recognizable artwork, such as that of the Starry Night by Vincent Van Gogh. I think that the cons of this application are found in its potential to minimize creativity when it comes to making artwork. However, I do not think that the cons outweigh the pros as I see this more as a fun activity than something that sincerely threatens the creativity of the art world.
Art Detector is currently designed such that it can detect five different art styles prominent during the western modern art period (1860s-1970s), including surrealism, expressionism, impressionism, cubism, and pop art. The image recognition technology was trained using Apple’s Create ML with each of the five categories of art containing 50 training images and 10 testing images for classification. The performance of the training and testing is shown below:
Considering the subjectivity of each individual artwork, the results were rather impressive. The model accurately classified 100% of the training images, 95% of the validation images, and 78% of the testing images. Models tend to have higher accuracy for the training dataset because they learn from that set, so it is unsurprising that the testing accuracy was lower.
Precision refers to the ratio of of correctly detected positives (in this case, images that actually belong to a particular art style category) to the total sum of all detected positives. Precision for each of the categories is at least 60% or higher (we got 60% for expressionism). This means that for each category, the application detects at least 60% of all works accurately.
Recall refers to sensitivity, or to the percentage of actual positives that are detected. In this case, it refers to the actual images that belong to a particular art style category. Recall for each of the categories is at least 50% or higher (we got 50% for surrealism), which suggests that besides surrealism, the application is good at finding all relevant examples of a category. It means that our app, for all categories, detects at least 50% of all true members of each category.
Introducing the design of Art Detective:
Users will begin their experience with Art Detective on the home screen as shown above. Once they press begin, they will be able to scan or upload an image which will then lead to results such as these:
Above is an example of the result of a user’s input of Murnau Street With Women, 1908 Wassily Kandinsky. The image recognition technology detects that the image reflects visual tendencies of expressionism, and therefore assigns expressionism as its label. Similar results could be found even with the input of a lesser-known artist as the technology relies solely on visual cues of the art in order to determine style, rather than the knowledge of the artwork itself.
The user is also provided with red x icon and a green checkmark icon to provide the application with feedback about its results. If the user finds the application’s assessment accurate or useful, they can use the green checkmark, and if they find that it needs improvement, they can use the red x icon. This feature intends to improve the capabilities of the visual recognition technology as the app will continue to be trained as it learns more from the users’ inputs through this feature.
In this case, above, the user inputted an undetectable image. Therefore, the application responds by informing them that it is unable to match this input with an art style, and that they should try again.
Shown above, the user prompts Art Detective to recognize the style of I and the Village, 1911 Marc Chagall. The visual recognition technology assesses that this work best reflects its understanding of cubism, and provides the user with its assessment. In addition, as with any of the styles, the user may tap on the art style definition to find more information about it. The result of this action is shown below:
Each of the art style definitions may be expanded to offer the user a brief profile of the style, overviewing its definition, popular artists of the movement, as well as more examples of artwork from that style. This feature intends to engage the user beyond simply informing them of the name of style which their input matches, but rather providing them with a general understanding of that style. This way, the user will hopefully walk away with more knowledge, and even more curiosity about art.
