Exploring Augmented Creativity
Seeing drawings that were made over 20'000 years ago has a haunting effect. Human history has been profoundly shaped by our ability to create visual content - assisted by tools. This experiment and report explore a new generation of creative tools, that help us draw.
Drawing is one of the oldest forms of human expression. According to Wikipedia it is a “visual art in which a person uses various instruments to mark paper or another medium”. Drawing helps us to communicate. The history of drawing is intertwined with the evolution of tools. From Rocks to Photoshop: Tools help us to change perspective and draw differently.
Experiment: Classifying Drawings
This experiment started with an exploration of the research project “How Do Humans Sketch Objects?” by Mathias Eitz, James Hays and Marc Alexa. The researchers collected 20,000 sketches across 250 categories (741h drawing time) and build a system that classifies drawings in realtime. For the following video, I asked a professional Illustrator to test the system:
A primary aim of this experiment was to explore how humans draw differently, when a machine continuously is trying to guess what is being drawn. To intensifying the user/machine feedback-loop, a Text2Speech system was added. Illustrator: KedamaMi - Music: Oscar Woods.
Entertaining side-note: Youtube’s A.Is classified this video as “copyright infringing in over 150 countries”, due to the used song. It was composed in 1937 by Oscar Woods and is now Public Domain, according to Archive.org.
Experiment: Memorability Prediction
This experiment started with an exploration of the research project “Understanding and Predicting Image Memorability at a Large Scale” by Researchers from MIT’s Artificial Intelligence Laboratory (CSAIL). Their system can predict how memorable or forgettable an image is.
As the current system is trained on photos, performance on drawings is poor. You can try it out with your own photos, by using their online app. This experiment serves as pointer to a future, where machine assistants can tweak drawings (& photos) on-the-fly, to make them more memorable.
Augmenting human skills with machines has a long history. Our ability to draw has continuously been expanded with new techniques. While we are still feeling the aftershocks of tools like Photoshop & AutoCad, A.I assisted drawing tools are emerging that radically redefine creative processes.
Research in “Assisted Drawing” technology dates back to the early 1960ties, where Douglas C. Engelbart famously called for the Augmentation of the Human Intellect. In recent years, research has substantially accelerated: A wide range of experiments are taking place in machine learning, human computer interaction, psychology, design and art.
20 Assisted Drawing Examples
The following is a collection of 20 Assisted Drawing Projects. It is in no way complete and aims simply at sparking a debate about vision & opportunity.
01. Freehand Drawing with Realtime Guidance
02. Reinforcement Learning Drawing Agents
03. Assisted Creativity for Novices
04. Style Transfer
05. Exploring Latent Spaces
06. 3D painting simulation at the bristle level
07. Autocomplete hand-drawn animations
08. Computer Assisted Animation of Line and Paint
09. Animating drawings with face recognition
10. Drawing in Virtual Reality
11. Drawing with Games
12. Handwriting Beautification
13. Robotic Handwriting Assistance
14. Generating Handwriting
15. Live stream of Drawing Robot controlled by Users.
16. Drawing Physical Stuff
17. Memorability Prediction
18. Adaptive Drawing
19. Live stream of Human Drawing in-front of 3 million people.
20. Drawing with Elephants
Le Corbusier once said: “I prefer drawing to talking. Drawing is faster, and leaves less room for lies”. The advent of powerful assisted drawing tools is enabling us to draw very differently and explore yet unseen perspectives. The implications of A.Is augmenting human creativity are profound & exiting!