Assisted Drawing

Exploring Augmented Creativity

Image for post
Image for post

Seeing drawings that were made over 20'000 years ago has a haunting effect. Human history has been profoundly shaped by our ability to create visual content - assisted by tools. This experiment and report explore a new generation of creative tools, that help us draw.

Image for post
Image for post

Drawing is one of the oldest forms of human expression. According to Wikipedia it is a “”. Drawing helps us to communicate. The history of drawing is intertwined with the evolution of tools. From Rocks to Photoshop: Tools help us to change perspective and draw .

Image for post
Image for post

Experiment: Classifying Drawings

This experiment started with an exploration of the research project “” by Mathias Eitz, James Hays and Marc Alexa. The researchers collected 20,000 sketches across 250 categories (741h drawing time) and build a system that classifies drawings in realtime. For the following video, I asked a professional Illustrator to test the system:

A primary aim of this experiment was to explore how humans draw when a machine continuously is trying to guess what is being drawn. To intensifying the user/machine feedback-loop, a Text2Speech system was added. Illustrator: KedamaMi - Music: Oscar Woods.

Image for post
Image for post

Experiment: Memorability Prediction

This experiment started with an exploration of the research project “Understanding and Predicting Image Memorability at a Large Scaleby Researchers from MIT’s Artificial Intelligence Laboratory (CSAIL). Their system can predict how memorable or forgettable an image is.

Image for post
Image for post

As the current system is trained on photos, performance on drawings is poor. You can try it out with your own photos, by using their online app. This experiment serves as pointer to a future, where machine assistants can tweak drawings (& photos) on-the-fly, to make them more .

Image for post
Image for post
Caves in Lascaux, France

Research

Augmenting human skills with machines has a long history. Our ability to draw has continuously been expanded with new techniques. While we are still feeling the aftershocks of tools like Photoshop & AutoCad, A.I assisted drawing tools are emerging that radically redefine creative processes.

Image for post
Image for post

Research in “Assisted Drawing” technology dates back to the early 1960ties, where Douglas C. Engelbart famously called for the Augmentation of the Human Intellect. In recent years, research has substantially accelerated: A wide range of experiments are taking place in machine learning, human computer interaction, psychology, design and art.

Image for post
Image for post

20 Assisted Drawing Examples

The following is a collection of 20 Assisted Drawing Projects. It is in no way complete and aims simply at sparking a debate about vision & opportunity.

01. Freehand Drawing with Realtime Guidance

02. Reinforcement Learning Drawing Agents

03. Assisted Creativity for Novices

04. Style Transfer

05. Exploring Latent Spaces

06. 3D painting simulation at the bristle level

07. Autocomplete hand-drawn animations

08. Computer Assisted Animation of Line and Paint

09. Animating drawings with face recognition

10. Drawing in Virtual Reality

11. Drawing with Games

12. Handwriting Beautification

13. Robotic Handwriting Assistance

14. Generating Handwriting

15. Live stream of Drawing Robot controlled by Users.

16. Drawing Physical Stuff

17. Memorability Prediction

18. Adaptive Drawing

19. Live stream of Human Drawing in-front of 3 million people.

20. Drawing with Elephants

Thanks to @zachlieberman @graphific @genekogan @hardmaru @kastnerkyle @alexjc @udibr @alecrad @worrydream @KedamaMi @nervous_system @inconvergent @auriea and many more for inspiration!

Image for post
Image for post

Final Thoughts

Le Corbusier once said:”. The advent of powerful assisted drawing tools is enabling us to draw and explore yet unseen perspectives. The implications of are profound & exiting!

Image for post
Image for post
Image by Roelof Pieters: Creative AI & Multimodality

Get in touch here: twitter.com/samim |http://samim.io

Sign up for the Newsletter for more experiments like this!

Supplement

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Follow Up Discussion

Written by

Designer & Code Magician. Working at the intersection of HCI, Machine Learning & Creativity. Building tools for Enlightenment. Narrative Engineering.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store