Final Project Proposal

For my final project, I want to create an interactive project where a user draws something random on a p5.js canvas without any fill, decoration, or color and the resulting output is their drawing colored or filled with parts of famous paintings (imagine a drawing of an owl having Vincent Van Gogh’s Starry Night as its body color). The thought of fusing simple art (user’s input) and complex/abstract art (the famous paintings) was something fun to think about when I reflected on the simplicity of drawings that I drew for the Google Quick Draw.

One Sentence Summary: A fun, personal interactive interface where the user draws a simple drawing and it artistically unique by transferring textures and styles of famous paintings, using the Fast Style Transfer model of Runway.

Once the user draws something simple on canvas, it becomes filled with a painting from Runway

Audience: At the current level, this interface would be best suitable for those who want to recontextualize a simple drawing they have just made and look at it from a different point of view. For children, it’s a learning tool that teaches them the artistic potential of any drawing. It’s also meant for people who want an indirect interaction between a famous painter and themselves, as both art fuse into one.

The process looks like this:

1. The user draws something simple on p5.js canvas within 10–15 seconds (or is given a prompt)
2. RunwayML transfers style of famous painting to the p5.js canvas
3. The painting fills the empty spaces of the inputted drawing
4. Depending on what painting was used, change the title label to make the user feel satisfied with his/her work

Another inspiration for this idea was something I read on the Internet, about a father using Photoshop to make his child’s drawings of real-life objects look real:

I wanted to explore a similar idea except filling in the drawing with famous paintings rather than the image of the object/being represented in the drawing in order to inspire people to think outside the box by deviating from the reality for a moment.

Challenges

How will Runway detect where to place the painting image on the canvas and resize according to the inputted drawing? I am currently exploring how to do that. One way I thought of doing is to create a small canvas (100x100) so that it will limit the size of the drawing by the user and also limit the scope of the painting from Runway. But if I draw something with multiple edges, how will I code so that the paintings only show within the drawing and not bleed to the rest of the canvas?