Five Pieces of Advice for Your Creative Coding Project
An Emmy Award-winner, fine artists, a web app developer, and an AI entrepreneur share perspectives on using emerging technologies as an artistic medium.
CLOUDS documented the creative coding community with volumetric capture software. Glitch Textiles made strange moments in code into woven blankets. The Songularity offered a pop album written with predictive text. On Kickstarter, artists and developers have brought visions for unexpected and groundbreaking creative coding projects to life.
We recently hosted our first Creative Coding Project Jam to bring together like-minded creators to chat with fellow coders, meet potential collaborators, and get inspired by new ideas and technologies. Here, we round up the perspectives and tips that came from our expert speakers, and encourage you to incorporate them into your own creative practice.
Use residencies to iterate on your ideas
“Someone once called me a professional artist-in-resident,” says Sougwen Chung. She’s helped companies like Nokia Bell Labs, Google, Eyebeam, Japan Media Arts, and Pier 9 Autodesk deepen their understandings of the interpersonal impact of technology, while researching layered contexts and questions in her own art practice. Since 2014, she’s been evolving her practice with three different generations of D.O.U.G. (Drawing Operations Unit: Generation_1, 2, and 3), a robotic arm she’s programmed to mimic her gestures, speculate at robotic memory, and paint in a multi-robot swarm.
“I’ve been able to fund some of these self-motivated experiments by working with different institutions. My practice is part of a creative ecosystem, it is informed by thinking about the multiplicity of relations that comprise working with technology today. It involves engaging with different types of engineers, different types of robots, like industrial and collaborative robotic arms.” At Autodesk that involved using existing machines like Kuka industrial arms. At Bell Labs, it involved hooking up custom units to established computer vision software.
And these residencies do more than just help fund her projects and provide access to proprietary technologies. Her presence promotes creative approaches and reflection at companies where “technology seems to often be the answer, but it’s not always clear what the question is.”
Acknowledge that art and entrepreneurship can be one and the same
James George co-directed CLOUDS, an interactive virtual reality documentary about code and creativity. James describes CLOUDS “as a video game masquerading as a documentary. It’s a VR world of interviews and artworks, all tagged, interlinked, and navigable in the same nonlinear form as the web.”
In developing CLOUDS with his co-director Jonathan Minard, the pair used openFrameworks creative coding software to develop the 3D depth camera, which captures hologram-like 3D video portraits. “I realized that this new device could potentially change the way that filmmaking works,” George explains.
“I came to appreciate that being an artist or an entrepreneur didn’t have to be separate. Building tools which allow other people to express themselves can be a work of art in itself.”
George has gone on to develop the software for CLOUDS into a full-fledged tool called Depthkit, which allows creators to capture hologram-like video with standard 3D sensors like the Kinect or Intel RealSense, a camera, and a laptop. Over 10 titles using Depthkit technology were at major film festivals around the world this year, in many cases bringing new voices to filmmaking. Asad Malik made Terminal 3 as a college student just coming from Pakistan; Illya Szilak directed Queerskins on her days off — she’s a doctor on Rikers Island.
Misuse your tools, not your collaborators
When Michael Mandiberg speaks about their code-based artworks, the artist brings technical collaborator Danara Sarioglu along, too.
“When somebody brings in a consultant to help them, or has a collaborator, an assistant, or an intern, I think a lot of labor is often effaced,” Mandiberg says. “I stand resolutely against that.”
Mandiberg is all for creatively misusing the platforms where so much of that digital labor happens, though. Sarioglu has joined in on projects like Print Wikipedia, which downloads all of Wikipedia and offers on-demand printed volumes through publishing platform lulu.com. And with Postmodern Times, they commissioned Fiverr freelancers to remake scenes from Charlie Chaplin’s Modern Times, and a quantified self portrait that tracked all of Mandiberg’s digital labor.
“One thing I end up doing a lot is putting more information into a platform or program than is supposed to be there. In many ways, that produces a kind of emotional valance for the data.”
Bring personal projects to your professional platform
“My job leading the team that builds Glitch.com is to showcase really awesome creators and all the things that they’re building,” explains Jenn Schiffer. Glitch is a platform for “basically anything that you can use code to create. We mostly support Node and JavaScript but we have Python projects, C projects, and, since I’m a troll and former PHP developer, we have PHP projects.”
To show off what the platform can do, she’s not shy about putting her personal projects — from erotic art to local community engagement initiatives — on Glitch.
“I make erotic pixel art, and I’ve always been making tools to facilitate the creation of art. My biggest passion project is make8bitart.com, a pixel art editor in the browser. When I heard about Glitch two years ago, the first project I started was to make checkbox art.”
Glitch is also a tool she’s used to cheekily challenge some of tech’s deepest issues. “We call Glitch ‘the friendly community where you’ll find and build the app of your dreams,’ because we want to signal that we expect people to treat each other respectfully. Most social platforms don’t set that bar, so we’re trying to get ahead of that. The question of how to maintain a positive community energy keeps me up at night, thinking, ‘How are people going to use my awesome, friendly platform to harm people? How do we not make the mistakes that platforms like Twitter are doing, where they’re just not fixing it?’ So I was like, ‘How do I prototype the perfect community that doesn’t require any moderation?’”
Her answer was jennchat. It’s not super practical, but it is indeed difficult to misuse. “You can only say three words in the chat. They’re the three words that I use most commonly on Slack: cool, nice, and siiick. The ‘siiick’ has to have three ‘i’s in it. It’s also case-sensitive because I don’t ever use uppercase.”
Make structures for yourself, keep pushing their limits
Phillip Stearns had his first real taste of Kickstarter success with Fragmented Memory, glitch art throw blankets that he dreamt up — and continued to develop — through iterative experiments and commitment to a daily practice.
Stearns started out as an audio engineer, dipped into musical composition, and then began exploring hardware-based media art. He started in an engineering context, which led to circuit bending (opening up toys and rewiring them to understand how they worked) and, finally, building original systems. “I started thinking about how I can work with the digital image in a similar process. I worked with people who were doing data bending and datamoshing, and decided that I would explore this process daily, which became a yearlong data project,” he says. It ultimately turned out to be too daunting, but he credits that initial motivation and structure with getting him going — and the resulting exhaustion with pushing him to new projects.
“The takeaway was that I found a good format to get me started. I had the ambition to do something big which led to some great, an unexpected, output. After the ‘year in code,’ I had enough of looking at a screen that I decided to pick up metal working.”
Reflecting on his process of “breaking things and exploring the nooks and crannies of how they’re constructed to find the latent potential within, I realized that was essentially what security hackers are doing.” He began to understand “just exactly how the internet I grew up with and loved had become weaponized.” This led him to work on OpenVault, a speculative malware that puts some of the world’s most famous computer viruses in ‘90s-style software product packaging. “The project has an old school feel, but it’s purporting to sell actual, real-life cyber weapons. It’s not necessarily a call to action, but certainly a signpost indicating: ‘This is happening. These weapons are in the world.’”
Give the people what they want
Cristóbal Valenzuela has been working on Runway, a toolkit that makes AI intuitive for creators of all kinds, since he moved to New York to attend NYU’s ITP (Interactive Telecommunications Program). “I don’t have a background in computer science. I was just curious about AI, specifically in how it could impact creativity.”
One of his first projects was a neural network tool to describe the content of videos and suggest similar scenes in other videos/films. For example, you can input your own video of a sidewalk and get back similar sidewalk shots from movies like Children of Men or Her. Next, in collaboration with Anastasis Germanidis, he made “Uncanny Road,” a web tool that used Generative Adversarial Neural Networks to generate a never-ending photorealistic road. Text2image, simple web application that generates images as you type them, followed. When he put each of these online, he says, they “blew up.”
As people started using his products, he recalls, “I was figuring out that what was compelling for me was not creating experiments or art projects so much as creating tools that allow other people to create their own content.”
Eventually all those creative explorations and experiments lead to Runway, a tool that allows creators of all kinds to use powerful AI algorithms and connect them to programs like Photoshop, Premier, Illustrator, Unity, or whatever other software they use to create content. They can search through models like one that turns black and white images into color images or another that recognizes faces, and then run them without writing a single line of code.
The platform is still in beta, but the idea is to expand its AI capabilities with existing and new research in the machine learning space and allow users to easily train their own ML models. Like with his other experiments, he wants to see how AI can power human creativity and how tools can spark new ways of collaborating with algorithms.
If you have a project you’d like to discuss or your own advice to share, email creators@kickstarter.com.
