Art & Machine Learning in NYC
What is art?
It’s a challenging question that gets at the heart of what makes us human and one that countless philosophers have tried to answer. If we came across a gallery containing the portrait below of a man who went to war and came back with stories that he wouldn’t share and scars that he wouldn’t show, you might feel something, something slight that tugged at your heart strings and made you linger for just a minute more. Is that art? That seems like it should be art.
What is an artist?
This reformulation is more than a single stroke on a canvas. The discussion now shifts from the story to the storyteller. How often do you remember the artist’s name behind the portrayal? They only parts that we seem to consider are the subjective quality of the work, the collective reaction to it, and the story we know of its origins. In the above, I never mentioned anything about the artist, not whether they were male or female, black or white, robot or human.
— — —
South Park Commons hosted a special event in New York City recently. The event was about understanding where Art and Machine Learning are coinciding and geared towards folks who identify as artists, technical experts, and machine learning specialists. Our goal was to show each crowd that there is something in this space for them, as well as highlight where it is heading at large. This isn’t just about technology, but also how people and society shape and are shaped by that technology. When talking about art, it’s important to remember that it is a reflection of the time in which it is birthed and developed. Just as Jacque Louis David and the Neoclassicist movement told a (very real) story of 18th century Europe, today’s artists are expressing a story intertwined with technology that is more and more a fabric of modern life. The machine learning exposition desires to eventually interject into this tale chapters involving new sentience, and it is society’s artists who are most on the front lines exploring how that affects us and is affected by us. We were lucky to have two such artists, Philipp Schmitt and Helena Sarin, prefacing the event with demos, and were graciously headlined by talks from speakers Prof. Hod Lipson, Gene Kogan, and Cristobal Valenzuela.
Why did we do this and why now? Joe, an alum of SPC and founder of Facet.ai, mentioned in a previous post how machine learning is augmenting human creative work and highlighted the awesome efforts from Google Brain Magenta. There’s a lot more going on underneath that tip of the iceberg and most of the crowd that could benefit from this doesn’t know about it.
In research, there has been a deep well of new papers exploring ways that could augment artists. We are no longer in the days of style transfer and deepdream dominating the headlines, but now can edit video like we do text, reliably transcribe piano music from scratch, and generate a strikingly good score of music with long-term coherence. On this front, Prof. Lipson spoke to about his adventures making robot artists over the past two decades. He is a professor at Columbia and has spearheaded research into creativity, curiosity, and self-awareness in that time, and has a rare view on what it means to be an artist. The most recent result is Pix18, a robot that is capable of painting images like you see above of the “man returning from war”. Pix18 forces us to ask questions like “What is an Artist [if not this]?” and “Are machines creative?” That has been a commonly cited separation between man and machine, but we can foresee a time when that line will blur, and arguably it’s already beginning.
A parallel is with animals. We frequently don’t credit them with creativity. Then they do something we consider creative and we assign the same attribute to the animal itself. However, the animal was creative all along if only we could understand its perspective better. When we suddenly do grasp how its understanding affects its actions, we gain an awareness of our own biases that we then update with this new information. Robots are no different from animals in this aspect.
A select group of artists are riding this wave of new possibilities and producing increasingly complex works. Phillip Schmitt’s Chair Project is a collaboration between AI and human to create chairs. In this scenario, the machine creates the chair and the human gives it life.
Mario Klingemann’s work is among my favorite. His Memories of Passerby is a machine that shows a woman and a man on either side, both of whom are constantly transforming into other people from the 17th-19th centuries. The catch is that none of these people exist, but all of them make the audience feel a little nostalgic.
Gene Kogan spoke to us on this topic. He briefed us first on his class The Neural Aesthetic, wherein he teaches machine learning to aspiring artists at ITP. You can find all of the material for that on the website. However, he was most interested in being the artist in the room and excitedly conveyed his dream of a decentralized autonomous art organization that utilizes machine learning to create an infinite, never-repeating, self-sustaining set of pieces. These aren’t statically set at the beginning but get better over time through feedback. It’s only the beginning for that adventure, but struck me already as an exemplar of the far-reaching nature that I ascribe to an artist’s vision.
Finally, we have the product side. It is not widely known how rapidly this space is growing, but we have noticed lots of companies springing up around how to best bridge the gap between the machine learning and the practitioners. This is absolutely essential to the ecosystem because what good is all of this technology if it can’t be effectively wielded? SPC graduate Facet is an example of this, and so is RunwayML co-founded by Cristobal Valenzuela, who gave our third talk. His talk was geared around all of the impressive capabilities built into Runway. They have a library of models for motion capture, image synthesis, segment masking, and more, which all make it trivially easy to manipulate these capabilities and bring your creativity to life.
We ended with a panel where the audience was given free rein to ask questions to the three speakers. One answer that was common to all of the speakers stood out. They were asked about what or whom is not getting publicity that we should not know about.
Their answer was not a specific person, but rather it was “the long tail of everyone who can now become this kind of artist.” I was struck by that because it’s reminiscent of what happened with the electric guitar. At the beginning, it was just a few people messing around with this crazy new sound. But once it hit its stride, it enabled people for whom this “crazy new sound” resonated extremely well. The same goes for the synthesizer as epitomized by Robert Moog’s story and designs, famously harnessed by Giorgio Moroder. There is a parallel here — there is a niche (but wide) audience for whom machine learning assisted art will unleash their inner artist in a way that other mediums never have. Organizations like RunwayML, Facet.ai, and Google Brain Magenta are pushing this forward and have developed tools for artists over the past few years. The latter also recently announced an exciting collaboration with Yacht and The Flaming Lips.
This story is far from complete. We haven’t discussed many other artistic mediums, including literature and games among others. In all of this, what we are seeing is that new approaches to the art are being metamorphized by what machine learning makes possible. The medium is not changing, but advancing, and with that comes a new message, one spoken by the artist and through the art. Or is it spoken by the art and through the artist?
South Park Commons is a community that helps entrepreneurs and technologists freely learn and start ambitious projects. We bring together talented people to share ideas, explore directions, and realize opportunities that occur in an environment that helps people take risks. If you would like to be a part of future SPC events in NYC, please sign up here.
