Machine relearning
What I’ve learned from one year of workshops
Over the past year, I’ve been teaching a workshop I’ve usually called “machine learning for artists,” which is also the name of a free, online book I’m building with collaborators, containing supporting materials and educational resources about a subject I’ve been interested in for a long time which has received a groundswell of public interest in the last two years.
I’ve been very lucky to be able to turn this into nearly a full-time job. I didn’t intentionally set out to do that, but each workshop seemed to bring about another invitation for one elsewhere. Over the past 14 months, 29 workshops have been organized, taking place collectively over 76 days and roughly 500 hours in session, not counting the hours of spontaneous hacking which would occasionally follow. Many of them have been co-taught with friends and collaborators [1][2][3][4][5][6], steering into numerous subtopics and application areas. Three were recorded and published online, to go with the screencasts I posted from a class of the same name I taught at the ITP program at NYU last year, the first of its kind that I did.
Workshops have varied considerably in duration and location. In one organized with School of Machines in Berlin, we met every day for a month, whereas others were scheduled over just a few hours in a day. They’ve taken place at universities, companies, makerspaces, conferences, festivals, and occasionally even outdoors (when the weather permitted).
Some were decidedly high-level, focused on conveying the theoretical and mathematical foundations of the research to a more general audience, or examining the social dilemmas posed by our increasing integration of these technologies into everyday life. But most workshops have emphasized making and hacking, appropriating these methods into the areas that participants care about, and encouraging a more active and personal engagement with the tools themselves. From design to architecture to music and many others, the applications within creative domains are numerous, diverse, and growing continuously.
This emphasis on hands-on experimentation has led to a virtuous circle, whereby workshops have fed back into the development of ml4a’s tools, culminating in a recent beta release of nearly 30 standalone applications which handle common machine learning tasks. These generic tools help enable students to build prototypes much more rapidly, even if they have little or no prior coding experience. I’m always amazed to see what people of varying backgrounds, interests, and skillsets cook up with them, applying them in ways I would have never dreamed up myself. Last month during a workshop at Copenhagen Institute for Interaction Design co-taught with Andreas Refsgaard, the students — in groups of three — managed to build from scratch a collection of mature projects that last year I wouldn’t have believed possible to finish in just a few days of practice.
Strategic unpreparedness
Early on, my workshops were planned carefully to be as prepared as possible to make the most use of the always-too-little time we had. As I got used to doing them, I began to loosen up, so as to let sessions go in the directions that students favored, something which neither they nor I could easily anticipate ahead of time. I began modularizing my teaching materials — both the presentation slides and software — into interconnected nodes, likening them to a graph through which we could find a path most suitable to their impulses, frequently leading into unforeseen territory. For example, demonstrations of computer vision software would unexpectedly provoke a spirited debate over ethics in machine learning, or a review of recent research papers would lead to speculation over potential art applications. One of the most serendipitous instances of this occurred during a workshop at OpenDot Lab in Milan last November. Literally minutes before the second session began, I read news about the now-famed pix2pix paper and codebase by Phillip Isola et al, which had just been publicly released the day before. I was so excited about it that I scrapped that morning’s plans and showed pix2pix to my students instead, sparking a brainstorming session that would lead to the spontaneous creation of a collaborative artwork we called Invisible Cities, in which we used pix2pix to generate satellite imagery and transfer the style of one city onto another.
Talks as tiny workshops
Over the same timeframe, I’ve also given almost as many “talks,” by which I mean a lecture of no more than an hour, although with the shortest workshops being only 3–4 hours themselves, the distinction is not always so clear. The lines are further blurred by running my talks from the same custom application I wrote to teach my workshops with. Built on top of openFrameworks along with some C++ deep learning libraries [1][2], some of my “slides” are actually self-contained real-time demos, interactively showing how to do things like identify objects in a camera feed and retrieve similar items, arrange and visualize large datasets of images and sounds, depict the internal states of convolutional networks, train your own to make music and play video games, and many others. By injecting my talks with fun demonstrations of these tools, I start to think of them as very short workshops in and of themselves, encouraging attendees to actively pursue a deeper study of the subject afterwards.
Aspiration vs. quality of life
As a personal aside, the workshops have taken place in many different cities around the world, allowing me to travel to places I had never been to before, make many new friends, and meet in-person with people whom I had only known through the internet before. But they’ve not been without some drawbacks. They require a lot of preparation and continual maintenance, taking time away from my own research and art practice, and have made it harder to pursue long-term commitments planted in one physical location. The lack of major institutional support means I am responsible for most of the administrative overhead as well, and I’m liable to losing entire afternoons just trying to keep track of my own invoices. The constant travel, although rewarding, can be draining as well, and I usually make very little time for sightseeing. On one trip to teach a workshop at Parsons New School in Paris, where I had never been before, I only managed to visit the Eiffel Tower in a mad scramble on my way back to the airport.
Sustainability and future prospects
The fees from the workshops have accounted for most of my income this year, allowing me to keep ml4a a free project — not only free in terms of costs, but free of obligations that could compromise its accessibility, openness, and respect for movements I support [1][2]. This is the most challenging and frustrating aspect of the endeavor, and my stubbornness probably costs me opportunities to expand its scope or reach new audiences. Not every potential contributor has as much privilege to slash and burn through in service of a free project as I do, and I probably spend too much of my time thinking about the next bit of content it needs, rather than its long-term sustainability. But as the book inches closer to a first draft, I’m beginning to consider crowdfunding platforms as a possible first step towards bridging this gap, and am actively seeking out feedback and advice from people with experience in self-publishing.
Additionally, I’m looking into ways of evolving the format into something which can be adapted and repurposed by anyone, and encouraging contributions from more participants in order to facilitate the creation of a common set of baseline tools that are broadly useful for many people. This initiative is inspired by the endless creativity of my workshop participants, and technologists out there more generally. There seems to be an ecosystem forming at the intersection of art and machine learning. The two disciplines bring very different legacies and vocabularies to the table, yet they have much to offer each other.
Complementary partners
The great paradox of machine learning for artists is its economics. In a field which is booming commercially and industrially, it can be challenging to make the case for spending time on being creative with technology in a way that has few immediate practical benefits, let alone for making “art for art’s sake.” Pursuing art as a career is difficult for most people; secure academic jobs require costly degrees, and the art world lacks the abundance of high-paying jobs that medicine and law can provide to justify similarly priced training. This lack of professional infrastructure leads most engineers and researchers to forgo any such practice, fearing it won’t lead to viable employment, a reasonable concern. On the other hand, my own artistic engagement with machine learning has quietly given me real-world experience in computer science, a field I never formally studied. More importantly, it’s given me intangible skills that outlive the ephemeral year-to-year overturn of the technology itself.
A creativity-driven approach engages people and helps communicate the topic to the general public. At the same time, the vitality of the machine learning sector within industry and commerce could potentially support outlets for people whose curious impulses are stifled by financial barriers. The mechanics of how a partnership between these two fields could work are still fuzzy to me, but the opportunity is impossible to ignore.
Next steps
ml4a’s release date has been delayed nearly a dozen times in the past year. Not since the final 45 minutes of the movie A.I. has the climax of something involving artificial intelligence dragged out for so long. But those who have followed its progress know that “release” is hardly more than a ceremonial term anyway, since the project is under near-constant public development. The questions I’m already looking ahead to are how can this project sustain itself into the future? How can it be structured in such a way that participants are incentivized to contribute back to it? Can ml4a evolve toward something like a blueprint or a kit, which others can freely unpack to organize their own workshops in their own communities? If I’ve learned nothing else from this year, it’s that I can’t really anticipate what others can build from these materials. I still don’t fully know what this project should become, but I’m open to ideas. Let this reflection be a first step towards finding out.
