Introduction to Runway ML — Machine Learning for Everyone

What if I told you that you could run a machine learning algorithm with only a few clicks of your mouse— no coding required?
That would be exciting, right?
You can now, thanks to an exciting new application called Runway ML.
Runway ML
Runway ML is a beautiful and easy-to-use application that makes machine learning more accessible and inclusive for a broader audience of people. It lets anyone with a computer start to explore and create using the latest AI / machine learning models. Here’s what it looks like:

Benefits:
- No coding required
- Works with images, videos, text, data (depending on the model used)
- No expensive hardware required
- Can run models locally or in the cloud using GPU credits
- Extensible to other pieces of software — i.e. a plugin for Photoshop
This software reduces the barrier to entry for machine learning, and lets folks start creating right away. People have already started making some really inspiring projects.
Getting Started
- Download the software from their website
- Create a Runway Account
- Start Playing!
Machine Learning Models
With Runway ML installed and running, the interface provides a quick way of searching for available models from the left-most tab on the interface.
Clicking on a model will bring up additional information describing its purpose. Users can then add this model to a workspace and begin creating.
Thanks to their excellent software developer kit, the list is always growing. Though some common types of models currently available are listed below:
- Object detection / Classification

- Text models

- Image generation

- Image post-processing

Using a Model
Once you’ve added a model to your workspace, you first need to run it. You can do so by clicking the button in the bottom-right corner of the interface. By default it will run on a GPU in the cloud using cloud credits by the minute. Though you are free to download the model and run it locally, provided you have a decent enough GPU.
When the model is running you can begin to use it. For this example, I want to colorize a black and white photograph that I found from the Library of Congress.
I click on the `File` icon at the top, choose a photo or video and wait for the model to return the result in the space below.

Each model has a number of parameters that you can modify on the right, including image options and model-specific parameters.
When you’re done, you can click on the little download button next to the image at the bottom to save it to your machine.

A lot of these models are fairly computationally expensive, and will take some time to run, so be wary of the size of image/video that you use.
Using Runway in a Creative Workflow
The Runway ML application also exposes the model input and output so that it can be integrated into other applications (using methods such as HTTP, Socket.IO, Javascript, or OSC).
There are some great examples on their website and their Github that show how you can integrate Runway ML into other applications.
Applications:
- p5js / Javascript
- Processing
- openFrameworks
- Touchdesigner
- PureData
There’s even a Photoshop plugin that someone created:
Publishing New Models
For more advanced users wanting to share their models with the world, Runway ML provides a very easy-to-use Software Developer Kit. With only a few lines of code, you can publish your saved model to the Runway ML ecosystem.
My Favorite Models
So far I’ve been loving Runway for my own creative work in making generative artwork / images. It has become another tool in my shed.
Here are a couple images i’ve made with the aid of some of my favorite models to work with so far:
- Arbitrary Style Transfer

- CycleGAN

- StyleGAN

As more models come online, the utility of Runway will continue to grow. Right now, I’m excited for this amazing tool to develop further and for the future of machine learning in creative practice.
Additional References:
- Runway Documentation: https://learn.runwayml.com/#/
- Runway Tutorials: https://www.youtube.com/channel/UCUBqu_z5uP0AZhYtuyFZB3g
- Runway for Creators: https://www.youtube.com/watch?v=ARnf4ilr9Hc
- Coding Train Intro: https://thecodingtrain.com/CodingChallenges/150-runway-rainbows.html
Please feel free to follow my accounts on instagram or follow me on twitter if you’d like to stay in touch!
Generative Art: https://www.instagram.com/cocos.graficos/
Twitter: https://twitter.com/c0c0_re1d