Text-to-Video: How to Write Prompts in Runway
Want to learn how to master text-to-video prompts for AI generated video in Runway Gen 2? You’ve come to the right place.
Medium is changing how they support contributors — if you like this content, claps are very much appreciated!
So you’ve tried out Runway and made some really cool videos, right?
But now you’re thinking, “I wish I understood these prompts enough to make this video better.”
You’ve landed in the right corner of the internet.
First, we’ll explain what Runway is and how you can use words or pictures to make amazing videos. Then, we’ll talk about the prompt specifics to make your videos even better!
What is Runway?
Runway is a generative video model that offers artists and creators a new way to generate novel video.
By using a variety of prompt modes — including text and image inputs — Runway allows you to craft videos that range from the visually stunning to the narratively compelling.
Also, it’s not just another editing tool — it’s a pretty major breakthrough that opens up a new frontier in digital storytelling and visual art.
The possibilities are as limitless as your imagination.
If you know how to prompt, that is.
Different ways to make videos in Runway
The platform offers a variety of prompt modes, each with its own set of nuances. Understanding these modes is important because they each offer some unique ways to manipulate your final product.
Text-to-video prompts
If you love to write, this is for you. You use words to tell Runway what kind of video to make. You basically write a story that turns into a video.
Image-to-video prompts
For those who think in pixels and color palettes, the image-to-video mode is awesome.
You can turn a single picture into a moving video. Imagine freezing a moment and then watching it come to life!
Image + text-to-video prompts
Want to generate video from an image, but also influence the final output with your text prompt?
This mode combines your text and image to create a video that’s as rich in narrative as it is in aesthetics.
Runway prompt techniques: modifiers, keywords, and cameras
I’ve created A LOT of content around text-to-image prompts in platforms like Midjourney, Stable Diffusion, and Ideogram.
There seem to be a lot of commonalities between generative image, and generative video.
For example, in my blog about mastering prompting in Ideogram, I asked GPT-4 to analyze all of the prompts on tons of different images that I liked, and create a set of best practices.
The core theme was that there are two really important parts of a good prompt: fundamental components, and augmentative components.
The same concepts apply to Runway, so lets break down what that means.
What are prompt action keywords in Runway?
Action keywords are fundamental to your prompt. These are words like ‘flying,’ ‘running,’ or ‘dancing’ that bring your video to life.
Imagine you’re telling Runway to make a video of a bird. If you use the word ‘flying,’ Runway understands that the bird should be soaring through the sky, not just sitting on a branch.
Action words give your video energy and make it exciting to watch. When you’re thinking of what you want in your video, consider what kind of action you want to see. It could make all the difference!
What are prompt modifiers in Runway?
Modifiers augment your final output to match a certain style.
These are words like ‘masterpiece,’ ‘cinematic,’ or ‘classical’ that give context clues to the model around the style of video that you’re trying to achieve.
For example, if you tell Runway to make a ‘cinematic’ video of a sunset, it’ll try to make the video look like something out of a movie.
Modifiers are really like the secret ingredients in a recipe. They can turn something good into something amazing.
Definitely don’t forget to use them when you’re writing your prompt.
How do you use camera specific terms in Runway?
These are words like ‘pan,’ ‘zoom,’ or ‘wide-angle’ that help you control how the camera moves in your video.
Imagine you’re the director of a movie. You get to tell the camera crew what to do — if you say ‘pan,’ the camera will move slowly across the scene, capturing everything in its path. If you say ‘zoom,’ it’ll focus closely on something specific, like a flower or a person’s face. And if you use ‘wide-angle,’ you’ll get a big, wide shot that shows a lot of things at once.
These camera terms let have much more control over your video, making sure the output is what you want it to be.
How to structure Runway prompts
According to Runway, a simple but effective way to structure your prompt is to use this format:
[base prompt] in the style of [style], [aesthetic]
For example, you could say “A serene beach at sunset in the style of a Monet painting, cinematic pastel colors.”
This gives Runway a clear idea of what you want, both in terms of content and style.
I’ve included a whole list of prompts and their output video at the end of the blog, but let’s finish talking about prompt best practices first.
Prompt simplification in Runway
With Runway, clarity is important. I know it’s tempting to load your prompt with intricate details and complex descriptors, but simplicity often yields the most accurate and useful results.
Think of your prompt as a clear set of instructions for Runway.
Overcomplicating it is like giving someone directions with too many turns and landmarks — it can lead to confusion.
Instead, aim for straightforward language that captures the essence of what you want.
For example, instead of saying:
“a bustling cityscape with towering skyscrapers illuminated by neon lights reflecting off the rain-soaked streets”
You could simplify it to:
“a bustling cityscape in the style of Blade Runner, cinematic neon lights”
This stripped-down version still conveys the core idea but makes it easier for Runway to understand and execute your vision.
Sample Prompts and Video
Note: none of these videos were upscaled, and they’ve all been turned into GIFs for Medium. Quality is not representative of the capabilities of the platform.
Beyond the camera: the community and ethics of Runway
The Runway platform also has a thriving ecosystem of artists, creators, and innovators. This community serves as the lifeblood of the Runway experience, offering a rich source of perspectives and ideas that can elevate your own creative endeavors.
In fact, there is an entire “Watch” page with user generated videos for you to take inspiration from.
Ethical do’s and don’ts of generative AI
As you dive deeper into the world of AI-generated art with Runway, it’s important to be mindful of ethical considerations.
While the technology offers lots of creative possibilities, it also comes with responsibilities.
For instance, always respect intellectual property rights. If you’re using text or images that aren’t your own as a basis for your prompts, make sure you have permission or that the content is free to use.
Additionally, be cautious when generating content that could be sensitive or controversial. AI-generated art should be a space for innovation and positive impact, not a platform for harm or infringement.
You can contribute to a more respectful and meaningful use of this transformative technology.
Wrapping Up
You’ve journeyed through the world of Runway, from mastering the art of prompt crafting to understanding the ethical nuances and the vibrant community that fuels this platform.
I hope you’ve found this enjoyed this content and found it helpful!
Do you like this type of content? Follow me for me! I post daily news, tips, and tutorials to help you navigate the digital world!
Related Content:
