Generate Cinematic Video with Runway Gen 2
Runway Gen-2 offers a real solution to create cinematic footage and B-roll almost instantly.
As a video content creator, I’ve always wondered what it would be like to be able to generate video with my mind.
I know that sounds weird, but few things are as frustrating as getting to the end of a video editing process to realize you have gaps in the timeline that you don’t have the right footage to close.
I’ve generally either gone back to get more footage at another time, or I’ve turned to platforms like Storyblocks or Envato to try to find footage that matches my scene, colors, etc.
It works, but it’s not perfect. If only I could just reference the rest of my footage, describe what I want to see, and then have that perfect clip magically materialize before my eyes
That’s real now.
Runway Gen-2 is about to change everything
Turns out, we can all do exactly what I just described.
Runway Gen-2 is a tool designed to assist in video creation, offering various modes to generate videos from text, images, or a combination of both.
For content creators, that means it lets you generate videos from an image reference, and use prompting to maintain a consistent style across your footage.
And, I can’t overstate this — that is INSANELY valuable for people that make videos.
What is B-roll
Long story short, B-roll is just the footage that you use in between your main shots to break up the scene, add depth, and create context.
For example, if you were creating a cinematic video of you walking in the wilderness, maybe a hiking video, you would have A-roll and B-roll.
A-roll is the footage of you or the subject.
B-roll would be the footage of the mountainscape. Maybe it’s a close up of the forest that you’re walking through. Maybe it’s a golden hour shot as the sun sets behind the tree line.
B-roll is essential to improving your content and achieving a cinematic look and feel.
However, creating good B-roll can often be time-consuming and expensive.
That’s where Runway Gen-2 comes in, offering a real solution to generate novel B-roll that matches your footage.
How to use Runway to create the perfect B-roll
There are three core methods to create video in Runway:
Text-to-Video for Conceptual Shots
One of the standout features of Gen-2 is its ability to convert text prompts into video footage. If you have a specific concept in mind, simply input the text and let Gen-2 generate a novel video.
Image-to-Video for Consistent Styling
Consistency is key in video production. Gen-2 allows you to use an image as a reference for generating B-roll. This ensures that the style and tone of your B-roll match your primary footage.
Text + Image-to-Video for Contextual Shots
For more nuanced requirements, Gen-2 allows you to combine text and images to generate footage. This is particularly useful when you need B-roll that not only looks good but also fits the context of your main content.
This is really where the “matching” part of the blog title comes in — you can craft a scene that matches the styling of your video, and you can also create the scene, motion, and characteristics that you’d like by combining the image reference with a text prompt.
Pretty wild, right?
Putting this method to work
First and foremost, you’ll go to www.runwayml.com and create an account.
There is a free version of the tool available, or a number of subscription levels depending on your needs. The free version is great to play around with, but there are some really nice features with the subscription services, namely video upscaling and watermark removal.
Once you have an account, you can dive right into playing around with prompts and generations.
Text-to-Video Prompts
This is fairly self explanatory if you’re familiar with text to image generators like Midjourney, Stable Diffusion, or Dall-E.
Simply describe to the model what you want in the video. I’ve found that prompt best practices from text to image generators also apply to Runway.
We won’t get into prompts in this blog, but this is a good start for photorealistic images, which you’ll want for cinematic B-roll:
Runway does a really good job of capturing what I was imagining for the B-roll!
Image-to-Video Prompts
This is where Runway really shines, and you’ve probably already heard the buzz about this feature on the internet.
This is how everyone is animating their Midjourney images. And it’s pretty cool, actually.
The first thing I did was animate a photorealistic Irish Bulldog bartender that I created for a series on photorealistic dog portraits:
Check out the dog portrait series here!
Text + Image-to-Video
As I mentioned before, this one could be a game-changer. You see, it’s cool to be able to have a tool that generates random video from a prompt. It’s also cool to take a nice image that you generated in Midjourney and animate it like our dog bartender earlier.
But it would be very useful to take an existing scene, create a new scene from it, and influence that scene with your prompt.
Let’s put it all together.
Creating scene specific B-roll with Midjourney and Runway
Say we have a video we’re working on. We spent a full day shooting it in New York City and then flew back to wherever we’re from.
Unfortunately some of our footage was corrupted when we tried to edit it, and we can’t go back to get more. Sigh, we lost some really good B-roll shots.
But, all is not lost! You can steal this workflow to create scene specific footage that will match the rest of your video.
Take a still screen snip of some of the footage that wasn’t corrupted. In this case, the video was cinematic and shot in New York City.
Our screen snip looks like this:
Now, we want some footage that fits in with this general scene.
Let’s use this photo as a Midjourney image prompt to get a starting point for our new B-roll.
1. Upload your image into Midjourney
Drag and drop your image into the chat interface and hit Enter/Return.
2. Copy the image address
Click on the image to display it in a different window, then right click and select “Copy Image Address.”
3: Create an image prompt
Copy and past the address back into Midjourney with the /imagine prompt to use this as a reference for the model to generate new imagery.
Note: I’ve used the ––chaos parameter to try to get images that are close in style, but different from the reference image.
As you can see, the resulting images have a similar style and color palette to the reference image that matches our video, but we have some new and interesting scenes to choose from.
Upscale your favorite one.
4. Use the new image to create your B-roll
Now we can simply return to Runway and follow either the image-to-video or text + image-to-video paths to get our B-roll footage.
As you can see, the Runway model has recognized the water ripples and created a slow motion rippling effect in the video.
This GIF can also be extended to 16 seconds in length, so you have plenty of footage to edit into your final B-roll clip.
The Wrap Up
Novel video generation is really cool and I can see huge implications soon from the technology.
Is it 100% perfect today? Absolutely not — it’s basically brand new, cutting edge tech.
But, where we are today feels like lightyears from where we were last year at this time, and if that’s any indication of what’s to come from platforms like Runway, we’re in for a wild ride.
I hope you’ve found this content helpful! If you did, I post daily news, tips, and tutorials to help you navigate the digital world. Follow me for more!
