Apple to introduce next-level professional VR video tools in Final Cut Pro X and Motion 5

On Friday Apple provided a sneak peek at features that will be in the next versions of Final Cut Pro X and Motion 5. Apple previewed HDR workflows, new colour tools and VR video features in Final Cut. Here’s a rundown of the VR video features coming to Final Cut Pro X by the end of the year.

Whereas spherical video tools have been available in editing applications before, the way Apple have implemented them means they are for the first time suited to editors — unlike the current tools which are designed for motion graphics animators.

VR video version 0: Use plugins to ‘fool’ old applications into working with spherical video

Current professional video editing and motion graphics applications are not able to handle spherical video on their own. These tools were designed many years before the popularisation of VR and spherical video. They are architected to work with rectangles of video that play back on rectangular displays.

As VR video became more popular, third-party developers took the opportunity to retrofit the ‘old’ applications to work with the new medium. That meant building plugins.

Fortunately for spherical video makers, the VR video industry settled on a standard way to map the video on the inside of 3D sphere to the rectangles of video that ‘flat video’ tools are designed to work with. ‘Equirectangular projection’ is a term from geography that defines how you distort a spherical source — such as a globe of the Earth — to a rectangular area — a flat map showing all the countries of the world. Because Earth’s poles have relatively little interest to most people finding their way using a map, those parts of the globe are be very much distorted in rectangular maps. There is no distortion on the equator line, the distortion increases as you get closer to the poles.

Up until now, the art of working with VR video has been dealing with this distortion. Systems that capture spheres of video output equirectangular video. Video applications didn’t ‘know’ they are dealing with spheres of video distorted into rectangles. The plugins that motion graphics editors applied to these rectangles of video ‘understand’ equirectangular distortion. The fact that the NLE is working with spherical video is known by the film maker, not encoded into the project they are working on.

Recently VR video plugins have been bundled with video applications, and the applications themselves have spherical video preview tools and can also prepare the videos for use in VR devices or for online distribution using YouTube and Facebook.

It is time for the video applications to treat spherical video as a peer of flat video.

VR video version 1.0: Applications that understand spherical video

The next version of Final Cut Pro X will be the first professional editing tool that inherently understands spherical video. As well as flat video timelines, there are spherical video timelines. Clips are spherical or flat. They work correctly in either kind of timeline. Where flat clips have position parameters, spherical clips have orientation parameters.

Currently to fix the orientation of a clip so its horizon line is level requires the editor to apply a plugin. In the next version of Final Cut, there is a Horizon overlay.

There is a new Reorient tool available to fix footage that what shot by cameras set up at the wrong angle.

Editors will be able to drag with the new tool in the viewer to reorient spherical video, so that distorted horizon lines are level.

Here are new spherical video plugins:

There is a new VR headset viewer. This adds a simulated headset view of the current spherical environment. This view can be swung around inside the sphere by the editor to get a simulation of viewers will see. If an HTC Vive VR headset is attached to the Mac, then it can be used by a director can use it to look around the sphere. The editor can choose to look in a direction of their choice on the screen viewer, or have it mirror the view of the director using an attached headset. This means that the editor can be looking at or even making changes to a part of the sphere the director isn’t looking at while the video is playing.

Apple have announced that the HTC Vive will be the only VR headset that works directly with Final Cut Pro X. However it will be still possible to view work in Final Cut on an Oculus DK2 VR headset: using the 360VR Viewer application from Dashwood Cinema Solutions.

New controls associated with this viewer in Final Cut Pro X include a field of view slider and an orientation reset button:

New Final Cut means new Motion

The next version of Apple Motion 5 can animate in spherical video environments. Motion users will be able to animate 2D objects, text and particles in 3D space inside a sphere of video.

Effects in Motion are known as filters. Motion filters can now be applied to spheres of video. Instead of special plugins that take the distortion of equirectangular projection on a rectangle of video into account, Motion filters are spherical video compatible as well as flat video compatible.

No more manipulation of equirectangular media

The main difference in these new tools is that the next versions of Final Cut Pro X and Motion 5 are working in real spheres of video. That means that although the way the sphere of video is imported and exported can change, the post production tools can remain the same.

Footage acquisition is currently based on light passing through lenses onto rectangular sensors. In the case of spherical video acquistion, the footage these rectangular sensors capture is combined together using a lot of complex maths to recreate a sphere of video.

Videos are displayed onto flat rectangular surfaces. In the case of most spherical video, the source sphere is mapped onto rectangles in front of individual viewer’s eyes in headsets and on mobile phones.

As aquisition and distribution methods change, all Apple need to is to add new import and export mechanisms. The post production metaphors remain the same — just as they have done since the first computer-based video editing systems were created. Over the years footage acquisition changed from SD video and film to digital UHD and higher. In the same period, post tools have exported to many kinds of videotapes and file types. The metaphor has remained working with a grid of pixels arranged in a 2D rectangle.

Now alongside rectangles, Final Cut Pro X and Motion can also work with groups of pixels arranged in 3D spheres. As workflows change and distribution changes, that metaphor will remain the same

Setting the stage for VR Video version 2.0

This update is much more than I expected.

Earlier this year Apple hired VR video expert Tim Dashwood. When that happened his advanced set of Motion and Final Cut Pro X plugins were suddenly made available for free from his distributor FxFactory.

In June Apple announced that Final Cut Pro X would have new spherical video features by Christmas. In that time period I would have expected that the Video Apps team would have had time to reimplement Tim’s plugins to be faster and more efficient. The team have done a great deal more.

They’ve enabled thousands of Motion 5 designers to create spherical video content by making the update simple: Want to make VR video? Animate inside this sphere, use the tools you usually use. Do beautiful work that you can share with Final Cut editors. Now the millions of Final Cut Pro X editors can use the application they already know how to use to create works inside spheres of video.

Once again, Apple have have taken on and hidden technical complexity so creative people can get on with telling their stories.

Final Cut Pro X ($299.99) and Motion 5 ($49.99) are available from the Mac App Store.