Montréal is a natural nexus between tech&the arts and… of VR. Part 2 of an interview with Marc Petit, 30 year veteran of VR

Ahead of the VR Salon on Oct 1–2, I wanted to get a larger perspective and sat down with Marc Petit, currently a partner and Entrepreneur In Residence at XPND Capital. One of the most interesting aspects of VR is entertainment and its reinvention of the visual grammar and vocabulary. Montréal’s place as a potential nexus between the traditional film&tv and the videogame storytelling grammars is exciting.

Marc has had a long career in the key building blocks of VR: co-CEO at Behaviour, SVP at Autodesk working on virtual post-production with James Cameron, VP at Softimage, vfx developer in the late 1980s. Marc’s 30 year experience in the 3D live action and animation space is unparalleled and we spoke of the challenges ahead for VR.

This is part 2 of the interview, part 1 can be found here.

Q: Content production and post-production is not without complexity in VR. You have a long background in that field, how do you see all of this play out ?

Marc Petit: There are definitely obstacles. In my mind there are two classes of content: live action VR and and CG VR. CG-based VR is an extension of gaming technology, the challenge is performance. Having a game run in VR at 120 fps is very different than running on a single HD screen. There’s currently a lot of development to raise the bar of the game engines by one or two notches, and Moore’s law is going to play in their favor by providing better hardware, but the problem remains the complexity of the content. You’ll need film quality assets at very high frame rates and the production budgets for VR experiences won’t be in the same range as vfx blockbusters. It creates a content pipeline problem, how do you reuse the film assets in a VR environment ? There are a lot of interesting tech developments, including Fabric Engine in which I’m an investor. There’s a big content pipeline there which is going to force the issue of the reuse of assets, the value of using film assets is that its forces virtual post-production. I love seeing the problem of performance and high quality assets finally being tackled outside of the game industry, and with new investment in new companies. This will be a fundamental shift for many industries, from design to film, to games themselves. VR is a powerful change agent and there’s a realization that solving these problems is going to yield value across many different markets. Within CG VR our friends at Epic Games created a very interesting position for themselves, and I think they could change the world. They have a very high quality engine and are able to move content across across multiple media.

Epic has the enabling technology to deliver high performance interactive content everywhere. I think they have an edge there on Unity, who have more distribution right now but from a tech perspective the UE4 engine is more scalable. Epic will definitely play an interesting role. In terms of CG VR, UE4 is going to play a central role, but there’s still a lot of technology to develop I’ll take one example: sound which is 70% of the immersive experience. The current sound quality we have is not good enough, it could break the immersion, there’s an interesting Montréal company working on sound, AudioKinetic. They have a fantastic opportunity to go into the immersive content space. Having seen the VR Smaug Experience from Epic and Weta at GDC where the animal is so close to you that you want to feel its breathing, the audio piece is key for the experience. There’s a lot of work to do to bring VFX quality animation to the virtual space including sound. You’re having to mix the same level of playback animation in games with film-level vfx. You want the creatures that you encounter in the virtual space to respond to your presence, a new animation technology has to be invented, which will be 80% playback and 20% interaction. This does not exist today, there is 100% interaction in games or 100% playback in visual effects. That’s one of the things we’re working on with Fabric Engine: how do you create an environment for mildly interactive content, where the creature will mostly acknowledge your presence and will be able to do something in reaction to it. I like the idea that it’s going to be a forcing function to get the two worlds of film and games to finally converge and work together, at all levels: graphics, rendering, animation. There was always a painful and big divide between film and game, when I was at Autodesk we were pioneers in virtual production and worked on James Cameron’s Avatar pipeline and trying to fit real time graphics into vfx production. I have a lot of personal scars in making this happen. When you finally see VR, you realize it’s going to solve many of these issues and as a result that the way to build media is going to be transformed forever, and for the better.

Q: What about live action VR ?

Marc Petit: Stereoscopy told us how to manage 2 overlapping fields of view and now with live-action 360 cameras we have 17 to 24 overlapping fields of view, it’s the same technology. The technology is there in terms of the pipeline, for example via The Foundry Nuke AR/VR. What kind of high quality preview do you make available on the set, how much of it is “in camera”, it’s all about efficiency. What is least figured out is sound capture and audio. The fact that the position of the viewer will change is quite new in the world of cinema, this has a lot of impact on editing but also on audio. The algorithms for stitching have first been developed for stereoscopy, we’re adapting them for VR, there’s more to compute but Moore’s law will help. Not everything should happen in post-production you would like as much as possible to happen in camera, what are the feedback mechanisms on the set ? That still needs to be figured out. Smoothing out the kinks will enable directors to learn the medium and do amazing things with it. What I like about this is that we’re not speaking about Panavision, we’re talking about GoPro. What’s very interesting in the 21st century is that consumer companies are leading innovation.

Q: Where does that leave Arri Alexa and the last wave of professional tools which upgraded to digital ?

Marc Petit: I think they live in different spaces. Previously things trickle down from the professional market down to the mass market it looks like it’s now the opposite: consumer to prosumer to professional. Already it’s possible to stick a few GoPros on a broomstick and produce 360 video. I’m not denying that there will be a market for professionals but there’s going to be a wide array of solutions available to hobbyists outside of Hollywood. This is how we are going to gain velocity because tools are widespread. If you’re not worried about the audio quality it’s perfectly possible today to do “crappy” 360 video. The question remains how fast can we democratize the high in tool so that they get in the hands of those who actually monetize the content. YouTube’s ad model will happen with 360 video, as far as the Netflix for VR, it’s still a bit open right now.

What still missing in terms of tools ? On the picture side it’s pretty much figured out thanks to stereoscopy. What’s not figured out is audio. Pro Tools falls on its face in VR, I think you will see a rush towards audio startups. Audio crowd tends to have a victim complex, this will be their day in the sunshine. This is where there’s room for lots of innovation. As an investor I love it when there’s a disruption in the market, because this is how you create value ! I’m very focused on audio in the 360 world.

Q: Montréal has a special role to play as a potential nexus between world-class film&tv talent, vfx/CG and the video games industry. Why do you think we can play such a special role ?

Marc Petit: All of this can be traced back to the Softimage days. Because we live in a francophone island, there’s always an overfunding of culture, in a place that is very favorable to technology and entrepreneurship. On a Canadian level, there is a lot of funding of culture and technology, for instance the role of the Canadian Media Fund is key. We have a lot going for us: Canada and Québec are good at funding culture and technology, we’re close to the US and a gateway to Europe, it’s a natural nexus between technology and the arts, it’s self-feeding.

We have Cirque du Soleil which is a form of immersive entertainment, Moment Factory is immersive as well. We’ve got games companies in Montréal, vfx tax credits which brought all the big vfx houses, tools companies like Autodesk and ToonBoom, Maxon, Unity now have big teams here. This also comes from university programs, who’ve done a good job in Montréal to stay close to industry needs.

Q: I want to go back to your first reference, which is Softimage, certainly well-known locally but others may have forgotten about it. It remains the biggest exit in Montréal.

Marc Petit: When Microsoft bought Softimage, it was the first time a consumer-focused tech company was going into the professional space. It was a visionary company and a very successful exit, in essence the fruits of this very special ecosystem we have in Montréal. Softimage really put Montréal on the map, it all started there, it’s part of the reason for Ubisoft coming here.

Q: My last question is logically, who’s the next Softimage ?

Marc Petit: The next Softimage ? Softimage was big and the next big one has been Unity because it commoditized CG. Next up is the commoditization of computer vision, augmented & virtual reality, the mix of video and CG. There’s an opportunity to create a commodity computer vision engine, that can run everywhere, cars, phones etc… both for entertainment and non-entertainment usage, that’s what we’re trying to do with Fabric Engine !

Show your support

Clapping shows how much you appreciated Frederic Guarino’s story.