Artificial Intelligence in Visual Effects
In my previous blog I explained how Artificial Intelligence has been affecting the music industry and how it has been used as a tool to improve the progress to develop songs, in this blog I will mainly talk on how Artificial Intelligence is transforming the VFX industry. VFX or Visual Effects are all the effects that movies or series use to implement CGI and extra effects in their videos. All the gunfire, aliens, monsters, sometimes even people are done with visual effects. Personally, I am really interested in how Artificial Intelligence is changing the industry, because I am really interested in working as a VFX artist in the future.
Artificial Intelligence in Content Creation
3D modelling is a very labour intensive process, and is even more so when you are trying to create something that looks and feels realistic. A clear famous example of Artificial Intelligence and machine learning in Visual Effects is none other than Thanos. The Marvel Avengers Villain has been created using artificial Intelligence that took advantage of facial-capture training data to transform josh brolin’s face into the lead character.
The use of machine and deep learning techniques in the creation of CG creatures and materials is still relatively new, but incredibly are promising. Pinscreen is an application that creates a whole 3D avatar or body based on one picture. This surprises me and interests me because I know how time consuming it is to create a whole 3D body. Pinscreen is also able to create a 3D model with a particular face expression that is found on a different photo, similarly like Thanos’s face expressions.
Ziva Dynamics, which offers physically-based simulation software called Ziva VFX, has been exploring machine learning, particularly in relation to its real-time solver technology. The machine learning algorithms enabled artists to interactively pose high-quality 3D characters in real-time. This is particularly interesting both from an interactive and artistic point of view, and the fact that you can animate more realistically and easily (Failes, 2019). I would really recommend to look at their demo video where they show the results.
Automation with Artificial Intelligence
Artificial Intelligence is not just creating 3D models and animation in the Visual Effects Industry. It is also helping with monotonous methods that every VFX artist has found labour-intensive. For all those VFX artists that hate rotoscoping (a method to separate something from the background in a video) because it is the most time consuming thing that I have learned in all my digital arts degree, you will be glad that plugins with Artificial Intelligence are being released that do the rotoscoping for you. Kognat, a company started by Rising Sun Pictures pipeline software developer Sam Hodge, has made its Rotobot deep learning rotoscope and compositing tool available for use with NUKE (a compositing and vfx program). The network analyzes the footage for the things that you want to rotoscope, for example all the people. Once the network knows which are the people in the footage it will rotoscope them in every frame. Arraiy, another software which specializes in rotoscoping, has trained its machine learning model by inputting vast amounts of specially-shot video footage, recreating in the process some of the thorniest problems in background extraction — like the most difficult thing to rotoscope, hair. The developers want Arraiy to do those hard rotoscoping jobs, and this is how they are gathering the data for that. They are putting a lot of complicated objects in front of their data capture, and running systems of equations that allow them to extract the exact ground truth (Anderson).
Personally I think that Artificial Intelligence in visual effects, is one of the areas in art and technology, where it has aided a lot. It is already being adapted by industry companies, and is being funded by a lot of great names in VFX, like Nvidia and Ziva Dynamics. Again we can see how Artificial intelligence is becoming a tool for a lot of artists both to create and develop and also to spend less time on particular methods. In my next final blog I will be talking about Artificial in the Fine Arts, mainly in Surrealism. I will be talking about the new movement that it is creating and how artists are adapting to use it to create great works. Stay tuned.
References
Anderson, M. (n.d.). A comprehensive guide to the state-of-art in how AI is transforming the visual effects (VFX) industry. Retrieved from https://rossdawson.com/futurist/implications-of-ai/comprehensive-guide-ai-artificial-intelligence-visual-effects-vfx/
Failes, I. (2019, May 11). The New Artificial Intelligence Frontier of VFX. Retrieved from https://www.vfxvoice.com/the-new-artificial-intelligence-frontier-of-vfx/
set AI for the VFX industry and whats it mean for you. (2019, January 7). Retrieved from https://onsetfacilities.com/on-set-ai-for-the-vfx-industry/
This Blog was created as part of an assignment for the final year of BFA (Hons) Digital Arts, 2020.
