AI-assisted art in CGI
51–477 S20
Introduction
The field of Computer Generated Imagery (CGI) covers many disciplines, including film production, video game development, and visual art creation. Although different software and techniques are used across these areas, there is a common trend between them of increasing standards as time goes on. This means more realism and interactivity in every level of production. 3D models must have higher detail, textures must have higher resolution, physics simulations must be faster and more accurate, worlds must be more expansive and unique. The cumulative effect is that modelers and animators cannot keep up with the demand by relying on standard software and workflows. The integration of Artificial Intelligence and Machine Learning into CGI has allowed artists and engineers to assign more types of lower-level work to software and focus more on high-level, creative aspects. These low-level tasks can include generating props and environments, or animating physics- or player-controlled objects. In this collection, I will show examples of how AI is being used today in different areas of CGI. This shift toward AI has been spearheaded by the video game industry the most, due to the exponentially increasing cost of triple-A game production. As a result, most of the examples are focused on video game creation, but there is significant overlap into 3D rendering and animation for other media.
Example-Based Workflow, Unity
The Unity game creation engine has recently announced a suite of AI and ML based tools to assist game creators. These tools rely on examples supplied by the developer, which the system learns rules from in order to create new instances of the required object. These objects might be props or textures, which the game needs a large set of unique examples. This allows the developer to create huge amounts of realistic content with ease, and save considerable time and effort.
“AI-assisted artistry solves the hard problem of producing and scaling high-quality content, giving 3D artists more time to focus on creativity.” https://blogs.unity3d.com/2020/03/10/artomatix-joins-unity-bringing-ai-assisted-artistry-to-material-creation-workflows/
Procedural Level Design, Independent Developers
These are three examples of how independent game creators are using procedural generation to generate environments for their games. The island creator by Oskar Stalberg uses a technique he calls “wave function collapse” to settle on a final design based on a set of premade assets which stitch together based on a random seed. Developer Josh Ge uses procedural generation to create the barebones network that his 2D game’s layout will follow. Lastly, 3D artist Simon Verstraete’s building generator can create endless unique structures which can fill an urban scene. Developers use tools like these to make their games replayable, since each experience will be in a unique setting, while still following rules to preserve gameplay.
Environmental Test, Embark Studios
This is another example of a procedurally generated environment, but uses a different method than the previous ones. Instead of using pre-made assets or an explicit rule, the developers here use a technique called photogrammetry to teach their program how to generate new spaces. Photogrammetry involves taking a large number of photos, which the software uses to gradually piece together what rules it needs to follow to generate similar examples. This includes the specific objects within a scene, such as rocks and plants, as well as the large-scale structure of the terrain.
Quadruped Motion Test, University of Edinburgh
Animating character movement to be realistic and fluid is a challenging area of work on its own. When the character is four-legged, the issues compound even further. There are multiple modes of movement which can be switched between at any time, and it’s difficult to record and model them separately. This paper shows a method which feeds unstructured motion capture data to an AI which learns the different types of movement and teaches them to a model creature. The result is realistic, real time quadruped movement which can be controlled by a user. Applications may include film, games, and robotics.
La Forge Project, Ubisoft
La Forge is a division of game creator Ubisoft focused on finding ways to use AI and ML in game creation. One project I found to be of note is their soft body physics engine. Materials like cloth and rubber can be simulated on the fly using live capture data which has been analyzed by an AI. This allows for simulations to be more efficient, in terms of both time and memory space.
