Facebook Open-Sources PyTorch3D CV Library
Although 3D computer vision is an increasingly important part of the computer vision research field, studies on 3D in deep learning have been limited by a lack of available tools and resources that can deal with the complexities involved in applying rich 3D data on neural networks. In a bid to simplify 3D deep learning and improve processing performance and efficiency, Facebook recently introduced an open-source framework for 3D computer vision. PyTorch3D is an efficient and reusable 3D computer vision library based on PyTorch that outperforms existing tools in many aspects of 3D modeling, rendering and other processing operations.
The Facebook researchers also propose a new 3D data structure, Meshes, for batching heterogeneous meshes in deep learning applications with better efficiency and flexibility. Meshes enable researchers to store and transform triangle meshes into different representation views easily and quickly.
The project includes an efficient new modular differential renderer for converting 3D scene properties into the pixel information of a 2D image. Traditional rendering methods are generally non-differentiable and so cannot be combined with deep learning. To address this issue, researchers developed an efficient, modular and differentiable renderer that is easy to customize, enabling users for example to change light and shadow effects during rendering. Like other PyTorch3D operators, the renderer can also handle batches of heterogeneous data.
The researchers performed optimization work to support heterogeneous batches of data inputs for 3D data processing. Users can import operators directly in PyTorch3D to speed up their experiment processes without re-creating or re-implementing the operators.
The Facebook team has produced four tutorials to help users to get started with PyTorch3D:
Facebook says it hopes PyTorch3D can promote the integrated development of deep learning and 3D. In their future work, the team intends to expand the operators provided in PyTorch3D, and has invited the broader computer vision and deep learning research communities to contribute to this process.
For more information about PyTorch3D, please check out the project page.
Author: Herin Zhao | Editor: Michael Sarazen
Thinking of contributing to Synced Review? Synced’s new column Share My Research welcomes scholars to share their own research breakthroughs with global AI enthusiasts.
We know you don’t want to miss any story. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.
Need a comprehensive review of the past, present and future of modern AI research development? Trends of AI Technology Development Report is out!
2018 Fortune Global 500 Public Company AI Adaptivity Report is out!
Purchase a Kindle-formatted report on Amazon.
Apply for Insight Partner Program to get a complimentary full PDF report.
