PIFuHD — FAIR Neural Network Restores a 3D Human Model
PIFuHD is a neural network architecture for reconstructing a 3D human model from a 2D image. The approach bypasses existing models for the realism of generated 3D models. The model was developed by researchers from Facebook AI Research. PIFuHD is based on the Pixel-Aligned Implicit Function (PIFu) method and a hierarchical layered neural network. The neural network takes into account the global and local contexts of the image, which allows achieving high accuracy of the final 3D model. PIFuHD captures details such as fingers, facial features, and wrinkles in the person’s clothing. Previous approaches were not capable of such detail.
What is the problem
Due to limitations in the memory of current electronic devices, past approaches more often took a compressed image as input. However, they produced less accurate predictions or predictions at low resolution. Researchers work around this limitation by using a two-tier architecture in PIFuHD. The model takes into account the global and local contexts.
The approach architecture
The neural network accepts an image of a person with a resolution of 1024 × 1024 as input. At the output, the approach gives a 3D model of the person. The architecture of the method consists of two levels of PIFu modules:
- A basic level that focuses on extracting global features from an image. This module is similar to PIFu;
- A refinement level that focuses on extracting local context information and adding precise detail to the 3D model