PyTorch3D : RoboLove
Using Facebook’s 3D Deep Learning Library to shape a robot’s heart!
FAIR (Facebook AI Research) recently introduced a Python library which works with PyTorch to facilitate the manipulation and processing of 3D data, such as 3D models. Read more about it on their release blog here or watch the video below:
Given that it’s Valentines day and my experience working with 3D data, apart from some half hearted attempts at using Blender some years ago, is close to none, I thought it’d be interesting to take one of the tutorials and ‘Valentinise’ it. So I downloaded the following models from the Free3D website.
The objective of the project is to use PyTorch3D’s sphere to dolphin tutorial as a base and adapt it to convert a robot into a heart.
The first step is to load these up into the Jupyter Notebook. I used Google Colab for the free access to GPUs and easiness of configuration. Also, I could directly clone the tutorial. Loading and structuring the files is really straightforward, however, the robot loaded in a lying down position by default.
To fix this, after some fiddling around, I managed to use the library’s RotateAxisAngle class to get him into an upright position. I used the following code to rotate the object, which, in hindsight, is really obvious:
Having learnt to use it, I thought I’d show off! Below we’ve got our source mesh, which is the robot object.
The images used to build this GIF were generated using this function:
On the other hand, we have the heart object (our target mesh), which looks like this:
Using the tutorial’s ‘optimisation loop’ code plus a very basic adjustment of the loss and learning rate hyper-parameters, I obtained the following mutation:
And of course, the final mesh rotating!
I managed to turn the robot into something that resembles a heart with antennas, so I can say this introduction to PyTorch3D was probably successful. However, it was interesting to work with FAIR’s library and I’m looking forward to using it for higher impact projects!
Applied Data Science Partners is a London based consultancy that implements end-to-end data science solutions for businesses, delivering measurable value. If you’re looking to do more with your data, please get in touch via our website. Follow us on LinkedIn for more AI and data science stories!
