Point Cloud Data Using Deep Learning

Raju Gudhe
Sep 3, 2018 · 3 min read

This is a follow up to an article I wrote, Point Cloud Data Introduction, after this highly effective introduction for the point clouds, I continued to learning on how to classify point clouds using Deep Learning.

This article focus on one of the pioneer work by the researchers of Stanford University, who introduced a novel architecture to process raw point cloud data.

In this article I tried to explain “PointNet” architecture, which is the stated to be the first deep neural network to act directly on the raw point cloud data for the classification and segmentation tasks.

Introduction:

Two dimensional Deep Learning has been vastly researched in the past few years, it achieved excellent progress in the classification, object detection tasks. With the development of three dimensional equipments, three dimensional deep learning has received great attention.

Typically, deep neural networks acts on the regular input formats such a image grids where as point clouds are not in regular format and cannot be directly feed into a neural network. Most of the researchers try to transform point clouds into 3D grids by few methods like voxelization and input the transformed 3D grids for a neural network. Transforming point clouds into 3D grids will results in poor performance as the number of parameters of the neural network is exponentially high. With the advent of the latest GPU’s it is always recommended to reduce the number of parameters to train the model effectively. Therefore we need a novel architecture that can act directly on 3D irregular data formats, It’s not a simple task to design such a network bearing the challenges associated with point cloud data.

Challenges:

I subdivided the challenges into two main categories: Neural network challenges and Data associated challenges.

Neural Network Challenges:

  • Unordered: The point cloud is a set of points with no specific order. For instance if a point cloud is Nx3 matrix, where N is the number of points, the network should feed N! permutations in an order to be invariant.
  • Global and Local features: A point in space will have a correlation with other points, by measuring the distance between we can capture the local and global relations.
  • Invariance to permutations: Geometrically, the oder of the point clouds set does not matter, however it does matter while representing them in a matrix structure.
Permutation Invariance

Data Associated Challenges:

  • Missing data: The scanned models are usually occluded and parts of the data are missing.
  • Noise: All sensors are noisy. There are a few types of noise which include point perturbations and outliers. It means that a point has some probability to be within a sphere of a certain radius around the place it was sampled (perturbations) or it may appear in a random position in space (outliers).
  • Rotation Invariance: Rotating and translating the points should not effect the classification.

Point Net Architecture:

Point Net is a novel architecture address above challenges while feeding raw point cloud data as input to a neural network. The main contributions of Point Net are:

  • A novel architecture for unordered input format.
  • Efficient to train a model for classification and segmentation tasks.
Point Net Architecture

The above figure illustrate the network architecture, as it is shown it address both classification and segmentation tasks and both share greater portion in the network. This article is highly related to classification task, the classification network takes n points as input, apply input and feature transformation and finally maxpooling to aggregate point features. Let’s dive into the network more closer to address the challenges while considering point clouds directly into a neural network.

Firstly I will address the neural network challenges associated with order. Point cloud data is a set of 3D points

https://gist.github.com/graju1401/1783dc7e5e2f4d4c9d6838f56ee5d6dc

Raju Gudhe

Written by

Deep learning and Sensor data fusion enthusiast

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade