Nerd For Tech
Published in

Nerd For Tech

Case Study— Object Detection Annotation in 3D Point Cloud

ByteBridge Object Detection in 3D Point Cloud

In the field of AI autonomous driving, accurate environment perception and positioning are the keys to reliable navigation, information decision-making, and safe driving in complex dynamic environments.

The two tasks require the acquisition and processing of highly accurate and informative data in real environments. To obtain such data, unmanned vehicles or mobile measuring vehicles are often equipped with multiple sensors, such as LiDAR or cameras.

Although LiDAR adopts the optical principle, it does not depend on light. It can operate 24 hours a day, which greatly enhances the adaptability of perception. The three-dimensional detection ability of targets increases the accuracy of detection, and it can capture both static and dynamic objects.

Here we will share an obstacle detection annotation project in 3D Point Cloud.

Annotation Guideline

Annotation types:

(1) Human:

Pedestrians: people riding scooters and balancing cars are labeled as pedestrians. The labels of people are divided into adults, children, and the elderly; The labels of occupation are divided into traffic police, delivery workers, couriers, and workers; Pedestrian status labels are divided into standing, walking, sitting, squatting, and lying.

Cyclists: motorcyclists, cyclists, and electric skateboarders.

Other people: non-real people, such as sculptures, dummies, and cartoon figures.

(2) Vehicle:

  1. Cars: SUVs, vans, pickup trucks, and common family passenger cars.

2. Buses.

3. Trucks: minivans and large trucks.

4. Engineering vehicles: sprinklers, engineering vehicles with mechanical arms (fire trucks, excavators, other engineering vehicles with arms), garbage trucks, tank trucks, trailers, other engineering vehicles, etc.

5. Motorcycles

6. Bikes

7. Electric scooters

8. Prams

9. Carts

10. Other vehicles

(3) Obstacle:

Conical obstacle, column obstacle, barrel obstacle, triangle warning sign, stone pier, construction warning sign, parking warning sign, road isolation pile, water safety barriers, and other obstacles.


Common animals, such as cats, dogs, chickens, etc.

(5) Related things and appurtenance:

Things related to pedestrians: suitcases, chairs, and trash cans.

Vehicle accessories: truck front doors, mini doors which are equivalent to small car, doors of a small car, truck container doors, express car doors, passenger car doors.

(6) The specification of annotation:

  • Children: the height is significantly shorter than that of adults; pre-school children without the ability to act autonomously; Children who ride scooters are labeled as children; According to the contrast with surrounding adults, when they cannot be distinguished, they should be labeled as adults.
  • Stooping older people with crutches is the labeling standard; if labelers are unable to tell, the person is labeled as an adult.
  • The judgment of the traffic police is based on whether they wear fluorescent clothing.
  • Obstacle subdivision includes cone obstacle, column obstacle, barrel obstacle, triangle warning sign, stone pier, construction warning sign, and no-parking sign.
  • When a pedestrian pushes a bicycle/electric bike/motorcycle, stroller, trolley, etc., the labeler needs to add a relationship between the person and the vehicle.
  • Need to add associations between pedestrians and their belongings, For example, people pushing trash cans, suitcases, chairs, etc.
  • The open door needs to be labeled separately. The open door is not labeled in the car body, and the relationship between the car body and the open door needs to be labeled.

Labeling Requirements

(1) The scope of annotation

All the objects that need to be labeled in the visible area are within 150 meters from the front view and 90 meters from other perspectives.

The objects are judged by the shape of the point cloud. If the number of point clouds is less than 3 points (excluding 3), there is no need to label.

For uncertain objects, do not imagine, if the object is still uncertain according to the frames before and after, then there is no need to annotate.

(2) Labeling requirements

  1. Overall requirements
  • Distinguishable objects need to be labeled separately, not as one object.
  • When the point cloud is complete and clear, the box should include all points of an object. When the point cloud is incomplete, the real size of the object needs to be completed through imagination.
  • Two-wheeled/three-wheeled vehicles, which are placed one by one and clearly visible, also should be labeled one by one. If they are connected together in the distance, label them as one object.

2. The requirements of special scenarios

  • Pedestrian holding umbrella: For a person holding an umbrella, workers need to label the umbrella and pedestrians together. For people holding umbrellas, label umbrellas and pedestrians together. If there are people outside the umbrella, label them separately.
  • The sitting positions of pedestrians: the height of the box should be determined according to the height of the actual object.
  • People get on and off the vehicle. When the person yet gets on the bicycle/electric scooter, with the feet on the ground, the bicycle/electric scooter and person are separately labeled as“human and bicycle” ;
  • People on the bicycle/electric scooter, with their feet not on the ground, are labeled as cyclists.
  • After the person gets off the bicycle/electric scooter, with the feet on the ground and separated from the bicycle/electric scooter, he/she is labeled as “human and bicycle”.
  • After people get off the car completely, with their feet on the ground and separated from the car (standing posture), people need to be labeled separately.
  • When people have part of the body in the car and thus are not completely separated from the car, people and the car should be together, labeled as “vehicle”.

3. 3D box

The crane boom should be labeled separately from the vehicle body. The main part is the vehicle body, and the rest part is labeled as the crane boom. Boxes must not overlap.

For bendable large vehicles, such as trailers and trucks, should be labeled separately, one for the trailer and one for the tow box. If they cannot be separated, label them as a whole.

4. Invalid data

The following circumstances can be regarded as invalid data and do not need to be labeled.

Point cloud problems (cluttered/scattered/jagged).

Parking lot/training ground/other non-normal road scenarios.

The number of pedestrians in the image is too large and too dense to be annotated.


Outsource your data labeling tasks to ByteBridge, you can get high-quality ML training datasets cheaper and faster!

  • Free Trial Without Credit Card: you can get your sample result in a fast turnaround, check the output, and give feedback directly to our project manager.
  • 100% Human Validated
  • Transparent & Standard Pricing: clear pricing is available(labor cost included)

Why not have a try?



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store



Data labeling outsourced service: get your ML training datasets cheaper and faster!—