Apollo 5.0 Technical Deep Dive
This is part two of the two-part technical blogs that summarize the major technical upgrades in Perception, Prediction and Planning modules in Apollo 5.0. Part one introduces the newly released developer tools and service offerings, read it here.
In Part One of the Apollo 5.0 technical blog, we talked about all the newly released features such as Smart Data Recorder, Apollo Data Pipeline, Large-scale Cloud Processing, and brand new service offerings like Vehicle Control Calibration Service, Control-in-the-Loop Simulation, Scenario Editor in Dreamland (Apollo cloud-based simulation platform) and the largest ever released Apollo Synthetic Dataset.
Besides all these exciting new service offerings and tools, the Apollo Team also implemented major changes in Perception, Prediction and Planning modules. Let’s dig in!
Perception
In Apollo 5.0, the Apollo Perception module incorporates the capability of using multiple cameras, radars (front and rear) and LiDARs to detect obstacles and fuse their individual tracks to obtain a final trajectory. Utilizing ego-motion of the self-driving vehicle (SDV), this sub-module estimates the trace of obstacles including a closest-in-path obstacle (CIPO). For lane line, we constructed an end-to-end lane line detector bypassing complicated postprocessing efforts. The detector can detect each instance of lane lines (e.g. L0, L1, L2, R0, R1, R2, …), classify multiple lane types (white/yellow, broken/solid, and curb), and recognize fork and split lane lines directly from a deep neural network. In addition, the lane line detector was co-trained to detect the vanishing point of each frame, which supports pose estimation and online calibration of cameras in SDV.
In addition, while deep networks of Apollo perception used to be deployed only using Caffe for inferences, now Apollo also supports both Caffe and PaddlePaddle, an open-source platform developed by Baidu to support its various deep learning models including point cloud based obstacle recognition, image-based obstacle recognition, and image-based lane line recognition. Furthermore, the Apollo Team developed an offline camera calibration tool that the community can easily calibrate the orientation of cameras with a push of a button. Last but not the least, we launched the Apollo Sensor Calibration Service, a versatile one-stop solution to calibrate your sensor extrinsics in a smoother, more efficient way. Send us an email at apollopartner@baidu.com to learn more about the service.
Prediction
The Prediction Module now will do model-building to different types of obstacles, including motor vehicles, non-motor vehicles, pedestrians, and etc. The obstacles are not classified into categories to make trajectory planning simpler and better managed. In terms of modeling, road conditions are also being recorded and analyzed. Semantic maps are generated to add road features to the deep learning network, aiding in the creation of models, such as behavior prediction, path prediction, intersection turning direction prediction, etc. At the same time, these models are optimized based on priority, and a more elaborate model will be used to deal with the obstacles of higher priority. Once such hierarchical architecture has passed the scene analysis, intent understanding and task prioritization, and through the model scheduling framework, the most appropriate model will be selected automatically to pursue the best prediction results possible.
Planning
In Apollo 3.5, we introduced the concept of scenario, which was well received and embraced by our developer community. In Apollo 5.0, we have implemented major upgrades to the four fundamental algorithms in the planning module.
The foundation of our planning module is to convert the world coordinate system into a reference-line coordinate system to optimize the trajectory problems. Therefore, the smoothness of the reference-line is crucial. In Apollo 5.0, a new discrete-point, optimization-based reference-line smoothing algorithm uses a quadratic programming optimization algorithm to replace the previous curve fitting, which is 7 times faster and achieves a more consistent smoothing effect. Furthermore, the new algorithm ensured the probability of crossing the boundary was reduced by 95%.
The new path bound decider and the optimizer are highly discretized in longitudinal and lateral directions to obtain the maximum solution space. It uses a very intuitive path bound decider to reformulate the constraints of the optimization problem. Therefore, the path is guaranteed to be smooth by this optimization process. This algorithm is highly versatile and can be used to solve the path planning problem for many different driving scenarios.
With a smooth path, we reuse the existing ST frame for speed planning, avoid obstacles to ensure absolute safety, and use a new optimization algorithm to optimize the distribution of speed, acceleration, and jerk over time, to ensure the best riding experience for passengers on board. In the Open Space Planner, we chose the model-based optimization scheme, re-presenting the obstacle shape and road boundary, and fully utilize the vehicle dynamics model. It is worth mentioning that we have modified the optimization solver IPOPT to improve the efficiency and robustness of the solver. We contributed this change to IPOPT open source community, which was very well received by their members. We are very glad this has become a successful collaboration between two open source communities.
All the above amazing upgrades are live and ready for you to explore today on our GitHub page. We hope you like this milestone release, and let’s keep working together to make a safer, more autonomous world together!
