Apollo 3.0: Entering the New Era of Autonomous Driving
On July 4th, 2018, Baidu unveiled the latest version of its autonomous driving platform, Apollo 3.0 in Beijing, China at its annual developer conference, Baidu AI Create. Apollo 3.0 introduced a production level solution for the low-cost, closed venue driving scenario that is used as the foundation for three upcoming products:
- Apolong, China’s first L4 self-driving minibus powered by Apollo platforn, has entered volume production stage with King Long (the second largest commercial bus maker in China). Together with the audiences, Baidu CEO Robin live-streamed the 100th bus rolling off the production line at the conference.
- Pand Auto, a car-hailing platform for new energy vehicles, provides vehicles and an operation platform for Baidu Apollo autonomous valet parking feature.
- Neolix, a Chinese startup company that works on the last mile delivery robots, will deploy their Apollo-powered autonomous delivery robots in China to provide last-mile delivering services to merchants and customers.
Besides the production level solution, Apollo 3.0 is packed with new features and some big changes to its software architecture. Before we dive into the new features of Apollo 3.0, let’s take a look at the architecture differences between Apollo 2.5 and Apollo 3.0:
And this is the architecture for Apollo 3.0:
As you can see, there are many new technologies were introduced in Apollo 3.0, and let’s dive in the software layer first.
Introducing the new module: Guardian
In Apollo 3.0, safety was one of our primary focuses, and hence we launched a new module called Guardian. Like its name, Guardian which means Protector, ensures the safety of the vehicle by carefully bring the vehicle to a stop during any emergency situations.
As you can see from the diagram above, Guardian works closely with the Monitor. Monitor is a surveillance system. It is very similar to the health monitor in the aviation industry. It receives signals from various modules for any fatal errors that might appear from both the hardware and software layers. If Monitor detects any software malfunction, such as the planning module’s publishing frequency drastically changing or no frequency being published for a sustained period time, it sends in an alert to Guardian as well as an audio and visual warning alert to the human driver to take over. In cases like these, Guardian has the ability to bypass the Control module and send a signal directly to CanBus, and gently bring the vehicle to a safe halt (gradual braking). Besides performing a soft halt, Guardian module also execute hard brakes under two scenarios:
- Ultrasonic sensor malfunction: Unlike other software module, Guardian constantly checks with ultrasonic sensor since it acts as the final gatekeeper. If the ultrasonic sensor does not respond or malfunction, irrespective of the car’s state of motion, Guardian issues a hard brake signal to CANBus to avoid a potential crash.
- Human driver failed to take over the vehicle: when Monitor detects any potential danger, besides requests Guardian to do a soft stop, it also works with HMI to issue a visual and audio warning to the human driver. If the human driver fails to intervene for a short period of time (e.g. 10 seconds) and the car failed to complete stop, Guardian would command a hard brake to prevent any further risk.
Major module update: Perception
In Apollo 3.0, Perception module introduced a few major features to provide more diverse functionalities and a more reliable, robust perception in AV performance.
- CIPV (Closest In-Path Vehicle) detection: this new feature enables the ego-car to detect the vehicle in front it and estimates its trajectory, thus it leads to a more efficient tailgating and lane keeping when the lane detection feature is unreliable.
- Asynchronous sensor fusion: unlike the previous version, Perception in Apollo 3.0 is capable of consolidate all the information and data points by asynchronously fusing LiDAR, radar and camera data. Such conditions allow for more comprehensive and various dynamic modes and reflect more practical sensor environments.
- Online pose estimation: This new feature can estimate the pose of an ego-car for every single frame, which helps to drive through bumps and slopes with more accurate 3D scene understanding.
- Ultrasonic sensors: Perception in Apollo 3.0 now works with ultrasonic sensors. The output can be used for Automated Emergency Brake (AEB), and vertical/perpendicular parking.
- Whole lane line detection: unlike the previous lane line segments, this whole lane line detection feature will provide more accurate and long range detetction of the lane line. The benefit of this inclusion, is to make lane-keeping much safer and provides more accurate prediction.
Next, let us disect the Hardware Development Platform updates:
In the previous versions, we called this layer the referencing hardware layer. In Apollo 3.0, we introduced the Hardware Development Platform. The platform now contains a brand-new hardware called Apollo Sensor Unit (ASU) and we added 15 new hardware devices which can be integrated with Apollo using ASU. These devices were tested and categorized into two different groups: Apollo Platform Supported devices and Apollo Hardware Development Platform Supported devices.
What’s the Apollo Sensor Unit (ASU)?
In previous versions, the hardware devices are centered around the Industrial PC (IPC). The number of sensors and hardware that can be integrated with Neousys IPC is limited which then restricts the opportunity of integrating additional types of sensors with Apollo’s platform. With the intention to remove the limitations of the IPC, we created the Apollo Sensor Unit (ASU) .
ASU is specifically designed to increase the flexibility of autonomous driving technology, and we built it with many automotive graded connectors. ASU will now serve as the new communication center of the hardware architecture, and developers can plug-in any hardware devices (sensors, PCs) to the ASU to integrate with Apollo.
There are 4 major features that we are particularly excited about:
- ASU provides up to 5 connections to FPD-link III cameras. This allows the 1080p resolution, external/software triggering mode for image capturing, and OTA (over-the-air) firmware updates.
- The ASU also provides a 4-channel CAN-interface. It works as a CAN-card to provide connections to Radar, Ultrasonic sensors and the vehicle’s CANBus.
- ASU replicates synchronization signals from the GPS receiver and transmits it to sensors that take in GPS signals, and enables them to synchronize for data fusion later on (see diagram below).
- The ASU has a Xilinx FPGA chip built-in, which provides computation power for data pre-processing. This allows us to standardize the format of incoming data to the computing unit (like an IPC).
Can’t wait to get your hands on one of the ASUs? Due to limited quantity, we currently only offer engineering samples to a small numbers of partners and developers. If you are interested, please send us an email at Apollo_hw@baidu.com, and our hardware team will get in touch with you.
New Hardware Categories: Apollo Platform Supported devices vs. Apollo Hardware Development Platform Supported devices
Starting with Apollo 3.0, all hardware devices will be tested and categorized into two groups:
Group #1 is called Apollo Platform Supported:
The hardware devices from this category have been tested and certified to be equivalent to the hardware devices listed in the reference design. Our developers can choose any of the above hardware devices and expect the same level of performance as the current model of Apollo being driven and tested by our internal teams. The devices from this category have been tested to work with the entire Apollo software stack.
Group #2 is the Apollo Hardware Development Platform Supported device:
Since the launch of Apollo 1.0, one of the highest requests that we received from our developers and partners around the world is for us to include more low-cost sensor solutions. After months of thorough research on the available products on the market, we are pleased to introduce 15 new hardware devices, and we call them the Apollo Hardware Development Platform Supported devices. This group contains many hardware devices that are cost-effective and were also tested to work with the rest of the reference hardware modules for data collection. Please note, the hardware devices from this group can only be used to collect data and cannot replace their counterparts in the reference design directly. If you want to feed the collected data to the Perception or other sofware modules, additional work is required (e.g. creating new models, annotation of the data, training the new models, and etc.). To view the complete list of the newly added devices, please visit our website here. If you want your hardware sensor to be certified by the Apollo platform, please email our hardware team at Apollo_hw@baidu.com as we are always welcome new hardware partners.
Now we are moving to the last layer of the architecture: the vehicle.
Introducing the Open Vehicle Certificate Platform
Another major announcement we had during the release of Apollo 3.0 is that we introduced Open Vehicle Certificate Platform. Why did we make the change? We want to provide a reference for the industry and the developer community with a simplified process of choosing and adding a development vehicle for their autonomous driving projects. Now you have access to a check-list where you can pre-evaluate whether the vehicle will be compatible with Apollo’s software stack. We believe by opening up the Apollo Drive-By-Wire Requirement list we can save a tremendous amount of time and effort for our developers and partners to find a suitable new drive-by-wire (DBW) vehicle. More importantly, it also makes the Apollo code’s porting process easier than ever, across different vehicles. Below is a simplified version of the list. For the complete list, please view here.
Now you have an overview of all the new features and functionalities of Apollo 3.0. But to truly experience everything, fork our code on Github today and dive in!
Access the new Apollo 3.0 code on our Github page now!
And stay connected: follow us on Twitter @ApolloPlatform and sign up for our newsletter below for future offline events!
A special thanks to Qi Luo, Yang Han, and Natasha D’Souza from the Apollo Team who helped with technical content of this blog post!
