5 Massive Changes That Have Come to Autonomy
Transcript
LUKE RENNER: This is advanced autonomy. I’m Luke Renner. My guest today is a colleague of mine here at Cyngn, Biao Ma. Biao is the VP of engineering and the head of autonomy. He spent his entire career in this space, bringing engineering expertise to the entire technology stack. This includes perception, planning and control, mapping and localization, and simulation. He was a software architect for Baidu’s autonomous driving division and has a master’s degree in software engineering from Carnegie Mellon University.
In this conversation, we’ll be talking about the five significant changescambios that have come to the autonomous vehicle sectoresfera in the last five years. Biao defines /significant/ as changes that either (a) didn’t exist five yearsaños ago or (b) have improved by many many multiples.
Hi Biao, welcome.
BIAO MA: Hey, Luke.
LUKE RENNER: You are here to walk us through the changes that have come to the self-driving space in recent years. So to kick us off, I wonder if you could just tell us what those changes are.
BIAO MA: Autonomy is a super-set of hardware, software platforms, tools, and a lot of others. So, first of all, I think the biggest significant advancement is the new computational methods in the autonomy space across the key subsystems.
Number two is the sensors. Take LIDAR, for example, the distancela distancia resolution and SNR (the signal-noise ratio) has increased at least two-to-three times.
The third is about the compute GPU. For example, comparing to the 97, 5 years ago, now the 3090 is 3x faster and, I remember, 3x bigger of the VRAM.
Also, five years ago, we didn’t have openabiertos platforms, open-source projects. open tools like Pytorch or Tensorflow for developers to develop. [Also, we have] way bigger and more data sets [of a] bigger size and better quality.
Last but not least is new communication technologytecnología such as 5G that has significantly changed the game. It will change the locality of how the architecturela arquitectura could be designed.
LUKE RENNER: So, we have new computational methods, more sensors, advances in computing power, the open ecosystem, and 5G. Is that right?
BIAO MA: Yep, yep.
LUKE RENNER: Okay. Alright so let’s dive into those. We’ll start with the computational methods. You know you can’t really talk about new advances in computational methods without talking about artificial intelligence so how are AI-based learning methods being used in different aspects of autonomy?
BIAO MA: So learning-based methods have significantly drawn attention and developed a lot in these five years. If you look at the autonomy stack, perception is the starting point. Being able to perceive objects, classify them, track, and predict objects is the place that really learning-based messages shine. At the end of the day, having learning-based methods being developed and optimized in these several years, we see significant change. It’s really an open door that we several years ago we didn’t have.
LUKE RENNER: Understood. So tell me a little bit about some of the new algorithms that have been developed?
BIAO MA: There are many algorithms developed in autonomy across the stack. For example, the architecture for a perception stack or specifically the object detection classification tracking stack. There we see generations of models. If you compare these state of art models today to not even five years ago or three years ago, there’s a significant improvement of accuracy, latency, and the efficient utilization of the compute unit — and we’re going to cover how the computer also getting better.
LUKE RENNER: One of the things that I know that has changed a lot in the last few years is the idea that the data sets for AI models have really changed. Can you talk about some of the transformations that have happened there?
BIAO MA: More than five years ago — well it’s about five to ten years ago —there’s one of the popular data set called the Kitti Dataset. If I remember correctly, the Kitty Dataset had about 7000 frames. Each of the frames was labeled with what object it is. It’s a combination of camera and LIDAR and they had a different variance.
Now, today in 2021, first of all, there is a big variety of open data sets. But take one of the popular datasets and you have about five-to-seven hundred thousand frames. So, you see a significant difference in terms of size. It’s bigger by several hundred times and also the quality is better.
If we know more about the perception stack, it’s not only about the label itself but also the registration or the calibration of different sensor combinations of LIDAR, RADAR. The quality also matters. We also see a significant change, not a small improvement, but uh is aligned way better.
To put it in simple terms, right? These datasets are registered, calibrated, and synchronized way better, right? And us an advanced way to synchronize the sensors so it’s a way bigger set of the data and much higher quality.
LUKE RENNER: I know that one of the promises of bringing artificial intelligence to the autonomous vehicle space is particularly with dealing with corner cases or the long-tail scenarios that a vehicle may have to deal with while driving autonomously. So, can you tell me a little bit about what are the challenges of corner cases and what are people doing to solve those problems?
Corner case challenges will be with us for a while. If we don’t do computation in an effective and scalable way, solving long-tail problems is extremely hard. That’s why methods and platforms are being built. Some are proprietary and some are open and do, at least three things in a very effective way.
One is to capture and create scenarios. So scenarios are really one key to the development of autonomous driving. Number two is a lot of them require automatic grading because, again, if you need a human to look at the scenario that manual effort will be bounded by human time, and the third is the big pipeline to trigger the computation in an effective way that comes with not only the data pipe but also the analytics, and effectively store the data that matter to us.
LUKE RENNER: So you mentioned sensors, I want to talk a little bit more about that. Can tell us what the major changes that have come to sensors in the last five years are?
BIAO MA: Let me give you one specific example: LIDAR. There’s been significant improvement in the number of how many beams of the LIDAR unit. Five years ago, there were about 16 lines — or up to 64 lines. As of today, there are more than 300 lines in the market. So you can see it’s several times more, right? And that is just one aspect.
The second is the distance. In the 16 line example, we can effectively do perception using that for about 25 to 30 meters. As of today, a lot of the popular units support a perception stack that goes beyond 100 meters so you can see it’s actually multiple-X improvement.
The third one is really about the SNR, the signal-to-noise ratio. It’s not only about how much you get but also it’s important how much noise you get, right? So based on the experiment that the data we have, there’s a significant improvement, almost cut half and actually more than that in terms of reduced noise of the signal. So that really changes the nature of the work of perception in LIDAR. Radar, cameras, and other sensors are enjoying the same significant advancement in the last five years but I’m just using LIDAR as one example.
LUKE RENNER: Yeah, yeah, I understand so in the last five years GPU computational power has increased threefold and the video memory size has also significantly increased. What would you say has been the impact for developers in autonomous driving?
BIAO MA: If you look at a learning-based model, its usefulness is bounded by how fast the model can go, right? Typically, we expect a learning-based model to go beyond 10 or 20 hertz to make it usable, right? If a model is running at two hertz or five hertz, it is too slow, and that is not only how fast they go. Think about the doors this increase in speed opens for different structures of the learning-based model. So now, with the latest generation of perception, models are really focusing more on using the latest generation GPU to make it efficient and fast.
So that is one side, the other aspect is we typically say the model is too big in the sense that it cannot fit in the GPU. We’re talking about the VRAM here. The ram's size really changes the size of the model you can fit into the GPU. If you cannot fit the model onto the chip, it’s not usable, right? So the fact that it’s way faster and much bigger really changes the variety and the richness of the learning-based model you can pick. And, of course, there are other details.
The latest generations, for example, have way more Cuda cores. There are also Tensor cores that we can leverage that are already optimized with platforms like Tensorflow or Pytorch. We use the software plus hardware optimization for that so this is the key difference. Before, things were either too slow or we just couldn’t fit the model in the GPU. Now everything has changed.
LUKE RENNER: I know that the rise of the open-source ecosystem has really made it easier for engineers to work cross-functionally. You know, you mentioned Tensorflow as one example. I’m wondering if you could talk about that for a little bit and tell us how open source projects have really changed the AV landscape.
Biao Ma: In the last five years, there is a list of open-source projects, big and small, coming out that have really given us, number one, the opportunity to learn from each other because we’re exposed to different angles and different way of implementations. We can see beyond the architectural decisions that they are looking at and/or we can infer the challenges that they’re having.
The second point is these are great tools and a great method for young engineers looking into autonomy as a starting point for their careers.
LUKE RENNER: Bringing us to the end of our five things — the last thing you mentioned that’s really transformed in the last few years is 5G. Now, I know a lot of people really assume that 5G is going to play a big role in this space and have a big impact on how we think about autonomous vehicles and their ecosystems but I don’t think people really understand what that change is going to look like, right? Like we know it’s going to be faster but maybe that is sort of the end of people’s understanding so I’m wondering if you could really explain it to us. How is 5G that much different than 4G and how’s it going to be that much different for AV development?
BIAO MA: 5G changes where things happen. Technically, this is called locality. It’s in the educational view of complicated architectures like autonomous systems. So, the reason it is so hard to make edge devices 2X or 10X more powerful is bounded by power itself. Think about an edge device in a car. There’s a limit both in terms of power and but also there’s a physical limit of how big and how power-hungry the device can be. So having the opportunity to put certain key components in a different place that is not bounded by the physical dimension and not bounded by power really opens doors for next steps.
Let’s look at two key numbers related to 5G to help us understand. Number one is the significant reduction of the latency. 4G is typically 50, 100, or even more milliseconds in terms of latency, which is basically how do I get something delivered to your hand. By comparison, 5G is 1 or 2 milliseconds or even less so it’s significantly faster. Again, that is not 10% faster that is 10X, many-X faster, right? So, this has really changed where things happen. And that is not the only change.
The second change is throughput. Throughput is not only how fast I can get data to you but also how big a load I’m getting to you. So in that sense, 5G is getting beyond a gigabit per second and to the scale of a gigabyte. So in that one second, a significant load of data is getting to your side.
By putting the latency and the throughput together, we’re about to see a significant change in where things could happen and where it is more optimized. One example is maybe part of the computation of autonomy could be from the light pole because we don’t need to worry about whether it is too late to deliver that information to the vehicle because of our bandwidth limitations.
LUKE RENNER: OK, 5G is going to make things so fast that, like, some of these AV computations, some of this decision-making and processing — doesn’t necessarily need to live right in the car. Alright, well, all of this is really fascinating. I really appreciate you walking us through all of that.
We’re a bit out of time here so I thought we would end with a question I’ve been asking everyone so far. This is our second podcast so two people. My question for you is what are you excited about in 2021? What are the breakthroughs that you predict might come to pass this year?
Oh, this is a great question. Uh, I feel very lucky to be able to study and spend my career in this amazing industry and this is just a starting point. I see autonomy from today towards um a full extent is we still have a long way to go. A 5-or-10-X improvement will continue happening.
LUKE RENNER: Yeah, absolutely. So is there anything else you want to tell us?
BIAO MA: So we are actually hiring across the stack, okay? If you are interested in an industry that doesn’t show 3-or-5-percent improvement, rather each area has several-X improvement, this is really a good time to look into this space and join us.
LUKE RENNER: So, if someone’s interested in working for us where should they go?
BIAO MA: Cyngn.com/careers.
LUKE RENNER: Alright, sounds good. Well, I really want to thank you for your time. It’s been super interesting. I look forward to speaking with you again. Talk soon.
BIAO MA: Alright, see you next time.
We are pleased to announce the launch of our new podcast, Advanced Autonomy. Follow along for fascinating discussions on the rise of self-driving vehicles, including interviews with industry leaders, technical conversations with engineers, and insider stories.
