DeepMap’s Co-Founder and CTO, Mark Wheeler, recently spoke with industry analyst Grayson Brulte for SAE Tomorrow Today, a podcast from SAE International. Today we are posting an excerpt of the wide-ranging conversation, which covered HD mapping, localization, safety, and more.
“A map can — in essence — help the vehicle look around corners and look ten seconds into the future.”
“Localization is a fundamental capability of self-driving. You need to know where you are in the world in order to go between two places.”
“What Tesla has done is incredible. I really like what they’ve done. But we do have a difference of opinion on where HD maps and LiDAR fit in into self-driving.”
Grayson Brulte: Mark, why do you believe that high-definition (HD) maps are the missing piece of the puzzle to solve full self-driving?
Mark Wheeler: All the main players in L4 driving rely on HD maps. And there’s a reason. Fundamentally the cars are moving around in real time, having to understand things and make decisions in a very short time period. Understanding is much more than pattern recognition. There’s got to be a deeper level of understanding and HD maps provide that.
If you think about all the things that the self-driving car has to do in real time, it has to ask a lot of questions about the world. And those answers have to be very, very reliable and precise and answered immediately. There’s not a lot of room for guessing. If you have a robot that’s controlling the vehicle, it has your life in its hands. HD maps can answer critical questions very reliably, much more reliably than any perception system could.
For example, sometimes you can’t see everything that you need to know about. Often, objects are obstructed. Sometimes things just disappear. And then there are sensors making mistakes, and perception systems making mistakes. So maps can bring about a much higher level of reliability and safety.
In addition, a map can — in essence — help the vehicle look around corners and look ten seconds into the future. So with all the planning and everything that goes on in a self-driving car, the HD map provides a lot of benefits which will make full self-driving vehicles tractable. And that’s a big point. To get to the point where the human driver can take their eyes off the road, the car has to be much, much more reliable in terms of making decisions. Otherwise, you’ll never be able to do it.
GB: How often are these maps updated?
MW: Updating HD maps is currently a very big challenge. With normal navigational maps, from Google and other mapping companies, you get updates happening on several month basis to a year basis. That won’t work for HD maps. When we started DeepMap, it was clear to us that map updates are one of the critical problems. This is one of the areas that we designed our approach around.
Conceptually, we believe that the only scalable solution is if the cars that are using the map are also creating and updating the map. The idea that you need to have a fleet of mapping vehicles, build the map and then give it to a self-driving car — we don’t think that is the solution. It will work on a small scale, but not on a large scale.
So, if you have cars driving around that can map and then update the map, how quickly can you do the map updates? Currently, we can do this in an order of minutes. The goal is to get it down to an order of seconds, where a change is detected and then the map is updated and communicated to the world.
If you’re in a self-driving car and there’s a change in the road, the first car that comes to that will detect the change and adjust its behavior. But the cars coming behind it should not have that problem. They should be just going through as if nothing changed. They’ll know exactly what to do.
GB: That’s fascinating. So there’s an individual riding in, say, the second or third vehicle that’s after the map update, they’ll just continue on that journey like there wasn’t an obstacle there?
MW: Yes, the vehicle will know precisely where the obstacle is and how to navigate around it.
GB: Wow. And what role does localization play in all this?
MW: Localization is a fundamental capability of self-driving. You need to know where you are in the world in order to go between two places. Now if you think about it, how accurately do you need to know where you are? Well, if you want to pull over and drop somebody off, you need to know very precisely where you can do that. If you are off by a yard or so, you could be violating a rule or putting a person in danger.
The other benefit that localization provides, that most people kind of miss, is that if you have accurate maps and accurate localization, you now can accurately tell what has changed in the world. And that is what we do. We can accurately and efficiently do change detection. It’s fundamental to DeepMap’s technology.
GB: Would a change detection be detecting whether a tree fell on the road or there was an object in the road?
MW: Yes. We can tell you the differences between what is in the map and what’s not. And we can do that in real time, pretty much instantly. And you reliably can see things like trees in the road and know to avoid them. So, this is a big thing that HD maps add to the equation.
GB: You mentioned centimeter accuracy. Could you define how accurate an HD map really has to be?
MW: When we started DeepMap, we talked to a lot of people working on AV efforts and tried to get a gauge on what accuracy they were expecting or needing. We got answers anywhere from five centimeters to 20 centimeters. People were generally talking five centimeters at one standard deviation or 20 centimeters at one center deviation.
The more we talked with people, the more it became clear that the people who had the most experience were asking for the highest accuracy. From the beginning of the company, we set our target towards the highest accuracy.
GB: And what is that exact accuracy?
MW: Our target is five centimeters at one standard deviation.
GB: We’re seeing growth in the area of autonomous delivery bots. Will that same centimeter level of accuracy be required for a delivery bot driving without a passenger?
MW: If it’s on the roads, yes. On sidewalks, maybe. There is less of a safety risk on the sidewalks. But sidewalks and crosswalks do go across roads. So you need to be careful there as well. The same principles and technology can apply to sidewalks as roads. We’ve had projects with different customers in the robo-delivery area.
GB: That’s fascinating. So DeepMap can clearly map a road. It can, for lack of a better term, map a sidewalk or a lane that an autonomous delivery bot is going to operate in?
MW: Yes. In fact, some of the self-driving car programs want the sidewalks mapped.
MW: The reason is, as you’re driving around using an HD map, you want to know where other traffic may come from. With our HD map, the vehicle knows about every single lane that could possibly intersect the lane it is driving in or planning to drive in, so it can be careful when there’s a possible oncoming car. And, it’s important to know where people tend to be or where to look out for people. That is why they want sidewalks mapped.
GB: On the map, let’s say there’s a high-traffic coffee shop or store. Can you mark that on the map so they know that there’s a lot of high foot traffic?
MW: We currently don’t do that, but we could imagine somebody wanting to do it. We mark areas such as school zones. You know when you are in a school zone, you have to obey different rules there. That’s one example. Another example is a tollbooth area.
GB: Recently, the senior director for AI at Tesla came out and said that a map-based approach for deploying self-driving cars en masse is non-scalable. What are your thoughts?
MW: First, what Tesla, and Andre Karpathy in particular, have done is incredible. I really like what they’ve done. But we do have a difference of opinion on where HD maps and LiDAR fit in into self-driving.
Building HD maps is a daunting task, even if you know what you’re doing. James Wu and I learned this at Google and James also experienced it at Apple. We understand why people would want to avoid having to depend on maps. If you can avoid depending on something that’s very hard, great.
But just because it’s hard, does not mean you can avoid it. Frankly, if it weren’t daunting, we probably would have had no reason to start the company because somebody would have done this already and we’d already have many more autonomous vehicles on the road.
We have a unique background and perspective on how to solve the HD mapping challenge and this is why we started the company.
GB: You publicly said HD mapping is less about mapping, and more about scalable infrastructure. Could you talk about that statement?
MW: The mapping part is hard enough. But the problem is, once you’ve figured that out, you have to manage it at a very large scale. The world is very large. We’re talking about mapping the world to five centimeters. This is an extremely large amount of data.
So we have to manage that and build processes to do it efficiently and economically. If you can’t do all of this economically, AVs don’t have a place in the world. They need to be safer than humans and cheaper. Maybe if they’re safer than humans and cost the same amount, there could be a role to play. But generally, AVs are not going to take off in a big way until they’re cheaper than having human drivers.
GB: You’re right about the economics and you’re right about safety. In your brilliant Safety in the Self-Driving Era blog post, you talked about distracted driving and pointed out how consumers are getting in more and more crashes because they’re looking at their phones. It seems that you’re laying the groundwork to define safety in the context of AVs. Still a long way to go, but you’re doing a great job, laying the groundwork to get there.
MW: I fear the AV industry is possibly paralyzed by the safety aspect. They have published these functional safety standards with the goal to have one fatal accident caused by a problem in the system every 30 billion miles.
That’s a very hard standard to prove. Everybody’s saying, how am I ever going to prove that? It will take forever to drive a billion miles, let alone 30 billion. But in that blog post, I looked at it a little bit differently. I generally think that with the right sensors and system and HD map and things like this, you can get to the point that it’s rare for a crash to be caused by the AV.
Mobileye has put out some work in this area in terms of managing safety envelopes around the car. You know, if the sensors are picking up all these obstacles moving around you much more frequently and accurately than a human, you can start to get to a point where you can pretty much avoid a collision.
If you can prove that you can avoid a collision in 20 million miles or one in 20 million miles, now you have a level of safety that’s like 10 times better, or more, than human. This reduces fatalities a huge amount and injury accidents a huge amount, and practically eliminates collisions as we know it.
GB: And it’s a great problem to solve, not only for individual lives saved, but also for all the families of the individuals who are impacted by accidents.
MW: In addition to saving lives, self-driving has the potential to save a huge amount of time. If you consider the amount of time spent, if you’re in even a minor collision, that has a major impact on you.
And these minor collisions happen very frequently. This is not one in a million kind of thing. It’s much more frequent and it’s traumatic. You know, some people can’t sleep after having an accident. Some people have whiplash, like my mom, for example.
People have been focused on reducing fatalities, of course. But the impact on timesaving from reducing the number of collisions will be huge.
GB: You’re right. Even if it’s a little fender bender, you’re stressed out, you’ve got to deal with insurance and other stuff.
MW: That’s right. If you’re even in a near-accident, where you have to slam on the brakes, you can be kind of traumatized, even though you didn’t hit anything. It can affect you throughout the week.
GB: You’re 100 percent right. And as you’re trying to improve this through HD maps, how accurate does localization have to be?
MW: Think about it this way. When you localize to the map, that basically places you in reference to the map. Everything around you is measured relative to that. If you have a five centimeter map and a one meter localization, you’re not in very good shape. Even 50 centimeters off going in a direction could greatly impact the planning and control for deciding how to bank and turn or accelerate or decelerate. So our aim is to also get localization in the five centimeter range. For most operating domains, we are in that range.
GB: Are you able to publicly talk about where some of your maps are deployed in geographical regions with leaving companies removed?
MW: I can’t give you specifics, but I can say we’ve mapped in eight countries and we don’t even need to go there to map. We’ve mapped in countries we’ve never been to.
MW: If our customers have a self-driving car, they drive their car around and send us the data. We do the map making, and then send them the map.
In addition, we have the capability to send a suitcase to people with our Portable DeepMapper (PDM) in it. They can put it on a car and map within a day. So, for example, if they’re planning a project and they need to collect data for simulation and training, they can get a PDM sent to them and they are off and running. We do that quite a lot.
GB: Will you sell me one?
MW: We can talk about it later.
GB: I’d just like to play with it.
MW: We might have some projects for you!
GB: This has been a really interesting conversation. We’ve focused a lot on ground vehicles, especially as it relates to moving passengers. We really haven’t dived into one of the fastest growing segments of autonomy — driverless trucks. Is DeepMap starting to get involved in the trucking space at all?
MW: Yes. Trucking is a big space. There’s a lot of potential for improvement in that market in terms of safety and utilization of trucks. We’re currently working with several trucking companies.
Trucks are a more complicated problem, and we’ve worked on our system to be able to accommodate that. Our map-making engine is able to support different truck configurations. Generally, the relationship between the sensors is not as stable as on a normal vehicle. You often have LiDAR very high in the air on these cabs that are bouncing around.
GB: It seems that some of these configurations are only in the front part and then some are around the back. Are there different configurations because there’s no standardization?
MW: We are working with our customers to figure out where the best mounting of the sensors is to make the mapping and localization reliable. The HD maps for cars and trucks are roughly the same. It’s just about making these trucks capable of consuming and creating and updating the maps.
GB: If you look at the general economics around the trucking industry, it seems like it’s growing faster than the robotaxi industry. On the other side of the trucking industry, you have the middle mile and last mile. Is that an area that DeepMap is also playing in as it relates to moving goods?
MW: Yes, we’re working with a few players in those spaces. The smaller delivery vehicles are very similar to what we would do with a car. If you get down to the sidewalk vehicles, then you’re talking about a very simple sensor set. Our system can work with that data as well.
GB: Mark, this has been a wonderful conversation. I’m really happy to hear that DeepMap is working on autonomous trucks, autonomous passenger vehicles, autonomous delivery bots, because DeepMap is clearly mapping the path towards full autonomy. Thank you for sharing your insights with us.
MW: Thanks for having me Grayson. It has been a pleasure.