During a recent trip to San Francisco, I caught a glimpse of a self-driving car. It wasn’t exactly difficult to spot thanks to a roof-mounted LiDAR suite and large orange lettering, but nevertheless my friends and I watched intently as it drove across the street. And then, just a few minutes later, I spotted another. And another. Our Lyft driver chuckled at my amazement.
“Those things are everywhere,” he said. “I actually cut in front of them, they usually let me through.”
Huh? They’re already here? Well, duh.
In the past few years, a combination of technological breakthroughs, increased testing on public roads, and astronomical investment have catalyzed the industry in a significant way. The question of self-driving cars isn’t a matter of if, but when.
The first cars — naively called horseless carriages — redefined communities and the ubiquity of transportation far beyond the wildest expectations. Similarly, self-driving cars are set to change the world. And I promise that’s not hyperbole. As this technology increases in prevalence, ripple effects will be wide and inescapable. The jobs of as many as 4 million Americans within the transportation sector will disappear while an expected $7 trillion will be added to the global economy.
In retrospect, it’s hard to rationalize my surprise in that moment. It wasn’t a reaction born out of ignorance, as I had generally kept up with the news. Rather, there was just something surreal about seeing it in action firsthand. Potentially the most impactful technology of our lifetimes was merging into the center lane just ahead — I couldn’t help but stare with awe! The experience got me thinking.
Current leaders in the world of self-driving consist of a variety of tech companies with futuristic-sounding names. Waymo. Argo. Cruise. The list goes on. But this technology wasn’t born in the labs of some Silicon Valley powerhouse. The origin of self-driving cars is a fascinating and peculiar one. It’s the story of a deadly war, a radical government agency, and a bunch of crazy scientists in the middle of the Mojave desert.
An unlikely beginning
Following the invasion of Afghanistan in the early 2000s, America had a critical problem on its hands. Hundreds of men and women were dying on the front lines due to improvised explosive devices that specifically targeted military convoys. Congress demanded a solution, and the obvious answer was unmanned ground vehicles. If military vehicles could traverse the off-road terrain of battlefields without the need for human drivers, surely the lives of countless soldiers could be saved.
Unfortunately, as promising as this vision was, execution proved to be incredibly difficult. The Pentagon burned through hundreds of millions of dollars in working with incumbent defense contractors by the likes of General Dynamics. Yet, after two full years of trying, results were largely nonexistent. Computing and sensor technologies were just not being improved upon quickly enough to create autonomous systems. It was time to look beyond traditional partnerships.
That’s when the Defense Advanced Research Projects Agency (DARPA) got involved. To understand DARPA, you’ll need to erase any preconceived notions of cumbersome bureaucracies that you may have. DARPA is a government agency unlike any other. It’s lean, with a size of just 220. It thrives on a sense of urgency — employees even have their “expiration date” printed on their ID badges as a reminder of their limited tenure.
Most importantly, though, DARPA has a stellar track record of thinking and delivering big.
The agency is best described as a deep-pocketed investment firm that seeks to accelerate technological breakthroughs that can provide value to the military.
Previous DARPA endeavors gave us GPS, speech recognition, and drones.
Oh, and the internet — no joke.
Tony Tether, then-director of DARPA, embodied the bold spirit of the agency wholeheartedly. He announced the 2004 Grand Challenge: a fully-autonomous 142-mile race through the Mojave Desert with a $1 million prize. The grueling race would take vehicles through switchbacks, steep gradients, and paths as narrow as 10 feet — capabilities that were hitherto only dreamt of. When someone asked, “Why are you making the race so hard?” DARPA responded, “That’s what makes it the Grand Challenge.”
The rough terrain of the race may have reflected the original military needs of the technology, but the similarities stopped there. Gone was any reliance on the traditional defense contractors that had failed the Pentagon — the Grand Challenge was open to anyone.
Really, I do mean anyone. The unique nature of challenge brought together droves of researchers, hobbyists, and even high school students. Tether himself was surprised by the support, and explained how the prospective competitors were united by a shared vision:
“These were just ordinary people who dreamed of having a car that was driverless.”
The Grand Challenge had it all. Solid funding, motivated participants, and a clear vision. What could possibly go wrong?
Rollovers. Fire. Tumbleweeds?
March 13th, 2004. The day of the Grand Challenge. 106 team applications had been reduced down to 15 finalists. What had they brought along?
The lineup of vehicles in the parched desert looked like a scene out of Mad Max. There were modified quad bikes, SUVs with sensors bolted on to all sides, a 14-ton military truck, and even a motorcycle. Carnegie Mellon’s Red Team, which had shone through in testing, brought along an absolute unit named Sandstorm: a red Humvee that had its roof and seats ripped out and replaced with 1,000 pounds of shock-resistant sensors and computers.
Some teams had spent as much as $3.5 million developing their vehicles. This was a culmination of a year’s hard work, and the air was buzzing with excitement. News crews waited patiently at the end of the 142-mile course — history was ready to be made.
As the driverless Frankencars set out, however, reality came knocking.
One of the vehicles, a Jeep Cherokee rigged with lasers, cameras, and even a couple of surfboards for looks, made it about 20 feet before abruptly making a U-turn and heading back to the starting line.
The lone motorcycle, which had promised to deliver on speed, fell over immediately after starting because its stabilization hadn’t been activated.
The sensor array of another vehicle got tangled up in barbed wire fencing due to a GPS issue.
Tether described the dismal failure of the biggest vehicle, after its laser scanners classified some brush ahead as an immovable obstacle:
“The Oshkosh truck backed up, and it saw another tumbleweed in the back. So it just went back and forth. Fourteen tons, taken down by two tumbleweeds.”
As the competition fell one by one, hopes of a self-driving future piled up on Carnegie Mellon’s Sandstorm. The Humvee had drawn attention from onlookers from the very start — speeding out of the gates at 40 miles per hour. The Pittsburgh-based team had spent the hours leading up to the competition ensuring that their state-of-art robotics components were working flawlessly.
So, did Sandstorm blitz through the entirety of the 142-mile course? Not quite.
How far did it make it? 100 miles? 50? 10?
No, no, and shockingly, no.
Sandstorm made it a whopping 7.4 miles, the furthest of any vehicle that day. A mechanical failure led it to drift slightly to the left. Red Whittaker, the team lead, watched on helplessly:
“The tire just spun and spun until it burned. Smoke was pouring off of it. The race officials hit the emergency kill, and the race was over for Sandstorm.”
Just like that, one empty fire extinguisher later, the 2004 DARPA Grand Challenge was over. Not a single car had made it the length of the whole course, leaving the million-dollar prize unclaimed.
Second time’s the charm
Had DARPA gone too far? Was the race too detached from reality, nothing more than a failed experiment?
It was easy to be discouraged. Headlines following the event were certainly grim, and naturally focused on the premature demise of the vehicles.
The real value of the Grand Challenge, however, lay in the individuals involved and the collective experience they had shared out there in the middle of the Mojave Desert.
These people — true enthusiasts — weren’t there for glory or money. They were passionate about seeing this technology being developed and were relentless in its pursuit. Teams would openly share their varying approaches with one another, establishing a decidedly collaborative environment from the very beginning.
It didn’t matter that no vehicle finished the 2004 Grand Challenge. The self-driving community had been born.
Recognizing this, Tether doubled down. Meeting with journalists after Sandstorm’s unfortunate fate, he announced:
“We’re gonna do it again, and this time it’s going to be a $2 million prize.”
The 2005 DARPA Grand Challenge was set to be held 18 months later at the same location. For many of the participants, competing again was a no-brainer. There were, of course, new teams as well.
Among the spectators at the first Grand Challenge was a German robotics scientist by the name of Sebastian Thrun. That’s a name you’ll want to remember. After witnessing the failures, Thrun believed he could do better. He threw down the gauntlet and entered the 2005 Challenge with a team from Stanford, where he was an associate professor of computer science.
To Thrun, the source of the failures was clear. He said:
“These vehicles didn’t fail because they weren’t rugged enough. They failed because they didn’t take in enough environmental information. None of them saw anything.”
To understand what he meant, let’s take a closer look at the traditional approach to building self-driving cars. These early robotic systems were designed to operate only in very constrained and predictable environments, and consequently relied on rule-based programming languages. In other words, they weren’t really autonomous.
In controlled testing, this worked great. On a desert race through loose gravel, well, the results of the 2004 Challenge spoke for themselves. The core fallacy was in trying to rely on the predictable aspects of an inherently unpredictable environment. As such, even the slightest erroneous sensor readings, unexpected obstacles, or tricky lighting conditions would throw the vehicles off course.
Thrun was on to something. His team created a log of human reactions and decisions and fed the data into a learning algorithm tied to the controls of their blue Volkswagen Touareg. As a result, their vehicle, named Stanley, was able to truly learn how to drive. Stanley sensed the world around it and easily overcame challenges, such as detecting shadows, that had proven insurmountable for other teams.
The 2005 Grand Challenge was ultimately a showdown between the traditional rule-based programming that dominated robotics and its clear antithesis: the rosy (yet unproven) promises of machine learning.
The differences between Carnegie Mellon and Stanford couldn’t be clearer. Once DARPA officials announced the waypoints of the race, the Pittsburgh-based team got to work manually hand tuning their Humvees with the specific GPS coordinates, as this was their primary source of navigation information. Stanley, on the contrary, only used GPS for general direction. The sensory-guided brains of the vehicle handled the rest. It was the closest thing to a truly autonomous vehicle that the world had ever seen.
Six hours and fifty-three minutes later, Stanley finished first. Four more cars, including both of Carnegie Mellon’s Humvees, would go on to finish the course.
It may have taken a second try, but history had finally been made.
Chasing the dragon
Fast forward a decade and a half, and we’re living at the precipice of a new technological revolution. The success of the self-driving car within the coming years is practically inevitable — technology that was for decades limited to the confines of science fiction is now considered ordinary in cities like San Francisco.
So, how much has DARPA contributed to where we are today? I reached out to someone who might know a thing or two about all of this: Sebastian Thrun himself. Even before the Grand Challenge, Thrun predicted its significance, saying that “100 years from now… I think this race will be heralded as a historical race.” So, was he right?
“I would say from today’s perspective, my prediction proved to be accurate. The Waymo team was heavily staffed with participants in the DARPA Challenges, including Chris Urmson, the CMU technical team lead. I would think that today’s many activities in the field — numerous start-ups, nearly all automotive OEMs, all major ride sharing companies, half a dozen tech giants like Baidu — would not be on this topic had it not been for Waymo’s great progress in this area — which is a direct offspring of the DARPA Grand Challenge.”
Thrun’s insight lays out the journey of autonomous vehicles. Technology once confined to research labs and the imagination of enthusiasts was shoved into the spotlight thanks to the DARPA Grand Challenge, and the community that was formed as a result became the backbone of the current self-driving car industry.
After the Grand Challenge, Thrun went on to lead Google’s self-driving car efforts, where he hired some of the best people he had met through DARPA. This endeavor went on to become Waymo, widely regarded as the front-runner in the space today. For many, like CMU’s Chris Urmson, the personal impact of the Challenge was clear. He told me:
“The DARPA GC was a galvanizing event — it pulled together a bunch of research and researchers and set a very concrete goal — drive 150 miles across the desert. My interest began because the problem was exciting and interesting. We were building vehicles to drive at high-speed across the desert. It was energizing, and novel. Over time I got to better understand the impact we could have in the world and that is what has created a long term motivation for me.”
Today, Urmson is the CEO of Aurora, a hot self-driving startup. Edwin Olson, who participated in a later iteration of the challenge as a doctoral student at MIT, explained to me the clearly impactful nature of the race:
“The Urban Challenge was incredibly compelling because it represented an inflection point of what we could build and make work. Many factors contributed to this, including better sensors, more CPU power, and also a better understanding of how to architect a system like this. Seeing our vehicle go out onto the course with no one in it, and having it come back to us several hours later, was an incredible thrill. In many ways I’m still chasing that dragon!”
Olson’s currently the CEO of May Mobility, a self-driving shuttle operator. You might have started to notice a trend here.
The CEO of Argo, CTO of Cruise, CEO of Marble, Co-Founder of Nuro, both the President and CTO of Optimus Ride, and CTO of Zoox all participated in the DARPA Challenges. Even critical supporting technologies, such as modern LIDAR systems, have their roots in DARPA: David Hall, the CEO of Velodyne, which provides advanced sensor systems for over 25 self-driving companies, decided to work on LIDAR after realizing how shortcomings in existing sensor technologies was bottlenecking autonomous systems.
The verdict is out. The DARPA Grand Challenges didn’t just encourage the development of this technology — it defined it.
The technological philosophy and leaders of this incoming revolution were minted in those desert races. We simply would not be where we are today without that deadly war, a radical government agency, and a bunch of crazy scientists united by dreams of a driverless future.
Sky’s the limit
The path forwards for self-driving cars seems pretty clear. With the current remarkable rate of development, we probably won’t be seeing any government-sponsored races in the near future. So what comes next?
Let’s shift our focus back to Sebastian Thrun. His journey reflects how the reins of self-driving technology have been handed over from research groups to private industry, but Thrun doesn’t work on self-driving cars with Google anymore. As of 2015 he’s been at the helm of Kitty Hawk, a startup aiming to create electric vertical take-off and landing vehicles (eVTOLs).
In other words, flying cars.
Kitty Hawk’s latest vehicle, Heaviside, promises travel times 10 times faster than road cars despite only using half the power. While this epitome of futuristic transportation has been garnering headlines and progressing on development, the field is yet to experience the kind of major revolution that we’ve seen with self-driving cars thanks to the DARPA Challenges.
Is there a place for similar events within the scope of eVTOLs? I asked Thrun, and he pointed me towards a NASA initiative with a surprisingly familiar name: the UAM Grand Challenge.
Set to begin in 2020, this new Grand Challenge will bring together the FAA and industry partners to accelerate the safe development of urban air mobility technologies. NASA understands that the incoming wave of eVTOLs from companies like Kitty Hawk will transform urban transportation, perhaps even more significantly than self-driving cars. Similar to the DARPA Challenges, participants will be expected to demonstrate the functionality of their vehicles via a series of timed challenges.
Will NASA’s attempt solve key roadblocks to flying cars? Or, will it fail to produce any tangible results?
If there’s anything that this story has taught us, it’s that it doesn’t matter.
When it comes to the moonshots of the future, overcoming inertia in development means taking a leap of faith and putting the technology out there for the world to see — even if that means failure. For decades, some of the smartest and most accomplished researchers had worked on self-driving cars, but it took a relatively simple race to bring about key innovations.
Let’s not forget, Thrun and others became so deeply invested in self-driving cars only after seeing them in action firsthand. If NASA can capture the imagination of the leaders of tomorrow, regardless of whether it be with demonstrations of wild success or of dismal failure, they’ve got a winner on their hands.
Humans are simple beings. Sometimes we’ve just got to see things with our own two eyes.