The Self-Driving Car Was Born In The Desert

Sohan Choudhury
Nov 12, 2019 · 14 min read

DARPA’s Grand Challenge sparked a revolution with a wild desert race. Nearly 15 years on, what comes next?

Image for post
Image for post
Illustration by Rounak Bhunia.

Horseless carriages

During a recent trip to San Francisco, I caught a glimpse of a self-driving car. It wasn’t exactly difficult to spot thanks to a roof-mounted LiDAR suite and large orange lettering, but nevertheless my friends and I watched intently as it drove across the street. And then, just a few minutes later, I spotted another. And another. Our Lyft driver chuckled at my amazement.

Cruise’s self-driving car on the streets of SF.
Cruise’s self-driving car on the streets of SF.
Cruise’s self-driving car on the streets of SF.

“Those things are everywhere,” he said. “I actually cut in front of them, they usually let me through.”

Huh? They’re already here? Well, duh.

In the past few years, a combination of technological breakthroughs, increased testing on public roads, and astronomical investment have catalyzed the industry in a significant way. The question of self-driving cars isn’t a matter of if, but when.

The first cars — naively called horseless carriages — redefined communities and the ubiquity of transportation far beyond the wildest expectations. Similarly, self-driving cars are set to change the world. And I promise that’s not hyperbole. As this technology increases in prevalence, ripple effects will be wide and inescapable. The jobs of as many as 4 million Americans within the transportation sector will disappear while an expected $7 trillion will be added to the global economy.

Argo AI’s self-driving car being tested in Detroit.
Argo AI’s self-driving car being tested in Detroit.
Argo AI’s self-driving car being tested in Detroit.

In retrospect, it’s hard to rationalize my surprise in that moment. It wasn’t a reaction born out of ignorance, as I had generally kept up with the news. Rather, there was just something surreal about seeing it in action firsthand. Potentially the most impactful technology of our lifetimes was merging into the center lane just ahead — I couldn’t help but stare with awe! The experience got me thinking.

Current leaders in the world of self-driving consist of a variety of tech companies with futuristic-sounding names. Waymo. Argo. Cruise. The list goes on. But this technology wasn’t born in the labs of some Silicon Valley powerhouse. The origin of self-driving cars is a fascinating and peculiar one. It’s the story of a deadly war, a radical government agency, and a bunch of crazy scientists in the middle of the Mojave desert.

An unlikely beginning

Congress turned to the Pentagon to protect vulnerable troops in the Middle East.
Congress turned to the Pentagon to protect vulnerable troops in the Middle East.
Congress turned to the Pentagon to protect vulnerable troops in the Middle East.

Following the invasion of Afghanistan in the early 2000s, America had a critical problem on its hands. Hundreds of men and women were dying on the front lines due to improvised explosive devices that specifically targeted military convoys. Congress demanded a solution, and the obvious answer was unmanned ground vehicles. If military vehicles could traverse the off-road terrain of battlefields without the need for human drivers, surely the lives of countless soldiers could be saved.

Unfortunately, as promising as this vision was, execution proved to be incredibly difficult. The Pentagon burned through hundreds of millions of dollars in working with incumbent defense contractors by the likes of General Dynamics. Yet, after two full years of trying, results were largely nonexistent. Computing and sensor technologies were just not being improved upon quickly enough to create autonomous systems. It was time to look beyond traditional partnerships.

That’s when the Defense Advanced Research Projects Agency (DARPA) got involved. To understand DARPA, you’ll need to erase any preconceived notions of cumbersome bureaucracies that you may have. DARPA is a government agency unlike any other. It’s lean, with a size of just 220. It thrives on a sense of urgency — employees even have their “expiration date” printed on their ID badges as a reminder of their limited tenure.

Most importantly, though, DARPA has a stellar track record of thinking and delivering big.

The agency is best described as a deep-pocketed investment firm that seeks to accelerate technological breakthroughs that can provide value to the military.

Previous DARPA endeavors gave us GPS, speech recognition, and drones.

Oh, and the internet — no joke.

DARPA’s Tony Tether at the Grand Challenge.
DARPA’s Tony Tether at the Grand Challenge.
DARPA’s Tony Tether at the Grand Challenge.

Tony Tether, then-director of DARPA, embodied the bold spirit of the agency wholeheartedly. He announced the 2004 Grand Challenge: a fully-autonomous 142-mile race through the Mojave Desert with a $1 million prize. The grueling race would take vehicles through switchbacks, steep gradients, and paths as narrow as 10 feet — capabilities that were hitherto only dreamt of. When someone asked, “Why are you making the race so hard?” DARPA responded, “That’s what makes it the Grand Challenge.”

The rough terrain of the race may have reflected the original military needs of the technology, but the similarities stopped there. Gone was any reliance on the traditional defense contractors that had failed the Pentagon — the Grand Challenge was open to anyone.

Really, I do mean anyone. The unique nature of challenge brought together droves of researchers, hobbyists, and even high school students. Tether himself was surprised by the support, and explained how the prospective competitors were united by a shared vision:

The Grand Challenge had it all. Solid funding, motivated participants, and a clear vision. What could possibly go wrong?

Rollovers. Fire. Tumbleweeds?

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Some of the wild participants of the 2004 Challenge. CMU’s Sandstorm is bottom right.

March 13th, 2004. The day of the Grand Challenge. 106 team applications had been reduced down to 15 finalists. What had they brought along?

The lineup of vehicles in the parched desert looked like a scene out of Mad Max. There were modified quad bikes, SUVs with sensors bolted on to all sides, a 14-ton military truck, and even a motorcycle. Carnegie Mellon’s Red Team, which had shone through in testing, brought along an absolute unit named Sandstorm: a red Humvee that had its roof and seats ripped out and replaced with 1,000 pounds of shock-resistant sensors and computers.

Some teams had spent as much as $3.5 million developing their vehicles. This was a culmination of a year’s hard work, and the air was buzzing with excitement. News crews waited patiently at the end of the 142-mile course — history was ready to be made.

As the driverless Frankencars set out, however, reality came knocking.

One of the vehicles, a Jeep Cherokee rigged with lasers, cameras, and even a couple of surfboards for looks, made it about 20 feet before abruptly making a U-turn and heading back to the starting line.

The lone motorcycle, which had promised to deliver on speed, fell over immediately after starting because its stabilization hadn’t been activated.

The sensor array of another vehicle got tangled up in barbed wire fencing due to a GPS issue.

Even the smallest challenges proved fateful for some. I mean, that embankment is tiny.
Even the smallest challenges proved fateful for some. I mean, that embankment is tiny.
Even the smallest challenges proved fateful for some. I mean, that embankment is tiny.

Tether described the dismal failure of the biggest vehicle, after its laser scanners classified some brush ahead as an immovable obstacle:

As the competition fell one by one, hopes of a self-driving future piled up on Carnegie Mellon’s Sandstorm. The Humvee had drawn attention from onlookers from the very start — speeding out of the gates at 40 miles per hour. The Pittsburgh-based team had spent the hours leading up to the competition ensuring that their state-of-art robotics components were working flawlessly.

So, did Sandstorm blitz through the entirety of the 142-mile course? Not quite.

How far did it make it? 100 miles? 50? 10?

No, no, and shockingly, no.

The yellow line marks the route of the Grand Challenge. The pink markers show how far the vehicles made it. Yeah, this was ba
The yellow line marks the route of the Grand Challenge. The pink markers show how far the vehicles made it. Yeah, this was ba
The yellow line highlights the route of the Grand Challenge. The pink markers show how far the vehicles made it. Yeah, this was bad.

Sandstorm made it a whopping 7.4 miles, the furthest of any vehicle that day. A mechanical failure led it to drift slightly to the left. Red Whittaker, the team lead, watched on helplessly:

Just like that, one empty fire extinguisher later, the 2004 DARPA Grand Challenge was over. Not a single car had made it the length of the whole course, leaving the million-dollar prize unclaimed.

Second time’s the charm

Had DARPA gone too far? Was the race too detached from reality, nothing more than a failed experiment?

It was easy to be discouraged. Headlines following the event were certainly grim, and naturally focused on the premature demise of the vehicles.

The real value of the Grand Challenge, however, lay in the individuals involved and the collective experience they had shared out there in the middle of the Mojave Desert.

Teams mingling at the 2004 Challenge.
Teams mingling at the 2004 Challenge.
Teams mingling at the 2004 Challenge.

These people — true enthusiasts — weren’t there for glory or money. They were passionate about seeing this technology being developed and were relentless in its pursuit. Teams would openly share their varying approaches with one another, establishing a decidedly collaborative environment from the very beginning.

It didn’t matter that no vehicle finished the 2004 Grand Challenge. The self-driving community had been born.

Recognizing this, Tether doubled down. Meeting with journalists after Sandstorm’s unfortunate fate, he announced:

The 2005 DARPA Grand Challenge was set to be held 18 months later at the same location. For many of the participants, competing again was a no-brainer. There were, of course, new teams as well.

German scientist, passionate roboticist. I present to you, Sebastian Thrun.
German scientist, passionate roboticist. I present to you, Sebastian Thrun.
German scientist, passionate roboticist. I present to you, Sebastian Thrun.

Among the spectators at the first Grand Challenge was a German robotics scientist by the name of Sebastian Thrun. That’s a name you’ll want to remember. After witnessing the failures, Thrun believed he could do better. He threw down the gauntlet and entered the 2005 Challenge with a team from Stanford, where he was an associate professor of computer science.

To Thrun, the source of the failures was clear. He said:

To understand what he meant, let’s take a closer look at the traditional approach to building self-driving cars. These early robotic systems were designed to operate only in very constrained and predictable environments, and consequently relied on rule-based programming languages. In other words, they weren’t really autonomous.

In controlled testing, this worked great. On a desert race through loose gravel, well, the results of the 2004 Challenge spoke for themselves. The core fallacy was in trying to rely on the predictable aspects of an inherently unpredictable environment. As such, even the slightest erroneous sensor readings, unexpected obstacles, or tricky lighting conditions would throw the vehicles off course.

Image for post
Image for post
Image for post
Image for post
Stanley! Back in a time when “machine learning” wasn’t such a buzzword.

Thrun was on to something. His team created a log of human reactions and decisions and fed the data into a learning algorithm tied to the controls of their blue Volkswagen Touareg. As a result, their vehicle, named Stanley, was able to truly learn how to drive. Stanley sensed the world around it and easily overcame challenges, such as detecting shadows, that had proven insurmountable for other teams.

The 2005 Grand Challenge was ultimately a showdown between the traditional rule-based programming that dominated robotics and its clear antithesis: the rosy (yet unproven) promises of machine learning.

The differences between Carnegie Mellon and Stanford couldn’t be clearer. Once DARPA officials announced the waypoints of the race, the Pittsburgh-based team got to work manually hand tuning their Humvees with the specific GPS coordinates, as this was their primary source of navigation information. Stanley, on the contrary, only used GPS for general direction. The sensory-guided brains of the vehicle handled the rest. It was the closest thing to a truly autonomous vehicle that the world had ever seen.

Six hours and fifty-three minutes later, Stanley finished first. Four more cars, including both of Carnegie Mellon’s Humvees, would go on to finish the course.

It may have taken a second try, but history had finally been made.

Chasing the dragon

Waymo’s self-driving car. The company is considered the clear frontrunner in the space.
Waymo’s self-driving car. The company is considered the clear frontrunner in the space.
Waymo’s self-driving car. The company is considered the clear frontrunner in the space.

Fast forward a decade and a half, and we’re living at the precipice of a new technological revolution. The success of the self-driving car within the coming years is practically inevitable — technology that was for decades limited to the confines of science fiction is now considered ordinary in cities like San Francisco.

So, how much has DARPA contributed to where we are today? I reached out to someone who might know a thing or two about all of this: Sebastian Thrun himself. Even before the Grand Challenge, Thrun predicted its significance, saying that “100 years from now… I think this race will be heralded as a historical race.” So, was he right?

Thrun’s insight lays out the journey of autonomous vehicles. Technology once confined to research labs and the imagination of enthusiasts was shoved into the spotlight thanks to the DARPA Grand Challenge, and the community that was formed as a result became the backbone of the current self-driving car industry.

Thrun allegedly moonlights as an artist in spare time. His favorite medium is chalk.
Thrun allegedly moonlights as an artist in spare time. His favorite medium is chalk.
Thrun allegedly moonlights as an artist in his spare time. His favorite medium? Chalk.

After the Grand Challenge, Thrun went on to lead Google’s self-driving car efforts, where he hired some of the best people he had met through DARPA. This endeavor went on to become Waymo, widely regarded as the front-runner in the space today. For many, like CMU’s Chris Urmson, the personal impact of the Challenge was clear. He told me:

Today, Urmson is the CEO of Aurora, a hot self-driving startup. Edwin Olson, who participated in a later iteration of the challenge as a doctoral student at MIT, explained to me the clearly impactful nature of the race:

Olson’s currently the CEO of May Mobility, a self-driving shuttle operator. You might have started to notice a trend here.

The CEO of Argo, CTO of Cruise, CEO of Marble, Co-Founder of Nuro, both the President and CTO of Optimus Ride, and CTO of Zoox all participated in the DARPA Challenges. Even critical supporting technologies, such as modern LIDAR systems, have their roots in DARPA: David Hall, the CEO of Velodyne, which provides advanced sensor systems for over 25 self-driving companies, decided to work on LIDAR after realizing how shortcomings in existing sensor technologies was bottlenecking autonomous systems.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Here’s a selection of companies with direct ties to the DARPA Challenges.

The verdict is out. The DARPA Grand Challenges didn’t just encourage the development of this technology — it defined it.

The technological philosophy and leaders of this incoming revolution were minted in those desert races. We simply would not be where we are today without that deadly war, a radical government agency, and a bunch of crazy scientists united by dreams of a driverless future.

Sky’s the limit

Kitty Hawk’s Heaviside, which promises to be 100 times quieter than a regular helicopter.
Kitty Hawk’s Heaviside, which promises to be 100 times quieter than a regular helicopter.
Kitty Hawk’s Heaviside, which promises to be 100 times quieter than a regular helicopter.

The path forwards for self-driving cars seems pretty clear. With the current remarkable rate of development, we probably won’t be seeing any government-sponsored races in the near future. So what comes next?

Let’s shift our focus back to Sebastian Thrun. His journey reflects how the reins of self-driving technology have been handed over from research groups to private industry, but Thrun doesn’t work on self-driving cars with Google anymore. As of 2015 he’s been at the helm of Kitty Hawk, a startup aiming to create electric vertical take-off and landing vehicles (eVTOLs).

In other words, flying cars.

Kitty Hawk’s latest vehicle, Heaviside, promises travel times 10 times faster than road cars despite only using half the power. While this epitome of futuristic transportation has been garnering headlines and progressing on development, the field is yet to experience the kind of major revolution that we’ve seen with self-driving cars thanks to the DARPA Challenges.

Is there a place for similar events within the scope of eVTOLs? I asked Thrun, and he pointed me towards a NASA initiative with a surprisingly familiar name: the UAM Grand Challenge.

Set to begin in 2020, this new Grand Challenge will bring together the FAA and industry partners to accelerate the safe development of urban air mobility technologies. NASA understands that the incoming wave of eVTOLs from companies like Kitty Hawk will transform urban transportation, perhaps even more significantly than self-driving cars. Similar to the DARPA Challenges, participants will be expected to demonstrate the functionality of their vehicles via a series of timed challenges.

Will NASA’s attempt solve key roadblocks to flying cars? Or, will it fail to produce any tangible results?

If there’s anything that this story has taught us, it’s that it doesn’t matter.

When it comes to the moonshots of the future, overcoming inertia in development means taking a leap of faith and putting the technology out there for the world to see — even if that means failure. For decades, some of the smartest and most accomplished researchers had worked on self-driving cars, but it took a relatively simple race to bring about key innovations.

NASA’s vision for the future. Disclaimer: It’ll probably take more than a few cool artist renderings to get this all to work.
NASA’s vision for the future. Disclaimer: It’ll probably take more than a few cool artist renderings to get this all to work.
NASA’s vision for the future. Disclaimer: It’ll probably take more than a few cool artist renderings to get this all to work.

Let’s not forget, Thrun and others became so deeply invested in self-driving cars only after seeing them in action firsthand. If NASA can capture the imagination of the leaders of tomorrow, regardless of whether it be with demonstrations of wild success or of dismal failure, they’ve got a winner on their hands.

Humans are simple beings. Sometimes we’ve just got to see things with our own two eyes.

Thanks for reading! Feel free to follow me / check out BOOST for more.

BOOST

Exploring the past, present, and future of transportation.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium