The Race to Fully Autonomous Cars
Here’s an overview on the Autonomy & Mobility space as it relates to self-driving cars — i.e. what does it mean, who’s ahead, what is the tech, how does it work, how much does it cost, what are the major hurdles to ubiquity, how is the tech currently performing, etc. This only covers through Dec. 1st, 2017 — notes on more recent developments are at the end.
What is Autonomy and Mobility? For this post, we will focus on self-driving cars. When someone mentions autonomy and mobility they are most likely referring to self-driving cars — think Waymo or Cruise Automation — but it is much more than that. The NHTSA says this technology, “…is so full of promise…” and defines it as, “…a future where vehicles increasingly help drivers avoid crashes. It’s a future where the time spent commuting is dramatically reduced, and where millions more — including the elderly and people with disabilities–gain access to the freedom of the open road. And, especially important, it’s a future where highway fatalities and injuries are significantly reduced.” Think of it as an easier way to travel from Point A to Point B on a road that has not been adapted for the technology. For example, you get into the car, tell it where to go and enjoy the ride while watching a movie or reading a book. Below is a diagram that the broader autonomy and mobility community uses to gauge the different levels of autonomy as defined by the NHTSA.
Why is it Important? Safety. Simplicity. Environmental Impact. There are many commercial reasons for the shift to autonomous vehicles — less traffic, faster commute, increased leisure time, etc. — but the main reason many industry leaders pursue this technology is to provide safer roads. In the US, more than 30,000 people die in traffic-related deaths every year, but self-driving cars could drastically reduce the number of accidents helping to save thousands of lives. In fact, if ~90% of cars on American roads were autonomous, the number of accidents would fall from 6 million a year to 1.3 million. Deaths would fall from 33,000 to 11,300, according to a study by the Eno Centre for Transportation. These numbers are based on the belief that if there will be less chance of accidents caused by human error, leading to less traffic congestion. It’s also expected that the rise of self-driving taxis will help decrease the total number of cars on the road, alleviating the overall traffic and since driver-less vehicles are designed to optimize efficiency in acceleration and braking, they will also help improve fuel efficiency and reduce carbon emissions. According to McKinsey, adoption of autonomous cars could reduce CO2 emissions produced by cars by as much as 300 million tons per year.
Who is Talking About it? Countries. Multi-national corporations. Thought leaders. Major countries such as the UAE, namely, the Dubai Future Foundation in conjunction with Dubai’s Roads and Transport Authority Launched the ‘Dubai Autonomous Transportation Strategy’. The strategy aims to transform 25% of the total transportation in Dubai to autonomous mode by 2030, involving 5 million daily trips, saving $6 billion in annual economic costs. Dubai recently ordered 50 Tesla Model S and Model X SUVs to turn them into fully autonomous vehicles ahead of the mentioned strategy above. The plan is to order another 150. Google’s subsidiary, Sidewalk Labs, in collaboration with Toronto’s southeastern district are working together to transform a 12-acre plot of land into a futuristic, urban development using a data-driven approach to creating a smarter city built around people and one that communicates with autonomous vehicles. If that goes according to plan, Google will do the same with an 800-acre plot nearby. Google will relocate their Toronto office to the 12-acre plot to better gauge the environment (i.e. collect data, self-immersion, etc.). According to a recent study by The Boston Consulting Group, 25% of all miles driven in the United States will be covered in self-driving cars by the year 2030. The BCG study predicts the combination of autonomous driving, electric power, and ride sharing will prove more potent economically than any of those factors on their own. BCG expects the shift toward autonomous electric cars to begin in the early 2020s, with the highest levels of adoption in cities with more than 1 million people. Large cities have the population necessary to keep fleet utilization high, and also theoretically stand to benefit the most from anticipated decreases in congestion, parking issues, and pollution.
How do Autonomous Vehicles Work? A Mix Of Artificial Intelligence And Sensors. The most prominent method industry leaders — such as Waymo and Cruise Automation — use to teach their current systems is to think like humans. This means the cars must identify the environment in real-time and understand how the environment will act in order to react accordingly. For example, if the car notices a pedestrian crossing the street, it will monitor the location of that pedestrian to the millimeter and anticipate their next step in order to avoid any collision. Ultimately, autonomous cars need to be better than humans to prevail. Below is a handful of modern technology that is crucial in making this process a reality.
How Does it Work Together? Artificial Intelligence. The sensors and systems noted above feed real-time data to a central AI software system that then observes the objects around the car, predicts their behavior and plans accordingly — like a human would do. Waymo, Google’s subsidiary and leader in the autonomous vehicle space, explains it in three main components.
- Perception — the part of the software that detects and classifies objects on the road, while also estimating their speed, direction and acceleration over time. The ability of software to correctly differentiate between people, cars, cones, etc. and accurately understand different fonts and colors helps to correctly predict the behavior of the object.
- Behavior — the software then models, predicts and understands the intent of objects on the road. Effective prediction of object behavior stems from millions of miles of testing. Waymo highlight’s how their software can accurately anticipate behavior of cyclists, pedestrians and motorcyclists despite looking similar (i.e. pedestrians move slower than motorcyclists, but can change direction more quickly).
- Planner — the car then plans the best course of action as a defensive driver would. Meaning Waymo, in particular, programs it’s cars to stay out of blind spots, provide extra room for cyclists and provide for a smooth ride in and out of traffic.
How Much do These Systems Cost? This depends on how advanced you want your car to be, but assuming you want to outfit an existing car to level 4 autonomy, expect to spend over $100,000. The main hardware needed to outfit the car is the main LiDAR system on top of the car (right). Velodyne is the manufacturer behind those that go on Google’s Waymo cars, but Google recently announced they found a way to reduce the cost from $75,000 to at least a tenth of that or ~$7,500 per unit. This has yet to be realized, but if it’s true, it’s a huge step in the right direction. Tack on four similar, but less important LiDAR sensors for roughly $6,000 and you’re up to $117,000 in LiDAR expenses. Beyond that there’s a radar ($10,000), cameras ($6,000 for two), and graphics cards and other hardware for another ~$10,000. In full, that’s nearly $143,000 plus the cost of the existing car, some more adaptable than others. There is a race among competitors to get this to market — expect lower prices in the near future.
Current Incumbents Relative to One Another. Navigant Research released a report on the 18 most influential autonomous car manufacturers, tech giants and startups addressing the autonomous car challenge. The study is based on a handful of criteria, but mainly strategy and execution (i.e. identifying the leaders). That is the basis for the structure of the chart here, but the findings are presented along with additional companies, ultimately altering the results presented by Navigant. The majority of leaders are large automakers but that’s only possible with the help of the startups building the software and hardware behind the scenes.
Leaders in the Race to Level 5 Autonomy:
Google / Waymo — started in ‘08. Testing level 5 in Arizona now. Next step: testing in the city. Launched eight years ago as a Google project, Waymo conducted it’s first driver-less test on public roads in October 2015, became its own company in December 2015 under the Alphabet umbrella and is currently offering rides in Arizona without oversight / someone behind the wheel. Waymo hasn’t disclosed how much it will charge customers, how long the rides will go for and what size territory it will cover in the process. Waymo is not in the business of physically building cars and intends to instead supply self-driving tech to other companies, but for now the goal is to test the technology as a ride-hailing app. Bringing its hardware efforts in-house instead of outsourcing from Velodyne, as well as, reducing the price of components is in-line with their strategy.
GM / Cruise Automation — Aiming to be the first company to test on NYC roads. In October, the company revealed plans to deploy a fleet of autonomous 2017 Chevrolet Volts at its Warren Technical Center campus in Michigan named MCity, from which GM employees will be able to reserve a car and input a destination using a mobile app. The company’s autonomous technology will drive passengers to their desired destination and park nearby for future use. GM also confirmed that its Super Cruise technology, which it has been developing since 2012, will be available in the 2017 Cadillac CT6. GM also announced plans for two car-sharing and ride-hailing pilot programs to help test its hardware and software products, and gain a better understanding of the user experience. Since GM’s driver-less cars are years away from hitting the road as part of Lyft’s service (GM invested $500M in Lyft), the automaker plans to provide short-term car rentals (for a day, week or month) to Lyft drivers in the immediate future. GM had to adjust Gen 3 Chevy Bolt 40% of the parts in the Chevy Bolt (the car used to test the system) to accommodate autonomous driving. GM acquired Cruise Automation for $1B in March of 2016 and plans to incorporate the Cruise technology into future production. Currently in effect, Cruise Anywhere is a robocar that drives 10% of the 250 San Francisco-based Cruise Automation employees to work from 7am to 11pm with a passenger to gauge customer behaviors, take note of any mishaps and course correct any errors. Cruise said it will be the first company to test autonomous vehicles in New York City.
Uber — Went live with their autonomous vehicles in Pittsburgh in September 2017. Uber set up shop in Pittsburgh after poaching several robotics experts from Carnegie Mellon in May 2015. After launching its Pittsburgh trial and in order to test autonomous vehicles in a safe and controlled environment, the company setup a 42-acre facility called the Almono in September. Uber currently tests via a similar facility in Arizona. Earlier this year, Uber partnered with nuTonomy and mentioned it was going to start developing it’s own software and hardware. In December, Uber got into a public dispute with the California DMV after launching a self-driving-car pilot in San Francisco without first obtaining an autonomous vehicles testing permit. Uber left California for Arizona after the DMV revoked registration of its 16 self-driving Volvo XC90s. In January, Uber formed a partnership with Daimler as a potential hedge in case their in-house tech doesn’t go according to plan. Waymo is suing Uber, claiming the ride-hailing service stole the intellectual property for its LiDAR system. Uber plans to buy 24,000 autonomous Volvo SUVs in the race for the driver-less future. Edison research stated that Uber’s tech forced drivers to take over at least once every mile. Whereas, Waymo’s tech only required intervention every 5,128 miles.
Delphi / nuTonomy — General Motors spin-off based in the UK. Delphi is a 100-member automated driving team that just added another 100 employees (70 of which are engineers and scientists) through its purchase of nuTonomy. nuTonomy, a Boston-based startup spun out of MIT in 2013, has been quietly making big moves in the self-driving-car space. In August 2016, nuTonomy became the first company to launch a fleet of self-driving taxis under a pilot program in Singapore. The startup has since partnered with Lyft to launch a pilot in Boston before the end of this year. nuTonomy has raised $20 million in venture funding through 2016. Investors include the government of Singapore and Fontinalis Partners. nuTonomy has deals with Lyft, Grab, and Groupe PSA, which owns European car brands Peugeot SA and Citroën. Delphi is working with Mobileye, the auto-vision company that was recently acquired by Intel, to build a self-driving car by 2019 (their annual report claims they will have a fully autonomous turnkey solution in market by 2019 — mentioned 6 times in the annual report). Namely, Delphi and Mobileye hope to offer automakers a system that can give less expensive cars and trucks the intelligence to drive themselves. At the center will be a package of Mobileye and Intel chips capable of computing about 20 trillion mathematical operations a second. They intend to increase that power 2 to 3 times in a later version.
Lyft — Just opened an 11,000 sqft location in Manhattan to focus on autonomous testing in NYC. On July 21st 2017, Lyft announced it would open a new division to focus on autonomous cars. On September 7th 2017, Lyft announced it will launch a fleet of self-driving cars outfitted with Drive.ai and only to select customers in San Francisco, but with a person in the car monitoring it at all times — per CA law. Drive.ai creates AI software for autonomous vehicles. It aims to build hardware and software kits powered by artificial intelligence for car-makers. Aside from Drive.ai, Lyft has scored partnerships with GM, a $25M investment and partnership with Waymo, plus a fleet of cars for testing from Jaguar Land Rover. Lyft just setup shop in Manhattan’s Chelsea district with an 11,000 sqft space and 80 employees (including autonomous driving engineers).
Ford — Ford wants to build a level 4 self-driving car by 2021. Ford has committed to expanding its research in advanced algorithms, 3-D mapping, radar technology and camera sensors. Since the launch of the program, Ford has announced four key investments and collaborations: Velodyne, SAIPS, Nirenberg Neuroscience and Civil Maps. Ford recently invested $1B in Argo AI aiming for “full autonomy by 2021”, hence the collaboration with Argo AI and other technology companies. The goal is to completely outfit Ford vehicles with self-driving technology. The Argo AI is the creation of an entirely separate company with an independent equity structure where Ford is the “majority stakeholder,” but will operate with “substantial independence.” Employees will receive equity in the company. The investment will be made over five years.
Audi — The most advanced Volkswagen Group affiliate has begun testing in Albany, NY. In May, the state of New York opened an application for companies that wanted to test autonomous vehicles there. Audi snagged the first license and began testing near Albany in the following weeks. Audi has tested in Nevada and California so this is not new to Audi. The A7 is set up to perform SAE Level 3 autonomous driving tasks, which means it can drive unassisted at posted highway speeds when conditions are good (i.e. no speeding, no snow, and a driver does still need to be behind the wheel and ready to take over when prompted). Audi’s Level 4 system, the A9 e-tron, which can drive the car at highway speeds and change lanes unassisted, is expected to be available by 2020–2021. To reach Level 4 autonomy by 2020, Audi will be relying on Nvidia to assist with artificial intelligence and the processing power required.
Tesla — Looking to prove themselves in 2018 after a handful of accidents relating to AutoPilot. Although Tesla’s self-driving features have come a long way, Navigant said there were reasons to doubt the automaker’s ability to achieve Level 4 autonomy. Tesla cars are currently being built with new hardware that will improve Tesla Autopilot, renaming the system Autopilot 2.0, and set the foundation for full autonomy. However, according to one Navigant researcher, “Autopilot 2.0 is still missing a lot of the functionality of the original version that relied more heavily on Mobileye from their vision system,” said Sam Abuelsamid, a senior research analyst at Navigant. “And from everything that we’ve seen, it does not sound like they have caught up to where Mobileye was a year and a half ago.” A Tesla will drive itself from Los Angeles to New York before mid-2018 to demonstrate the technology, the company claims. Tesla crash rates plummeted 40% since Autopilot was first installed in 2015.
Toyota — Picking up the pace and well positioned to develop fully autonomous vehicles. Toyota’s $1B investment in the Toyota Research Institute (TRI) highlights the Japanese automaker’s interest in autonomous driving. At the end of September, the company released Platform 2.1, its next-generation self-driving test car, a vehicle it says can more accurately detect objects and roadways. This comes months after its first launch of the initial version introduced March 2017. The quickening pace shows automaker’s desire to reach mass production and in the case of Toyota, their interest in early stage startups as the company recently teamed up with Luminar, a startup headed by a 22-year-old with connections to Peter Thiel (more on this later). According to Navigant, as one of the world’s largest and most profitable OEMs, Toyota has the resources and expertise to make fully automated vehicles.
Volvo — In the middle of signing one of the biggest car deals in history. Volvo plans to make its cars “death proof” by 2020 by rolling out semi-autonomous features over time. The automaker is letting families test self-driving Volvos in Gothenburg, Sweden and London this year as part of its Drive Me program. The company will also conduct an “advanced autonomous driving experiment” in China, where 100 volunteers will be able to test driverless Volvo XC90s on public roads, but the automaker hasn’t said when that trial will start. Volvo and Uber agreed to a $300 million alliance in August to develop autonomous vehicles, which are currently being tested in Arizona. The new deal with Volvo is said to be worth around $1.4 billion.
BMW — Pursuing fully autonomous cars with two crucial partners. BMW has released advanced driver assistance tech in its luxury vehicles, like the BMW 7-Series and 5-Series. BMW plans to release a fully driverless car in 2021 and has teamed up with Intel and Mobileye to do so. Called the BMW iNEXT, the first fully autonomous BMW car is going to lay down the foundations of BMW’s strategy on this front. Test drives will start next year. Intel vice president of the automated driving group Kathy Winter unveiled the first of 40 BMW 7-Series highly-automated cars it’ll be testing on the road. All the cars are expected to be on the road by the end of the year. All the vehicles will include multiple Mobileye cameras for road scanning.
Mercedes-Benz (Daimler) — Focusing on software / AI systems. Daimler is teaming up with Bosch to bring fully autonomous cars to urban roads by 2020. The focus of the partnership will be on the software and algorithms required to make those advanced driving systems safe and predictable. The goal is to create shared cars that can operate autonomously within designated areas of a city. Daimler participated in the $46M Series B for Momenta.AI, software for autonomous cars. The automaker also participated in financing rounds for Matternet, an autonomous drone startup, and Starship Technologies, a company building self-driving, last-mile delivery robots. Currently there is a suite of semi-autonomous features in brands like its Mercedes S-Class and E-Class cars. A Mercedes big-rig truck made history in 2015 when it drove itself on a public highway.
Honda — Lost a big deal (with Waymo) to Chrysler and slowly recovering. Honda is more focused on expanding its assisted driving features in its current vehicles rather than pushing for full autonomy. Honda said that it intends to have vehicles capable of Level 3 freeway driving on the market by 2020. The company has an autonomous vehicle testing permit in California, but only tested on closed courses in 2016. Whereas Waymo once considered using Honda cars, Honda lost the deal to Chrysler, the Pacifica Hybrid Minivan, in particular.
Renault-Nissan Alliance — A partnership to address the future of autonomy and mobility together. The alliance will begin selling vehicles in 2018 with “multiple-lane control,” which can autonomously negotiate hazards and change lanes during highway driving. By 2020, it says it will introduce vehicles that can navigate city intersections and heavy urban traffic without driver intervention. The automaker has released ProPILOT, a self-driving feature that lets cars drive autonomously on highways, in its production vehicles in Japan. Renault-Nissan also plans to roll out ProPILOT in Europe, the US, and China as well. Its ultimate aim is to keep adding autonomous features to ProPILOT until it’s cars are fully self-driving in 2020. Nissan is currently exploring the use of call centers to intervene if its self-driving car can’t handle a certain driving scenario.
PSA — Major European competitor focused on outfitting cars with ZF tech. PSA is the second-largest car manufacturer in Europe. With its three world-renowned brands, Peugeot, DS and Citroën, Groupe PSA, it sold 3 million vehicles worldwide in 2015. As Europe’s second largest car-maker, it generated sales of $63 billion in 2015. The company is planning to have fully driver-less cars on the road in 2020. Four of the automaker’s self-driving cars drove 360 miles between Paris and Bordeaux in France in October 2015. ZF, a German auto part manufacturer, announced in 2016 it will supply cameras, radar, and software for PSA vehicles with self-driving capabilities. Those cars will hit roads in 2018.
Porsche — Afraid to lose brand identity and focused on slowly incorporating autonomous features. The most tangible action Porsche has taken in the auto tech world is the introduction of The Mission E: fully electric car to combat Tesla, but no time-frame for mass production has been mentioned yet. Porsche said it will invest $1B+ to produce the Mission E. They have teamed up with Mazda on creating an autonomous car. Porsche’s CEO, Klaus Zellmer, likened an autonomous driving mode to features like cruise control and lane-keeping assist, and said that it will eventually be standard in every Porsche. “You will be able to press that button and the car will take you home, because our customers also experience traffic situations they don’t enjoy and they want to do something else,” he said. “You have to let the customers choose. We’ll deliver customers the possibility of autonomous driving mode.”
Hyundai — Launching its semi-autonomous features ahead of schedule. Hyundai has been deploying its “Highway Driving Assist 2” (HDA2) semi-autonomous features, like lane-keep assist, in vehicles like the 2016 Elantra. The HDA2 system is properly considered a Level 2 autonomous feature, similar to Tesla’s Autopilot. Hyundai Motor announced this week that it’s going to deploy autonomous vehicles using HD mapping technology at next year’s Winter Olympics at Pyeongchang, South Korea, to shuttle people between Seoul and the Olympic site, a distance of 78 miles. Hyundai plans to have a suite of self-driving features in production vehicles in 2020, but won’t commit to full autonomy until 2030. At CES 2017, Hyundai showcased an autonomous prototype of its Ioniq electric car.
Luminar / Austin Russel — Teaming up with Toyota and backed by Peter Theil. A 22-year old from Stanford and backed by Peter Theil that is replacing the existing LiDAR systems with single laser that can see further then the current systems. LiDARs used in Waymo’s system now can only see one second in front of the car whereas this system can see seven seconds in front of the car providing far more time to react in accordance to moving objects. Luminar’s technology uses a single laser that moves back and forth quickly to detect objects past 200 meters or about 656 feet — which Russell claims is 10 times the range of many lidars out there — with 50 times better resolution. The company launched in April 2017 with $36 million in funding and announced it is working with Toyota Research Institute on the newest version.
Comma.ai / George Hotz — Backed by Andreessen Horowitz ($3.1M) with intentions to open source cars. Probably better known for the amount of legal suits against him for hacking systems like iOS and Playstation’s PS3, George Hotz is the mastermind behind a new technology aiming to replace the hefty, bulky systems of Waymo with a $4,000 turn-key solution to be mounted on any vehicle. Instead of collecting fines for selling his untested Comma.ai systems, Hotz gave it out for free through a product called Openpilot to collect millions of miles more quickly. Whereas other companies are teaching their cars by defining different road conditions and manually labeling driving data — this is a passing lane; this is a stop sign — Comma.ai relies on the patterns and behaviors of everyday drivers to train the models used by Openpilot.
Top-Tier Suppliers:
Mobileye — One of two leading suppliers of hardware. The majority of the hardware technology in cars today is produced by Mobileye. Mobileye is an Israeli company that specializes in vision systems — bought by Intel for $15B in May 2017 — that is used by Tesla Motors, Uber, Fiat Chrysler, BMW, Delphi, and others. The company recently released a statement claiming they have found a way to program responsibility into the car effectively ensuring cars equipped with this software will never cause an accident. This is yet to be tested. According to Intel, the vehicles will combine Mobileye’s “computer vision, sensing, fusion, mapping, and driving policy” with Intel’s “open platforms and expertise in data center and 5G communication technologies to deliver a complete ‘car-to-cloud’ system.”
Velodyne — Only major competitor to Mobileye. Founded in 2005, Velodyne was the leading manufacturer behind the single most expensive hardware, LiDARs, used in Waymo, Ford and Baidu. Although now Waymo has said they will build these LiDAR’s in-house bringing the price to $7,500 versus the current $75,000. Last month, the company announced that it has increased its production capacity by more than 400 percent in order to meet growing global demand. The company is expecting a busy 2018 with orders for fully and semi-autonomous applications leading to expanding production facilities. A high-volume, automated plant that can run “with the lights out” will help drive prices down.
Qualcomm — Well positioned as leader in chip-maker battle for best. One of the more proactive chip-makers in the autonomous world, Qualcomm is contending with competitors like Intel and has plans to open another research center in Seoul in partnership with LG to explore the production and commercialization of 5G networks as it relates to autonomous vehicles. This is a crucial piece of the autonomous car revolution as the tech will let cars talk to each other in real-time and handle much more data than current systems.
Intel — Catapulting into the conversation with recent acquisition of Mobileye. Intel announced in December 2016 that it would invest $250 million in start-ups working on automated-driving technologies. In July it formed a partnership with Mobileye and the German automaker BMW to provide chips for a self-driving car that BMW intends to begin producing by 2021. Intel then bought Mobileye for $15B in August with the intention to define and deliver cloud-to-car solutions for the automotive market segment. Intel i7 processors are used in the Mobileye systems, but Intel plans to release a faster chip this year.
Nvidia — Crucial component in the overall production of autonomous cars. Nvidia makes a processing unit that Audi is putting into its newest models, and another that Tesla has just started using its cars. The Nvidia device used by Tesla, called the Drive PX2, can compute 24 trillion operations a second. Nvidia recently (article dated as of Dec 2016) demonstrated a more powerful version called Xavier. German auto maker ZF is also using the PX2 and plans to put it into fully autonomous cars by 2020.
Samsung — Looking for creative ways to get involved. Samsung is starting a new fund, Samsung Automotive Innovation Fund to invest in a range of connected car areas including sensors, machine vision, artificial intelligence, high-performance computing, cloud services, mobile connectivity, automotive-grade safety and security, the company said. The first investment will be $90 million in TTTech, a Vienna-based firm known for safe, highly reliable network computing systems used in the Boeing Dreamliner, Audi cars including the new Audi A8, and NASA spacecraft, it said. Previous Samsung auto-related investments include AImotive and Renovo for automated driving, Quanergy, for 3D cameras known as LiDAR, and Graphcore, a maker of high-performance artificial intelligence computing.
Baidu — Declining ad revenues highlights importance of new revenue streams, namely autonomous cars. Baidu, has been publicly testing its self-driving-car technology since 2015. The company allowed members of the public to take rides in its fleet of electric, autonomous cars for the first time in November 2016, but the trial only lasted a week. Baidu in September launched a $1.5 billion fund dedicated to autonomous-car development. The Beijing-based company plans to produce a limited number of autonomous vehicles for a shared shuttle service in 2018 and to mass produce self-driving cars in 2021. The company has an autonomous testing permit in California and an office in Sunnyvale. It now has 70 partners across several fields in the auto industry, up from 50 in July. Existing partners include microprocessors firm Nvidia Corp and mapping service TomTom NV.
Apple — Rethinking its strategy and shuffling management around. Whereas Apple originally wanted to design an entirely new car last year they are cutting efforts to produce an entire car and just focusing on the peripheral products that go into it (i.e.. Software, etc.). There were some spats internally between fully autonomous or semi-autonomous and unreal deadlines apparently that tore the team apart. The bout was between Steve Zadesy, an Apple Exec initially in charge of Titan (the project tasked with winning the autonomous car race) — who wanted semi-autonomous vehicles, and John Ive, Apple’s Chief Designer, who wanted to fully re-imagine the automobile all together therefore creating a new car from scratch. Google had a similar bout but the autonomous car won out as they don’t trust people to retake control in emergency situations. Eventually the company tapped Bob Manfield to lead the charge as he’s helped with countless hardware engineering. Main concern for Apple is holding onto to great engineers and that’s warranted as there are more attractive competitors out there.
Selected Historical Transactions (leading up to December 2017):
Delphi / nuTonomy — 2013 MIT spin-out focused on software for cars and robots. nuTonomy, a 2013 MIT spin-out, has also been operating autonomous taxis in Singapore since 2016, and recently received permission to test its self-driving vehicles in Boston. nuTonomy will combine more than 100 employees, including 70 engineers and scientists, to Delphi’s 100-member automated driving team. Delphi will have 60 autonomous cars on the road across three continents by years end. nuTonomy has deals with Lyft, Grab, and Groupe PSA, which owns European car brands: Peugeot SA and Citroën.
LeddarTech — Automotive-grade solid-state LiDARs. The round was led by Osram and included Delphi, Magneti Marelli and Integrated Device Technology as strategic investors. Representing the company’s largest capital raise to date, this round of funding will allow LeddarTech to enhance its ASIC development efforts, expand its R&D team and accelerate LiDAR development programs with select Tier-1 automotive customers for rapid market deployment.
Innoviz — Better, lower price point and smaller LiDAR sensor. Innoviz recently raised $65M in Series B funding, from strategic partners and leading auto industry suppliers Delphi Automotive and Magna International. As top-tier suppliers, both investors want to supply automakers with core autonomous driving components and systems. The new funding will help Innoviz continue to push towards mass production of their LiDAR solution, which uses a solid-state design for greater reliability over time and which also claims better sensing capabilities across different weather conditions, including challenging conditions.
FiveAI — LiDAR system aimed to bring autonomy to auto-OEMs. UK-based FiveAI — a partner in the U.K.’s StreetWise self-driving project — raised two tranches to fill out its plans for a two-part business in the world of self-driving services. FiveAI is building its own autonomous driving system; and second, FiveAI will use that AI-based platform to take on Uber and other transportation services with a fleet of self-driving taxis. Lakestar Capital — the firm founded by prolific investor, Klaus Hommels — led this Series A round.
Oryx Vision — Israeli-based LiDAR manufacturer. Next-Generation automotive LiDAR innovator, Oryx Vision, raised a $50 million Series B funding round. Third Point Ventures and WRV led the round and was joined by Union Tech Ventures, Bessemer Venture Partners, Maniv Mobility and Trucks VC. A mere 15 months after its first funding round, this fundraise brings the total investment in Oryx to $67 million.
Intel / Mobileye — Leading supplier of ADAS software with 25+ automaker partners. The vehicles will combine Mobileye’s “computer vision, sensing, fusion, mapping, and driving policy” with Intel’s “open compute platforms and expertise in data center and 5G communication technologies to deliver a complete ‘car-to-cloud’ system.” Mobileye was also an early partner of Tesla’s for its autonomous technology. Other investments that Intel has made in the space of cars include taking a stake in Here (which will feed into the mapping initiatives at Mobileye); acquiring Itseez and Yogitech for safety and navigation functionalities in autonomous cars; and making a commitment of at least $250 million to the space.
Momenta.ai — The brains / software behind the autonomous system. Only a year old. Led by Nio Capital and Shunwei Capital, Sinovation Ventures, Unity Ventures and Daimler also participated in the round. Momenta.ai is merely a year old, boasts three ex-Microsoft researchers and three PhDs focused on bringing billions of crowd-sourced data to develop ‘market-ready’ sensors and software. The amount and quality of the data is crucial as it expedites production of the software.
Nauto — AI powered camera network connected to the cloud. A Palo Alto company focused on retrofitting existing vehicle fleets with networked safety camera-equipped devices. The round was led by SoftBank and Reid Hoffman at Greylock. The company’s products focus on gathering data about human drivers and their behavior in order to improve safety practices right now, but their platform also has a second, potentially more lucrative purpose: building a huge data set that can prove valuable in the development of self-driving cars — that’s why SoftBank participated. The potential value of this data is a big reason why a number of automakers have also made strategic investments in Nauto, including General Motor Ventures, Toyota AI Ventures and BMW iVentures.
Swift Navigation — High-accuracy GPS for autonomous vehicles. Building “highly-precise, centimeter-accurate” Global Navigation Satellite Solutions (GNSS) at affordable prices and provides solutions to over 2,000 customers — including autonomous vehicles, precision agriculture, unmanned aerial vehicles (UAVs), robotics, maritime, transportation logistics and outdoor hardware applications. By moving the GPS positioning from custom hardware to a flexible software-based receiver, Swift delivers Real Time Kinematics (RTK) GPS that is 100 times more accurate than traditional GPS at a fraction of the cost of the competition. This round was led by NEA and included existing investors Eclipse and First Round. The company has raised $47.6M to date.
Luminar — LiDAR system with 1550 nano-wavelength vs. 905 nano-wavelength. Luminar, in stealth mode until recently, raised $36M in seed funding since its founding in 2012 from Canvas Ventures, GVA Capital, and the Peter Thiel-backed 1517 Fund, among others. The company makes its LiDAR’s from scratch making it more affordable and more accurate. Luminar ditched the conventional silicon chips opting for indium gallium arsenide chips instead (supposed to make the chips better). Luminar has 150 employees, opened a facility in Orlando, has acquired two companies including Open Photonics, whose co-founder, Jason Eichenholz, became Luminar’s CTO and is gearing up to manufacture its first 10,000-unit run of its latest device.
Autotalks — V2X (“Vehicle-to-Everything”) communication solutions expanding globally. Chipsets addressing upcoming regulation, with superior communication performance, strongest Cybersecurity, highest integration level, and many competitive features. Investors include: Magma Venture, Mitsui & Co., Liberty Media, Delek Motors, Fraser McCombs and Samsung Catalyst Fund. The funding round came at the heels of a USDOT issued NPRM (Notice of Proposed Rulemaking) that, in an effort to increase road safety, will mandate DSRC (Dedicated Short Range Communication) based V2V in all new light vehicles sold in the US by 2023.
Ford / Argo AI — Argo AI is an Artificial Intelligence company. Ford invested $1 billion in a joint venture with Argo AI, a Pittsburgh-based company with ties to Carnegie Mellon. The goal is to completely outfit Ford vehicles with level 4 self-driving technology. This is the creation of an entirely separate company with an independent equity structure. Ford is the “majority stakeholder,” but will operate with “substantial independence.” Employees will receive equity in the company.
Uber / Otto — This is the acquisition that resulted in legal action from Google. Otto has been focusing on self-driving technology that could be fitted into trucks that are already on the road now. This fits perfectly into Uber’s strategy as the company doesn’t want to become a car manufacturer. Instead, Uber has been looking at partnerships with existing car manufacturers, such as Volvo, in order to turn their cars into self-driving cars using Uber’s proprietary technology. Uber started to test self driving semi-trucks in 2016 when a truck using advanced technologies drove 120 highway miles along a specific highway route with Budweiser, marking the world’s first commercial shipment by self-driving truck and plans to continue testing semi-trucks.
GM / Cruise Automation — Cruise is the main competitor to Waymo. Cruise created a kit that allows buyers to convert certain types of cars — namely Audi A4 and S4 models — into autonomous vehicles for the highway and GM wants to incorporate the tech into their manufacturing process. GM has also invested $500m in Lyft and launched a new initiative called Maven aimed at taking on ride sharing companies like Uber. The GM-Cruise deal has led to the rollout of a fully integrated autonomous electric car — the tech is being built into Chevy Bolt EVs — that’s being tested in San Francisco, Detroit, and Phoenix, with NYC to follow in 2018.
Delphi / Ottomatika — Ottomatika is the brains behind Delphi. Ottomatika, a CMU spinoff provides software and systems development for self-driving vehicles. Ottomatika’s software is the brain powering Delphi’s advanced network of sensor technology for autonomous vehicles. The combined software from Delphi and Ottomatika enabled the longest drive by an automated vehicle in North America in April 2015. The Delphi vehicle completed a 3,400-mile trip from San Francisco to New York in autonomous mode 99% of the time in which the vehicle drove through construction zones, and adverse traffic and weather.
How Well Are Autonomous Vehicles Currently Performing?
California DMV report show signs of major improvements since last year. The California DMV recently released its annual stack of “disengagement reports” documenting the progress of the 11 companies warranted access to test autonomous vehicles on California’s public roads. Although the numbers don’t paint the entire picture, they do provide good insight into progress. The report covers December 2015 through November 2016 and shows how often drivers must intervene with autonomous vehicles. Google and General Motors are leading the class with cars capable of driving hundreds of miles at a stretch without trouble. But even those in the back of the pack are showing signs of improvements. Nissan’s robocars, for example, needed human intervention once every 247 miles, compared to once every 14 miles in 2015. Cruise, the startup leading GM’s autonomous driving efforts, did all its testing in San Francisco, where it ramped up from five miles in June 2015, to 400 in June 2016. By late last year, it was clocking hundreds of miles without a hitch. Most of Delphi’s trouble came while changing lanes in heavy traffic. Ford’s two autonomous cars in California only drive on the highway, during the day, with fine weather and road conditions, which explains why it only needed human help three times in 590 miles.
How is the AV Market Doing Compared to the Broader Auto-Tech Market? In 2016 and 2017 YTD, deal activity to semi-autonomous and autonomous technology eclipsed all other segments of auto tech combined.
Autonomous Vehicle (AV) Tech vs. Other Auto Tech — This year’s largest deals have been to companies spanning the AV ecosystem (i.e. sensors, vision, 3D mapping, etc.). A sustained flow of capital will likely be needed as players in these resource-intensive spaces hope to move closer to market. Zoox, the stealthy AV startup aiming to reinvent the concept of a vehicle itself, is said to be in talks with SoftBank for an investment of up to $1 billion. Prior to the takeoff of AV technology, investments had centered around connected car and fleet telematics startups. The latter field saw several large exits last year, with Verizon acquiring both FleetMatics and Telogis.
Auto Tech Global Annual Deal Share by Stage — Granted there is still time in 2017 (i.e. the data isn’t perfect), 2017 YTD is the first year where seed activity represented less than a third of deals. Auto tech is beginning to mature with startups seeded in years prior now receiving mid- and later-stage investments. Also, auto tech continues to draw a much deeper pool of backers, as the number of unique investors has jumped every year since 2012. The sharp rise in corporate and CVC investors reflects the entrance of auto OEMs, suppliers, and semiconductor and aerospace sectors. This shows the far-reaching effects of the AV tech as that segment continues to outpace traditional auto tech.
Notable Investors on the Autonomy and Mobility Space. Many of these investors have a strategic component and a narrow focus.
Other Applications of Autonomous Technology
Vertical Takeoff and Landing (“VTOL”). Although often associated with recreational drones, VTOL encompasses flying cars too. For example, Airbus launched project A3 in early 2016 in order to “open up urban airways by developing the first certified electric, self-piloted VTOL passenger aircraft.” Airbus anticipates “speeds will be 2–4x faster than cars or traffic, and have a flight range of about 50 miles (80 km).” The major hurdle for Airbus is not takeoff or flight, but rather it’s the ability to properly land in challenging circumstances. Many of the systems installed on the aircraft are the same as those used on autonomous vehicles: LiDAR, sensors, cameras, mapping / GPS, etc. but also things like inertial measurement instruments and perhaps more processing power than one would find in Waymo’s tech, for example. Uber recently announced its partnership with Aurora Flight Sciences, deemed “Uber Elevate”, to develop eVTOL (electric VTOL aircrafts). Aurora’s eVTOL concept is founded on its XV-24A X-plane program for the U.S. Department of Defense and other autonomous aircraft the company has developed. The partnership is to enable urban transportation solutions for the masses via the “Uber Elevate Network”. The first test flight of the aircraft was successful earlier this year and the goal is to deliver 50 aircraft for testing by 2020. Drones –non-passenger ones — have near-term commercial demand in a variety of use cases in media/entertainment, journalism, agriculture, real estate and public safety creating demand for “drones-as-a-service”. In this space, there are other types of opportunities such as maintenance, software, communication chips, insurance, entertainment and better sensors — to name a few. Security is another important area of the drone market. Fresh off a successful $11.5m Series A led by Andreesen Horowitz, Skysafe’s radio wave technology can detect and stop rogue drones from entering unauthorized areas like military bases, stadiums, prisons and airports.
Last Mile Commerce. The “last mile problem” has long been a thorn in the side of logistics providers, transportation companies, and retailers alike. Compared to the main legs of bulk shipping, train, truck, or aircraft transport, the final leg (or last mile) from logistics hubs to individual homes and offices has traditionally incurred the highest cost and complexity. Last mile challenges have only grown as the proliferation of online shopping strains capacity. Infrastructure failings also complicate last-mile delivery in developing markets and disaster-stricken areas. All of these startups are conceptually similar, building autonomous drones to drive down the cost and difficulty of last mile delivery. However, they are pursuing wholly different approaches in aerial (Flirtey and Matternet) versus ground-based solutions (Starship, Dispatch, Marble, and Robby). Aerial drones must contend with air-traffic regulation and safety concerns, while ground-based vehicles are more susceptible to vandalism and theft. Starship Technologies, a European technology start-up building a fleet of self-driving delivery robots, recently raised $17.2M in a Series A round from investors including Daimler, Shasta Ventures, Matrix Partners and others. The technology is designed to deliver goods locally within 30 minutes using ‘off the shelf’ components. The robots are lightweight and low-cost, enabling the company to bring the current cost of delivery down by 10–15 times per shipment.
Warehouse and Manufacturing. More companies are applying autonomous / self-driving technology to existing systems in warehouses (i.e. pay for the software not the wages and get better, more accurate results). For example, Brain Corp — a San Diego based software developer aimed at outfitting existing floor robots with proprietary software — that just raised $114M Series C led by SoftBank’s Vision Fund. Qualcomm Ventures also participated in the round. The $114 million will be used to develop AI technology for multiple types of commercial and consumer robots. BrainOS is the company’s proprietary operating system that integrates with off-the-shelf hardware and sensors to provide a cost-effective “brain” for robots. It’s essentially Android OS for robots. It has computer vision and A.I. that enable quick and efficient development of smart systems that learn and adapt to people and environment. Its technology provides advanced self-driving capabilities for cluttered and dynamic indoor environments.
Construction and Agriculture. Similar to the use-case for warehouse robots, the construction industry is seeing real impact from robots like the team of automated bulldozers helping the Japanese government prepare for the 2020 Olympics. In order to compensate for the lack of construction workers (partly stemming from stringent internationally labor laws) and rapidly approaching deadline, the team is using automated bulldozers guided by drones in the sky that map out the route and can measure the density of the terrain better then the bulldozers themselves. The drones are provided by Skycatch, which builds technology to autonomously capture, process, and analyze 3D drone data. According to CNBC, Goldman Sachs predicts farm technologies could become a $240 billion market opportunity for ag suppliers, with smaller driverless tractors a $45 billion market on its own. Goldman also predicts tens of billions could be spent on advanced tech for major farm uses such as precision fertilizer, planting, spraying and irrigation.
Smart Cities. The National Association of City Transportation Officials (NACTO) recently released a 60 page report on how the government entity envisions the future of smart cities, deemed “autonomous urbanism.” The organization is aiming to proactively plan for the autonomous future in order to better manage traffic, use less to do more, and provide mobility centered around people not cars. For example, if there were no median or traffic light as cars and the environment can communicate with one another there would be more space for commercial and residential purposes. Although NACTO is planning for the future, Google and it’s subsidiary, Sidewalk Labs, are building the future of cities. As mentioned previously, the tech giant will relocate the Toronto offices to the Quayside where Sidewalk Labs is set on redeveloping a 12-acre plot of land into a smart city. If that goes according to plan, Google has access to another 800-acres to do the same.
Mesh Networks. Car-to-Car (CxC) and Car-to-Everything (CxV) Communication Technology. In line with smart cities, a wireless mesh network (WMN) is a local network in which infrastructure nodes connect directly, dynamically and non-hierarchically to as many other nodes as possible and efficiently cooperate with one another to send data to / from clients in the network. Wireless mesh networks consist of clients, routers and gateways. Mesh nodes are often static objects in the infrastructure around the vehicle (i.e. routers), while mesh clients can be laptops, cell phones and any other wireless device. The gateways do not need to be connected to the internet and pass information through the network via the nodes and clients. The area of coverage resulting from the connection of nodes, clients and gateways is referred to as a mesh cloud and similar to a cloud, a mesh network is fluid. Meaning, if a node cannot secure a connection, the rest of the nodes can still communicate directly or indirectly through intermediate nodes. Wireless mesh networks work with different wireless technologies including 802.11, 802.15, 802.16, cellular technologies and do not need to be restricted to one technology or protocol. A company and industry leader currently testing this technology is Veniam. Veniam raised a Series B funding round of $22 million in February 2016 led by Verizon Ventures. Union Square Ventures, True Ventures, Cisco Ventures, Liberty Venture and others participated in the round. The company is aiming to build city-scale networks of connected vehicles that expand wireless coverage and bring terabytes of physical data to the cloud. With 80 patents in place, 780k unique users to date, 17 million internet sessions and 43 million connected kilometers, Veniam‘s systems can cover an entire city in seamless Wi-Fi coverage and brought the first car-based mesh network to New York City. Already, a pilot program in Portugal, Veniam is serving 110,000 people a month.
How Does It Work? According to Veniam.
- Broadband Connection: Veniam leverages existing internet infrastructure by adding supercharged wireless routers throughout the city. Since the routers broadcast on a frequency reserved for transportation systems, they can cover a wider range (~1,600) than a private network would have.
- Mobile Hot Spots: Buses, cabs, garbage trucks, police cars, etc. are outfitted with Veniam’s proprietary NetRider routers and these routers receive wireless signals from the access points, creating nimble hot spots on the go in order to fill in the gaps between stationary routers.
- Mesh Networks: One major benefit of the mesh network is its ability to share connections with vehicles that cannot connect directly to a stationary router. Meaning, a vehicle without a secure signal from a stationary router can piggyback on the connection of another vehicle within range. This is how mesh networks currently span major cities like Singapore and Porto, Portugal — they are all on the same network.
Main Hurdles to The Adoption of Autonomous Technology.
- Computer Software and Hardware. Self-driving cars have to be better than humans. They not only have to see what is going on around them, but they must use that data to make better choices for EVERYTHING around them. The systems in an autonomous car must be robust and flexible. Unlike your laptop where you can reboot after a crash, the car and safety of those in the car is liable too. This is where the majority of capital is going — to make the technology better.
- Mapping and Navigation. Google embeds existing 3D maps of the environment into the car’s computers to free up computing space for the slew of radars and sensors pick up other vehicles and moving objects. However, cityscapes and the natural environment around cars is subject to change and continually updating maps will be cumbersome and costly. As the technology evolves these systems will become more agile and adjust real-time. Google plans to periodically update a central database as new cars are released to the fleet, while Tesla plans to lean on imaging and real-time processing to avoid collisions.
- Better, Affordable Technology. Like many cutting edge technologies, it takes time to lower the cost of the systems to a point where consumers will buy the tech. Some sensors used in autonomous cars — LiDAR, radars and lasers — like Veodyne’s LiDAR systems currently cost $75,000 per unit (as of now each car needs one). This device is the single most expensive piece of hardware on the vehicle by far, but Google reportedly found away to reduce the cost to 1/10th of that or ~$7,500. Advancements like this are necessary to make the technology an option to consumers and automakers. This will be a moving target — like the rest of the ‘hurdles’ on this page — but startups and multi-national automakers are racing to solve the problem.
- Car-to-Everything / Mesh Network Communications. Many industry leaders, startups, tech companies, etc. believe one major way to reduce crashes is to make cars communicate effectively with the world around it. Car-to-car communication lets cars broadcast their position, speed, steering-wheel position, brake status and other data to other vehicles within a few hundred meters. The other cars can use such information to build a detailed picture of what’s unfolding around them, revealing trouble that even the most careful and alert driver, or the best sensor system, would miss or fail to anticipate and avoid.
- Safety. Computers do not get fatigued, drunk or distracted, but humans do. Under mounting pressure, the technology will have to prove it is safer then human drivers before it is adopted by users and law makers. Also, driving requires complex social-interactions — eye contact, subtle gestures, etc. — which are still tough for robots to comprehend.
- User Acceptance. One of the biggest barriers to the widespread acceptance of self-driving cars will be whether customers choose to use and coexist with the technology (i.e. as pedestrians, cyclists, etc.). For some, including those with disabilities, the technology represents mobility in the purest form, while for others it is a threat to their freedom. The use-case is not the same for everyone — just because I want an autonomous car doesn’t mean others do too. Some people enjoy driving and would prefer the experience of learning stick shift, for example. The way companies choose to introduce the technology will also impact the way other customers pay for technology (i.e. the high initial cost of autonomous ride hailing services may increase prices for everyone).
- Legal Issues. Once the technology is in place, companies need to determine who is responsible if a self-driving car gets in an accident. Is it the “driver,” is it the company that built the car, or perhaps the operator of the ride sharing service? It’s a legal dilemma that will have to be addressed before private ownership of autonomous cars becomes commonplace. Will you need a driver’s license to operate a self-driving car? There’s not only the cost of repairing damages to consider, but also civil or criminal liability if someone is injured or killed. Volvo has announced that it will accept liability for collisions when drivers are using the IntelliSafe Autopilot system that is expected in 2020. There are also state and federal laws that regulate the testing and operation of autonomous forms of transportation. That patchwork of differing rules hampers the testing of the technology.
- Ethics. When a collision is unavoidable, should a self-driving car protect those outside the car or inside the car? Humans make this decision in tough times and computers will be expected to do the same. However, with self-driving cars, these decisions will need to be hard-coded into the software’s ‘DNA’ and the car will be expected to make the right decision every time. This is referred to as “algorithmic morality”. Will customers put themselves in cars that are programmed to save the most lives, which in some cases may favor those outside the car more than those inside the car?
Recent Updates:
- Here’s an important (and entertaining) review by Rachel Metz highlighting some of the current autonomous vehicles showcased at CES.
- Apparently GM is one step away from launching a ride-sharing service at commercial scale in 2019. That one step? Approval from the DOT to produce cars without steering wheels or pedals.
- Rebounding from an early disappointment in the Fisker Karma debuted in 2012, Fisker released the new EMotion at CES with the confidence that it’s partnership with Quanergy’s LiDAR sensors will execute level 4 autonomy.
- Mercedes-Benz finally revealed their Smart Vision EQ Concept at CES after first showing it off at the Frankfurt Motor Show and Tokyo last year.
- Ford (as well as most of the big car companies) is focusing on monetizing autonomous cars with existing partnerships such as Domino’s and Postmates.
There’re likely more updates to follow as CES wrapped up last week.
Additions and thoughts are welcome!
This story is published in The Startup, Medium’s largest entrepreneurship publication followed by 288,884+ people.
Subscribe to receive our top stories here.
