End-to-End Models’ Impact On Robotic Eng’s Career

Most of you are likely familiar with Tesla’s Version 12 software, which famously slashed the onboard code from 300,000 lines to just 2,000. While Tesla’s advancements are well-known, companies like Wayve and 1x are also pioneering similar advancements in the automotive and humanoid robotics sectors, signaling a seismic shift — Robotics 2.0 — that is poised to redefine the careers of robotics engineers dramatically.

Originally, the robotics framework was a product of academic development in the early 2000s, comprising discrete modules for perception, localization, planning, and control. This approach was first showcased in the DARPA self-driving challenge, leading to the founding of Waymo in 2008, and soon became the industry standard.

Today, we’re witnessing a paradigm shift to large, integrated models that amalgamate these once-separate functions into a single, robust system capable of synchronizing information fusion, prediction, and planning — although control remains somewhat segmented. This integration is rendering traditional roles like localization redundant, as vehicles now rely primarily on GPS for navigation, avoiding obstacles based on sensor data rather than precise location.

Despite concerns that AI might never fully achieve Level 4 autonomy due to unpredictability, the success of Tesla’s Version 12 demonstrates that with substantial data, AI can achieve remarkable results. These models are advancing rapidly, and at the end of the day, a statistically better product is a better product.

This shift presents a crucial question for robotics engineers: What will our future roles look like in this new landscape? While speculative, it is likely that many engineers including myself will find themselves in “secondary” or “supportive” roles unless they are directly involved in designing and refining AI models. The necessity for vast data sets to be labeled and fed into training pipelines underscores a burgeoning demand for skills in machine learning and deep learning, particularly off-board tasks aimed at identifying and tagging critical scenarios for training.

Simulations, too, are set to play an indispensable role, especially for modeling edge cases that are impractical to capture in real-world testing. Training robust AI models also requires a deep understanding of the physical world, potentially revitalizing interest in SLAM-related techniques for generating accurate “ground-truth” data.

In addition, new roles are created to help better understand how these “black box” models work. One example of such, is achieved by connecting the model to a Large Language Model (LLM), so the model can present its reasoning in natural language.

I believe, in the immediate future, there will be an increased demand for skills related to processing and analyzing camera data — a trend driven by the focus on vision-only systems by industry leaders like Tesla, Wayve, and 1x. These systems benefit from the lower cost and relative maturity of computer vision technologies, facilitating easier scaling and data accumulation. Although they don’t enjoy the benefit of redundant sensor suites, they do enjoy scalable data flywheels — and they are spinning fast.

To summarize, I’ll quote a statement on company 1x’s website:

… They (engineers) represent a new generation of “Software 2.0 Engineers’’ who express robot capabilities through data instead of writing code

I encourage my colleagues in the robotics field to consider how the landscape of our careers might transform over the next decade.