Real2Sim
Autonomous robots have long been a mainstay of science fiction, but the problem has proven to be remarkably hostile to AI systems.
A big part of the problem is the ambiguity and variation in perceiving the real world for manipulation tasks. To that end, AI systems can perform much better in simulated environments, and a lot of work has gone into transferring promising results from simulations to real world environments, so-called “Sim2Real”.
However, this line of research has been challenging, see for example, the comic below:
The joke here is that it’s easier to try and make reality look like simulations rather than it is to try and get better at transferring simulations to reality.
However, I will argue that recent advancements in LIDAR technology will enable the joke of “Real2Sim” to become real and integral to robotics progress.
Historically, LIDAR has been used for robotics applications such as autonomous vehicles and general robotic manipulation.
For example:
It’s pretty obvious that the clump of dots that are read out by today’s LIDAR systems are nowhere near what is necessary to be useful for solving the perception stack fully, and is nowhere near “Real2Sim”.
However, advancements in LIDAR systems, driven by perceived demand from automotive has resulted in short-range LIDAR that is qualitatively different from the LIDAR of old.
Short range LIDAR has improved tremendously, leveraging advances in flash LIDAR technology to achieve much high resolutions than previously possible.
Such high resolution LIDAR systems will result in robots being able to perceive the world in a new way, more akin to viewing a simulation than reality, hence “Real2Sim”. Just as MEMS sensors were the “peace dividends of the smartphone wars” that enabled quadcopters, so too will LIDAR enable many robotics applications to become a reality.
Finally, looking more towards the future, newer LIDAR systems are smaller than ever, and despite the packaging expensive of silicon photonics, integrated LIDAR systems will be cheap and small enough to embed everywhere, even on the “fingertips” and other surfaces of robotic manipulators, enabling something even more powerful than a sense of touch.
