Key Takeaways from IROS 2023: Insights into the Future of Robotics

Principal Engineer (Autonomy), Tan Je Hon, distills key ideas from the International Conference on Intelligent Robots and Systems (IROS) 2023, which showcased the latest advancements in robotics, covering key themes like multi-agent systems, machine learning, perception, legged locomotion.

Teaming Paradigms

One of the pivotal discussions at IROS 2023 centered around the various paradigms for teaming in robotics. As the field continues to expand, the need for effective coordination among multiple autonomous agents becomes increasingly apparent. Here are the three primary paradigms discussed:

1. Pre-coordination: In this approach, agents follow scripted behavior and operate autonomously but under pre-defined coordination. While this method is straightforward to implement, it may lack the adaptability required for complex scenarios.
2. Decentralized Coordination: This paradigm involves training agents together and enabling them to coordinate autonomously during execution. It offers greater adaptability in a wide range of scenarios, allowing for dynamic responses to changing conditions.
3. Ad-hoc Teaming: Ad-hoc teaming takes a different route by independently training agents, providing high flexibility. However, this flexibility may come at the cost of suboptimal performance in certain situations.

Photo by Mulyadi on Unsplash

Multi-Agent Reinforcement Learning (MARL) Strategies

MARL strategies were a focal point of discussion, given their pivotal role in enabling effective multi-agent coordination. Here are the key MARL strategies discussed at IROS 2023:

1. Fully Centralized MARL: In this strategy, all agents are trained and executed as a single entity, simplifying implementation. However, it necessitates centralized control, which may not always be feasible.
2. Centralized Training, Decentralized Execution (CTDE): Agents train together but operate independently during execution, coordinating as needed. This strikes a balance between centralized and decentralized approaches.
3. Decentralized Training and Coordination: This strategy involves separate training with coordination mechanisms, allowing for both autonomous operation and effective teamwork.
4. Fully Decentralized: Agents are trained separately without coordination and align only during execution. This approach offers robustness in diverse scenarios but requires sophisticated coordination mechanisms.

Photo by Craig Sybert on Unsplash

Modularized Reinforcement Learning

Emerging from Google DeepMind’s research in robot soccer, modularized reinforcement learning was a topic of great interest. This approach entails training individual skills before combining them to achieve complex behaviors. It addresses the limitations of end-to-end reinforcement learning by enhancing adaptability and training efficiency.

Distributed Perception and Planning

Innovations in distributed perception and planning are simplifying and scaling distributed consensus algorithms. Approaches, such as Gaussian belief propagation and distributed neural network optimization, are gaining traction for their applicability to various tasks and their ability to maintain data privacy.

Verifiable Safety and Trustworthiness

Ensuring the safety and trustworthiness of multi-agent systems were key concerns at IROS 2023. Researchers are focusing on “correct-by-construction” algorithms that employ formal methods to guarantee safe outcomes in multi-agent environments.

Conference presentation highlights

1. DiNNO: Distributed Neural Network Optimization for Multi-Robot Collaborative Learning — A distributed algorithm for the collaborative optimization of a neural network by multiple robots over a mesh network.

2. Distributing Collaborative Multi-Robot Planning with Gaussian Belief Propagation — A novel high-performance method of distributed planning using Gaussian Belief Propagation. The same technique is applicable to distributed state estimation.

3. Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning — Findings from a long-running effort at DeepMind on robot soccer. Demonstrates the strength of modularised, skills-based RL.

4. Aerial Swarm Defense using Interception and Herding Strategies — A drone-on-drone aerial defence system using a non-learning, heuristics and optimization-based approach that produce deterministic performance with theoretical guarantees.

5. Semantics-Aware Mission Adaptation for Autonomous Exploration in Urban Environments — Semantic cues in from perception trigger task switching to enable more efficient, targeted exploration.

6. Risk-Tolerant Task Allocation and Scheduling in Heterogeneous Teams — Method of task allocation for robot team of heterogeneous capability that can account for uncertainties and guarantee probability of failure is below user-specified threshold.

7. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control — Transformer-based vision-language model trained on internet-scale data, fine-tuned to directly output robot action tokens, enabling direct end-to-end robot control. Focuses on object manipulation using arm with gripper end-effector. Demonstrates surprising level of generalization to new objects and robot types, and signs of improvisation. Paves the way for potential foundational models in robotics.