Creating a Simulation Environment for Autonomy Verification and Validation for Unmanned Surface Vessels (USV).
Celeste Wong, Chew Yihang and Wong Yuhin created a simulation environment to verify and validate autonomous behaviours on Unmanned Surface Vessels (USV). The team looked in to employing the Virtual RobotX (VRX) environment and evaluated three different methods for simulation. The preferred setup used Robot Operating System 2 (ROS 2) and Gazebo Sim within a Virtual Machine (VM), achieving a Real Time Factor (RTF) between 7.0–9.0%. Additionally, configurations for USV teleoperation are outlined. The team was mentored by Chng Zhen Hao, Head Capability Development (Unmanned Maritime Systems) from the Naval Systems Programme Centre.
Introduction
Verification and validation (V&V) of autonomy in unmanned surface vessels (USVs) traditionally occur after construction, involving extensive and costly testing over long durations. By integrating simulation environments early in the project phase, this paper illustrates how virtual testing can expedite the development process, detect issues earlier, and reduce costs by reducing the costs involved in testing physical prototypes.
The Virtual RobotX (VRX) Environment
Developed in collaboration with Open Robotics and the Naval Postgraduate School, the VRX environment is a Gazebo-based simulation designed for developing and testing USVs. It supports dynamic oceanic conditions and is utilized in the VRX competition, which challenges participants to creatively address navigational problems within this simulated setting.
Methods for Setting Up the VRX Environment
Three approaches to configuring the VRX environment are presented:
- Base Gazebo: This method involves manually adding models to a basic Gazebo environment. It’s straightforward but can be labor-intensive and does not integrate with ROS 2.
- ROS 2 Humble and Gazebo Garden in a VM: This setup is recommended for its balance of ease of use and comprehensive support through the VRX competition’s resources. It requires installation of specific packages and versions but offers a complete simulation setup.
- ROS 2 Humble and Gazebo Garden in a Docker container: Similar to the VM method but more resource-efficient as it operates directly on the host OS. This method simplifies package management though it can introduce complexities with Docker’s configuration.
Evaluation of the Results
Each method was implemented and evaluated for its effectiveness in simulating a functional USV environment. The qualitative measure we used was the ability to simulate collisions and support simulator plugins. The quantitative measure we used was the Real Time Factor (RTF).
The RTF is a metric defined as the ratio of simulated time to real time. An RTF greater than 1 indicates that the simulation runs faster than real-time, suggesting efficient computational performance or a simplified model that requires less processing power. Conversely, an RTF less than 1 means the simulation is slower than real-time, which may occur due to the hardware’s computational limitations or because the simulation involves complex processes like fluid dynamics that demand more intensive calculations.
The VM method using ROS 2 and Gazebo Garden emerged as the most practical due to its simpler setup and adequate performance, despite a lower than desired RTF.
Teleoperation of the Unmanned Surface Vehicle
With the simulation environment established, the USV was configured for teleoperation using a gamepad. The process was straightforward, although the low RTF resulted in delayed responses in the USV’s movements, highlighting the need for further optimization.
With the final simulation environment set up, teleoperation of the USV was configured. To configure teleoperation, the ROS 2 Humble teleoperation dependencies have to be installed and launched. The process of setting up teleoperation is simple and movement of the USV can be controlled through the joysticks of a gamepad. The USV was able to teleoperate however, due to the low RTF of the simulation, the movement of the USV was extremely delayed. The position of the USV shown in the first of two images below. USV’s commences it’s journey from the starting point and it is then moved toward the buoy (circled in red). When the USV first made contact with the buoy, Fig. 4 was taken, the photographs were taken 88s apart, showing that the USV takes a significant amount of time to move from point to point.
Future Improvements
Future work could enhance the Base Gazebo method to include collision elements, potentially increasing the RTF and enabling accelerated testing. Additionally, strategies to improve the overall RTF without sacrificing environmental detail would benefit the realism and utility of the simulation.
Conclusion
The implementation of the VRX environment through various methods demonstrates the feasibility of using simulation for the V&V of autonomous systems. While the current setup offers a functional testing environment, optimization is necessary to fully realize the benefits of accelerated testing and enhance the fidelity and responsiveness of the simulation.
Game for a similar challenge? Step into your future
Excited by what you’ve read? There could be a thinker and tinkerer in you that seeks a greater challenge. Learning never stops — chart your next adventure, and push the envelope in defence tech with us through the Young Defence Scientist Programme.
Slide into our DMs here to fuel your passion for science and technology and be mentored by Singapore’s top engineers and developers.
References
- Araujo, Hugo, et al. “Testing, Validation, and Verification of Robotic and Autonomous Systems: A Systematic Review.” ACM Transactions on Software Engineering & Methodology, vol. 32, no. 2, 2023, doi:10.1145/3542945.
- Bingham, B., et al. “Toward Maritime Robotic Simulation in Gazebo.” OCEANS 2019 MTS/IEEE SEATTLE, 2019, doi:10.23919/OCEANS40490.2019.8962724.
- Koenig, N., and Howard, A. “Design and use paradigms for Gazebo, an open-source multi-robot simulator.” 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2004, doi:10.1109/IROS.2004.1389727.
- Yadav, Anuj Kumar, et al. “Docker Containers Versus Virtual Machine-Based Virtualization.” Advances in Intelligent Systems and Computing, 2018.
