Hello AirSim’s World
Three years ago, Microsoft released one of its first open-source projects. AirSim, drones, and ground vehicles simulator to accelerate autonomous vehicle research. Since then, AirSim’s popularity overgrew with many opensource projects in AI, deep learning, computer vision, and reinforcement learning. The simulator can be used as a simulation environment with different available Binariess each simulates a diverse, visually realistic environment. It can also be used as a hardware-in-loop with popular flight controllers such as PX4.
AirSim supports many sensors like Cameras, Lidars, GPS, IMU, Magnetometer, Barometer, and Distance sensor. And as you can see from the video above, you can get Depth and segmented images from the cameras. This Makes AirSim an excellent tool for data collecting.
AirSim exposes C++ and python APIs so you can interact with the vehicle in the simulation programmatically. You can use these APIs to collect sensors data, control the car, and control the weather.
AirSim Block:
Let’s get our hands dirty and start building an AirSim Block. But if you want to test it without making a new one, you can grab it from YonoStore.
This Block is a base block for AirSim Simulator with no input ports. You can add Your input ports and handle the inputs throughon_new_messages(self, messages) of your Block. or add
AirSimm Environment:
Before building our Block, we should develop its environment and install all the required dependencies. So let’s open YonoEBuilder and click on “Create Environment.”
From the Requirments, tab makes sure to choose “Ubuntu 18.04, GPU with Accelerated Rendering, Python 3 (x86_64)” this base image uses cudagl to add OpenGL support through libglvnd. Because AirSim requires OpenGL or Vulkan installed in the system.
Then we can install AirSim from the GitHub repo, Airsim package from pip, and other dependencies.
AirSim Block Code:
The Full Code for my Block can be found here
We use the AirSim Python Client to handle all the communication with AirSim. But before we use it, we should have an AirSim Binary running.
But how can we achieve this? Let’s look atdownload_run_airsim(self)function
We get the latest AirSim Binaries from Github, unzip it, and run it as non-root users, which we will create later.
The two environment variables we set at the first two lines of the function are to ensure AirSim will run in headless mode.
We also use two properties in our block for the simulation configuration — one for the AirSim environment selection and the other for the AirSim settings.json file.
This function is called in our Block on_start(self) function.
Now we can use AirSim Client to handle the communication between our Block and AirSim.
First, we create a non-root user, which we used in the previous function to run AirSim. We can use this user also to connect to the Block using SSH.
Starting from line #12, we check the simulation vehicle type (Car or Multirotor) then we can create the appropriate client type, and poll it till AirSim simulation is running.
Then we define some variables to control the flow of our Block.
I have created a textbox property for the user to enter commands to the vehicle. In the case of Car, these commands are Throttle, Steering angle, timeout_sec. If a multirotor is used instead, these commands are x, y, z, Velocity, timeout_sec.
This is handled inside on_button_clickedfunction of the block.
Next, we are going to code the main loop of the block. The run(self) function. We create an infinite loop and a timer to handle the frequency of reading/publishing data. We also check if the user has clicked no “Execute Control” Button and handle the sent command. Then it’s time to read some data from the simulation and publish it on our Block’s ports. Using self.client.SimGetImages() we can read images from several cameras defined in the vehicle. This function returns a compressed png as an array of bytes. We can convert into a NumPy array, then ROS message, and publish it to our ports.
AirSim client has many APIs to read from different sensors. You can check them from here. The next API we are going to use is self.client.getLidarData() to get the Lidar PointCloud and publish it as sensor_msgs/PointCloud2, I’ve created a helper function to do the conversion in the code above parse_lidardata(lidardata).
This all for our Block. But feel free to add more sensors, read them, and publish to your Block’s outputs ports.
That’s It, Enjoy Your AirSim Block and don’t forget to share with us the great pipelines you made.
About YonoHub:
Yonohub is a web-based cloud system for development, evaluation, integration, and deployment of complex systems, including Artificial Intelligence, Autonomous Driving, and Robotics. Yonohub features a drag-and-drop tool to build complex systems, a marketplace to share and monetize blocks, a builder for custom development environments, and much more. YonoHub can be deployed on-premises and on-cloud.
Get $25 free credits when you sign up now. For researchers and labs, contact us to learn more about Yonohub sponsorship options. Yonohub: A Cloud Collaboration Platform for Autonomous Vehicles, Robotics, and AI Development. www.yonohub.com
If you liked this article, please consider following us on Twitter at @yonohub, email us directly or find us on LinkedIn. I’d love to hear from you if I can help you or your team with how to use YonoHub.
