Learning Soft Robotic Assembly Tasks with a Handful of Trials

We would like to introduce our project about learning assembly tasks with soft robots, which has been accepted to IEEE International Conference on Robotics and Automation 2020 (ICRA 2020).

Masashi Hamaya, Robert Lee, Kazutoshi Tanaka, Felix von Drigalski, Chisato Nakashima, Yoshiya Shibata, and Yoshihisa Ijiri, “Learning Robotic Assembly Tasks with Lower Dimensional Systems by Leveraging Physical Softness and Environmental Constraints,” accepted to ICRA 2020 [paper]

This paper was written with the supports of our intern, Robert Lee, a Ph.D. student at the Australian Center for Robotic Vision at Queensland University of Technology in Brisbane, Australia.

Overview

Robotic manipulation for assembly is essential for industrial applications. Although many researchers have explored control methods, these tasks remain challenging due to low tolerance and complex models such as contacts and jamming between the assembly parts. Typical approaches have developed position and force controllers using rigid robots. The position controllers might generate large forces with small position errors. Moreover, the force controllers would require high-frequency controllers or precise force/torque sensors, therefore, they largely depend on their hardware performance. If the robots could more easily interact with the environment without such controllers or sensors, the robot system would be much simpler.

To this end, we use physically soft robots (including compliant components such as springs and silicon) for the assembly tasks because the softness allows the robot to contact the environments safer without the need for significant engineering effort. Meanwhile, designing the controllers manually is difficult due to the model complexity derived from the soft components. One promising approach is to use certain data-driven approaches. However, if we directly apply such approaches to the robots, we would be required to collect a large amount of real robot data to learn the complex dynamics in such non-linear and high dimensional systems.

To deal with this problem, we propose a novel soft robotic control framework for assembly tasks. The key ideas are as follows;

1. Leveraging the softness and environmental constraints: the robot can complete the tasks with simplified dynamics, which leads to sample efficiency. Given peg-in-hole tasks, which is one of the most common assembly tasks, we can divide the sequence of the peg-in-hole tasks into subtasks called manipulation primitives(e.g., fit, align, and insertion) [1], which allow us to consider only important state space. Moreover, our insight is that we can ignore unnecessary action space thanks to the softness, which allows stable contacts.
2. Sample-efficient model-based reinforcement learning: we employed Probabilistic Inference for Learning Control (PILCO) [2] to obtain each primitive with a few learning trials given the simplified dynamics model. PILCO has demonstrated remarkable sample efficiency by explicitly considering the model uncertainty using Gaussian processes.

A proposed framework for assembly tasks with a soft robot. We develop simplified dynamics models using softness and environmental constraints to make learning tractable. Then, we apply a sample-efficient reinforcement learning method.

After learning for a peg-in-hole task in Box2D simulation.

Based on the ideas, we performed a simulation and a real-robot experiment. The experimental results demonstrated that our method successfully the peg-in-hole tasks with a few learning trials.

Leaning fit primitive (left) and after learning (right) in a real robot.

Presentation video

The short presentation can be available in the following video.

Future works

We will consider extending the proposed method to deal with unseen tasks or environments (e.g., different goals or peg sizes). Combing a transfer reinforcement learning method such as MULTIPOLAR can be a promising extension.

We also explore how to deal with the pose uncertainty of the grasped parts. Using tactile sensors or an in-hand pose estimation method can be useful.

References

[1] L. Johannsmeier, M. Gerchow, and S. Haddadin, “A framework for robot manipulation: Skill formalism, meta learning and adaptive control,” in IEEE International Conference on Robotics and Automation, 2019, pp. 5844–5850.

[2] M. P. Deisenroth, D. Fox, and C. E. Rasmussen, “Gaussian processes for data-efficient learning in robotics and control,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 2, pp. 408–423, 2015.

Call for interns!

Now we are looking for intern students who are interested in soft robotics applications, robot learning, or machine learning for real-world applications. The detail is available from here. Please contact us for internship opportunities and collaborations!

Post based on

Masashi Hamaya, Robert Lee, Kazutoshi Tanaka, Felix von Drigalski, Chisato Nakashima, Yoshiya Shibata, and Yoshihisa Ijiri, “Learning Robotic Assembly Tasks with Lower Dimensional Systems by Leveraging Physical Softness and Environmental Constraints,” ICRA 2020 [paper] (some typographical errors were fixed in this version)