Learning Powder Weighing Manipulation
We are thrilled to announce that our paper has been accepted to the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023) (acceptance rate of 43.3%)!
Yuki Kadokawa, Masashi Hamaya, Cristian Beltran-Hernandez, and Kazutoshi Tanaka, “Learning Robotic Powder Weighing from Simulation for Laboratory Automation” [paper] [code] [project page]
Kadokawa Yuki is a Ph.D. student at Nara Institute of Science and Technology and was a research intern at OMRON SINIC X. This work has been done during this internship.
This blog briefly introduces our proposed method.
Background
This study addressed a robotic powder weighing task for laboratory automation. In this task, researchers or workers typically scoop the powder from a bottle and dispense a portion of it into the weighing dish on the scale. This task is essential in various fields, including chemical and pharmaceutical sciences. However, this task is time-consuming as human researchers need to prepare many samples with a milligram-level mass.
This study aims to train the robot to weigh various powders without additional settings and specialized dispensing machines. The robot is expected to perform accurate powder weighing, including the powders that were never weighed, and thus produce adaptive scooping and dumping motion based on the powder states.
Powder weighing also poses several challenges for robotic manipulation. The powder dynamics vary significantly between materials; for example, smaller-size particles tend to be more cohesive, leading to different fluidity. Moreover, for precise weighing of small target masses, the robot must strike a delicate balance between aggressive and conservative actions. For instance, the robot may need to shake the spoon vigorously to dump cohesive powders but not too much to avoid dumping an excessive amount of powder. Hence, learning-based approaches are promising; however, many interactions with real-world environments require significant manual efforts to clean the spread powders.
This study adopted a sim-to-real transfer learning approach to enable the robot to weigh various powders with a small target mass and alleviate the burden of collecting data in a real-world environment. To this end, we employed a Domain Randomization (DR) technique, which can compensate for the sim-to-real gap and adapt to several unseen real environments without additional training. We developed a powder weighing simulator with Isaac Gym and selected the dynamics parameters to be randomized, such as particle friction and gravity.
Problem setting
To enable a robot to automate this task, we developed the setting, as depicted in Fig. 1. A robot is equipped with a dispensing spoon to scoop and dump powder from a bottle on an electric scale. The objective of the powder weighing task is to adjust the amount of powder in the spoon to a specific target mass ranging between 5 and 15 mg through scooping and dumping actions. The scale measures the weight of the powder in the bottle, and the powder weight on the spoon is indirectly obtained by monitoring the scale measurements of the amount of powder scooped from and dumped into the bottle.
Notably, the setup presented here differs slightly from that of an actual powder weighing scenario. The reason for this deviation is to save time on powder transportation and facilitate easier evaluation in real-world experiments. Nevertheless, our proposed setting can address the fundamental problem of how to adjust the amount of powder through dumping.
Learning Powder weighing
To train the policy, we used one of the most widely used sample-efficient model-free deep reinforcement learning methods, soft actor-critic (SAC). To consider the time-series powder state, we applied a long short-term memory (LSTM) structure to the actor and critic network. The use of an LSTM-based structure was beneficial in balancing the selection of aggressive and conservative actions. Specifically, if the robot dumped too much powder, the policy undertook more conservative actions in the subsequent timestep. Conversely, if the amount of powder remained constant for several timesteps, the policy selected more aggressive actions.
We designed and implemented a simulator for powder weighing tasks using Isaac Gym. The simulator enables us to adjust the amount of powder on the spoon by inclining and shaking the spoon. Unlike real-world scenarios, the robot arm model and the process of scooping powder from the bottle are excluded. Instead, powder particles spawn on the spoon by randomly changing the particle number. The powder weight on the spoon is calculated by counting the number of particles on the ground.
This study applies a DR technique to obtain the dumping policy to adjust the amount of powder on the spoon in the simulator. Then, the transfer policy is utilized in real-world environments without additional interactions. We selected and randomized eight parameters during train-
ing in the simulator, including the target powder amount, as described in Table I. These parameters were uniformly randomized at every episode.
To consider the variations in the physical properties of the powder, powder friction, mass, and radius were randomized (Table I). Particle number was used for the variation in the initial amount of the powder. Spoon friction is also important to exhibit the various fluidity of the powders. The powder’s movement in the air was mimicked using gravity. The shake speed weight was used to ensure the robustness of the robot’s response to noise and delay in the system. The range of the values of the physical parameters was experimentally determined by observing the powder behavior during the weighing task performed by humans.
Experiments
For the real-world evaluation, we prepared four types of powders that were less harmful and exhibited different fluidity: wheat flour, salt, coal, and rice flour. Table II lists the weighing performances with the four powders. The results show that the accuracy for rice flour and salt was within 0.5 mg, and the performance for coal was approximately 0.5 mg. Wheat flour is the most susceptible to moisture in the air and is prone to the agglomeration phenomenon; therefore, its performance was relatively low. Nevertheless, it achieved an accuracy of approximately 0.5 mg on average.
Conclusion
This study proposed a powder weighing simulator and utilized it for learning transfer policy with Domain Randomization for sim-to-real. The experimental results showed that the proposed method was applicable to obtain transfer policies, and learning was achievable in a reasonable
time without hard real-robot initialization and maintenance.
If you are interested in our internship, please check our website!
If you are interested in collaborating with us, please send us an e-mail! contact@sinicx.com
