Rethinking Contact Simulation for Robot Manipulation
Hydroelastic Contact in Drake
By: Michael Sherman
Today’s robots have a tough time manipulating objects in complex human environments, including homes and factories. TRI is studying how to fix that, and we believe effective simulation is a critical part of the solution. Simulation allows us to quickly explore novel designs and methods, and to train controllers on vast quantities of synthetic data — but for that to work, behavior observed in simulation must match behavior in reality.
When we began working on Drake in 2016 the lore among roboticists was, “You can’t use simulation for manipulation applications.” Manipulation results in simulation often failed to predict results in the real world. As Russ Tedrake explained in an earlier blog post, simulating contact for manipulation is much harder than contact for wheeled or legged locomotion — contact details matter when grasping a coffee cup; not so much when stepping on a floor.
Today, Drake is used routinely for simulating manipulation applications. It is fully open source with examples you can try here. Read on to see how we tackled this problem.
Problems with point contact
Robot simulators conventionally simulate contact between two objects by assuming (a) that the objects are rigid, and (b) that contact occurs at a “contact point” (or at a few such points). Forces are calculated that prevent the rigid objects from penetrating one another at the contact point. Frictional forces are calculated and applied at the same point. Initially that’s what we did in Drake too, and we were disappointed with the results. Point contact results are qualitatively different from real results. Non-physical artifacts are common, such as discontinuous changes in force direction (see top animation of Figure 3) and artificially anisotropic friction; effects such as torsional friction and rolling resistance are missing; simulations are plagued by bouncing and dropping instabilities and occasional failures to find a solution.
to determine force direction (green arrow), resulting in
non-physical discontinuities. In contrast, the net force produced
by a contact patch (bottom) evolves smoothly. (3 views)
The problem is that “contact point” is not a good abstraction for contact. In reality, one or both of the contacting objects deform in response to pressure where they meet, and then contact occurs over one or more finite-area patches (surfaces) between the objects. Pressure is then distributed unevenly over the patch, resulting in smoothly-evolving normal and frictional forces that can produce complex effects (see bottom animation of Figure 3). These effects contribute significantly to the behavior of a manipulating robot since typically the gripper is covered with a soft material and often the manipulands deform as well.
What about FEA?
There is already a well-developed technology that can simulate these effects: nonlinear finite element analysis (FEA). FEA can model the deformation (strain) of materials as a result of contact forces, and calculate the extent of the contact patch and the pressure distribution (stress) across the contact surface and within the objects.
FEA is commonly used in structural analysis, but unfortunately it is ill-suited to typical problems in robotics since each computation can take minutes or even hours to perform. Roboticists expect to model rapidly changing behaviors at near real time rates, that is, hundreds or thousands of computations per second. Additionally, modeling and proper meshing of FEA contact is a specialized skill not common among roboticists — roboticists typically work with coarsely-meshed geometry or primitive shapes and can provide only rough estimates of compliance and friction coefficients. In Drake, those limited inputs must be accepted while still producing physically-realistic contact behavior at high speeds.
A brief history of Hydroelastic Contact
In early 2018, Evan Drumwright and I were struggling with this problem at TRI. We recognized that our needs were in many ways simpler than what FEA provides. For example, we didn’t need all the detailed internal stress information provided by FEA. Also, in most cases of interest to roboticists, the deformations were small and had little effect on mass distribution — for example, consider a metal gripper with a rubber coating. All we needed was the location and extent of the contact patch and the pressure distribution over it. With that we could integrate over the patch to produce net forces and moments on the contacting objects, which could then be treated as rigid for predicting motion.
By summer 2018, we had a good statement of the problem, but were still lacking a good solution. In keeping with scientific tradition, we foisted the problem onto our intern Ryan Elandt, who was then a graduate student in Professor Andy Ruina’s lab at Cornell University (TRI has provided sponsored research funding to Prof. Ruina’s laboratory). This is how we specified the problem for Ryan:
Provide a physically-justifiable compliant contact model that can be applied directly to coarsely-meshed rigid objects in collision.
- Results must be calculated from the undeformed overlap region, without requiring calculation of the deformed mesh.
- Effects must be local, that is, independent of size and shape of objects far from the contact region.
- Force magnitude and direction must be continuous with respect to configuration.
- Artifact free: no weird behavior, at least in normal use.
- Accurate in simple cases, reasonable in others.
- Uses familiar theory-of-elasticity parameterization where possible.
- Fast enough to be useful to roboticists.
We also provided some ideas based on overlap volume that turned out to be useless.
Prof. Ruina provided a key insight that helped move us in the right direction. He said the problem reminded him of the hydrostatic pressure that keeps boats afloat.
That model has several great properties:
- the pressure at any point on the hull is calculated locally just from knowing the depth of that point (it’s the weight of the displaced column of water above that point),
- the force direction is locally perpendicular to the hull, and
- no matter what shape or sharp edges on the hull, the forces are always continuous — small changes in position and orientation of the boat produce small changes in force magnitude and direction.
Unfortunately, solid objects in contact don’t have a convenient fluid whose displacement can be easily calculated. Ryan had the key insight for extending this to contact. He noticed that the Elastic Foundation (“bed of springs”) model of materials provides a similar local determination of pressure, based on depth. (These two insights are the etymology of the name “Hydroelastic Contact” that we use for our implementation in Drake.)
Ryan’s brilliant idea was to precalculate a “pressure field” throughout the interior volume of a compliant object. At any point in the interior, the precalculated pressure could approximate the pressure that would occur if the object’s surface were deformed to that point, but without performing the expensive deformation calculation. The effect looks like this (in 2D):
The precalculated pressure field in the compliant gray object allows us to calculate forces on the contacting rigid blue box. We can read out the pressure on the surface of the box from the object’s pressure field, just as we could if the box were floating in a liquid. When two compliant objects are in contact, we can quickly find the contact surface since Newton’s third law requires it to be where the pressure field values of both objects are equal.
How to calculate a pressure field
The intent of the pressure field is to approximate strain, which (roughly speaking) is the fraction of a material that has been deformed (so given as a percent). That deformation causes stress, which has units of pressure (force per unit area). So at any point P within the pressure field we want an approximation of the stress that would be produced by deforming the object so that a surface point has been pushed to point P. Then by sampling that over an area we can calculate a force. For simple shapes we can generate an adequate pressure field analytically. But how can we compute a pressure field for an arbitrary shape?
Ryan addressed that using Laplace’s equation. That is more commonly used to solve steady-state heat flow problems but has the properties we want. For an arbitrary object we know its outer boundary will have strain=0 everywhere (that’s like an ice bath at 0°C). We pick an internal boundary where strain=100% (like a source of constant temperature). Solving Laplace’s equation with those boundary conditions gives us a smooth field (like a temperature gradient) whose value everywhere approximates strain. Here is an example:
Then we scale the strain field by the elastic modulus, which (again roughly speaking) is the material stiffness, to get the pressure field. We then approximate the pressure field using a coarse tetrahedral mesh to obtain a representation we can evaluate very quickly. For details, see our 2019 paper.
Fun with pressure fields
After Ryan prototyped the idea, the Dynamics and Simulation team at TRI developed a refined implementation (2022 paper) for incorporation into Drake. To get a sense of what this enables us to do, the animation below shows the contact patches we calculate as we force a soft sphere to encounter several soft objects (no physics here). Each object is represented internally by a coarse tetrahedral (tet) mesh carrying a precalculated pressure field. You can see the realtime-calculated contact patches that result from fast tet-tet intersections. These patches approximate where the deformed surfaces would meet if the two objects actually deformed, and how pressure would be distributed. Despite sharp edges and coarse meshes, contact evolves smoothly as it does in reality. Then in Drake’s Simulator we can integrate the pressure to calculate forces, then apply those forces to the objects to produce motion.
Applying these forces in a Drake simulation yields meaningful behavior as shown in the figure below.
The spatula example we started with shows how useful this can be when simulating robot manipulators. With the right contact abstraction (patches, not points!), simulation can now be used successfully to predict behavior in complex contact situations.
If you’d like to try Hydroelastic Contact yourself, Drake is fully open source, actively supported by Toyota Research Institute, and available here. We would love to hear how this works in your application, answer your questions, and get your ideas for improvement.
