Manipulation of Deformable Objects

Context

Following the Industry 4.0 paradigm, robots are expected manipulate various objects in real-world settings. In this context, providing robots with the ability to manipulate soft objects has many practical uses. This particularly concerns deformable linear objects (DLOs), which are one-dimensional soft objects such as cables, plants, or beams. Typical applications are related to cable harnessing, hose manipulation, or plant stem bending for harvesting.

Challenges

Modeling DLOs for robot manipulation remains a challenge. In fact, such objects exhibit nonlinear deformations that are difficult and computationally expensive to accurately model. To avoid modeling DLO deformations, a line of research explored deep reinforcement learning (DRL) approaches. A model-free approach such as DRL is however limited by the sim-to-real gap. When considering DLOs, it is worth mentioning the following challenges:

DLO have some singular configurations that cannot be simulated accurately. This is the case when vertical force is exerted on a straight DLO (no curvature). The DLO direction of deformation cannot be predicted. Consequently, the policies learned in simulation are no longer valid in real-world settings.
Resolving this sim-to-real gap is still an open problem since there are no accurate and standard simulation environments for deformable objects. Simulated DLOs differ significantly from real DLOs. First, mechanical parameters (Young’s modulus, Poisson coefficient, mass, friction, etc.) are only valid for one instance of a DLO. Second, real DLOs may be elastoplastic and partially maintain deformations.
Most recent studies on DRL usually control the position of the gripper only. However it is more intuitive and natural to also actuate the gripper orientation. A 6 DOF-gripper is less restricted and can subsequently achieve more complex deformations than a 3 DOF-gripper. Unfortunately, using more DOFs leads to the well-known curse of dimensionality inherent in DRL approaches: the action space grows exponentially with the number of controlled DOFs. It becomes more difficult to find an optimal policy to achieve the desired DLO deformations.

Scientific Contributions

Manipulating Deformable Linear Objects in 3D with a DRL-based framework

To mitigate the above issues, we proposed MultiAC6, a multi actor critic deep deterministic policy gradient (DDPG), which is able to achieve large 3D deformations in real-world setting, on different types of DLOs without fine-tuning or retraining. MultiAC6 is a multi-agent algorithm based on a action space decomposition framework. The proposed action space decomposition. In our settings, a first agent is trained to achieve a given desired DLO tip orientation. This desired orientation is known to lead to the desired DLO deformation while avoiding singularities. In parallel, from the desired orientation of the DLO tip, a second agent is trained to control the translation velocity of the gripper to deform the DLO into the desired shape. This provides a straightforward but still efficient way to bridge the sim-to-real gap for DLO manipulation. Such a strategy combines the advantages of a 6 DOF control of the gripper with the benefits of a limited action space. Indeed, by decoupling the gripper control over two agents, each of them only explores a limited action space, allowing them to find useful learning signals to achieve their respective task.

MultiAC6 principle

Our results showed that MultiAC6 is more accurate and efficient than a single agent controlling only the gripper position (3DOF), and a single agent controlling the gripper pose (6DOF).

MultiAC6vsfailed.mp4

MultiAC6 works better than single agent frameworks

MultiAC6 is robust and can be used on different DLOs without finetuning or training. The same model is used for all the below experiments

MultAC6_variousDLO.mp4

MultiAC6 on different types of DLOs

Related Publications

Mélodie Daniel, Aly Magassouba, Miguel Aranda, Laurent Lequièvre, Juan Antonio Corrales Ramón, Roberto Iglesias Rodriguez, and Youcef Mezouar. Multi actor-critic DDPG for robot action space decomposition: A framework to control large 3D deformation of soft linear objects. IEEE Robotics and Automation Letters, volume 9, pages 1318–1325. IEEE, 2024.