Manipulation of Deformable Objects

Context

Following the Industry 4.0 paradigm, robots are expected manipulate various objects in real-world settings. In this context, providing robots with the ability to manipulate soft objects has many practical uses. This particularly concerns deformable linear objects (DLOs), which are one-dimensional soft objects such as cables, plants, or beams. Typical applications are related to cable harnessing, hose manipulation, or plant stem bending for harvesting.


Challenges

Modeling DLOs for robot manipulation remains a challenge. In fact, such objects exhibit nonlinear deformations that are difficult and computationally expensive to accurately model. To avoid modeling DLO deformations, a line of research explored deep reinforcement learning (DRL) approaches. A model-free approach such as DRL is however limited by the sim-to-real gap. When considering DLOs, it is worth mentioning the following challenges:  

Scientific Contributions

Manipulating Deformable Linear Objects in 3D with a DRL-based framework

To mitigate the above issues, we proposed MultiAC6, a multi actor critic deep deterministic policy gradient (DDPG), which is able to achieve large 3D deformations in real-world setting,  on different types of DLOs without fine-tuning or retraining. MultiAC6 is  a multi-agent algorithm based on a action space decomposition framework.  The proposed action space decomposition. In our settings,  a first agent is trained to achieve a given desired DLO tip orientation. This desired orientation is known to lead to the desired DLO deformation while avoiding singularities. In parallel, from the desired orientation of the DLO tip,  a second agent is trained to control the translation velocity of the gripper to deform the DLO into the desired shape. This provides a straightforward but still efficient way to bridge the sim-to-real gap for DLO manipulation.  Such a strategy combines the advantages of a 6 DOF control of the gripper with the benefits of a limited action space. Indeed, by decoupling the gripper control over two agents, each of them only explores a limited action space, allowing them to find useful learning signals to achieve their respective task.

MultiAC6 principle

Our results showed that MultiAC6 is more accurate and efficient than a single agent controlling only the gripper position (3DOF), and a single agent controlling the gripper pose (6DOF). 

MultiAC6vsfailed.mp4

MultiAC6 works better than single agent frameworks

MultiAC6 is robust and can be used on different DLOs without finetuning or training. The same model is used for all the below experiments

MultAC6_variousDLO.mp4

MultiAC6 on different types of DLOs

Related Publications