Deformable object manipulation by “photometric shape servoing”

Context:
 
A major challenge in robotics is the interaction with deformable objects. Indeed, nowadays most of the robot control frameworks are built for rigid objects observed in the scene. Therefore, in order to extend the manipulation capabilities of robots, the main objective of this thesis is to investigate a new robotic control approach for manipulating soft objects to apply a desired deformation. The general idea is to use visual feedback to estimate and track in real time the deformations of the object of interest, and to develop a visual servo-control approach to control one or more robotic manipulators in order to apply a desired deformation to the object. 

Automatically controlling the deformation of soft object would enable numerous robotic applications such as gripping a deformable object with a robotic hand, assembling flexible elements, accurately manipulating elastic materials or food products.

Recently, a number of studies have proposed solutions for applying controlled deformations to soft objects by robots, leading to the emergence of a new field of robotics research that is called "Shape servoing". 
Controlling the deformation of soft materials requires the knowledge of how displacements of the robotic manipulator are translated into material deformation. This relationship can be either estimated from past visual observation with data-driven approaches [1] or expressed by a physics-model of the object such as finite element [2] or mass-spring models [3-4].
 
Envisaged Activities:
 
The various existing approaches need to extract visual geometric features to represent the object current deformations. For example they use 3D points of the objects surface provided by an RGB-D camera. Therefore, one of their limitations is the requirement of a real-time process that extracts and tracks these geometric features. This process usually involves a segmentation process which is only optimized and efficient for a specific type of object. 

To avoid the need for extracting geometric features, we will propose in this thesis to consider directly the photometric information as well as the depth information provided by the RGB-D camera as the visual features to be regulated by the control law. This was already done for positioning a camera with respect to a rigid object by the “direct visual servoing” concept introduced in [5-6]. However, at our best knowledge photometric features were never considered for shape visual servoing.

This thesis will therefore focus on the elaboration of a new shape servoing framework that we call “photometric shape servoing”. One of the scientific contributions will concern the modelling of the interaction that provides the variation of the observed photometric information with respect to the motion of the robot acting on the soft object. This interaction model will then be considered in the design of a robot controller that will allow the autonomous shaping of a soft object by several robotic arms. A reduction of the dimensionality of the photometric shape features will be also considered thanks to projection approaches [7-8].
The methods studied will be developed, tested and validated on an experimental bench consisting of deformable objects, depth cameras, 2 robotic arms with 6 degrees of freedom, each equipped with an force sensor and a robotic griper/hand.

Skills/Requirements:

The candidate must have an excellent track of records and a Master Degree (or equivalent) in robotics and computer vision.
The candidate must have the following qualifications:
•    Strong background in robotics
•    Experience with computer vision, physical robots, or 3D simulation 
•    Excellent programming skills in C++
•    Strong proficiency in both written and spoken English
•    Ability to perform experimental validations
•    Ability to work independently as well as collaboratively

Conditions:

The position is full-time for 3 years and will be paid according to French salary rules for PhD students.

How to apply? 

Please send before to alexandre [*] krupainria [*] fr and eric [*] marchandirisa [*] fr the following elements in a zip file:

•    Your CV
•    Your motivation letter including an explanation of interest in the research we conduct and why you believe you are suitable for the position, a short description of graduate projects, details of any relevant work experience
•    List of marks (even preliminary) of your Master 2 or engineer formation
•    Letters of recommendation and contact details of professional references (mail, address, position)