Vous êtes ici

Mixed tactile-force feedback for safe and intuitive robotic teleoperation

Equipe et encadrants
Département / Equipe: 
Site Web Equipe: 
https://www.irisa.fr/lagadic/
Directeur de thèse
Paolo Robuffo Giordano
Co-directeur(s), co-encadrant(s)
Claudio Pacchierotti
Contact(s)
NomAdresse e-mailTéléphone
Claudio Pacchierotti
claudio.pacchierotti@irisa.fr
+33 2 99 84 25 47
Sujet de thèse
Descriptif

Context

Teleoperation systems are employed to sense and mechanically manipulate objects at a distance by virtually relocating the human operator at a place other than his or her true location [1]. They are composed of a slave system, which interacts with the remote environment, and a master console, operated by a human. The human operator should then receive enough information about the slave system and the remote environment to feel physically present at the remote site. This condition is commonly referred to as telepresence [2]. Achieving a good illusion of telepresence is a matter of technology. If the slave system transmits sufficient information back to the operator, displayed in a sufficiently natural way, the illusion of telepresence can be compelling. This can be achieved through different types of feedback information, that flow from the remote scenario to the human operator. One piece of this flow is haptic force feedback, that conveys information about the forces exerted at the slave side of the system, providing the operator with the compelling sensation of touching the remote environment.

Haptic feedback is usually provided through desktop haptic interfaces, which are very precise and can exert high forces. However, desktop haptic interfaces also suffer from several mechanical and control issues, such as workspace limitations, the presence of effective masses the operator needs to move around, and the undesired vibrations and force generated by the motors, cables, friction and bearings. The form factor and weight of grounded devices also drastically limit the possibility of engaging in multi-contact interactions and operations [3]. Finally, as grounded interfaces directly control the position of the slave robot, any problem in the force feedback is reflected to the position of the slave robot, affecting the stability of the teleoperation loop and possibly leading to dangerous oscillations at both sides of the system [4].

To address the abovementioned issues, researchers have started to work on innovative designs and control algorithms for haptic interfaces. In this respect, tactile haptic feedback has recently received great attention from researchers looking for a safe and effective alternative to force feedback. Tactile stimuli are detected by mechanoreceptors in the skin, enabling humans to recognize the local properties of objects such as shape, edges, and texture. Delivering this type of haptic cues to the operator’s skin has been preliminary proved to convey rich information and does not affect the safety of the teleoperation system [5, 6].

PhD topic

In this thesis, we propose to study the most effective and safe ways of providing haptic feedback in robotic teleoperation, with the objective of maximizing the information provided, the user comfort, the operating workspace, and the system’s safety. At the same time, we will integrate these haptic systems in two paradigmatic applications: teleoperation of a fleet of Unmanned Aerial Vehicles (UAVs) and teleoperation of a humanoid robot.

To this end, the thesis will proceed by developing five main key aspects:

  • Perception of multiple haptic stimuli. At first, we will study the effectiveness of combining multiple haptic stimuli, focusing on force, vibrations, normal indentation, and skin stretch. This perceptual work aims at evaluating the role of different combinations of haptic stimuli in the considered teleoperation tasks, and it is propaedeutic to all the succeeding studies along this line of research. We will focus on stimuli being able to provide multi-directional information, applied to different parts of the body, such as the hand, wrist, and forearm. In the literature, extensive research has been done on the use of force feedback for robotic teleoperation using desktop haptic interfaces, such as the Omega or the Phantom interfaces [7]. However, very little research has been carried out on the role of using solely tactile cues or mixed force-tactile cues.
     
  • Haptics for telemanipulation and wearable technologies. The result of the above experimental evaluation will be useful to devise novel wearable tactile interfaces able to provide the human operator with effective and unobtrusive information coming from the remote environment. Wearable devices can significantly enlarge the workspace of the human operator, reducing the effective mass at the end-effector, easily enabling multi-contact interactions [8].
    However, due to their small size, it is not possible to include many sensors and actuators. To overcome these technological limitations, we will have to study novel haptic rendering and control algorithms able to tackle the specific challenges due to the underactuation and undersensing of these devices.
     
  • Safety and stability.  Using tactile stimuli in teleoperation will also open new exciting opportunities for the design of novel stability control techniques, especially when mixing force and tactile feedback. We will work to improve existing stability control approaches to take into account for the additional tactile stimuli, focusing on time-domain energy-based techniques, with the objective of maximizing transparency while guaranteeing the overall safety of the system.
     
  • Forward mapping (positions). In order to map the motion of the human operator into a motion of the slave robots (e.g., a UAV fleet or a humanoid robot), we need to address the problem of mapping positions between systems with very dissimilar kinematics (e.g., five fingers vs. three drones). The mapping can be designed taking into account the environment domain with the objective of maximizing the similarity of the action between master and slave. The interaction points on the master system can be used to estimate a homogeneous transformation which describes the master motion. The contact points on the slave system are then used to transfer this transformation on the slave robot(s); and a motion of the human hand causes a displacement of the interaction points on the master side.
     
  • Backward mapping (forces). To map the force sensed by the slave robot(s) to the human operator, we plan to study mapping techniques based on postural synergies [9]. The objective is matching, in terms of change of internal forces, the action produced by the robot on the environment to the human operator.

Below we show two-mock ups, available at Irisa, representing the two envisaged applications. The PhD student will validate his works on these platforms.

 

Fig. 1. Mock-up of a human-UAVs teleoperation system using a team of quadrotor UAVs, five tactile fingertip device, and one vibrotactile bracelet.

 

Fig. 2. Mock-up of the human-humanoid robot teleoperation system exploiting the Romeo humanoid
robot available at Lagadic.
Bibliographie
[1] T.B. Sheridan. "Teleoperation, telerobotics and telepresence: A progress report," Control Engineering Practice 3(2):205–214, 1995.
[2] T.B. Sheridan. "Telerobotics," Automatica 25(4):487– 507, 1989.
[3] L. Meli, G. Salvietti, G. Gioioso, M. Malvezzi, and D. Prattichizzo. “Multi-contact bilateral telemanipulation using wearable haptics,” in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, pp. 1431-1436, 2016
[4] C. Pacchierotti. "Cutaneous haptic feedback in robotic teleoperation. Springer Series on Touch and Haptic Systems," Springer International Publishing, 2015.
[5] D. Prattichizzo, C. Pacchierotti, and G. Rosati, “Cutaneous force feedback as a sensory subtraction technique in haptics,” IEEE Transactions on Haptics, vol. 5, no. 4, pp. 289–300, 2012.
[6] C. Pacchierotti, D. Prattichizzo, K. J. Kuchenbecker. "Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery," IEEE Transactions on Biomedical Engineering, 63(2):278-287, 2016.
[7] J. Park and O. Khatib, “A haptic teleoperation approach based on contact force control,” The International Journal of Robotics Research, vol. 25, no. 5-6, pp. 575–591, 2006.
[8] D. Prattichizzo, F. Chinello, C. Pacchierotti, M. Malvezzi. "Towards wearability in fingertip haptics: a 3-DoF wearable device for cutaneous force feedback," IEEE Transactions on Haptics, 6(4):506-516, 2013.
[9] G. Salvietti, L. Meli, G. Gioioso, M. Malvezzi, D. Prattichizzo. "Multi-Contact Bilateral Telemanipulation with Kinematic Asymmetries," IEEE/ASME Transaction on Mechatronics, 2016.
Début des travaux: 
01/10/2017
Mots clés: 
teleoperation, haptics, tactile interfaces, wearables, stability, shared-control, postural synergies
Lieu: 
IRISA - Campus universitaire de Beaulieu, Rennes