Aerial Robots with the Sense of Touch

Aerial robots (commonly called “drones”) are nowadays extensively used to see the environment for surveillance and monitoring in applications like agriculture, mapping, etc. But if aerial robots were also able to effectively manipulate the environment (by physically interacting with it), the application domains could be further extended toward new areas like contact-based inspection, assembly and construction, and so on. But contact is today synonymous for crash, and thus avoided. The unstable nature of aerial robots, as well as their non-linear dynamics and limited vision-based perception, makes the manipulation problem in real environments extremely difficult and delicate.

To show the feasibility of Aerial Physical Interaction (APhI), the research community has focused in the last years on the design and control of aerial manipulators, that is, drones equipped with an onboard manipulator arm. This opened the door to new applications, e.g., contact-based inspection. However, current methodologies are still limited to very simple interaction tasks, involving limited contact behaviors with static and rigid surfaces (e.g., touching a flat wall with a stick attached to the robot). Furthermore, although the preliminary results are very encouraging, they have also been obtained in extremely controlled lab conditions. Current experiments mostly rely on external Motion Capture Systems (MOCAPs) that guarantee precise state estimation and environmental models. However, in real scenarios robots can only rely on on-board sensors, which today do not provide at all the required accuracy

Thanks to the investigation of novel haptic-sensing for aerial robots, combined with the well-known vision-sensing, this Thesis has the ambition to drastically enhance aerial physical interaction capabilities for complex manipulation in real environments. We aim at bringing aerial robots much closer and faster to real applications than what the current investigation based on vision-only systems could do.

For a grounded robot, which is very precise and stable, manipulating articulated objects might be an easy task, but for an aerial robot it is extremely complex. These challenging tasks require in-flight physical interaction whit millimeter precision. However, because of limited payload, aerial robots can employ only “cheap” vision-based sensors which, subject to vibrations of propellers, can provide only centimeter accuracy. Unfortunately, only vision-based methods are not enough to perform complex manipulation tasks.

On the other hand, in nature, it has been shown that the sense of touch is a key element to perform manipulation. Even humans, with a very accurate vision system and dexterous hands, would have problems performing manipulation tasks without the sense of touch.

Nevertheless, so far no one dared to extend haptic and touch capabilities to aerial robots, and even less to exploit touch sensors as direct feedback to perform manipulation. We will address these challenges in the following work-packages:

WP1 - Touch sensor: On one side we have to answer the question if and which “touch sensors” are feasible for flying robots. The vibrations and the air gusts produced by the propellers might drastically deteriorate the measurements making them useless (this, for instance, has been the case with many vision-based sensors that required specific modifications to be used on aerial robots, e.g., event-based cameras). We will review and evaluate different touch sensors mounted on the end-effector of an aerial manipulator. We will analyze the quality of contact and interaction force measurements, designing appropriate filters and mechanical supports to make sensors readings free of propellers’ vibrations.

WP2 - haptic-based control: On the other side, we have to understand how to use those measurements at best. Currently, they are used only to improve models of the environments, based on which standard control methods are applied. This introduces an overhead in terms of both computational time and resources. We instead foresee the use of the sense of touch to directly drive actions. Skin-like sensors produce a matrix where every cell corresponds to the pressure value. One can notice the similarity with cameras. Therefore, skin-like sensors provide “images of the contact”. Inspired by this correlation, we will investigate even further the similarities and differences, to then try to transfer the rich knowledge on vision-based perception to the purely explored area of haptic-based perception. We will take inspiration from visual-servoing and reinforcement-learning control methods to define a new haptic-servoing paradigm, where the robot actions are defined to reproduced a desired haptic feeling. The expertise of the Rainbow group on both visual-servoing and haptics will be helpful. We want to study if it is possible to transfer to robots the capacity of humans in using the sense of touch for manipulation.

WP3 - experimental evaluation: The proposed methodologies will be validated on real aerial manipulators. Their effectiveness will be demonstrated through complex interaction tasks in real-world scenarios, like opening a door with only onboard sensors and minimal knowledge of the environment. We will further assess their practical value by investigating their performance when sensors’ accuracy degrades, as well as their limitations.