"Tactile Images": relief on your images

Have you ever dreamt of being able to touch the images or pictures displayed on your computer and feel their relief? Well, it is now possible, and you can even try it at home (see the examples/demonstrations).

An Inria research scientist from Irisa Rennes proposes a technique that makes it possible to simulate the tactile sensations related to the textures or relief of the 2D images displayed on your computer screen. Usually, when you move the cursor of your mouse over an image, you feel absolutely nothing. Today, however, the results obtained with this technique provide the feeling of touching the images, feeling their grain and bumps just as if you were running your finger over them!

The applications of this technique are numerous. Let us mention first of all, the perception of picture relief for entertainment or professional purposes, such as for example image retouching. When used on the Internet, the technique adds effects of an entirely new kind to web pages having to do with the cursor: relief and contour perception, attractive or repulsive areas or html links. The technique can also be used to better perceive the elements of your computer desktop, for example to feel the textures of icons, buttons, or the limits of a window of your graphical user interface. Other applications can be envisioned, such as assistance to visually impaired persons in helping them access computers by facilitating the use of the mouse, or video games in which the player's immersion could be enhanced by providing new, more tactile sensations.



The technique in question consists in modifying the movement and speed of the cursor when it crosses an image, either slowing it down or speeding it up. If you want to give the illusion that the mouse cursor is climbing a slope, then it is slowed down. Conversely, if you want to simulate the cursor goind down the slope, it is accelerated. If the cursor hits a "wall", then its movement is completely stopped. For example, to simulate the cursor crossing a bump, the cursor is decelerated until it reaches the top of the bump. Once it is past the top, the cursor is accelerated until it reaches the foot of the bump. After that, the cursor recovers its normal speed. In the end, you will have had the impression to have brushed this bump with the tip of your fingers!

It is possible to simulate the relief of any texture or 2D image using the proposed technique. The generic algorithm developed at Irisa/Inria Rennes consists in using a topography or height map of the image computed from its colors or grayscale levels for example. The trajectory of the mouse is analyzed in order to identify the height ot the points (or pixels) that the cursor is going to cross. The algorithm then works on the theoretical cursor trajectory point by point (or pixel after pixel) and slows down or accelerates the cursor depending on the heights crossed. For example, when the cursor is about to go from a low pixel (a dark pixel for instance) to a high pixel (a clear pixel), then it is decelerated. The final position visually reached by the cursor takes into account all these elementary slow downs and speed ups.

This technique is based on recent research results in the field of human visuo-tactile perception and in the field of macroscopic texture perception. It has been shown that the perception of efforts supersedes that of the height. The technique was assessed via a three step experimental validation phase. A first experiment measured the capacity to identify simple shapes such as bumps and hollows. A second experiment made it possible to assess the capacity to recognize a relief solely based on cursor movements. A final experiment made it possible to study relief perception more precisely by asking subjects to draw the profile of simulated surfaces. The three experiments showed that practically everyone is capable of recognizing and precisely drawing the relief simulated by this technique.