Reflectance and Illumination Estimation for Augmented Reality

Contact: Salma Jiddi, Philippe Robert, Eric Marchand

This is a joint work with Technicolor.

Demonstration description

In this paper, we propose a method that compactly estimates surface material properties and lighting conditions in order to correctly render virtual objects in a real scene. Using a Kinect sensor, we are able to estimate both, diffuse and specular reflectance components, as well as their respective parameters for every 3D point. Furthermore, we estimate the position of light sources responsible for specular phenomena and use this information to propose a novel photometry-based classification for all 3D points. Our algorithm allows convincing AR results such as realistic synthesized specu- larities and shadows as well as proper illumination occlusions.


Other videos are available on my page:

The left video is a basic compositing where shadow maps have been computed only with a random light source position and shadowing coefficient. Moreover, the specular reflection is still visible throughout the entire sequence. The right video demonstrates our photometric contribution to the realism of the virtual object. One can notice a correct specular occlusion as well as a realistic virtual shadow using the estimated 3D position of the light source, the diffuse reflectance component and the shadowing coefficient.

| Lagadic | Map | Team | Publications | Demonstrations |
Irisa - Inria - Copyright 2009 Lagadic Project