3D Reconstruction and Applications

Automatic 3D reconstruction from natural video sequences

F. Galpin,

contact: F. Galpin, L. Morin

Goal

The goal of our work is the recovering of 3D information from sets or sequences of uncalibrated images in the context of a static scene. This issue is of interest in many applications, such as compression or navigation in multimedia applications [Oisel95].

Approach

Instead of constructing a realistic model of a 3D real scene through a tiresome and expensive modeling procedure, a video sequence of the scene could then be automatically processed to produce such a model. Using real texture extracted from the video also prevents from sophisticated and expensive texture computation. The criterion for evaluating the quality of the reconstructed model is based on visual appreciation and does not require precise 3D reconstruction. The presented algorithm takes a video sequence as input, and outputs several model in VRML format [Vrml96] for display facilities.

Algoritm

First the fundamental matrix (denoted F) associated with the epipolar geometry is computed in order to constrain a dense disparity field estimation [Oisel98c]. This dense matching between the 2 images is based on an algorithm derived from optical flow estimation, which provides a robust and regularized estimated field. A projective reconstruction can then be obtained from the association of this field and the fundamental matrix. Then, fixing some camera parameters, we obtain the 3D coordinates of each point, i.e. a pseudo-Euclidean reconstruction of the scene. In order to minimize artifacts and to make navigation through the scene easier, a segmentation is performed, based on a 2D triangulation. This triangulation is iteratively computed either using the dense field or 3D coordinates of each point. Different choices of criteria and constraints for finding this triangulation have been compared. The last part consists in obtaining a realistic VRML model.

Experimental Results

VRML Models on natural sequence

First and last images in the video sequence		Original video sequence	A virtual sequence obtained with the VRML model (click to load full VRML model)

You can get a VRML plug-ins for netscape at the following URL

VRML Models on synthetic sequence

First and last images in the video sequence		Original video sequence	A virtual 3d (anaglyph) sequence obtained with the VRML model(click to load full VRML model)

You can get a VRML plug-ins for netscape at the following URL

References

[Oisel95]

L. Oisel: "Stereoscopic data compression using projective geometry", Proceedings of the International Workshop on Stereoscopic and Three Dimensional Imaging

[Oisel98c]

L. Oisel: "Planar facets segmentation using a multiresolution dense disparity field estimation", icip, Chigago, USA, 1998

[Vrml96]

G. Bell, A. Parisi and M. Pesce: "The Virtual Reality Modeling Language", Version 1.0 Specification

[Galpin99a]

F. Galpin, Luce Morin, Lionel Oisel: "Génération d'un modèle VRML texturé à partir de séquences non calibrées", ORASIS99, 1999 (downloadable: PS version)

[Galpin99b]

F. Galpin, Luce Morin, Lionel Oisel: "VRML model generation from disparity field", IWSNHC3DI99, 1999 (downloadable: PS version)