Where: TEMICS / DEMOS / 3D4 /

3D RECONSTRUCTION AND APPLICATIONS

Computed 3D Models for Very Low Bit-rate Video Coding and Video Manipulations

F. Galpin,

Goal

The goal of this work is the representation of video sequence with estimated 3D models. We assume unknown static scene captured with a monocular moving camera, with unknown motion and unknown internal parameters. We propose a fully automatic method using several 3D models in order to reconstruct the original video sequence or a virtual one. A complete scheme for video sequence compression is presented and compared with classical compression schemes. We also show results on video manipulations which are quite easy in the proposed representation.

Approach

The main idea is that instead of aiming at reconstructing a unique realistic model of the real 3D scene which needs too much assumptions, the video sequence is processed to produce several 3D models which are valid for a section of the sequence called GOP. These 3D models are then used for reconstructing the original video sequence with a very low bitrate or used for video manipulation.

More information

You can get a more detailed description of our method and links to our papers

Video Compression

Street sequence

Street sequence is provided by Thomson Multimedia/CCETT.


Frame from original video sequence	H26L compressed video sequence (very low bitrate coder) 112kb/s, CIF, 25Hz (click to load a MPEG sequence part)	Rec3D compressed video sequence 114kb/s, CIF, 25Hz (click to load a MPEG sequence part)

Further in very low bitrate :


Rec3D compressed video sequence 60kb/s, CIF, 25Hz (click to load a MPEG sequence part)	Rec3D compressed video sequence 25kb/s, CIF, 25Hz (click to load a MPEG sequence part)	Rec3D compressed video sequence 16kb/s !!, CIF, 25Hz, original path is approximated (click to load a MPEG sequence part)

Stairway sequence

Stairway sequence : a bit harder due to horizontal lines, water etc... Encoded sequence is stabilized.


Original video sequence shaky acquisition, 25Hz (click to load a MPEG sequence part)	Rec3D compressed video sequence 96kb/s, CIF, 25Hz (click to load a MPEG sequence part)	Rec3D compressed video sequence 32kb/s, CIF, 25Hz (click to load a MPEG sequence part)

Thabor sequence

Thabor sequence : rotation + translation along z-axis


Original video sequence shaky acquisition, 25Hz (click to load a MPEG sequence part)	Rec3D compressed video sequence 100kb/s, CIF, 25Hz (click to load a MPEG sequence part)	Rec3D compressed video sequence, stabilized 32kb/s, CIF, 25Hz (click to load a MPEG sequence part)

Video Manipulation

Virtual path


Rec3D virtual video sequence Constrained path (click to load a MPEG sequence part)	Rec3D virtual video sequence Free path (click to load a MPEG sequence part)

Virtual light


Night promenade in the city Some lights added in the scene (click to load a MPEG sequence part)	Exploring the park with a flash-light The scene was retimed and a moving light was added. (click to load a MPEG sequence part)

Virtual objects


Rec3D augmented reality video sequence A cube was added in the scene (click to load a MPEG sequence part)	Rec3D augmented reality video sequence A statue was added in the scene (click to load a MPEG sequence part)

Stereo visualization

Stereo visualization is automatic with this representation.


Rec3D stereo video sequence Anaglyph video sequence (click to load a MPEG sequence part)	Rec3D stereo video sequence Anaglyph video sequence (click to load a MPEG sequence part)

3D Models

Here are some of the estimated 3D models which make up our 3D representation.


3D model from Street sequence (click to load a VRML model)	3D model from Stairway sequence (click to load a VRML model)

You can get a VRML plug-ins for netscape at the following URL

You can get a Unix VRML plug-ins for netscape at the following URL

You can get a Unix MPEG plug-ins for netscape at the following URL

Previous results

Webmaster:
Last time modified: 2008-03-14