3D RECONSTRUCTION AND APPLICATIONS

3D reconstruction Computed 3D Models for Very Low Bit-rate Video Coding and Video Manipulations

F. Galpin,

contact: F. Galpin, L. Morin

Goal

The goal of this work is the representation of video sequence with estimated 3D models. We assume unknown static scene captured with a monocular moving camera, with unknown motion and unknown internal parameters. We propose a fully automatic method using several 3D models in order to reconstruct the original video sequence or a virtual one. A complete scheme for video sequence compression is presented and compared with classical compression schemes. We also show results on video manipulations which are quite easy in the proposed representation.

Approach

The main idea is that instead of aiming at reconstructing a unique realistic model of the real 3D scene which needs too much assumptions, the video sequence is processed to produce several 3D models which are valid for a section of the sequence called GOP. These 3D models are then used for reconstructing the original video sequence with a very low bitrate or used for video manipulation.

More information

You can get a more detailed description of our method and links to our papers

Video Compression

Street sequence

Street sequence is provided by Thomson Multimedia/CCETT.

original image H26L compressed image Rec3D compressed image
Frame from original video sequence H26L compressed video sequence (very low bitrate coder)
112kb/s, CIF, 25Hz
(click to load a MPEG sequence part)
Rec3D compressed video sequence
114kb/s, CIF, 25Hz
(click to load a MPEG sequence part)

Further in very low bitrate :

Rec3D compressed image Rec3D compressed image Rec3D compressed image
Rec3D compressed video sequence
60kb/s, CIF, 25Hz
(click to load a MPEG sequence part)
Rec3D compressed video sequence
25kb/s, CIF, 25Hz
(click to load a MPEG sequence part)
Rec3D compressed video sequence
16kb/s !!, CIF, 25Hz, original path is approximated
(click to load a MPEG sequence part)

Stairway sequence

Stairway sequence : a bit harder due to horizontal lines, water etc... Encoded sequence is stabilized.

original sequence Rec3D compressed image Rec3D compressed image
Original video sequence
shaky acquisition, 25Hz
(click to load a MPEG sequence part)
Rec3D compressed video sequence
96kb/s, CIF, 25Hz
(click to load a MPEG sequence part)
Rec3D compressed video sequence
32kb/s, CIF, 25Hz
(click to load a MPEG sequence part)

Thabor sequence

Thabor sequence : rotation + translation along z-axis

original sequence Rec3D compressed image Rec3D compressed image
Original video sequence
shaky acquisition, 25Hz
(click to load a MPEG sequence part)
Rec3D compressed video sequence
100kb/s, CIF, 25Hz
(click to load a MPEG sequence part)
Rec3D compressed video sequence, stabilized
32kb/s, CIF, 25Hz
(click to load a MPEG sequence part)

Video Manipulation

Virtual path

Rec3D virtual sequence Rec3D virtual sequence
Rec3D virtual video sequence
Constrained path
(click to load a MPEG sequence part)
Rec3D virtual video sequence
Free path
(click to load a MPEG sequence part)

Virtual light

Rec3D relight sequence Rec3D relight sequence
Night promenade in the city
Some lights added in the scene
(click to load a MPEG sequence part)
Exploring the park with a flash-light
The scene was retimed and a moving light was added.
(click to load a MPEG sequence part)

Virtual objects

Rec3D sequence Rec3D sequence
Rec3D augmented reality video sequence
A cube was added in the scene
(click to load a MPEG sequence part)
Rec3D augmented reality video sequence
A statue was added in the scene
(click to load a MPEG sequence part)

Stereo visualization

Stereo visualization is automatic with this representation.

Rec3D stereo sequence Rec3D stereo sequence
Rec3D stereo video sequence
Anaglyph video sequence
(click to load a MPEG sequence part)
Rec3D stereo video sequence
Anaglyph video sequence
(click to load a MPEG sequence part)

3D Models

Here are some of the estimated 3D models which make up our 3D representation.

Rec3D VRML model Rec3D VRML model
3D model from Street sequence
(click to load a VRML model)
3D model from Stairway sequence
(click to load a VRML model)

You can get a VRML plug-ins for netscape at the following URL

You can get a Unix VRML plug-ins for netscape at the following URL

You can get a Unix MPEG plug-ins for netscape at the following URL

Previous results

See also previous results: old results on compression aspects, reconstructed videos and a global scheme of the method , reconstructed models .

Webmaster: Valid CSS! Valid XHTML IRISA
Last time modified: 2008-03-14