This paper describes a novel approach
to partially reconstruct high-resolution 4D light fields from a stack
of differently focused photographs taken with a fixed camera. First, a
focus map is calculated from this stack using a simple approach
combining gradient detection and region expansion with graphcut. Then,
this focus map is converted into a depth map thanks to the calibration
of the camera. We proceed after this with the tomographic
reconstruction of the epipolar images by back-projecting the focused
regions of the scene only. We call it masked back-projection. The
angles of back-projection are calculated from the depth map. Thanks to
the high angular resolution we achieve, we are able to render puzzling
perspective shifts although the original photographs were taken from a
single fixed camera at a fixed position. To the best of our knowledge,
our method is the first one to reconstruct a light field by using a
focalstack captured with an ordinary camera at a fixed viewpoint.
We designed a two-steps
algorithm for focus map estimation: first, we detect strong gradients
on every image of the focus stack, then we use a graph-cut algorithm to
expand the zones where strong gradients have been detected.
Thanks to the prior calibration of the camera, we can then transform
this focus map into a depth map.
A conventional photograph is a
projection of the 4D light field. Thus, a focal stack can be viewed as
a collection of projections of the 4D light field along different
directions. We propose to retrieve the epipolar images from their
projections with an algorithm that we name masked back-projection.
The conventional backprojection method is based on the Radon transform.
The original image intensity is recovered from its projections by first
filtering the projections, and then back-projecting them onto the plane
of the image to be reconstructed.
In order to achieve an accurate reconstruction with the traditional back-projection algorithm, a high number of projections is needed, whereas the number of projections is limited by the number of images in the focal stack in our problem. Nevertheless, the epipolar image to be reconstructed has a special structure, which can be approximated as a set of overlapping lines with different slopes.
We thus propose a modified version of the back-projection algorithm adapted for our problem, which exploits this prior information on the epipolar image. We first back-project entirely the background. Then, we backproject only the in-focus parts of the projections, from the second farthest to the foreground, in the order of decreasing depth, as depicted in the figure below.