Hierarchical joint estimation/segmentation of optic-flow

Contacts: E Mémin, P Pérez


We address here the intricate issue of jointly recovering the apparent velocity field between two consecutive frames and its underlying partition. The joint estimator we designed is modelized as the minimization of a global cost functional including robust estimators. These estimators have two main advantages. First of all, they enable the method to be robust to the large deviations occurring in the different terms composing our energy function. Secondly, they also offer the possibility to introduce a simple coupling between a dense optical flow field and a segmentation process through auxiliary variables. We reinforce also this coupling by a parametric likeness term applied on each regions of the partition. The underlying minimization at hand is conducted efficiently by a multigrid optimization algorithm. The resulting estimation-segmentation model operates finally a tight cooperation between a local estimation process and a global modelization.}


We presents here results obtained for a parking lot sequence:

final segmentation Dense motion filed Parametric motion field
Segmentation results at the different grid level Parametric motion field Dense vector fields

We presents also a movi showing the evolution of the hierarchical segmentation on the yosemite sequence:
evolution of the segmentation 

The table below gathers the results obtained on the basis of Barron et. al criterion ("performance of optical flow techniques" IJCV vol. 12 1994), for the cropped Yosemite sequence (without the sky). This criterion represents a discrepancy between the estimated flow field and the actual one.
Motion field Mean angular error Standard deviation
Parametric 1.58 1.21
Dense 1.92 1.59

Results obtained with other methods are recalled below:
Estimateur Résultats Références
Szeliski et Coughlan moy= 2.45 sigma= 3.05 "Hierarchical spline based image registration". CVPR'94
Black et Anandan moy= 4.46 sigma= 4.21 "Robust incremental optical flow", CVPR'92
Black moy= 3.52 sigma= 3.25 "Recursive non linear estimation of discontinuous flow field", ECCV'94
Black et Jepson moy= 2.29 sigma= 2.25 "Estimating optical flow in segmented images using variable-order parametric models with local deformations", PAMI vol. 18, 1996
Ju, Black et Jepson moy= 2.16 sigma= 2.0 "Skin and bones: multi-layer locally affine, optical flow and regularization with transparency", CVPR'96
Lai et Vemuri moy= 1.99 sigma= 1.41 "Reliable and efficient computation of optical flow", IJCV 29(2) 1998


  1. E. Mémin, P. Pérez. Joint estimation-segmentation of optic flow.  Proc. of the 5th European Conf. on Computer Vision, ECCV'98, Volume 2, pages 563-580, Freiburg, Germany, June 1998. (postscript)
  2. E. Mémin et P. Pérez. Hierarchical estimation and segmentation of dense motion fields. submitted to Int. Journ of Comp. Vision. (postscript)