Abstract
This paper deals with full-frame estimation of optical flow in a generalized imaging system by exploiting probabilistic subspace constraints on the flow. We deal with the extended motion of the imaging system through an environment that we assume to have some degree of statistical regularity. For example, in autonomous ground vehicles the structure of the environment around the vehicle is far from arbitrary. We exploit this regularity to predict the perceived optical flow due to platform motion. The subspace constraints hold not only for perspective cameras, but in fact for a very general class of imaging systems, including catadioptic and multiple-view systems. Using minimal assumptions about the imaging geometry, we derive a probabilistic subspace constraint that captures the statistical regularity of the scene geometry relative to an imaging system. We propose an extension to probabilistic PCA (Tipping and Bishop, 1999) as a way to learn this subspace from a large amount of recorded imagery, and demonstrate its use in conjunction with a sparse optical flow algorithm. To deal with the sparseness of the input flow, we use a generative model to estimate the full-dimensionality subspace using only the observed flow measurements. Additionally, to identify and cope with image regions that violate the subspace constraints, such as moving objects or gross flow estimation errors, we employ a per-pixel Gaussian mixture outlier process. We demonstrate results of finding the optical flow subspaces and employing them to estimate full-frame flow and to recover camera motion, for a variety of imaging systems in several different environments.
Publications
Learning General Optical Flow Subspaces for Egomotion Estimation and Detection of Motion Anomalies. Richard Roberts and Christian Potthast and F. Dellaert. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR, oral presentation, 4.1% acceptance). 2009. [ pdf ]
Memory-Based Learning for Visual Odometry. R. Roberts and H. Nguyen and N. Krishnamurthi and T. Balch. IEEE International Conference on Robotics and Automation (ICRA, oral presentation), 2008. [ pdf | Video mov ]
Videos
Submitted to the CVPR Video Review.
This video illustrates estimating dense optical flow from sparse flow, and also ego-motion estimation on a mobile robot. This video shows sparse flow vectors labelled as inliers (green) and outliers (red). In the first half of the video, the right side shows esimated dense flow, while in the second half of the video the right side shows the recovered platform trajectory (blue path) and approximate ground truth (green path).
