Much work has been done by the vision community on extracting camera motion and scene parameters from relatively short video sequences. There are many problems that must be dealt with in order to use the theory that has been developed for short sequences on extended sequences.
There are two main characteristics of extended video sequences that we want to take advantage of. First, the coupling between camera/scene parameters can vary significantly. If many images are taken from roughly the same position, they will be highly coupled to each other and will also be highly redundant. Conversely, some images are very weakly coupled to each other and do not contain much redundant information.
Our first contribution to the field was developing a framework for adding new images from a video sequence into a SFM solution incrementally. Our paper, Propagation of Innovative Information in Non-linear Least-Squares Structure from Motion, outlines this framework. With our technique, when a new camera or scene observation is added to the system, only camera or scene parameters that are highly correlated to the new observation will need to be updated.
In our current work, we are further extending our framework to exploit the redundancy of parameters. We are developing a principled approach to grouping and optimizing parameters that are correlated/redundant and thereby further compressing the dimensionality of the error function.
[abstract, pdf, ps] D. Steedly and I. A. Essa. Propagation of Innovative Information in Non-Linear Least-Squares Structure from Motion. International Conference on Computer Vision 2001.
D. Steedly, I. A. Essa and F. Dellaert. Spectral Partitioning for Structure
From Motion. International
Conference on Computer Vision 2003.