3D Boundary Reconstruction from Multiple Views

Publications

• “Reconstruction of Objects with Jagged Edges through Rao-Blackwellized Fitting of Piecewise Smooth Subdivision Curves” by M. Kaess and F. Dellaert. In Proceedings of the IEEE 1st International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, (Nice, France), Oct. 2003, pp. 39-47. Details. Download: PDF.
• “MCMC-based Multiview Reconstruction of Piecewise Smooth Subdivision Curves with a Variable Number of Control Points” by M. Kaess, R. Zboinski, and F. Dellaert. In Eur. Conf. on Computer Vision, ECCV, (Prague, Czech Republic), May 2004, pp. 329-341. Acceptance ratio 34.2% (190 of 555). Details. Download: PDF.

Approach

Given a set of images of an object we want to fit a 3D curve to the boundary of the object. The cameras are calibrated using standard computer vision approaches, based on the calibration pattern partially visible in the background.

We use subdivision curves as a powerful curve representation. Depending on an averaging mask, different types of curves can be generated that range from B-splines to fractals.

Tagging allows easy modeling of non-smooth features by locally using a different averaging mask.
We use a generative model and define the likelihood function based on the Chamfer distance, that can efficiently be precalculated.
Because of the huge combined space of continuous 3D control point locations and discrete taggings, we choose to sample from the posterior distribution. Rao-Blackwellization makes efficient sampling possible by integrating out the continuous part of the state space.
We do not know how many control points are needed to represent the outline sufficiently well, i.e. we are faced with a model selection problem. A reversible-jump Markov chain Monte Carlo sampler solves this problem by allowing adding and removing of control points.

No manual interaction is needed in this process, as the object is automatically segemented from the calibration background by a Markov Random Field approach.