Michael Kaess
Center for Robotics and Intelligent Machines, Georgia Tech GT CoC IC GVU RIM@GT BORG

please see my MIT page for up-to-date information

3D Boundary Reconstruction from Multiple Views

Publications

  • “Reconstruction of Objects with Jagged Edges through Rao-Blackwellized Fitting of Piecewise Smooth Subdivision Curves” by M. Kaess and F. Dellaert. In Proceedings of the IEEE 1st International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, (Nice, France), Oct. 2003, pp. 39-47. Details. Download: PDF.
  • “MCMC-based Multiview Reconstruction of Piecewise Smooth Subdivision Curves with a Variable Number of Control Points” by M. Kaess, R. Zboinski, and F. Dellaert. In Eur. Conf. on Computer Vision, ECCV, (Prague, Czech Republic), May 2004, pp. 329-341. Acceptance ratio 34.2% (190 of 555). Details. Download: PDF.

Approach

Shard on calibration pattern.
Given a set of images of an object we want to fit a 3D curve to the boundary of the object. The cameras are calibrated using standard computer vision approaches, based on the calibration pattern partially visible in the background.

Subdivision process in 1D.
We use subdivision curves as a powerful curve representation. Depending on an averaging mask, different types of curves can be generated that range from B-splines to fractals.

A tagged 2D subdivision curve
Tagging allows easy modeling of non-smooth features by locally using a different averaging mask.
We use a generative model and define the likelihood function based on the Chamfer distance, that can efficiently be precalculated.
Because of the huge combined space of continuous 3D control point locations and discrete taggings, we choose to sample from the posterior distribution. Rao-Blackwellization makes efficient sampling possible by integrating out the continuous part of the state space.
We do not know how many control points are needed to represent the outline sufficiently well, i.e. we are faced with a model selection problem. A reversible-jump Markov chain Monte Carlo sampler solves this problem by allowing adding and removing of control points.

Automatically segemented shard image.
No manual interaction is needed in this process, as the object is automatically segemented from the calibration background by a Markov Random Field approach.

Samples, visualized by reprojection.
We visualize the development of the 3D curve by reprojecting it into three different views (columns). Note that we show the initial guess (circular configuration of the control points), and the fifth and 250th sample, while the complete set of samples approximates the posterior distribution and can for example be used to obtain marginal distributions.