NOTE : This is a GVU-internal page intended only for viewing by department members. Other interested parties should contact Gabriel Brostow (CoC Room #214 or email@example.com) or Spencer Reynolds (firstname.lastname@example.org)
3D Puppet Reconstruction (under construction)
The purpose of this project is to develop a method where an animated 3D model of a puppet/actor can be generated from video footage taken of the moving subject from several, synchronized, and calibrated views. The final CG reconstruction will include both surface and joint characteristics that represent those of the physical model. This method is being presented as an alternative to markered motion capture.
Our protagonist. (Click for larger image)
The primary steps include: capturing the video footage from many angles, calibrating the cameras (to determine their intrinsic and extrinsic parameters), segmenting the character from the background, generating the model in voxel space, and looking for consistencies in the motion. Creation of a voxel representation is accomplished by "carving" the shape of the character using the silhouettes of the model from the many views. Motion consistencies are used to determine the locations of joints and their ranges of motion. The inital test was to determine the feasability of the voxel carving process.
Capturing the video footage (Click for larger image)
Segmentation (Click for larger image)
Camera calibration (Click for larger image)
The test was carried out using only three cameras, but the results are promising assuming that more cameras are used.
(Click on images for larger size)
Here are some more recent results of the research.
One of the camera setups (Click for larger image)
MOVIE CLIP : Virtual camera setup in Maya
MOVIE CLIP : "Bone" segments and joints
MOVIE CLIP : Surface normals of the voxel camel
More surface normals of the voxel camel (Click for larger image)