Motion Based Decompositing of Video
by Gabriel J. Brostow, Irfan A. Essa

College of Computing, GVU Center,
Georgia Institute of Technology,
Atlanta, GA 30332-0280 USA

Presented at ICCV 1999 in Corfu, Greece.

We present a method to decompose video sequences into layers that represent the relative depths of complex scenes. Our method combines spatial information with temporal occlusions to determine relative depths of these layers. Spatial information is obtained through edge detection and a customized contour completion algorithm. Activity in a scene is used to extract temporal occlusion events, which are in turn, used to classify objects as occluders or occludees. The path traversed by the moving objects determines the segmentation of the scene. Several examples of decompositing and compositing of video are shown. This approach can be applied in the pre-processing of sequences for compositing or tracking purposes and to determine the approximate 3D structure of a scene.

PowerPoint Slides Used in Oral Presentation of this paper at the conference. (Videos not embedded. Samples are linked below instead.)

PS Version of the paper.
PDF Version of this paper.

Sample input data and processing results (QuickTime movies)

Motivation for our approach:
Click to see an MHI of this sequence. Note that the chair is easy to spot because it is occluding the walking motion.

Simple example of pushing into a background layer. (5.7Mb)                 Step-by-step decomposition of "toy" sequence. (17Mb)
                     &nb sp;                              Note both pushing and popping occur here.

Original sequence (2.5Mb):
                 &n bsp;     Results of processing and demonstration of recompositing application. (2.5Mb)

Original sequence (5Mb):                                       &n bsp;                                           ;                           Hiding-edge detection (3Mb)
                    ;        Results of processing (1.5Mb)                 

Original sequence (6Mb):
Results appear in PowerPoint presentation.