Efficient Hierarchical Graph-Based Segmentation of RGBD Videos

26th IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, June 2014


Teaser [PNG]
Authors
Steven Hickson
College of Computing, Georgia Tech
shickson [at] gatech.edu
Stan Birchfield
Microsoft
stanleyb [at] microsoft.com
Irfan Essa
College of Computing, Georgia Tech
irfan [at] cc.gatech.edu
Henrik Christensen
College of Computing, Georgia Tech
hic [at] cc.gatech.edu

Abstract
We present an efficient and scalable algorithm for seg- menting 3D RGBD point clouds by combining depth, color, and temporal information using a multistage, hierarchical graph-based approach. Our algorithm processes a moving window over several point clouds to group similar regions over a graph, resulting in an initial over-segmentation. These regions are then merged to yield a dendrogram us- ing agglomerative clustering via a minimum spanning tree algorithm. Bipartite graph matching at a given level of the hierarchical tree yields the final segmentation of the point clouds by maintaining region identities over arbitrarily long periods of time. We show that a multistage segmentation with depth then color yields better results than a linear com- bination of depth and color. Due to its incremental process- ing, our algorithm can process videos of any length and in a streaming pipeline. The algorithm’s ability to produce robust, efficient segmentation is demonstrated with numer- ous experimental results on challenging sequences from our own as well as public RGBD data sets.

Paper
Paper [PDF]
Download PDF

Video

Poster
Poster [PDF]
Download Poster

Code/Data
The code requires OpenCV and PCL and should be compatible in Windows and Linux with the most time-of-flight and projected infrared devices. It can be downloaded at https://github.com/StevenHickson/4D_Segmentation

Citation
S. Hickson, S. Birchfield, I. Essa, and H. Christensen (2014), “Efficient Hierarchical Graph-Based Segmentation of RGBD Videos,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
@inproceedings{2014-Hickson-EHGSRV,
       Author = {Steven Hickson and Stan Birchfield and Irfan Essa and Henrik Christensen},
       Booktitle = {Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
       Month = {June},
       Organization = {IEEE Computer Society},
       Title = {Efficient Hierarchical Graph-Based Segmentation of RGBD Videos},
       Year = {2014}
}

Acknowledgement
The authors would like to thank Prof. Mubarak Shah and his PhD Student Gonzalo Vaca-Castano for their mentorship and guidance to the primary author of this paper, when he participated in the National Science Foundation funded " REU Site: Research Experience for Undergraduates in Computer Vision" (#1156990) in 2012 at University of Central Florida's Center for Research in Computer Vision .
In addition, we would also like to thank TUM and NYU for providing datasets. Details and links forthcoming.

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without explicit permission of the copyright holder.


COC@GT Logo  ---  --- RIM@GT Logo