LEGO Tracking

Abstract

Visual tracking is an important problem in computer vision which aims to track the 2-D or 3-D spatial configurations of the targets given a sequence of images. In this project, our goal is to develop a real-time markerless 3-D LEGO tracker on hand-held device for augmented reality applications. This tracker will be the first step to realize many human-LEGO interaction applications. For example, if we can keep tracking how users manipulate and assemble LEGO blocks into complex models, users can download the resulting digital LEGO models to their computers. In a more advanced application, we can analyze users strategies and behaviors of building complex models from simple LEGO blocks. In contrast to conventional tracking applications which focus on tracking targets with distinct appearance or rich texture, our tracking target, LEGO, is textureless and have only relatively reliable edges on their surfaces. This property poses much difficulty for tracking because reliable correspondences on LEGO blocks are not available. To resolve this problem, we use particle filtering to maintain a posterior distribution for the LEGO's 3-D position. It can alleviate the drifting problem encountered in the algorithms based on Kalman filters. Edge, color features and particle filters are used. OpenGL shader is used to speedup the likelihood evaluation.

Demo