Current Research

I am interested in exploring fundamental research problems at the intersection of Computational Perception, Statistical Learning, and Ubiquitous Computing. In particular, I want to designing systems that can perceive, learn, and predict what is happening around them. Recently, I have been exploring various computational mechanisms for the perceptual understanding of human activities. A brief description of my research interests can be found in my Research Statement.

Here are some of the specific topics I have explored so far.

Structure from Statistics - Suffix Trees for Activity Analysis

Models for activity structure for unconstrained environments are generally not available a priori. Recent representational approaches to this end are limited by their computational complexity, and ability to capture activity structure only up to some fixed temporal scale. In this work, we propose the usage of Suffix Trees as an activity representation to efficiently extract structure of activities by analyzing their constituent event-subsequences over multiple temporal scales.

[Publication]

Unsupervised Analysis of Activities using Event-Motifs

For an active environment, how can one transform semantically agnostic low-level perceptual inputs, into some mid-level abstractions that sufficiently encode the activity structure? How can one represent such activity structure over a continuum of temporal resolutions? Finally, how can one automatically detect event subsequences that are locally atypical in a structural sense? In this work, we investigate these questions in the context of understanding everyday activities.

[Publication]

Discovery & Characterization of Activities from Event Streams

A key step towards understanding what is happening in an active setting, is to discover the various kinds of frequently occurring similar activities in that domain. Equally important is the question of finding efficient characterizations for these different kinds of activities. In this work we tackle the question of activity class discovery and characterization, in the backdrop of analyzing everyday activities.

[Publication]

Anomaly Explanation - Activities as Bags of Event n-grams

Anomalies are sets of rare events which, for any reasonably unconstrained situation, are hard to completely define as a prior. For the reasons of rarity and large within-class variation of anomalies, techniques which try to model them, either statistically or through a set of rules, often prove to be brittle and over-fitted. We formulate the problem of Anomalous Activity Explanation by proposing a novel representation of activities as bags of n-grams of discrete events.

[GVU Brown Bag Talk] [Publication] [Project Page]

Probabilistic Graphical Models for Human Activity Recognition

A novel framework for recognizing complex multi-agent activities using probabilistic graphical models is presented. We employ statistical feature based particle filter to robustly track multiple objects in cluttered environments. Spatio-temporal features extracted from tracking are thereon used to build probabilistic graphical models for characterization of these activities.

[Project Page] [Publication]

 

Previous Research Projects

Following are few of the research projects I have explored while collaborating with different research groups.

Taylor Expansion Based Classifier Adaptation: Application to Person Detection

 

Due to the large variation in the physical attributes of different environments, a generic classifier trained on extensive data-sets my still perform sub-optimally in a new test environment. In this work we present a general framework for classifier adaptation that allows an already trained generic classifier to perform better in new test environments. The work was done at Microsoft Research, 2007.

Accepted for CVPR 2008.

Weighted Ensemble Boosting for Robust Activity Recognition

 

The weighted Ensemble Boosting method combines Bayesian Averaging strategy coupled with Boosting framework, finding useful conjunctive features-combinations and achieving lower error rates than traditional Boosting algorithm. The method demonstrates a comparable level of stability with respect to the classifier selection pool. We compare its performance with different classifier combination methods, including Approximate Bayesian Combination, Boosting, Feature Stacking and the more traditional Sum and Product rules. The work was done at Mitsubishi Electronic Research Lab, 2005.

[Publication]

Programming Context Aware Applications by Demonstration

Programming context aware applications incorporating both implicit and explicit modes of input in which the design of the application is put in the hands of the end users themselves. The work was done for Intel Research Lab Berkeley, 2003.

[Publication]

A Variational Approach to Audio-Visual Flow Estimation

The flow field of a moving sound source not only has an optical component, but also an audio component; something we call audio-visual flow. We present a common structure-tensor based variational framework for dense audio-visual flow-field estimation.

[Publication]

Automatic Automobile Occupancy Detection

Decision Tree based Object Classifiers for automatic automobile detection system. The project was a collaborative effort between General Motors, & Techlogix Inc.  The project resulted in a US patent and a publication.

[Publication]

Mobile ADVICE: Design of an accessible mobile device

The visually impaired have limited access to the world of mobile devices. Our goal was to design a handheld mobile device to overcome limitations such as reliance on visual display and lack of audio and tactile feedback. We built a prototype handheld device using a combination of tactile feedback and auditory display based on preliminary research and testing.

[Publication]