Research
[Home] [Research] [Students] [Teaching] [Contact]
[Presentations] [Publications/CV] [Prospective Students] [CPL]

 

This is a Research Overview page; it has a partial listing of projects and selected publications.    The publications are way out of date. There is also a not so out of date complete listing of my  publications.

For a complete set of projects at CPL, go to the CPL Project page. 

  Also for some of my older work not mentioned below you might try the MIT Media Lab Vision and Modeling Projects page.

General Overview

For the last several years my work has focused on the visual machine perception of action. Lately I've taken to thinking of the different motion understanding problems as a taxonomy consisting of movement, activity, and action.  Some general papers are:

  1. Bobick, A. "Movement, Activity, and Action: The Role of Knowledge in the Perception of Motion." Phil. Trans. Royal Society London B, 352, pp.1257-1265, 1997. Postscript
  2. Bobick, A. "Computers Seeing Action." British Machine Vision Conference, Edingburgh, Scotland, pp. 13-22, September 1996.

Recognition of Human Movement

Movement is the most primitive form of motion that can be interpreted semantically. Movements are typically atomic, with no distinct parts. To recognize movements one only needs a description of the appearance of the motion. Time can typically be handled by only linear scaling. Sitting down or swinging a bat are good examples.  Lately we have begun work on gait recognition and gait.

Some basic human movement recognition ideas and body tracking:

bulletCampbell, L. and A. Bobick, "Recognition of Human Body Motion Using Phase Space Constraints." Fifth International Conference on Computer Vision, Cambridge, MA, 624-630, June 1995. Postscript
bulletBobick, A. and J. Davis, "The recognition of human movement using temporal templates," IEEE Transaction on Pattern Analysis & Machine Intelligence, 23(3), March 2001.
bulletKwatra, V., A.F. Bobick, and A. Johnson, " Temporal Integration of Multiple Silhouette-based Body-part Hypotheses” Proceedings of IEEE Computer Vision and Pattern Recognition, Kauai, Hawaii, December 2001.

Work on gait.  There is a Human ID  gait page at the CPL projects site.  Here are some papers where we introduce Expected Confusion and show how it relates t ROC curves (the work with Amos Johnson and Jie Sun))  Also are some papers about how gait varies with speed.

bulletBobick, A.F., and A. Johnson  "Gait recognition using static activity-specific parameters" ” Proceedings of IEEE Computer Vision and Pattern Recognition, Kauai, Hawaii, December 2001.
bulletJohnson, A. and A.F. Bobick, "A Multi-view Method for Gait Recognition Using Static Body Parameters" 3rd International Conference on Audio- and Video Based Biometric Person Authentication, 301-311, Halmstad, Sweden, June 2001.
bulletJohnson, A. and A.Bobick, “Reation between Expected Confusion and ROC curves,” Int’l Conference of Pattern Recognition, Quebec, Canada, August, 2002.

Gesture recognition - We have done a variety of state-based  recognition work on gesture that extend HMMs or define a more explicit set of states. 

bulletBobick, A. and A. Wilson, “A State-based Approach to the Representation and Recognition of Gesture.” IEEE Transaction on Pattern Analysis & Machine Intelligence, Vol. 19, No. 12, December 1997.
bulletWilson,A., and A.. Bobick, “Hidden Markov Models for Modeling and Recognizing Gesture Under Variation”, Int’l J. of Pattern Recognition and Artificial Intelligence 15(1): 123-160 (2001)
bulletWilson, A. and A. Bobick, "Parametric Hidden Markov Models for Gesture Recognition," IEEE Transaction on Pattern Analysis & Machine Intelligence, 21(9), September 1999, pp. 884-900.
bulletWilson, A. and A. F. Bobick, "Real-time online adaptive gesture recognition," International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, Corfu, Greece, September 1999.
bulletWilson, D. and A. Bobick, "Nonlinear PHMMs for the Interpretation of Parameterized Gesture", To Appear in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition , Santa Barbara, CA, June 1998. Postscript
bulletWilson, D., A. Bobick and J. Cassell, "Temporal classification of natural gesture and application to video coding", Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, pp. 948-954, June 1997. (Closest abstract and postscript.)
bulletWilson, A. and A. Bobick, "Learning Visual Behavior for Gesture Analysis." Proceedings of the IEEE Symposium on Computer Vision, Coral Gables, FL, pp. 229-234, November 1995. Postscript

Activity Recognition and Surveillance

For some demos and other group publications go to the Action Recognition site at CPL. Some papers that include new methods of representing and recognizing activities, especially appropriate for surveillance or monitoring domains:

  1. Ivanov, Y. and Aaron F. Bobick, “Recognition of Visual Activities and Interactions by Stochastic Parsing”, IEEE Transactions on Pattern Analysis & Machine Intelligence, 22(8), August, 2000.
  2. Ivanov, Y. and A. F. Bobick, "Recognition of Multi-agent Interaction in Video Surveillance", Proc. of Intl Conference on Computer Vision, Corfu, Greece, pg. 169-176, September 1999.
  3. Ivanov, Y., C. Stauffer, A.F. Bobick and W.E.L Grimson, "Video Surveillance of Interactions", Workshop on Video Surveillance, June 1999, Ft. Collins, CO
  4. Pinhanez, C. and A. Bobick, "Human Action Detection Using PNF Propagation of Temporal Constraints", Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, pp. 898-904, June 1998.

Complex Actions

Actions are the high-level entities that people typically use to describe what is happening. Two examples from my own work are "The chef is mixing the ingredients." or the "NE Patriots just ran a p51-curl play.'"  Representing and recognizing actions requires qualitative descriptions of time, such as intervals. Context becomes fundamental, though how much one has to reason about context is unclear. Currently we are focusing on actions that can be recognized by what they look like as opposed to needing to reason about such unsavory things as intentionality and the like.

  1. Intille, S. and A.F. Bobick, “Recognizing planned, multi-person action”, Computer Vision and Image Understanding 81, 414–445 (2001)
  2. Pinhanez, C. and A. Bobick, “Interval Scripts,” Presence, to appear.
  3. Pinhanez, C. and A. Bobick, "Human Action Detection Using PNF Propagation of Temporal Constraints",Proceedings of IEEE Conference on Computer Vision and Pattern Recognition , Santa Barbara, CA, June 1998. Postscript
  4. Bobick, A. and C. Pinhanez, "Controlling View-Based Algorithms Using Approximate World Models and Action Information." Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, pp. 955-961, June 1997. (Closest abstract and postscript.)
  5. Pinhanez, C. and A. Bobick, "Approximate World Models: Incorporating Qualitative and Linguistic Information into Vision Systems." Proceedings of the AAAI '96, Portland, OR, pp. 1116-1123, August 1996. Postscript

Interactive Environments

Work includes both representation of the action of the environment (a notion of scripting) and some vision methods particularly suited for such environments.

Representation of action and story:

  1. Pinhanez, C., J. Davis , S. Intille, M. Johnson, A. Wilson, A. Bobick, B. Blumberg, “Physically Interactive Story Environments, IBM Systems Journal, 39 (3&4): 438-455, 2000
  2. Bobick, A., S. Intille, J. Davis, F. Baird, C. Pinhanez, L. Campbell, Y. Ivanov, A. Schutte, A. Wilson, “The KidsRoom: A Perceptually-based Interactive and Immersive Story Environment,” PRESENCE: Teleoperators and Virtual Environments, 8(4), August 1999. pp. 367-391.
  3. Pinhanez, C. and A. Bobick, “Intelligent Studios: Modeling Space and Action to Control TV Cameras.” Applications of Artificial Intelligence, Vol. 11(4), pp. 285-306, 1997.
  4. Pinhanez, C. and A. F. Bobick. "'It/I': A Theater Play Featuring an Autonomous Computer Graphics Character", Proc. of the ACM Multimedia'98 Workshop on Technologies for Interactive Movies, Bristol, England, pp. 22-29. September. 1999.
  5. Pinhanez, C. and A. Bobick, " Using Computer Vision to Control a Reactive Computer Graphics Character in a Theater Play", Proceedings of International Conference on Vision Systems (ICVS’99), Las Palmas, Gran Canaria, Spain, January 1999.
  6. Pinhanez, C. and A. Bobick, " It/I: An Experiment Towards Interactive Theatrical Performances", Proceedings of CHI '98, Los Angeles, CA. pp. 333-334, April 1998. (Closest abstract and postscript.)
  7. Pinhanez, C., K. Mase and A. Bobick, "Interval Scripts: A Design Paradigm for Story-Based Interactive Systems." Proceedings of CHI '97, Atlanta, GA, pp. 287-294, March 1997. Postscript

Technologies for interactive environments:

  1. Davis, J. and A. Bobick, "A Robust Human-Silhouette Extraction Technique for Interactive Virtual Environments", IFIP Workshop on Modeling and Motion Capture Techniques for Virtual Environments (CAPTECH98), November 1998.
  2. Intille, S., J. Davis and A. Bobick, "Real-Time Closed-World Tracking." Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 697-703, Puerto Rico, June 1997. Postscript
  3. Pinhanez, C. and A. Bobick, "Intelligent Studios: Modeling Space and Action to Control TV Cameras." Applications of Artificial Intelligence, Vol. 11(4), pp. 285-306, 1997. Postscript
  4. Ivanov, Y., A. Bobick, and J. Liu, "Fast lighting independent background subtraction," International Journal of Computer Vision, 37(2), June 2000, pp. 199-209.
  5. Johnson, M.P., A. Wilson, B. Blumberg, C. Kline, and A. Bobick, "Sympathetic interfaces: using a plush toy to direct synthetic classes", Proceedings of Computer Human Interface, Pittsburgh, pp. 152 – 158, May 1999

[Home] [Research] [Students] [Teaching] [Contact]
[Presentations] [Publications/CV] [Prospective Students] [CPL]

This page was last updated on 11/28/03 01:46:36 AM.