Propagation Networks for Recognizing Partially Ordered Sequential ActivityYifan Shi, Yan Huang, David Minnen, Aaron Bobick, Irfan Essa
|
||
This work is presented in CVPR'2004, (PDF, BIB)
Represent and fuse human knowledge of daily activities with noisy perceptual features | |
Detect and recognize an activity | |
Pinpoint components of the activity and detect missing or improperly performed steps |
Difficulty in modeling parallel streams
| |||||||||||
Difficulty in handling noisy sensor data
|
Handle multiple, parallel streams | |
Naturally represent partially ordered activity components | |
Directly model duration of components | |
Deal with noisy underlying vision sensor data | |
Classify whole sequence as well as identify individual components | |
Applied to recognition of actions in the medical domain |
Use links to represent temporal partial ordering; | |
Use nodes with duration model to represent temporal interval; | |
Represent parallel activity stream with multiple, simultaneously active nodes; | |
Use conditional probability function to represent temporal/logical relationship between nodes; | |
Embed duration model in activation triggering function; | |
Use hidden observation model for underlying vision uncertainty |
Local Maximal Search Algorithm
| |||||||||||
D-Condensation
|
Model the complex LifeScan glucose calibration process with a P-Net using simple vision detectors | |||||||
3 subjects with 41 videos | |||||||
3 task categories:
|
P-Net Definition for Glucose Meter Calibration Task |
System Architecture |
Detection rate for individual nodes over all samples |
Ability to locate critical moment |
TurnOn [44] |
Unscrew Cap [229] | Drop Solution [256] | Screw Cap [288] | Insert Strip [329] | Mark Ctrl [655] |
Snapshots from example sequence and corresponding detection results |
Subject1 correct sequence 1 subject1_right1.avi | |
Subject1 correct sequence 2 subject1_right2.avi | |
Subject1 correct sequence 3 subject1_right3.avi | |
Subject1 correct sequence 4 subject1_right4.avi | |
Subject1 correct sequence 5 subject1_right5.avi | |
Subject1 correct sequence 6 subject1_right6.avi | |
Subject1 correct sequence 7 subject1_right7.avi | |
Subject1 correct sequence 8 subject1_right8.avi | |
Subject1 correct sequence 9 subject1_right9.avi | |
Subject1 correct sequence 10 subject1_right10.avi |
Subject2 correct sequence 1 subject2_right1.avi | |
Subject2 correct sequence 2 subject2_right2.avi | |
Subject2 correct sequence 3 subject2_right3.avi | |
Subject2 correct sequence 4 subject2_right4.avi | |
Subject2 correct sequence 5 subject2_right5.avi | |
Subject2 correct sequence 6 subject2_right6.avi | |
Subject2 correct sequence 7 subject2_right7.avi | |
Subject2 correct sequence 8 subject2_right8.avi | |
Subject2 correct sequence 9 subject2_right9.avi |
Subject2 missing-one-step sequence 1 subject2_noshake1.avi | |
Subject2 missing-one-step sequence 2 subject2_noshake2.avi | |
Subject2 missing-one-step sequence 3 subject2_noshake3.avi | |
Subject2 missing-one-step sequence 4 subject2_noshake4.avi | |
Subject2 missing-one-step sequence 5 subject2_noshake5.avi | |
Subject2 missing-one-step sequence 6 subject2_noshake6.avi | |
Subject2 missing-one-step sequence 7 subject2_noshake7.avi |
Subject2 missing-six-step sequence 1 subject2_noapply1.avi | |
Subject2 missing-six-step sequence 2 subject2_noapply2.avi | |
Subject2 missing-six-step sequence 3 subject2_noapply3.avi | |
Subject2 missing-six-step sequence 4 subject2_noapply4.avi | |
Subject2 missing-six-step sequence 5 subject2_noapply5.avi |
Subject3 correct sequence 1 subject3_right1.avi | |
Subject3 correct sequence 2 subject3_right2.avi |
Subject3 missing-one-step sequence 1 subject3_noshake1.avi | |
Subject3 missing-one-step sequence 2 subject3_noshake2.avi | |
Subject3 missing-one-step sequence 3 subject3_noshake3.avi |
Subject3 missing-six-step sequence 1 subject3_noapply1.avi | |
Subject3 missing-six-step sequence 2 subject3_noapply2.avi | |
Subject3 missing-six-step sequence 3 subject3_noapply3.avi | |
Subject3 missing-six-step sequence 4 subject3_noapply4.avi | |
Subject3 missing-six-step sequence 5 subject3_noapply5.avi |
Copyright © 1997-2005 |
Last Updated Dec 31, 2004.