Understanding of Captured Experiences
The Automated Understanding of
Captured Experiences project is aimed at reducing
substantially the human input necessary for creating and accessing large
collections of multimedia, particularly multimedia created by capturing
what is happening in an environment.
This project is Funded by the National Science Foundation (Grant # 9806822)
[Sept 1, 1998 - Aug 31, 2001]
(The material presented on this WWW page is based upon work supported
by the National Science Foundation under Grant No. 9806822)
The objective of this research is to reduce substantially the human input
necessary for creating and accessing large collections of multimedia, particularly
multimedia created by capturing what is happening in an environment. The
existing software system which is being used as the starting point for
this investigation is Classroom 2000, a system designed to capture what
happens in classrooms, meetings, and offices. Classroom 2000 integrates
and synchronizes multiple streams of captured text, images, handwritten
annotations, audio, and video. In a sense, it automates note-taking for
a lecture or meeting. The research challenge is to make sense of this flood
of captured data. The project explores how the output of Classroom 2000
can be automatically structured, segmented, indexed, and linked. Machine
learning and statistical approaches to language are used to attempt to
understand the captured data. Techniques from computational perception
are used to try to find structure in the captured data. An important component
of this research is the experimental analysis of the software system being
built. The expectation is that this research will have a dramatic impact
on how humans work and learn, as technology aids humans by capturing and
making accessible what happens in an environment.
Salber, Post-Doctoral Researcher 1999-2000. Now at IBM Research NY
Lionel Reveret, Post-Doctoral
Researcher 2000-2001, Now at INRIA France
Gary Boone, Graduated
with PhD 2000, Now at Accenture, CA
Graduated with PhD 2002, Now a Research Scientist at GA Tech.
Anind Dey, Graduated with PhD 2001, Now at Intel
Darnell Moore, Graduated
with PhD 2000, Now at Texas Instruments
Jen Mankoff, Graduated with PhD 2001, Now Assistant Prof at Berkeley
Graduated with MS 2000, Now graduate student in Berlin
- Undergraduate Students
- Scott Carter
- Frank Casteneda
- Jahir Pabon
- Yanir Hershberg
Related Research Groups at GA Tech
Compaq CRL (R. Nikhil and J. Rehg)
Texas Instruments (Darnell Moore)
Intel Corporation (Anind Dey)
NIST (Vince Stanford)
PI Information Request 2002
Gregory D. Abowd, "Classroom 2000: An Experiment with the Instrumentation of a
Living Educational Environment", IBM Systems Journal
Gregory D. Abowd and Elizabeth D. Mynatt, "Charting past, present and future
research in ubiqutous computing", ACM Transactions on Computer-Human Interaction
(special issue on HCI research in the new millenium), p. 29, vol. 7/1, (2000).
Gregory D. Abowd and J. Sterbenz, "Report on the Inter-Agency Workshop on
Research Issues for Smart Environments", IEEE Personal Communications, p. 36,
vol. 7, (2000). ) Published
Jennifer Mankoff, Gregory D. Abowd, Scott E. Hudson, "OOPS: A toolkit supporting
mediation techniques for resolving ambiguity in recognition-based interfaces",
Computers and Graphics (special issue on Calligraphic Interfaces: towards a new
generation of interactive systems), p. 819, vol. 24/6, (2000).
Maria da Graca Pimentel, Yoshihide Ishiguro, Bolot Kerimbaev, Gregory D. Abowd,
Mark Guzdial, "Supporting long-term educational activities through dymanic web
interfaces", Interacting with Computers (special issue on interfaces for the
active Web), p. 353, vol. 13/3, (2001).
Anind K. Dey, Daniel Salber, Gregory D. Abowd, "A conceptual framework and a
toolkit for supporting the rapid prototyping of context-aware applications",
Human-Computer Interaction (HCI) Journal (anchor article of a special issue on
Context-Aware Computing), p. , vol. 16, (2001).
James M. Rehg, Kathleen Knobe, Umakishore Ramachandran, Richiyur S. Nikhil,
"Integrated Task and Data Parallel Support for Dynamic Applications", Journal of
Scientific Programming, p. 1, vol. 1, (2000).
K. Knobe, J. M. Rehg, A. Chauhan, R.S. Nikhil, U. Ramachandran, "Scheduling
Constrained Dynamic Applications on Clusters", Supercomputing 99, p. 1, vol. 12,
S. Nikhil, U. Ramachandran, "Garbage Collection of Timestamped Data in
Stampede", Principles of Distributed Computing 2000, p. 1, vol. 7, (2000).
I. Essa, "Computers Seeing People", AI Magazine, p. 69, vol. 20/1, (1999).
Irfan Essa, "Ubiquitous Sensing for Smart and Aware Environments", IEEE Personal
Communications (special issue on Networking the Physical World, p. 47, vol. 7/5,
Jennifer Mankoff, Scott Hudson, Gregory D. Abowd, "Providing integrated
toolkit-level support for ambiguity in recognition-based interfaces",
Proceedings of CHI 2000, p. 368, vol. 1, (2000).
Robert Orr, Gregory D. Abowd, "The smart floor: A mechanism for natural user
identification and tracking", Companion Proceedings of CHI 2000 (short paper),
p. 1, vol. 2, (2000).
Gregory D. Abowd, Christopher G. Atekson, Aaron Bobick, Irfan A. Essa, Blair
MacIntyre, Elizabeth D. Mynatt, Thad Starner, "The Future Computing Environments
Group at the Georgia Institute of Technology", Proceedings of CHI 2000
(companion proceedings) / Organizational overview, p. 1, vol. 2, (2000).
Maria Pimentel, Gregory D. Abowd, Yoshihide Ishiguro, "Linking by Interacting: A
paradigm for authoring hypertext", Proceedings of the 11th ACM Conference on
Hypertext and Hypermedia 2000, San Antonio, Texas, p. 1, vol. 1, (2000).
A.K. Dey, G.D. Abowd (principal research by Dey, shared authorship), "CyberMinder:
A Context-Aware System for Supporting Reminders", Proceedings of the 2nd
International Symposium on Handheld and Ubiquitous Computing (HUC2K), p. 172,
vol. 1, (2000).
Jennifer Mankoff, Scott E Hudson, Gregory D Abowd, "Interaction techniques for
ambiguity resolution in recognition-based interfaces", Proceedings of the
Symposium on User Interface Software Technology - UIST 2000, p. 11, vol. 1,
D Moore, I Essa, M Hayes, "Exploiting Human Actions and Object Context for
Recognition Tasks", Proceedings of IEEE International Conference on Computer
Vision 1999, p. 1, vol. 1, (1999).
A. Haro, I. Essa, M. Flickner, "A Non-invasive Computer Vision System for
Reliable Eye Tracking", Proceedings of ACM CHI 2000 Conference, late breaking
short paper, p. 1, vol. 2, (2000).
A. Haro, I Essa, M. Flickner, "Detecting and Tracking Eyes by Using their
Physiological Properties, Dynamics and Appearance", Proceedings of IEEE Computer
Vision and Pattern Recognition 2000 Conference, p. 1, vol. 1, (2000).
S. Stillman, R.Tanawongsuwan, and I. Essa, "A System for Tracking and
Recognizing Multiple People with Multiple Cameras", in Proceedings of AVBPA
Conference, Washington DC. March 1999. [Available as GVU
Tech Report # 98-25 (Abstract
| PDF |
D. Moore, I. Essa, and M. Hayes, "Object Spaces: Context Management for Human
Activity Recognition", in Proceedings of AVBPA
Conference, Washington DC. March 1999.[Available as GVU
Tech Report # 98-26 (Abstract
| PDF |
Gregory D. Abowd, “Classroom 2000: An Experiment with the Instrumentation
of a Living Educational Environment”, IBM Systems Journal, Accepted
J. Brotherton, K. Truong, and G. Abowd. “Supporting capture and access
interfaces for informal and opportunistic meetings.” Technical Report GIT-GVU-99-06,
GVU Center, Georgia Institute of Technology, January 1999.
A. Chervenak, V. Vellanki, I. Yanasak, N. Harel, R. Rodenstein, G. Abowd,
J. Brotherton, and K. Ramachandran. “A scalable workload model of media-enhanced
classrooms.” Technical Report GIT-CC-99-06, College of Computing, Georgia
Institute of Technology, February 1999.
Jason Brotherton, Janak R. Bhalodia, and Gregory D. Abowd, “Automated Capture,
Integration, and Visualization of Multiple Media Streams”, (1998). conference,
Published Collection: Proceedings of the IEEE Multimedia and Computing
Systems '98 Conference Bibliography: pages 54-63
Daniel Salber, Anind Dey and Gregory Abowd, "The Context Toolkit: Aiding
the Development of Context-Enabled Applications", (1999).conference, Accepted
Collection: Proceedings of CHI'99 Bibliography: To appear
Gregory D. Abowd, “Software engineering issues for ubiquitous computing",
(1999). conference, Accepted Collection: Proceedings of the International
Conference on Software Engineering - ICSE'99 Bibliography: To appear
Heather Richter, Pascal Schuchhard and Gregory A, "Automated capture and
retrieval of architectural rationale", (1999). position paper at conference,
Published, Collection: First Working IFIP Conference on Software Architecture
R. S. Nikhil, K. Ramachandran, et al., “Space-Time Memory: A parallel programming
abstraction for interactive multimedia applications,” In Proc. ACM Principles
and Practices of Parallel Programming, May 1999, Atlanta, GA.
R. S. Nikhil, K. Ramachandran, et al., “Space-Time Memory,” In New Ideas
Sesson held at the Eighth International Conference on Architectural Support
for Programming Languages and Operating Systems}, October 1998, San Jose,
R. S. Nikhil, K. Ramachandran, et al., “Stampede: A programming system
for emerging scalable interactive multimedia applications,” In The 11th
International Workshop on Languages and Compilers for Parallel Computing,
August 1998, Chapel Hill, NC.