Investigating Research Issues in Ubiquitous Computing: The Capture, Integration, and Access Problem

Gregory D. Abowd

College of Computing & GVU Center
Georgia Institute of Technology


801 Atlantic Drive
Atlanta, GA 30332-0280
Phone: (404)894-7512
Fax : (404)894-9442


Research group


3. Other Communication Modalities.


Ubiquitous computing, education, multimedia, capture, integration, pen-based interaction


The interest in ubiquitous computing has surged over the past few years. The defining characteristic of ubiquitous computing is the attempt to break away from the traditional desktop interaction paradigm and move computational power into the environment that surrounds the user. As Mark Weiser has observed, ``Applications are of course the whole point of ubiquitous computing.'' The Future Computing Environments Group at Georgia Tech, under the guidance of the PI, is taking this applications perspective to guide the discovery of fundamental research challenges in ubiquitous computing.

In this project, we focus on automating the capture of individual and group experiences in order to facilitate access to a richly integrated record of events. We refer to this as the automated capture, integration and access problem for ubiquitous computing. We take an applications focus in approaching this problem. Natural candidates for the capture, integration and access tools are collaborative activities ---meetings, education, design. Whereas previous research in ubiquitous computing has focused on support for a single individual in the midst of a collaborative exchange, we will apply capture, integration and access tools to support many simultaneous collaborators.

We have initiated a large-scale ubiquitous computing effort at Georgia Tech ---entitled Classroom 2000--- to provide automated environments and tools to capture, integrate and access the rich interaction of the classroom experience. This effort will grow to support a wide variety of classes and assist students throughout their educational career and beyond. We are investigating open issues, such as the effect of granularity of capture, support for revision during access, and how interaction between collaborators affects capture and integration. We will plan and carry out an evaluation plan to determine the impact of capture, access, and integration on educational goals. We are specifically interested in determining if this technology supports note-taking practices that contribute to near- and long-term retention of knowledge. We hope that our work will provide a standard of evaluation for the experimental ubiquitous computing community as well as provide insight into more effective and modern teaching and learning methods. Finally, we will further demonstrate the generality of the capture, access, and integration problem through its application to other domains beyond education.

Much of the research focus in this proposal has an immediate relevance toward education. We expect to make a significant impact on educational practices by evaluating the effectiveness of ubiquitous computing technology ---handheld devices, pen-based interfaces and cross-platform software delivery. As students and teachers experiment with the tools and new modes of interaction that are enabled by our technology, we expect new forms of education to emerge if we can show that they provide ways to motivate and energize both teacher and student.


Gregory D. Abowd, Chris Atkeson, Ami Feinstein, Cindy Hmelo, Rob Kooper, Sue Long, Nitin "Nick" Sawhney and Mikiya Tani. Teaching and Learning as Multimedia Authoring: The Classroom 2000 Project. In the Proceedings of the ACM Multimedia'96 Conference, November 1996. HTML Version

Gregory D. Abowd, Chris Atkeson, Ami Feinstein, Yusuf Goolamabbas, Cindy Hmelo, Scott Register, Nitin "Nick" Sawhney and Mikiya Tani. Classroom 2000: Enhancing classroom interaction and review. GVU Center, Georgia Institute of Technology, Technical Report GIT-GVU-96-21. September 1996. Postscript version

Gregory D. Abowd, Anind K. Dey, Rob Orr, and Jason Brotherton. Context-awareness in Wearable and Ubiquitous Computing. GVU Technical Report GIT-GVU-97-11. May 1997. HTML version

Gregory D. Abowd. Ubiquitous Computing: Research Themes and Open Issues from an Applications Perspective GVU Technical Report GIT-GVU-96-24. December 1996. Postscript version

Future Computing Environments Group. Classroom 2000 WWW Home page,

Nitin "Nick" Sawhney, Gregory D. Abowd and Chris Atkeson. Can Electronic Notebooks Enhance the Classroom? GVU Technical Report GVU-96-06, February 1996. HTML version


Rather than force the user to search out and find the computer's interface, ubiquitous computing suggests a different paradigm in which the interface itself can take on the responsibility of locating and serving the user. The history of computing is filled with examples of radical paradigm shifts in the way humans interact with and perceive technology [Dix et al. 93, Ch. 4]. The vision of ubiquitous computing ---first expressed by Weiser [Weiser 91] and grounded in experimental work done at Xerox PARC--- holds the promise of yet another interaction paradigm shift.

What is ubiquitous computing technology? Our general working definition is any computing technology that permits human interaction away from a single workstation. This includes pen-based technology, hand-held or portable devices, large-scale interactive screens, wireless networking infrastructure, and voice or vision technology.

As Weiser points out, ``Applications are of course the whole point of ubiquitous computing.'' [Weiser 93] There are three major research themes that generalize some of this applications focus:

The work on automated capture, integration, and access has been greatly influenced by the work at Xerox PARC in ubiquitous computing [Weiser 91, Weiser 93] and tools to support electronic capture and access of collaborative activities [Minneman et al. 95, Moran et al. 96, Moran et al. 97].

The Marquee note-taking prototype developed at PARC [Weber and Poon 94], the Filochat prototype developed at Hewlett-Packard Labs [Whittaker et al. 94], and the Dynomite personal note-taking environment from FX-PAL [Wilcox et al. 97], provide a personal electronic, pen-based note-taking device . Augmented Paper-based solutions to note-taking have also been researched by Stifelman [Hindus and Schmandt 92] and at Apple [Degen et al. 92].


L. Degen, R. Mander, and G. Salomon. Working with audio: Integrating personal tape recorders and desktop computers. In Proceedings of ACM CHI'92 Conference, pages 413-418, May 1992.

A. Dix, J. Finlay, G. Abowd and R. Beale. Human-Computer Interaction, Prentice-Hall International, 1993.

D. Hindus and C. Schmandt. Ubiquitous audio: Capturing spontaneous collaboration. In Proceedings of ACM CSCW'92 Conference, pages 210-217, 1992.

S. Minneman, S. Harrison, W. Janssen, G. Kurtenbach, T. Moran, I. Smith, and W. van Melle. A confederation of tools for capturing and accessing collaborative activity. In Proceedings of the ACM Multimedia'95 Conference, pages 523-534, November 1995.

T. Moran, P. Chiu, S. Harrison, G. Kurtenbach, S. Minneman, and W. van Melle. Evolutionary engagement in an ongoing collaborative work process: A case study. In Proceedings of ACM CSCW'96 Conference, 1996.

T. P. Moran, L. Palen, S. Harrison, P. Chiu, D. Kimber, S. Minneman, W. van Melle and P. Zelweger. "I'll Get That Off the Audio": A Case Study of Salvaging Multimedia Meeting Records. Proceedings of the ACM CHI'97 Conference, pages 202--209, March 1997.

K. Weber and A. Poon. A tool for real-time video logging. In Proceedings of ACM CHI'94 Conference, pages 58-64, April 1994.

L. J. Stifelman. Augmenting real-world objects: A paper-based audio notebook. In Proceedings of ACM CHI'96 Conference, pages 199-200, April 1996. Short paper.

M. Weiser. The computer of the 21st century. Scientific American, 265(3):66-75, September 1991.

M. Weiser. Some computer science issues in ubiquitous computing. Communications of the ACM, 36(7):75-84, July 1993. S. Whittaker, P. Hyland, and M. Wiley. Filochat: Handwritten notes provide access to recorded conversations. In Proceedings of ACM CHI'94 Conference, pages 271-277, April 1994.

L. Wilcox, B. Schilit and N. Sawhney. Dynomite: A Dynamically Organized Ink and Audio Notebook. In Proceedings of ACM CHI'97 Conference, March 1997.



There is a very good fit here with any work on content-based retrieval of a multimedia record. Such research would use many of the captured streams of information from an interactive setting, including handwriting, audio and video signals. It is also an inviting challenge to determine how to visualize a large corpus of multimedia information to assist a user in browsing and locating areas of interest.

The assessment of impact of a ubiquitous computing system is a challenging topic and provides a good bridge between more futuristing interactive systems and conventional usability evaluation techniques.