Audio-only Augmented Reality

Introduction

Over the past years, we (and others) have built a number of augmented reality systems to address many problem domains. For specific examples, see our papers. We would like to continue exploring the uses of augmented reality, as well as investigating certain general issues in greater depth (such as how to interact with the environment, suitable rendering techniques, etc).

When considering Augmented Reality, one usually thinks of visually overlaying graphics on a user's view of the world. However, it is not clear that this is necessary (or desirable) for many tasks, especially "everyday-use" wearable AR systems that might be continuously worn. In such situations, it may be better to have an "audio-only" system that does not try to place information visually in front of the user. A good example of such a system is a system Dr. Mynatt built at Xerox PARC called Audio Aura.

Project

The goal of this project is to design and build a simple audio-only augmented reality system. We have numerous high quality head-tracking systems in the Augmented Environments Lab, ranging from orientation only systems to one that allows the position and orientation of a users head to be tracked with high accuracy and low latency over a 10' cube (basically, one end of our lab). For this system, you can either use one of these systems, or create a synthetic environment that the user can move about in.

In this prototype, we would like you to create an environment that contains a large set of audio clips attached to 3D positions in the world. These clips will be grouped together into two groups, "primary clips" (that anchor the story being told by all of these audio clips) and "secondary clips" (that serve to expand upon the information in the primary clips). You will build an audio only interface that would allow a novice user navigate through the space and make sense of the information. The difficulty is in deciding what information to present when, so that the user does not become confused. You should discuss your plan with Dr. MacIntyre before deciding to do this project.

Once you start, here is what you need to do.

1. Read the following papers:
   • Azuma, Ronald T. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments 6, 4 (August 1997), 355 - 385 (available from Ron's old web page here in pdf format)
   • Feiner, Steven, MacIntyre, Blair and Seligmann, Doree. ``Knowledge-Based Augmented Reality.'' In Communications of the ACM 36(7): 53-62, July, 1993. (if you are not an ACM member, see Dr. MacIntyre for a copy. However, you should join; the digital library is useful, and student membership is cheap.)
   • Audio aura: light-weight audio augmented reality; Elizabeth D. Mynatt, Maribeth Back, Roy Want and Ron Frederick; Proceedings of the 10th annual ACM symposium on User interface software and technology, 1997, Pages 211 - 212 (http://www.cc.gatech.edu/fce/ecl/projects/mynatt/AudioAura/aa-uist97.ps)
   • Designing audio aura ; Elizabeth D. Mynatt, Maribeth Back, Roy Want, Michael Baer and Jason B. Ellis; Conference proceedings on Human factors in computing systems, 1998, Pages 566 - 573 (http://www.cc.gatech.edu/fce/ecl/projects/mynatt/AudioAura/aa-chi98.pdf).
2. Decide how you would like to implement your prototype.
3. Meet with Dr. MacIntyre regularly to discuss your progress.

Background

• Augmented Environments Lab web page
• Experience Java and computer audio is a must. VR experience would also be useful.

Deliverables

• Meet with Dr. MacIntyre a few times to discuss the project you are interested in pursuing
• Write a 1-2 page report addressing the following issues:
  • Describe your system
  • Describe the design decisions you made
• Submit your code, demonstrate your system and discuss it with Dr. MacIntyre

Evaluation

Based on the code, demonstration and report turned in to the sponsor of the project by the due date.