Future Computing Environments

Personalized Computing

Dealing With The Explosion In Computer Resources
Version 1.0
Chris Atkeson

The following is a position paper describing a potential project. We hope to begin actual work in this area soon.

I cannot program our home VCR or stereo system. I don't know how to use DOS, Windows, OS/2, or Macintoshes, since I seem only able to keep track of how to use one type of computer at any one time. At last count there were 4 remote controls on our coffee table. I know how to use only one of them. If things continue on their current path I and most everyone else will become increasingly technologically dysfunctional. In our homes, offices, cars, and even outdoors we will be able to communicate at all times with computer resources provided by different sources, at varying bandwidths, and with a variety of interfaces (character, GUI, on-screen programming, handwriting, speech, gestures, ...). Unfortunately, we will have to attend night school in order to make use of any of this. The challenge for computer science will be to unify the multiple interfaces to disparate resources loosely connected on a variety of networking mediums. I believe we can take advantage of this window of opportunity so that in 5-10 years we are viewed as leaders in "PERSONALIZED computing".

As a College of Computing we can afford to investigate many ways to address this challenge. The option I am excited about is to make dealing with computers as much like dealing with another human being as possible. In fact, I only want to deal with one other "human being", who plays the role of a "valet/butler/cook/secretary/parent". This person is the same no matter what I am trying to do with my computer. In the morning I want to talk to this person about scheduling my day. I want this person to screen my phone calls, faxes, and email, and to know when to interrupt me for an urgent message. I want this person to remind me what to do and when to do it, and to minimize the amount of work I have to do by trying to do the job first itself if possible. I want this person to recommend movies and TV shows, and to remember to videotape Star Trek when I miss it. I also want this person to remember to feed my fish, and to alert me if there is a fish behaving abnormally. I want to be able to ask what is in the refrigerator when I am driving home, and to be told a grocery list if I decide to stop at the store.

There are may ways to implement this type of interface. Let me outline the environment I expect, in order to make the discussion more concrete. I assume computing resource not unlike what we have now, networks of workstations connected via the Internet. The Internet will extend into our homes with the bandwidth ranging from ISDN rates (100Kbits/second) to current Ethernet rates. When we are outdoors or in our cars we will have more limited bandwidth ranging from current modem rates over cellular phone links to faster data radio networks. Indoors or in our cars we may not have to carry any special device or interface, as our home, office, and car may be instrumented with microphones, video cameras, touch screens, and projection equipment to allow a Star Trek like interface ("Computer, how soon until I run out of gas?). Or we may carry a small (credit card size?) device that has a microphone, speaker, video camera, display screen or projection device (the Time Trax interface model). It is not clear to me whether my computer "servant" will exist in one place, move from computer to computer tracking me as I move around, or be a distributed entity.

The idea of a personalized machine "servant" is a very old one, probably predating the idea of computers. What has changed is that we can now afford to put relatively powerful computing and interface devices in every room of our homes, offices and classrooms, in fact in every home appliance and car as well. Furthermore, the networking infrastructure for using phone lines, cable TV wiring, and radio waves to communicate relatively cheaply is coming into place. There is no longer a technological or cost requirement to sit in front of a workstation and type on a keyboard in order to interact with computing resources. I imagine a future in which my keyboard is mounted on my office wall as a treasured relic of the "old days". There have been previous shifts in computing paradigms, such as the shift from batch to time-sharing computing, and then the shift from time-sharing computing to individual workstations. These shifts were accompanied by, and in some cases driven by shifts in user interfaces: submitting jobs (via punch cards, for example) gave way to interactive terminals. The graphical interfaces made individual workstations a requirement. It is true that other places are already players in this arena (Xerox Parc's "Ubiquitous computing", Microsoft's "Bob", CMU's "Wearable computers", MIT's "Intelligent Room", and "Intelligent Agents at many places, for example). However, there is room for many different visions and many players. This revolution is going to happen, in one form or another. It is important for us, and the students we train, to be experienced with what future possibilities might be before they happen.

RESEARCH: This challenge could be used to link various research efforts and groups in the College. Some areas that I see as potentially being involved:

Learning and AI: Learning will be a critical component in personalizing the interface to a particular user. Learning can occur both at the levels of voice, handwriting, or gesture recognition and also at the level of user modeling, predicting what the user will want or do next. This interface could provide a focus for AI activities and technology demonstrations. The more intelligent we can make the interface the more successful it will be.

Robotics: This interface is a robot. It will gather data, reason about the real world, and act by interacting with its master. It may also have direct control of devices such as household appliances. What techniques from robotics will be useful?

Usability: I have outlined only one approach to personalizing the interface. Is it a good way? Are there better ways?

Graphics, Visualization, and Animation: To what extent should the interface output be graphical, and perhaps even animated? What are the benefits over a text display or a disembodied voice?

Networks and Systems: How should we distribute data, computing, and resources over a network of heterogeneous computing resources to give the appearance of a low latency and responsive person? Are there special purpose architectures or network protocols that will be useful?

Software Engineering: How does the servant interact with other resources? How do we program the servant?

Databases: These servants will collect a tremendous amount of data. How should that be organized? What should be remembered, what should be forgotten? What should be public, and what should be private? What economies of scale may be obtained by sharing databases? How do humans interact with the database?

STUDENTS: Undergraduates at all levels could be involved in work on this project. I am interested in engaging the students in something with a larger scale than quarter courses. Imagine the building full of students taking notes and communicating using hand held interfaces as part of this project. The project could also be used to link courses in the College. For example, the microprocessor systems courses could focus on these applications. The AI courses could use these applications for projects. Some undergraduate theses could focus on this area. If there is interest, software engineering, networks, database, and systems courses could also become involved.

IMMEDIATE AGENDA: We can start focusing attention on this challenge without needing a lot of new money, space, or people. I think we can get a lot of mileage out of simply achieving a consensus that this is an area of common interest, and then using our existing resources effectively. We have rooms full of workstations with audio input and output, and a room full of workstations with video input and output. We have an existing wire network. We can simulate other types of indoor networks (radio and infrared) to allow more mobile indoor use, or install prototype networks in the College. I propose that we install demonstration hardware and software in the entryway of the College, to attract the attention of the many students and visitors who pass through. An initial emphasis might be on providing a unified personal interface to various forms of communication (email, phone, fax, paging, netnews, WWW).

Future Computing Environments Georgia Institute of Technology