Perceptual User Interfaces

Problem
In recent years, researchers and companies have become interested in perceptual user interfaces; that is, user interfaces that use perception to support the user interaction. Recently, we created a vision-based system that detects and tracks eyes robustly in indoor environments. A user interface that is aware of the number of people interacting with it or in the scene is valuable in many applications. If your computer knew when you were looking at it, it would also be able to look at your face and see if you were getting frustrated, for example.

This project entails creating the foundation for such a system. In it, you will take the existing eye tracker and use the outputs (a list of detected pairs of eyes) to create a user interface mockup on a display tablet that knows when a user is looking at it.

Background

Take a look at an overview of our eye tracker at: http://www.cc.gatech.edu/cpl/projects/pupil/index.html. Also, read our paper on the eye tracker to get an idea of how it works: http://www.cc.gatech.edu/cpl/projects/pupil/pupil.pdf. Note: you do not have to understand the paper fully to do this project; reading it will help you get a high-level understanding of the code, which will be useful since you will be modifying a small portion of it.

What you need to do:

1. Email Antonio Haro (haro@cc) to get a copy of the code, the camera, and a tablet.
2. Create a user interface that uses the data returned from the tracker. The best way to do this will be to modify the code directly. The user interface can be implemented in Visual C++ and can be as simple as a dialog box initially.
3. Get your system to pop up a dialog box saying "Hello" to users if they are looking at the tablet.
4. Think of some other things for your user interface to do if the user is there. What can you do now that you know when people are looking at the tablet as well as how many people are looking at it?
5. Write up your findings and submit your documented code.