Activity Recognition of Object Play for Early Identification

Child'sPlay is a technology for supporting the automatic recording, recognition, and quantification of toddlers' object play behaviors to assist with the early identification of children with developmental delays. My prototype system consists of six varieties of toys augmented with wireless sensing capabilities and a mobile computing platform which uses statistical pattern recognition techniques to automatically classify sensed play behaviors. This research explores choices in toy design both in form factor as well as sensing capabilities along with the development of suitable recognition algorithms. Related Publication

ActionGSR: multimodal sensing for GSR in active environments

The galvanic skin response (GSR), measures changes in electrical resistance across two regions of the skin. GSR can measure arousal levels in children with autism; however, the GSR signal may be overwhelmed by the vigorous movements of the children. The ActionGSR sensor is a wireless sensor capable of measuring both GSR and acceleration simultaneously in an attempt to disambiguate valid GSR signals from motion artifacts. Related Publication

Wireless Sensors to Support the Diagnosis of Autism

This project involves the development of a wireless on-body sensor system and algorithms for recognizing autistic self--stimulatory behaviors. Autism is a developmental disorder affecting a child's ability to communicate and socially develop. Children with autism exhibit self-stimulatory behaviors (``stimming'' for short) that typically involve extreme repetitive body motions and/or repetitive vocal stutters. Researchers would like to correlate the occurrenc of stimming behaviors with other variables such as mood, treatment, environment and other physiological indicators. Currently we are using three 3-axis wireless accelerometers and Hidden Markov Models to recognize and automatically index the occurrence of self-stimulator behaviors. Related Publication

Recognizing Song Based Blink-Patterns

A small population of people possess such severe disabilities that the only voluntary muscle movement they retain is control of their eyes. Blinking patterns with the eyes might be an effective mode of communication or remote device control for such individuals. Our previous work suggested that such individuals could potentially interact with their environment by using songbased blink patterns to control devices. Using songs as the basis of blinked commands also allows for more information to be encoded. Not only does the blink itself provide information (ie: number of blinks per sequence), but its temporal relation with other blinks in the pattern (ie: the rhythm) also encodes information. The interval between the blinks can be used to distinguish between sequences with the same number of blinks. To help aid in recall, the songs selected should have some meaningful association to the device it interfaces. For example, a user might blink to the cadence Frosty the Snowman to lower the temperature setting of the thermostat. Related Publication

Voluntary Behavioral Based Biometrics

Blinking is a biological function that can be both voluntary and involuntary. While some research has explored the use of involuntary blinks as a biometric indicator, this work focuses on the use of voluntary blinking as a form of personal identification. Voluntary blinking as a behavioral biometric indicator measures how the eye region changes as individuals blink specific patterns. The blinking cadence chosen, in essence, is similar to a Personal Identification Number (PIN) -- which associates identity with numerical patterns. However, unlike PINs, identifying the user with blink-based biometrics requires more information than just knowing the correct pattern. The blinked pattern must also be performed correctly. Correct performance of the pattern is dependent on the time between blinks, how long the eye is held closed at each blink, and other physical changes the eye undergoes (which are specific to the user) while blinking. We refer to these characteristics as a person's blinkprint -- a "blinking fingerprint" -- which is used to perform identification. Related Publication

Georgia Tech Gesture Toolkit (GT2K)

Gesture recognition is becoming a more common interaction tool in the fields of ubiquitous and wearable computing. Designing a system to perform gesture recognition, however, can be a cumbersome task. The Georgia Tech Gesture Toolkit (GT2k) provides a publicly available toolkit for developing gesture--based recognition systems. The Georgia Tech Gesture Toolkit GT2k leverages Cambridge University's speech recognition toolkit, HTK, to provide tools that support gesture recognition research. GT2k provides capabilities for training models and allows for both real--time and off--line recognition Related Publication

Gesture Panel

Interior distractions are a large factor in automobile accidents in the United States. Research is currently underway to determine if gesture recognition can be used to control devices in the automobile such as the radio and the climate control system. We are developing a system called the Gesture Panel, which will allow the driver to control devices using simple, gross gestures with a minimal amount of visual distraction. The gesture panel has been designed to operate in the cabin of a car which is constantly subjected to illumination changes. A black and white camera points at a grid of infrared light emitting diodes (IR LEDs). The aperture of the camera is reduced so that only the light from the IR LEDs is visible. The gesture is performed in between the camera and the grid. Thus, as the gesture is made light from some of the LEDs is occluded by the hand. Gestures can be recognized based on the various patterns of occlusions through time. Preliminary experiments with a prototype gesture panel have demonstrated that the light from the IR LEDs is brighter then any reflected sunlight that is present in the car. This implies that this method of gesture recognition can work in various lighting conditions. Related Publication

Billiards Assistant

We have developed a collaborative interface that combines the Perceptive Workbench with a physical game of pool ina two-player telepresence game. The objective of the game is for the player at the billiards table to sink all of the balls while avoiding a virtual obstacle controlled by the other player at the workbench. The billiard table is augmented with a projector/camera setup. A camera positioned above the table tracks the type and position of the pool balls, while a projector in a similar location can create visual feedback directly on the playing surface. The current state of the billiard table is transmitted to the workbench client and rendered as a 3-D model. As the game progresses, the workbench updates this model continuously, using streaming data from the billiards client. Related Publication