| Jim Davies | Irfan Essa | Terry Maple |
|---|---|---|
| College of Computing (GVU/IS) | College of Computing (GVU) | School of Psychology (Zoo Atlanta) |
The ultimate goal of the Primatech project is to establish permanent installation at Zoo Atlanta and maybe other locations. This Primatech installation will be a fascinating educational tool for zoo visitors. Our interest in this project follows the developments related to the orangutan Chantek (http://www.cc.gatech.edu/~jimmyd/images/primatech/lyn-and-chantek.JPG). Chantek will be the first primate with language ability to be on public display at any zoo, and with Primatech visitors can learn about Chantek and experience what it is like to talk with him. In addition to supporting interaction and communication with Chantek, Primatech will introduce zoo visitors to sign language and perhaps teach them a few signs.
In our prototype, we will provide users with an image of a male orangutan on a large screen. When the user talks or signs to the image, the orangutan responds in American sign language just as Chantek would. In the installation version, this system will be in close proximity to Chantek.
Further benefits include scientific contributions in many fields: graphics, sign language recognition, artificial natural language understanding and generation, and animal behavior. Finally, this unusual and accessible project will increase publicity for GVU and Zoo Atlanta.
Animation: We will develop a 3D graphical model of Chantek. This graphical model will then be animated with joints for movements based on actual orangutan anatomy. Underlying will be system for moving the orangutan in predefined ways. For example, if the model is sent the word "banana," the model knows what sequence of actions to take such that the model appears to be signing. The high-level behaviors of the model will be controlled by first implementing existing theories of animal behaviors as presented by animal behaviorist and at present being studied by various A-life researchers. We will extend these to account for language-type interaction, which is our real goal. We will initially rely on existing technologies to support this part and may use some infrastructure used by GVU's Virtual Environment Groups work on the Virtual Gorilla and expertise from GVU's Animation Lab.
Intelligence: We will develop and artificial intelligence-based system to model the language-using behavior of Chantek as closely as possible, including both language understanding and generation. The inner workings of the model will be compatible with what is known about language use, particularly in non-human primates. This research will be pioneering in that computational modeling of primate language use has never been done before. It should shed light on the controversial issue of language use in apes. Jim Davies will complete this component in the coming year.
Voice/Sign Language Recognition: Like Chantek, the installation will recognize a subset of spoken English and American Sign Language. A hidden camera and microphone will take input from the user; the recognizer will parse the signs and words and send them to the intelligence/language module. Separate modules will be used to recognize speech and signs, but in cases where the user both speaks and signs, the two modules can cross-validate.
Such a system is being built by several researcher in the Computational Perception Lab under the supervision of Dr. Essa. This system uses cameras to track hands in real time. The system is trained for various grammars and vocabularies using a hidden Markov model (HMM). Finger spellings, which are integral part of sign-language recognition, are done using Principle Components Analysis (PCA) on localized and aligned images of hands. We use commercially available packages (HTK by Entropics Inc.) for HMM-based recognition of hand movements and speech recognition.
Primatech will use a version of this system. To optimize the ability of the recognizer, studies can be done to see what people will tend to say or sign at this installation. The sign language recognizer will require finding the solution to many problems such as variable height of the users, inexperienced signers, a `noisy' visual atmosphere, etc. The recognizer is not meant to be a model of orangutan perception.
Following is a list of people who have agreed to contribute to this project.
Irfan Essa's expertise is in the area of computer vision, computational perception, and computer graphics/animation. He has worked extensively in the area of face, facial expression and gesture recognition from video signals. For more information on his work see his WWW page at http://www.cc.gatech.edu/~irfan
Terry Maple is director of Zoo Atlanta and an animal behavior specialist in the school of psychology. He specializes in primates, and has written the book "Orangutan Behavior".
Jim Davies is a Ph.D. candidate in the College of Computing. Jim will lead this project under the guidance of Dr. Essa, Dr. Miles, and Dr. Maple. He is trained in cognitive modeling and animal behavior.
Dr. Lyn Miles is an employee at Zoo Atlanta, a professor at the University of Tennessee and the trainer of Chantek. She signs, has intimate knowledge of Chantek and will consult on the quality of the graphic and cognitive model. In addition she can design and supervise any further experimentation or observation with Chantek.
Gabe Brostow and Wasinee Rungsarityotin, CoC Ph.D. students, and Tara Stoinski, a Ph.D. student in animal behavior, will also support this project.
Our hope is that Primatech becomes a permenant exhibit like those in science museums: visually interesting, educational, and interactive. Zoo Atlanta and other places committed to scientific research, technology, and education will love this project as much as the visitors who will experience it. More importantly, this will also allow us to study the concepts of interactions, language understanding, and animal behavior with a larger audience than just our colleagues.
