Kim Xu

HCI 575x Final Project: Signing Buddy

An Application for Speech-to-ASL and ASL-to-Speech Education and Communication

Signing Buddy is an application that allows users to learn sign language and enables simple communication between the hearing and hearing impaired. A user can speak into a microphone and the monkey avatar will sign what is spoken using the ASL manual alphabet. When a web camera is connected to the computer the user can also sign words, and the application will repeat the words signed as speech.

Click here to watch a video.

Monkey Signing the letter V Monkey Signing the letter I Monkey Signing the letter A Monkey Signing the letter B Monkey Signing the letter L Monkey Signing the letter E
Kim signing the letter 'o' to be recognized.  Helen speaking to the application for recognition.

The picture on the left shows me signing the letter 'o' for recognition. The right-hand picture shows Helen speaking to the application for sign generation.

The picture below and to the left shows the application upon start up. The monkey waves hello and verbal instructions are given. The picture on the right shows the screen that is visible during signing. The letter B is currently being signed.

Monkey waving hello at application startup. The signing screen. Note the letter 'b' in the window.
Poster for signing buddy