Word spotting: An interface to facilitate searching audio

Problem
In the Summer of 1997, the Classrooom 2000 project developed the capability to search the audio of a lecture. This functionality will be provided in two ways. The purpose of this project is to create some sample interfaces for allowing a student to search the audio of one or more classes.

The audio search system was developed by Peter Cardillo, a graduate student in ECE. The system works by using a Hidden Markov Model to convert an audio file (as a WAV file) to a sequence of timestamped phonemes. A phoneme dictionary is used to convert words into their corresponding phoneme representation. The phoneme representation is then used to perform a search of the audio, returning a series of ranked hits. The final stage of the system then delivers a pointer to stream-based audio player (such as RealAudio) that plays the audio from where the word was spotted. Here is a demonstration of the word spotting system with a simple interface.

In this project, you are to build on this simple interface to allow for the following new features:

• Provide a nicer visualization of the matched word hits. We suggest providing a timeline that is decorated with the hits.
• The system is not perfect, and it will sometimes return false hits. Provide the user with the ability to remove such false hits. This means that the results of the search need to be saved to some file and then a CGI or Java-based interface will allow the modification of the contents of this file.
• Allow searches of multiple classes and provide a nice visual representation of the hits over all classes. You might start with a series of timelines for each class.

• You will want to gain familiarity with the output routine of the word spotting system written by Peter Cardillo. Peter can help you understand this. His system was written in C++. You can either write your system in C++, or you can write it in Java and have Peter deliver you some agreed format.
• You should look at the work done by Janak Bhalodia, a Tech undergrad, who has been working with Classroom 2000 to provide timeline interfaces. There are also some other prototype interfaces that Dr. Abowd can show.

• a Web page that serves as a user manual for the system and a pointer to a demonstration of the system
• all source code
• a brief discussion of how your system can fit into the rest of the Classrooom 2000 system for accessing class notes.