This computer-based interactive dance tutor is envisioned to be an aid in teaching ballet technique. Generally, dance is taught in a communal classroom setting, where one teacher is responsible for teaching at least 10 students. Students receive limited individual attention, and are encouraged to practice on their own to master the technique. However, strict alignment and corrections are necessary to master ballet technique and to avoid creating misaligned "muscle memories," so practicing without a coach may not be the best approach. Here may be an ideal situation for a computational system to aid in teaching.
Our proposed work require integration of several different modules, each requiring work in a different area. We describe these briefly here.
Animation: To be an interactive tutor our system will present movement sequences--in this case, standard ballet technique exercises--to the user. These will be presented as 3D animations and/or video sequences from various viewpoints. The user will be allowed to choose which viewpoint to view the movement from.
Perception: Interactive guidance is only possible if the system can perceive the user's movements. Such perception can be achieved through computer vision and sensor tracking techniques to track the user. The tracking on a dance move by the user will be interpreted by the system within the context of possible movements and the system will provide feedback to the user about his or her performance and how to best correct it. We will also study the importance of music and audio in this form of interaction.
Interface: This research project will examine both the functional and user requirements for the interface of such a system. We will look at the interface issues from both a computational and a psychological perspective. Currently, we are investigating the proper style of presentation for the system. That is, we are investigating the optimal way for the system to display movement sequences to a user so that the user will be able to learn those sequences. This is being done by examining both novice and expert users and their ability to recognize and recreate movement sequences from both videotape and computer animations.
In addition to the above issues we are also looking at possible control mechanisms for such a system. Users are not going to want to type into a keyboard or click a mouse while they are practicing steps, so conventional methods of interacting with a computer may not be conducive to the end goal of the system, i.e., learning dance. Thus, several non-conventional controls need to be researched (such as speech recognition and gesture recognition), prototyped, and then user-tested to provide the most optimal method of user control.
The benefits of this research are:
The government agencies may include NSF (particularly the Interactive and Intermedia Technology Program and the Robotics and Human Augmentation program), DARPA (Program on Smart Spaces) and ONR. Additionally, there are several inter-organizational programs (e.g. STIMULATE program) and programs for interactive training that will be interested in this type of work. Recently there has been much interest in the industrial sector in this kind of work due to the growth of the Entertainment and the Educational applications. We foresee several major industrial labs (Xerox PARC, Microsoft Research, IBM, TI, DEC, MERL) interested in this work. Additionally, due to the interactive art-technology aspect of this work, we will also pursue setting this work up as an installation.
In addition, this work will lead to several publications. We are at present considering submissions to CHI and SIGGRAPH.
Gabriel Brostow is currently a first year student in the Human Computer Interaction (HCI) Masters Program. He is also an active member of the Computational Perception Laboratory (CPL). Upon completion of his MS, he will embark on his doctoral work with Dr. Essa.
Katherine Sukel is a first year doctoral student in the School of Psychology, concentrating in the area of Engineering Psychology, specifically in HCI related issues. She is supervised by Dr. Catrambone, and is a member of the Computation Perception Laboratory (CPL). Additionally, she has studied ballet for over 15 years, and is currently studying with the Atlanta School of Ballet.
Wasinee Rungsarityotin is a first year doctoral student in the College of Computing and is also a member of the Computational Perception Laboratory (CPL). She is also currently studying beginning ballet. Wasinee's areas of interest are computer vision, computer animation of human movement and performance technology.
Dr. Irfan Essa is an assistant professor in the College of Computing, and the head of the Computational Perception Laboratory (CPL). His expertise lies in the areas of computer vision, computational perception and computer graphics/animation.
Dr. Richard Catrambone is an associate professor in the experimental area of the School of Psychology. His research primarily focuses on problem solving, as well as training and learning environments.
All researchers are members of the Graphics, Visualization, and Usability (GVU) Center.