Investigations into the Design and Implementation of a Computer-based Dance Tutor

G. Brostow (CoC), K. Sukel (Psychology), W. Rungsarityotin (CoC)
R. Catrambone (Psychology), I. Essa (CoC)

GVU Center, Georgia Tech

Proposed Research:

As technology continues to flourish, we will be able to use our computers for increasingly unexpected and complex interactions as illustrated by the scope of this project, a computer-based interactive dance tutor.

This computer-based interactive dance tutor is envisioned to be an aid in teaching ballet technique. Generally, dance is taught in a communal classroom setting, where one teacher is responsible for teaching at least 10 students. Students receive limited individual attention, and are encouraged to practice on their own to master the technique. However, strict alignment and corrections are necessary to master ballet technique and to avoid creating misaligned "muscle memories," so practicing without a coach may not be the best approach. Here may be an ideal situation for a computational system to aid in teaching.

Our proposed work require integration of several different modules, each requiring work in a different area. We describe these briefly here.

Animation: To be an interactive tutor our system will present movement sequences--in this case, standard ballet technique exercises--to the user. These will be presented as 3D animations and/or video sequences from various viewpoints. The user will be allowed to choose which viewpoint to view the movement from.

Perception: Interactive guidance is only possible if the system can perceive the user's movements. Such perception can be achieved through computer vision and sensor tracking techniques to track the user. The tracking on a dance move by the user will be interpreted by the system within the context of possible movements and the system will provide feedback to the user about his or her performance and how to best correct it. We will also study the importance of music and audio in this form of interaction.

Interface: This research project will examine both the functional and user requirements for the interface of such a system. We will look at the interface issues from both a computational and a psychological perspective. Currently, we are investigating the proper style of presentation for the system. That is, we are investigating the optimal way for the system to display movement sequences to a user so that the user will be able to learn those sequences. This is being done by examining both novice and expert users and their ability to recognize and recreate movement sequences from both videotape and computer animations.

In addition to the above issues we are also looking at possible control mechanisms for such a system. Users are not going to want to type into a keyboard or click a mouse while they are practicing steps, so conventional methods of interacting with a computer may not be conducive to the end goal of the system, i.e., learning dance. Thus, several non-conventional controls need to be researched (such as speech recognition and gesture recognition), prototyped, and then user-tested to provide the most optimal method of user control.

The benefits of this research are:

Future:

The funding from this seed grant will provide the necessary base for exploration into the optimal methods of designing and implementing the computer-based dance tutor. This is a unique project that encompasses many interesting areas. We could foresee funding from both government and industrial agencies interested in any or all of the following areas: computational perception, interface design for non-conventional systems, and the learning and training of complex motor skills.

The government agencies may include NSF (particularly the Interactive and Intermedia Technology Program and the Robotics and Human Augmentation program), DARPA (Program on Smart Spaces) and ONR. Additionally, there are several inter-organizational programs (e.g. STIMULATE program) and programs for interactive training that will be interested in this type of work. Recently there has been much interest in the industrial sector in this kind of work due to the growth of the Entertainment and the Educational applications. We foresee several major industrial labs (Xerox PARC, Microsoft Research, IBM, TI, DEC, MERL) interested in this work. Additionally, due to the interactive art-technology aspect of this work, we will also pursue setting this work up as an installation.

In addition, this work will lead to several publications. We are at present considering submissions to CHI and SIGGRAPH.

Personnel:

The proposed system will require extensive collaboration between the Computational Perception Laboratory and the School of Psychology. This project will be led by Gabriel Brostow, a graduate student in the College of Computing, Katherine Sukel, a graduate student in the School of Psychology, and Wasinee Rungsarityotin a graduate student in CoC. Research will be supervised by Dr. Irfan Essa from the College of Computing, and Dr. Richard Catrambone from the School of Psychology. A large part of the system will be prototyped and implemented by Wasinee Rungsarityotin. Gabe Brostow will aide in the development of the animation module and Katherine Sukel will work with Wasinee on the interface evaluations and subject studies.

Gabriel Brostow is currently a first year student in the Human Computer Interaction (HCI) Masters Program. He is also an active member of the Computational Perception Laboratory (CPL). Upon completion of his MS, he will embark on his doctoral work with Dr. Essa.

Katherine Sukel is a first year doctoral student in the School of Psychology, concentrating in the area of Engineering Psychology, specifically in HCI related issues. She is supervised by Dr. Catrambone, and is a member of the Computation Perception Laboratory (CPL). Additionally, she has studied ballet for over 15 years, and is currently studying with the Atlanta School of Ballet.

Wasinee Rungsarityotin is a first year doctoral student in the College of Computing and is also a member of the Computational Perception Laboratory (CPL). She is also currently studying beginning ballet. Wasinee's areas of interest are computer vision, computer animation of human movement and performance technology.

Dr. Irfan Essa is an assistant professor in the College of Computing, and the head of the Computational Perception Laboratory (CPL). His expertise lies in the areas of computer vision, computational perception and computer graphics/animation.

Dr. Richard Catrambone is an associate professor in the experimental area of the School of Psychology. His research primarily focuses on problem solving, as well as training and learning environments.

All researchers are members of the Graphics, Visualization, and Usability (GVU) Center.

Budget:

This proposal requests 1/2 time GRA funding for fiscal year 1999 for Wasinee Rungsarityotin. This funding will be approximately $1566.67 per month, for a grand total of $14,100.03 for the nine month school year. Both Gabriel Brostow and Katherine Sukel will be fully funded through the GVU's National Science Foundation HCI Traineeship, and will therefore not need funding. The researchers envision Wasinee's contribution as prototyping and implementing the system. Her strengths lie in implementing non-invasive real-time computational tracking of natural movement. As our system will need to have functional tracking and feedback to adequately test the control mechanisms, the addition of this member to our research team will provide a strong base for exploration and discovery that will help to create a realistic user interface for this system, and will move us towards obtaining external long-term funding.