Hook up a multi-modal sketching system to a portable electronic whiteboard

Problem

Low-cost, portable electronic whiteboards are now available.  However, using these whiteboards with older sketching technologies may involve modifying interface assumptions, since with these whiteboards it is often not possible to offer visual feedback to the user.

Background

The nuSketch system [1,3] is a multi-modal sketching system that combines sketch-like gestures with speech recognition. By saying "add mountains" while drawing a  mountainous region, for example, the system can understand the purpose of the drawn area, and make inferences based on its boundary characteristics and location.

nuSketch uses a digital tablet for sketching input.  For this project, I want you to extend nuSketch to work with a portable electronic whiteboard (see www.Mimio.com).  The electronic whiteboard has a number of advantages over a digital tablet: the system itself is extremely portable and can be used with any whiteboard.  In addition, the larger size facilitates collaborative drawing, which is common in military and architecture diagrams, two key target domains of the system.

At the same time, an electronic whiteboard is also more limited because the system cannot write to the whiteboard as it does to a screen. Thus, no visual feedback is available to the system, so nuSketch's visual cues must be replaced with other cues (perhaps audible cues) to compensate. For this reason, your summary report should address these interface issues specifically.

Deliverables

• Modify the nuSketch system to take input from the Mimio whiteboard.  This involves a study of the SVG vector graphics format, and some minor Lisp programming.
• Study how well the system works with Mimeo by attempting to use the system for an existing nuSketch domain.
• Write up a report (4-7 pages) which describes the benefits and limitations of this approach, and make proposals for more effective long-term changes in the interface.

Evaluation will by done by the sponsor based on the report and the submitted code.

References (Available from sponsor)

      [1]   Forbus, K. D., Ferguson, R. W., & Usher, J. M. (2000). Boundary-based multimodal input for geographic planning sketches. In P. Healy (Ed.), Proceedings of the First International Workshop on Interactive Graphical Communication . London: Queen Mary College, University of London.

      [2]   Ferguson, R. W., Rasch, R. A. J., Turmel, W., & Forbus, K. D. (2000). Qualitative spatial interpretation of Course-of-Action diagrams, Proceedings of the 14th International Workshop on Qualitative Reasoning . Morelia, Mexico.

      [3]   Forbus, K. D., Ferguson, R. W., & Usher, J. M. (2000). Towards a computational model of sketching, Proceedings of the International Conference on Intelligent User Interfaces . Sante Fe, New Mexico.

      [4]   Ferguson, R. W., & Forbus, K. D. (2000). GeoRep: A flexible tool for spatial representation of line drawings, Proceedings of the 18th National Conference on Artificial Intelligence. Austin, Texas: AAAI Press.