GeoRep: A flexible tool for spatial representation of line drawings, Proceedings of the 18th National Conference on Artificial Intelligence . Austin, Texas: AAAI Press. Abstract Full text
A key problem in diagrammatic reasoning is understanding how people reason about qualitative relationships in diagrams. We claim that progress in diagrammatic reasoning is slowed by two problems: (1) researchers tend to start from scratch, creating new spatial reasoners for each new problem area, and (2) constraints from human visual processing are rarely considered. To address these problems, we created GeoRep, a spatial reasoning engine that generates qualitative spatial descriptions from line drawings. GeoRep has been successfully used in several research projects, including cognitive simulation studies of human vision. In this paper, we outline GeoRep's architecture, explain the domain-independent and domain-specific aspects of its processing, and motivate the representations it produces. We then survey how GeoRep has been used in three different projects-a model of symmetry, a model of understanding juxtaposition diagrams of physical situations, and a system for reasoning about military courses of action.
Forbus, K. D., Ferguson, R. W., & Usher, J. M. (2000). Towards a computational model of sketching, Proceedings of the International Conference on Intelligent User Interfaces . Sante Fe, New Mexico.
Sketching is a powerful means of interpersonal communication. While many useful programs have been created, current systems are far from achieving human-like participation in sketching. A computational model of sketching would help characterize these differences and better understand how to overcome them. This paper is a first step towards such a model. We start with an example of a sketching system (nuSketch COA Creator) designed to aid military planners, to provide context and a source of examples. We then describe four dimensions of sketching, visual understanding, conceptual understanding, language understanding, and drawing, that can be used to characterize the competence of existing systems and identify open problems. The issues involved will be illustrated by examples from our experience with nuSketch. Three research challenges are posed, to serve as milestones towards a computational model of sketching that can explain and replicate human abilities in this area.
Ferguson, R. W. (2000). Modeling orientation effects in symmetry detection: The role of visual structure, Proceedings of the 22nd Conference of the Cognitive Science Society . Hillsdale, New Jersey: Erlbaum.
Symmetry detection is a key part of human perception. One incompletely understood aspect of symmetry detection concerns orientation effects. The best-known orientation effect is the preference for vertical symmetry, where symmetry around a vertical axis is detected more quickly and accurately than symmetry at other orientations. Current symmetry detection models have difficulty explaining this effect. Using MAGI (Ferguson, 1994), we show how orientation effects may be caused by interactions between the perceived visual relations and the current reference frame. As evidence for this explanation, we simulate several orientation characteristics, including the preference for vertical symmetry and Wiser's (1981) theory of "intrinsic axes". Finally, we successfully simulate the results of a classic study by Palmer and Hemenway (1978) which explores the relationship between the preference for vertical symmetry, multiple symmetries, and inexact symmetry. Collectively, these results show that orientation effects may be due to characteristics of detected visual relations rather than either exact point-to-point equivalencies or the bilateral symmetry of the visual system.
Ferguson, R. W., & Forbus, K. D. (1998). Telling juxtapositions: Using repetition and alignable difference in diagram understanding. In K. Holyoak, D. Gentner, & B. Kokinov (Eds.), Advances in Analogy Research (pp. 109-117). Sofia, Bulgaria: New Bulgarian University.
Diagrams often use repetition to convey points and establish contrasts. This paper shows how MAGI, our model of repetition and symmetry detection, can model the cognitive processes humans use when reading repetition-based diagrams. MAGI, which is based on the Structure Mapping Engine, detects repetition by aligning both visual and conceptual relational structure. This lets visual regularity of form support an understanding of the conceptual regularity such forms often depict. We describe JUXTA, which uses this insight to critique a class of diagrams that juxtapose similar scenes to demonstrate physical laws.
Ferguson, R. W., Forbus, K. D., & Gentner, D. (1997). On the proper treatment of noun-noun metaphor: A critique of the Sapper model. Poster presented at the 19th Annual Meeting of the Cognitive Science Society.
Ferguson, R. W., Aminoff, A., & Gentner, D. (1996). Modeling qualitative differences in symmetry judgments, Proceedings of the Eighteenth Annual Conference of the Cognitive Science Society . Hillsdale, NJ: Lawrence Erlbaum Associates.
Symmetry perception is an important cognitive process across many areas of cognition. This research explores symmetry as a special case of similarity—self-similarity—and proposes that qualitative relationships play a role in the early perception of symmetry. To support this claim, we present evidence from two psychological studies where subjects performed symmetry judgments for randomly constructed polygons. Subjects were faster and/or more accurate at detecting asymmetry for stimuli with qualitative asymmetries than for stimuli with equivalent quantitative asymmetries. Aspects of this effect are replicated using the MAGI computational model, which detects symmetry using a method of structural alignment. The results of this study suggest that qualitative information influences early perception of symmetry, and provides further support for the MAGI model.
Ferguson, R. W. (1994). MAGI: Analogy-based encoding using symmetry and regularity. In A. Ram & K. Eiselt (Eds.), Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society (pp. 283-288). Atlanta, GA: Lawrence Erlbaum Associates.
Analogy has always been considered a mechanism for interrelating distinct parts of the world, but it is perhaps just as important to consider how analogy might be used to break the world into comprehensible parts. The MAGI program uses the Structure-Mapping Engine (SME) to flexibly and reliably match a description against itself. The resulting mapping pulls out the two maximally consistent parts of the given description. MAGI then divides out the parts of the mapping and categorizes the mapping as symmetrical or regular. These parts may then be used as the basis for new comparisons. We theorize that MAGI models how people use symmetry and regularity to facilitate the encoding task. We demonstrate this with three sets of examples. First, we show how MAGI can augment traditional axis detection and reference frame adjustment in geometric figures. Next, we demonstrate how MAGI detects visual and functional symmetry in logic circuits, where symmetry of form aids encoding symmetry of function. Finally, to emphasize that regularity and symmetry detection is not simply visual, we show how MAGI models some aspects of expectation generation in story understanding. In general, MAGI shows symmetry and regularity to be not only pretty, but also cognitively valuable.
Forbus, K. D., Ferguson, R. W., & Gentner, D. (1994). Incremental structure mapping. In A. Ram & K. Eiselt (Eds.), Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society (pp. 313-318). Atlanta, GA: Lawrence Erlbaum Associates.
Many cognitive tasks involving analogy, such as understanding metaphors, problem-solving, and learning, require the ability to extend mappings as new information is found. This paper describes a new version of SME, called I-SME, that operates incrementally. I-SME is inspired by Keane's IAM model and the use of incremental mapping in Falkenhainer's PHINEAS learning system. We describe the I-SME model algorithm and discuss tradeoffs introduced by incremental mapping, including parallel versus serial processing and pragmatic influences. The utility of I-SME is illustrated by two examples. First, we show that I-SME can account for the psychological results found by Keane on a serial version of the Holyoak & Thagard attribute mapping task. Second, we describe how I-SME is used in the Minimal Analogical Reasoning System (MARS), which uses analogy to solve engineering thermodynamics problems.
Forbus, K. D., Ferguson, R. W., & Usher, J. M. (2000). Boundary-based multimodal input for geographic planning sketches. In P. Healy (Ed.), Proceedings of the First International Workshop on Interactive Graphical Communication . London: Queen Mary College, University of London.
Interactive sketching is a powerful means of interpersonal communication. Current computer-based sketching systems, while falling short of human-like interaction, successfully exploit the limited gesture and speech recognition technology that is currently available. These sketching systems succeed through calculated bets about the nature of human gesture, language, and spatial interpretation. In this paper, we evaluate such tradeoffs in our own interactive sketching system, nuSketch, which is currently under development. nuSketch uses a technique called boundary-based multimodal sketching to build representations of a domain. This technique is well suited to nuSketch's current task: a geographic military planning domain called course-of-action (COA) sketches. As part of our analysis of nuSketch, we identify three dimensions of sketching - visual understanding, language understanding, and conceptual understanding - that are useful for characterizing the abilities of sketching systems and identifying open problems in this area.
Ferguson, R. W., Rasch, R. A. J., Turmel, W., & Forbus, K. D. (2000). Qualitative spatial interpretation of Course-of-Action diagrams, Proceedings of the 14th International Workshop on Qualitative Reasoning . Morelia, Mexico.
This paper demonstrates qualitative spatial reasoning techniques in a real-world diagrammatic reasoning task: Course-of-Action (COA) diagrams. COA diagrams are military planning diagrams that depict unit movements and tasks in a given region. COA diagrams are a useful test bed for researching diagram understanding due to their composable symbology, their intrinsically spatial task, and their use across many types of military planning. We constructed two COA diagram interpreters using our qualitative spatial reasoning engine, GeoRep. The first system uses GeoRep to interpret individual COA glyphs. The second system, building upon the first, takes pre-classified symbol input and then uses GeoRep to describe geographic relationships implied by the symbol arrangements. This latter system, in a recent DARPA initiative, answered dozens of geographic queries about many different COA diagrams. This research shows that qualitative spatial reasoning, through tools like GeoRep, provides a useful substrate for complex diagrammatic reasoning.
Forbus, K., Ferguson, R. and Usher, J. (2000). Towards a computational model of sketching. Proceedings of QR2000, Morelia, Mexico.
Previous version of Forbus, Ferguson & Usher IUI-2000 paper.
Ferguson, R. W., & Gentner, D. (2000). Analogical encoding: A framework. Poster presentation at Thinking-2000 conference. Durham, England.
Ferguson, R. W., Rasch, R. A. J., Turmel, W., & Forbus, K. D. (2000). Qualitative spatial interpretation of Course-of-Action diagrams. Poster presented at the Intelligent Systems Demonstrations, 18th National Conference on Artificial Intelligence, Austin, Texas.
This is a two-page version of the Qualitative Reasoning Workshop paper.
Ferguson, R. W., & Forbus, K. D. (1999, June, 1999). GeoRep: A flexible tool for spatial representation of line drawings. Qualitative Reasoning Workshop, Loch Awe, Scotland.
Earlier version of AAAI-2000 paper.
Ferguson, R. W., & Forbus, K. D. (1995). Understanding illustrations of physical laws by integrating differences in visual and textual representations. Presented at the Fall Symposium on Computational Models for Integrating Language and Vision., Cambridge, Massachusetts.
An important problem in the integration of vision and language is comprehending explanatory diagrams, such as those found in science and engineering textbooks. One class of diagrams, which we call juxtaposition diagrams, illustrate a physical principle by comparing two similar situations that vary in a carefully chosen way. This paper describes research in progress on a computational model, JUXTA, which analyzes juxtaposition diagrams. JUXTA performs its analysis by finding the interesting differences in a figure, and then relating those differences to differences stated in the diagram caption. By using the visible differences in the figure as reference points for the qualitative relationship given in the caption, JUXTA is able to intelligently label the relevant parts of the figure. JUXTA also critiques the figure for understandability, warning of differences in the figure which may confuse the reader, and noting visible differences in the figure which are irrelevant and may be removed.
Ferguson, R. W., Aminoff, A., & Gentner, D. (Submitted). Early detection of qualitative symmetry .
Does symmetry detection involve a structural alignment process? We investigate this possibility by comparing human participants' ability to detect two classes of asymmetry in randomly constructed polygons. If symmetry detection utilizes structural alignment, then asymmetric figures containing qualitative differences (such as polygons with misaligned or missing vertices) should be more easily judged asymmetric than asymmetric figures that lack qualitative differences. Results of two experiments confirmed that participants were consistently more accurate or faster in detecting asymmetry when figures contained qualitative differences. This effect was significant even after accounting for many competing measures of asymmetry, such as difference in area or radial distance. A computer model of alignment-based symmetry detection successfully simulated the results.
Gentner, D., Brem, S., Ferguson, R. W., Markman, A. B., Levidow, B. B., Wolff, P., & Forbus, K. D. (1997). Analogical reasoning and conceptual change: A case study of Johannes Kepler. The Journal of the Learning Sciences, 6(1), 3-40.
The work of Johannes Kepler offers clear examples of conceptual change. In this article, using Kepler's work as a case study, we argue that analogical reasoning facilitates knowledge change in four ways: (a) highlighting, (b) projection, (c) rerepresentation, and (d) restructuring. We present these four mechanisms within the context of structure-mapping theory and its computational implementation, the structure-mapping engine. We exemplify these mechanisms using the extended analogies Kepler used in developing a causal theory of planetary motion.
Ferguson, R. W. (1997). Book Review of The Subtlety of Sameness, by Robert M. French. The Knowledge Engineering Review, 12(4), 417-420.
Forbus, K. D., Gentner, D., Markman, A. B., & Ferguson, R. W. (1997). Analogy just looks like high level perception: Why a domain-general approach to analogical mapping is right. Journal of Experimental and Theoretical Artificial Intelligence.
Hofstadter and his colleagues have criticized current accounts of analogy, claiming that such accounts do not accurately capture interactions between processes of representation construction and processes of mapping. They suggest instead that analogy should be viewed as a form of high level perception that encompasses both representation building and mapping as indivisible operations within a single model. They argue specifically against SME, our model of analogical matching, on the grounds that it is modular, and offer instead programs like Mitchell & Hofstader’s Copycat as examples of the high level perception approach. In this paper we argue against this position on two grounds. First, we demonstrate that most of their specific arguments involving SME and Copycat are incorrect. Second, we argue that the claim that analogy is high-level perception, while in some ways an attractive metaphor, is too vague to be useful as a technical proposal. We focus on five issues: (1) how perception relates to analogy, (2) how flexibility arises in analogical processing, (3) whether analogy is a domain-general process, (4) how should micro-worlds be used in the study of analogy, and (5) how best to assess the psychological plausibility of a model of analogy. We illustrate our discussion with examples taken from computer models embodying both views.
Gentner, D., Brem, S., Ferguson, R. W., Wolff, P., Markman, A. B., & Forbus, K. D. (1997). Analogy and creativity in the works of Johannes Kepler. In T. B. Ward, S. M. Smith, & J. Vaid (Eds.), Creative Thought: An Investigation of Conceptual Structures and Processes (pp. 403-459). Washington, DC: American Psychological Association.
This is longer version of the article which appeared in The Journal of the Learning Sciences.
Qualitative spatial reasoning in diagrams. Talk at International Computer Science Institute, University of California at Berkeley. February 28, 2000.
Qualitative spatial reasoning in diagrams. Talk at Xerox Palo Alto Research Center. February 29, 2000.
Qualitative spatial reasoning in diagrams. Learning Science Brown Bag. Department of Education, Northwestern University, Evanston, IL. March 6, 2000.
GeoRep: A flexible system for spatial representation of line drawings. Talk at the University of Cambridge Computer Laboratory. Cambridge, UK. August 27, 2000.
GeoRep: A flexible system for spatial representation of line drawings. Talk at the School of Computing, University of Leeds. Leeds, UK. September 4, 2000.