Research Projects

Research Interests

I am interested in visual reasoning and analogical problem solving. I am investigating problems of visual and spatial reasoning in the context of analogy, and in particular multi-modal analogy in the context of design problem solving, combining functional and behavioral models with visual and spatial information. I am also interested in problems of sketch and diagram understanding and diagrammatic reasoning. (see also my list of publications).

Archytas

For my dissertation work I am developing a theory of compositional visual analogy in design, implemented in system called Archytas, that attempts to address some visual aspects of design understanding in engineering design. Archytas attempts to read in a line drawing of a mechanical device and reconstruct a model by analogy of the basic and composite shapes present in the drawing as well as the design of the device, accounting for the structural elements, components and connections and their properties and variable parameters; behaviors and causal interactions among components; and the function of the device. This is does by analogy to a known drawing with a known model. The models are based on so-called structure-behavior-function (SBF) models of device designs.

There have been two versions of Archytas: the first version computed spatial relations over geometric shapes in a line drawing using a forward-chaining rule system, and matched source to target drawing using a maximum common subgraph (MCS) algorithm. Model transfer then proceeded on this basis. The second version uses a substantially different algorithm that begins by attempting to derive a representation of shapes analogically, grouping multiple symmetric mappings at the level of line and intersection, and it is this version that implements the theory of compositional visual analogy.

Patrick W. Yaner and Ashok K. Goel, "Visual Analogies at Multiple Levels of Abstraction." In Proceedings of the 29th Annual Meeting of the Cognitive Science Society (CogSci-07), August 2007. gzipped postscript (56k), PDF (95k)

Patrick W. Yaner and Ashok K. Goel, "Understanding Drawings by Compositional Analogy." In Proceedings of the 20th International Joint Conf. on Artificial Intelligence (IJCAI-07), pp. 1131-1137, January 2007. gzipped postscript (136k), PDF (184k)

Patrick W. Yaner and Ashok K. Goel, "Interpretation of Design Drawings by Analogy." 21st National Conference on Artificial Intelligence (AAAI-06), Boston, MA, July 2006. gzipped postscript (41k), PDF (54k)

Patrick W. Yaner and Ashok K. Goel, "From Form to Function: From SBF to DSSBF", In Design Computing and Cognition '06 (DCC-06), pp. 423-441, Springer, July 2006. gzipped postscript (144k), PDF (280k)

Patrick W. Yaner and Ashok K. Goel, "From Diagrams to Models by Analogical Transfer", In Diagrammatic Representation and Inference: 4th International Conference Proceedings (Diagrams 2006), volume 4045 of Lecture Notes in Artificial Intelligence, pp. 55-69, Springer, June 2006. gzipped postscript (62k), PDF (136k)

Patrick W. Yaner, "Multimodal Analogies in Modelling and Design." PhD Thesis Proposal, Technical Report GIT-CC-05-08, College of Computing, Georgia Institute of Technology, April 2005. gzipped postscript (312k), PDF (312k)

The Name Archytas

For those interested in the name: Archytas was an ancient Greek mathematician, statesman, and philosopher from Tarentum, Magna Graecia (modern Taranto, in the arch of the boot of Italy), and a contemporary of Plato, born about 428 BCE, died about 350 BCE. He was a pupil of Philolaus, a Pythagorean, and so followed the Pythagorean belief that mathematics provided the path to the understanding of all things. Like most ancient authors, Archytas's writings survive only in fragments, and much of what we do know we do so from other writers. He worked on the harmonic mean and the problem of duplicating the cube. He is also sometimes said to be the founder of mechanics (in the theoretical and mathematical sense of that term), and invented two mechanical devices: a mechanical bird, and a rattle for children.

Geminus

The Geminus system was an analogical retrieval and mapping system I made in my first years as a graduate student. Geminus means "twin" in Latin, which does seem appropriate, but in fact, in this case it is once again the name of an ancient Greek mathematician (yes, a Greek with a Latin name; apparently that was the cool thing for Greek parents to do in his day). My Geminus system read in a target (or query) line drawing from XFig, generated a relational description of this drawing, and attempted to match this description with descriptions of known drawings in memory. There were two versions. Geminus 1 used resolution theorem proving: the target drawing description was treated as a "theorem" to be proved of a given source drawing description from memory, and a successful proof yielded a result. This was done over every image in memory.

Geminus 2 used a constraint satisfaction method. The target description was treated as a labelled graph, relations as edges, shapes as nodes, where each shape is a "variable" to be assigned a "value". The values are the shapes in images in memory, and the constraints are that the relationships between the shapes in each image must match. Mathematically, this is computing the same function that the old resolution algorithm did, but it was much faster, as it ran across all of memory at once, enabling some pruning of the search space to be done early on, discrimination tree indexing to be used, and a fast backtracking method to be used.

Patrick W. Yaner and Ashok K. Goel, "Visual Analogy: Viewing Analogical Retrieval and Mapping as Constraint Satisfaction Problems." Applied Intelligence 25(1):91-105, 2006. gzipped postscript (120k), PDF (24k)

Patrick W. Yaner and Ashok K. Goel, "Visual Analogy: Reexamining Analogy as a Constraint Satisfaction Problem." In Proceedings of the 26th Annual Meeting of the Cognitive Science Society (CogSci-04), pp. 1482-1487, August 2004. gzipped postscript (49k), PDF (47k)

Patrick W. Yaner and Ashok K. Goel, "Visual Analogy: Viewing Retrieval and Mapping as Constraint Satisfaction Problems." In Visual and Spatial Reasoning in Design III, pp. 233-253, Key Center of Design Computing and Cognition, University of Sydney, July 2004. gzipped postscript (168k), PDF (192k)

Patrick W. Yaner and Ashok K. Goel, "Visual Case-Based Reasoning I: Memory and Retrieval." In Proc. IICAI-03, December 2003. gzipped postscript (85k), PDF (520k)

Patrick W. Yaner and Ashok K. Goel, "Using Spatial Structure in the Associative Retrieval of 2-D Line Drawings." Technical Report GIT-CC-02-70, College of Computing, Georgia Institute of Technology, December 2002. gzipped postscript (112k), PDF (128k)

Patrick W. Yaner and Ashok K. Goel, "Retrieving 2-D Line Drawings by Example." In Diagrammatic Representation and Inference: Second International Conference Proceedings (Diagrams 2002), volume 2317 of Lecture Notes in Artificial Intelligence, pp. 97-99, Springer, 2002. gzipped postscript (84k), PDF (51k)