Publications

Journal

• N. Dantam and M. Stilman. The Motion Grammar: Analysis of a Linguistic Method for Robot Control. Transactions on Robotics (T-RO). 2013.

  We present the Motion Grammar: an approach to represent and verify robot control policies based on Context-Free Grammars. The production rules of the grammar represent a top-down task decomposition of robot behavior. The terminal symbols of this language represent sensor readings that are parsed in real-time. Efficient algorithms for context-free parsing guarantee that online parsing is computationally tractable. We analyze verification properties and language constraints of this linguistic modeling approach, show a linguistic basis that unifies several existing methods, and demonstrate effectiveness through experiments on a 14-DOF manipulator interacting with 32 objects (chess pieces) and an unpredictable human adversary. We provide many of the algorithms discussed as Open Source, permissively licensed software.

      @article{dantam2013motion,
        author = {N. Dantam and M. Stilman},
        title = {The Motion Grammar: Analysis of a Linguistic Method for Robot Control},
        journal = {Transactions on Robotics (T-RO)},
        year = {2013}
    }
  

Conference

• N. Dantam, Ayonga Hereid, Aaron Ames, and M. Stilman. Correct Software Synthesis for Stable Speed-Controlled Robotic Walking. Robotics: Science and Systems. 2013.

  We present a software synthesis method for speed-controlled robot walking based on supervisory control of a context-free Motion Grammar. First, we use Human-Inspired control to identify parameters for stable fixed speed walking and for transitions between fixed speeds. Next, we build a Motion Grammar representing the discrete-time control for this set of speeds. Then, we synthesize C code from this grammar and generate supervisors online to achieve desired walking speeds, ensuring correctness of discrete behavior. Finally, we demonstrate this approach on the Aldebaran NAO, showing stable walking transitions with dynamically selected speeds.

      @inproceedings{dantam2013correct,
        author = {N. Dantam and Ayonga Hereid and Aaron Ames and M. Stilman},
        title = {Correct Software Synthesis for Stable Speed-Controlled Robotic Walking},
        booktitle = {Robotics: Science and Systems},
        year = {2013}
    }
  

• N. Dantam and M. Stilman. Robust and Efficient Communication for Real-Time Multi-Process Robot Software. International Conference on Humanoid Robots (Humanoids). 2012.

  We present a new Interprocess Communication (IPC) mechanism and library. Ach is uniquely suited for coordinating drivers, controllers, and algorithms in complex robotic systems such as humanoid robots. Ach eliminates the Head-of-Line Blocking problem for applications that always require access to the newest message. Ach is efficient, robust, and formally verified. It has been tested and demonstrated on a variety of physical robotic systems, and we discuss the implementation on our humanoid robot Golem Krang. Finally, the source code for Ach is available under an Open Source permissive license.

      @inproceedings{dantam2012ach,
        author = {N. Dantam and M. Stilman},
        title = {Robust and Efficient Communication for Real-Time Multi-Process Robot Software},
        booktitle = {International Conference on Humanoid Robots (Humanoids)},
        year = {2012}
    }
  

• N. Dantam, I. Essa, and M. Stilman. Linguistic Transfer of Human Assembly Tasks to Robots. Intelligent Robots and Systems (IROS). 2012.

  We demonstrate the automatic transfer of an assembly task from human to robot. This work extends efforts showing the utility of linguistic models in verifiable robot control policies by now performing real visual analysis of human demonstrations to automatically extract a policy for the task. This method tokenizes each human demonstration into a sequence of object connection symbols, then transforms the set of sequences from all demonstrations into an automaton, which represents the task-language for assembling a desired object. Finally, we combine this assembly automaton with a kinematic model of a robot arm to reproduce the demonstrated task.

      @inproceedings{dantam2012assem,
        author = {N. Dantam and I. Essa and M. Stilman},
        title = {Linguistic Transfer of Human Assembly Tasks to Robots},
        booktitle = {Intelligent Robots and Systems (IROS)},
        year = {2012}
    }
  

• N. Dantam, C. Nieto-Granda, H. Christensen, and M. Stilman. Linguistic Composition of Semantic Maps and Hybrid Controllers. International Symposium on Experimental Robotics (ISER). 2012.

  This work combines semantic maps with hybrid control models, generating a direct link between action and environment models to produce a control policy for mobile manipulation in unstructured environments. First, we generate a semantic map for our environment and design a base model of robot action. Then, we combine this map and action model using the Motion Grammar Calculus to produce a combined robot-environment model. Using this combined model, we apply supervisory control to produce a policy for the manipulation task. We demonstrate this approach on a Segway RMP-200 mobile platform.

      @inproceedings{dantam2012composition,
        author = {N. Dantam and C. Nieto-Granda and H. Christensen and M. Stilman},
        title = {Linguistic Composition of Semantic Maps and Hybrid Controllers},
        booktitle = {International Symposium on Experimental Robotics (ISER)},
        year = {2012}
    }
  

• N. Dantam and M. Stilman. The Motion Grammar Calculus for Context-Free Hybrid Systems. American Control Conference (ACC). 2012.

  This paper provides a method for deriving provably correct controllers for Hybrid Dynamical Systems with Context-Free discrete dynamics, nonlinear continuous dynamics, and nonlinear state partitioning. The proposed method models the system using a Context-Free Motion Grammar and specifies correct performance using a Regular language representation such as Linear Temporal Logic. The initial model is progressively rewritten via a calculus of symbolic transformation rules until it satisfies the desired specification.

      @inproceedings{dantam2012calc,
        author = {N. Dantam and M. Stilman},
        title = {The Motion Grammar Calculus for Context-Free Hybrid Systems},
        booktitle = {American Control Conference (ACC)},
        year = {2012}
    }
  

• N. Dantam and M. Stilman. The Motion Grammar: Linguistic Perception, Planning, and Control. Robotics: Science and Systems (RSS). 2011.

  We present and analyze the Motion Grammar: a novel unified representation for task decomposition, perception, planning, and control that provides both fast online control of robots in uncertain environments and the ability to guarantee completeness and correctness. The grammar represents a policy for the task which is parsed in real-time based on perceptual input. Branches of the syntax tree form the levels of a hierarchical decomposition, and the individual robot sensor readings are given by tokens. We implement this approach in the interactive game of Yamakuzushi on a physical robot resulting in a system that repeatably competes with a human opponent in sustained game-play for the roughly six minute duration of each match.

      @inproceedings{dantam2011yama,
        author = {N. Dantam and M. Stilman},
        title = {The Motion Grammar: Linguistic Perception, Planning, and Control},
        booktitle = {Robotics: Science and Systems (RSS)},
        year = {2011}
    }
  

• N. Dantam, P. Kolhe, and M. Stilman. The Motion Grammar for Physical Human-Robot Games. Intl. Conf. on Robotics and Automation (ICRA). 2011.

  We introduce the Motion Grammar, a powerful new representation for robot decision making, and validate its properties through the successful implementation of a /physical/ human-robot game. The Motion Grammar is a formal tool for task decomposition and hybrid control in the presence of significant online uncertainty. In this paper, we describe the Motion Grammar, introduce some of the formal guarantees it can provide, and represent the entire game of human-robot chess through a single formal language. This language includes game-play, safe handling of human motion, uncertainty in piece positions, misplaced and collapsed pieces. We demonstrate the simple and effective language formulation through experiments on a 14-DOF manipulator interacting with 32 objects (chess pieces) and an unpredictable human adversary.

      @inproceedings{dantam2011chess,
        author = {N. Dantam and P. Kolhe and M. Stilman},
        title = {The Motion Grammar for Physical Human-Robot Games},
        booktitle = {Intl. Conf. on Robotics and Automation (ICRA)},
        year = {2011}
    }
  

• P. Kolhe, N. Dantam, and M. Stilman. Dynamic Pushing Strategies for Dynamically Stable Mobile Manipulators. Intl. Conf. on Robotics and Automation (ICRA). 2010.

  This paper presents three effective manipulation strategies for wheeled, dynamically balancing robots with articulated links. By comparing these strategies through analysis simulation and robot experiments, we show that contact placement and body posture have a significant impact on the robot's ability to accelerate and displace environment objects. Given object geometry and friction parameters we determine the most effective methods for utilizing wheel torque to perform non-prehensile manipulation.

      @inproceedings{kolhe2010pushing,
        author = {P. Kolhe and  N. Dantam and M. Stilman},
        title = {Dynamic Pushing Strategies for Dynamically Stable Mobile Manipulators},
        booktitle = {Intl. Conf. on Robotics and Automation (ICRA)},
        year = {2010}
    }
  

Workshop

• N. Dantam, M. Egerstedt, and M. Stilman. Make Your Robot Talk Correctly: Deriving Models of Hybrid System. RSS Workshop on Grounding Human-Robot Dialog for Spatial Tasks. 2011.

  Using both formal language and differential equations to model a robotic system, we introduce a calculus of transformation rules for the symbolic derivation of hybrid controllers. With a Context-Free Motion Grammar, we show how to test reachability between different regions of state-space and give several symbolic transformations to modify the set of event strings the system may generate. This approach lets one modify the language of the hybrid system, providing a way to change system behavior so that it satisfies linguistic constraints on correct operation.

      @inproceedings{dantam2010talk,
        author = {N. Dantam and M. Egerstedt and M. Stilman},
        title = {Make Your Robot Talk Correctly: Deriving Models of Hybrid System},
        booktitle = {RSS Workshop on Grounding Human-Robot Dialog for Spatial Tasks},
        year = {2011}
    }
  

Technical Reports

• Neil Dantam, Irfan Essa, and Mike Stilman. Algorithms for Linguistic Robot Policy Inference from Demonstration of Assembly Tasks. Georgia Insitute of Technology. GT-GOLEM-2012-002. 2012.

  We describe several algorithms used for the inference of linguistic robot policies from human demonstration. First, tracking and match objects using the Hungarian Algorithm. Then, we convert Regular Expressions to Nondeterministic Finite Automata (NFA) using the McNaughton-Yamada-Thompson Algorithm. Next, we use Subset Construction to convert to a Deterministic Finite Automaton. Finally, we minimize finite automata using either Hopcroft's Algorithm or Brzozowski's Algorithm.

      @techreport{dantam2012algorithms,
        author = {Neil Dantam and Irfan Essa and Mike Stilman},
        title = {Algorithms for Linguistic Robot Policy Inference from Demonstration of Assembly Tasks},
        number = {GT-GOLEM-2012-002},
        institution = {Georgia Insitute of Technology},
        year = {2012}
    }
  

• Neil Dantam and Mike Stilman. Ach: IPC for Real-Time Robot Control. Georgia Insitute of Technology. GT-GOLEM-2011-001. 2011.

  We present a new Inter-Process Communication (IPC) mechanism and library. Ach is uniquely suited for coordinating perception, control drivers, and algorithms in real-time systems that sample data from physical processes. Ach eliminates the Head-of-Line Blocking problem for applications that always require access to the newest message. Ach is efficient, robust, and formally verified. It has been tested and demonstrated on a variety of physical robotic systems. Finally the source code for Ach is available under an Open Source BSD-style license.

      @techreport{dantam2011achtech,
        author = {Neil Dantam and Mike Stilman},
        title = {Ach: IPC for Real-Time Robot Control},
        number = {GT-GOLEM-2011-001},
        institution = {Georgia Insitute of Technology},
        year = {2011}
    }
  

• N. Dantam, P. Kolhe, and M. Stilman. Equations of Motion for Dynamically Stable Mobile Manipulators. College of Computing. Georgia Institute of Technology. GT-GOLEM-2010-002. 2010.

  This paper derives the equations of motion for Sparky, a mobile manipulator robot. These equations are used in the manipulation analysis, simulation, and experiments of "Dynamic Pushing Strategies for Dynamically Stable Mobile Manipulators."

      @techreport{dantam2010equations,
        author = {N. Dantam and P. Kolhe and M. Stilman},
        title = {Equations of Motion for Dynamically Stable Mobile Manipulators},
        number = {GT-GOLEM-2010-002},
        institution = {College of Computing. Georgia Institute of Technology},
        year = {2010}
    }
  

• N. Dantam and M. Stilman. The Motion Grammar: Linguistic Perception, Planning, and Control. College of Computing. Georgia Institute of Technology. GT-GOLEM-2010-001. 2010.

  We present the Motion Grammar: a novel unified representation for task decomposition, perception, planning, and hybrid control that provides a computationally tractable way to control robots in uncertain environments with guarantees on completeness and correctness. The grammar represents a policy for the task which is parsed in real-time based on perceptual input. Branches of the syntax tree form the levels of a hierarchical decomposition, and the individual robot sensor readings are given by tokens. We implement this approach in the interactive game of Yamakuzushi on a physical robot resulting in a system that repeatably competes with a human opponent in sustained game-play for matches up to six minutes.

      @techreport{dantam2010yamatech,
        author = {N. Dantam and M. Stilman},
        title = {The Motion Grammar: Linguistic Perception, Planning, and Control},
        number = {GT-GOLEM-2010-001},
        institution = {College of Computing. Georgia Institute of Technology},
        year = {2010}
    }