Human-like Action Segmentation
Video: Human-like action segmentation
Contributors: J. Shim, A.L. Thomaz

Robots learning interactively with a human part- ner has several open questions, one of which is increasing the efficiency of learning. One approach to this problem in the Reinforcement Learning domain is to use options, temporally extended actions, instead of primitive actions. In this work, we aim to develop a robot system that can discriminate meaningful options from observations of human use of low-level primitive actions. Our approach is inspired by psychological findings about human action parsing, which posits that we attend to low-level statistical regularities to determine action boundary choices. We implement a human-like action segmentation system for automatic option discovery and evaluate our approach and show that option-based learning converges to the optimal solutions faster compared with primitive-action-based learning.

J. Shim and A.L. Thomaz, "Human-like Action Segmentation for Option Learning." In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2011.

Learning Task Goals from Demonstrations
Video: Simon Learns Task Goals, Simon learns where things go (CHI 2010), Concept grounding in Kitchen Tasks
Contributors: C. Chao, M. Cakmak, A.L. Thomaz

We aim to build robots that frame the task learning problem as goal inference so that they are natural to teach and meet people's expectations for a learning partner. The focus of this work is the scenario of a social robot that learns task goals from human demonstrations without prior knowledge of high-level concepts. In the system that we present, these concepts are grounded from low-level continuous sensor data through unsupervised learning, and task goals are subsequently learned on them using a Bayesian approach. Our grounded concepts are derived from the structure of the Learning from Demonstration (LfD) problem and exhibit degrees of prototypicality. These concepts can be used to transfer knowledge to future tasks, resulting in faster learning of those tasks. Using sensor data taken during demonstrations to our robot from five human teachers, we show the expressivity of using grounded concepts when learning new tasks from demonstration. We then show how the learning curve improves when transferring the knowledge of grounded concepts to future tasks.

C. Chao, M. Cakmak and A.L. Thomaz, "Towards Grounding Concepts for Transfer in Goal Learning from Demonstration." In Proceedings of the International Conference on Development and Learning (ICDL), 2011.

Turn-taking and Contincgency Detection in HRI
Videos: Contingency detection
Contributors: C. Chao, J. Lee, J.F. Kieser, M. Begum, A.F. Bobick, A.L.Thomaz

We present a novel method for the visual detection of a contingent response by a human to the stimulus of a robot action. Contingency is defined as a change in an agent’s behavior within a specific time window in direct response to a signal from another agent; detection of such responses is essential to assess the willingness and interest of a human in interacting with the robot. Using motion-based features to describe the possible contingent action, our approach assesses the visual self-similarity of video subsequences captured before the robot exhibits its signaling behavior and statistically models the typical graph-partitioning cost of separating an arbitrary subsequence of frames from the others. After the behavioral signal, the video is similarly analyzed and the cost of separating the after-signal frames from the before-signal sequences is computed; a lower than typical cost indicates likely contingent reaction. We present a preliminary study in which data were captured and analyzed for algorithmic performance.

C. Chao, J. Lee, M. Begum and A.L. Thomaz, "Simon plays Simon says: The Timing of Turn-taking in an Imitation Game." In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2011.

J. Lee, J.F. Kieser, A.F. Bobick and A.L.Thomaz, "Vision-based Contingency Detection." In Proceedings of the International Conference on Human-Robot Interaction (HRI), 2011.

C. Chao and A.L. Thomaz, "Turn-Taking for Human-Robot Interaction." In the poster session of the AAAI Fall Symposium on Dialog with Robots, 2010.

Life-like Robot Motion
Videos: HRI 2011: Spatio-temporal correspondance, Task-aware variations, RO-MAN 2010: Stylized motion, RO-MAN 2010: Secondary action
Contributors: M.Gielniak, C.K. Liu, A.L.Thomaz

In social interactions with human partners, humans communicate with each other through their motion. Multimodality and communicative motion benefit interaction through increased partner internal state transparency, improved teammate timing synchronization, and reduced training costs for collaborative robots.
We hypothesize that believable motion increases communication, improves interaction, and advances task completion for social robots interacting with human partners. The theory is that improved motion appearance communicates a cleaner message to the human partner (i.e. improved signal-to-noise ratio).
Misclassification of motion information by human partners occurs when the communicative information is outside a bounded region. Therefore, communication in motion must be added in a specific and correct way to increase interaction task performance.
The main challenge is lack of a metric to measure believable motion. Thus, the objectives of our research are to study motor coordination (i.e. spatiotemporal correspondence) as a viable metric for believable motion; use the metric to develop a real-time, dynamic, autonomous motion algorithm, which systematically composes communicative signals to robot motion using minimal prior information; and quantitatively prove how this metric and algorithm improve interaction task performance.

M.J. Gielniak and A.L. Thomaz, "Anticipation in Robot Motion." In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2011.

M.J. Gielniak, C.K. Liu and A.L. Thomaz, "Task-aware Variations in Robot Motion." In Proceedings of the International Conference on Robotics and Automation (ICRA), 2011.

M.J. Gielniak and A.L. Thomaz, "Spatiotemporal Correspondence as a Metric for Human-like Robot Motion." In Proceedings of the International Conference on Human-Robot Interaction (HRI), 2011.

M.J. Gielniak, C.K Liu and A.L. Thomaz, "Stylized Motion Generalization Through Adaptation of Velocity Profiles." In Proceedings of the 19th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2010.

M.J. Gielniak, C.K Liu and A.L. Thomaz, "Secondary Action in Robot Motion." In Proceedings of the 19th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2010.

Active Learning in Human-Robot Interaction
Video: Simon CHI 2010 Demo
Contributors: M. Cakmak, C.Chao, A.L.Thomaz

We address some of the problems that arise when applying active learning to the context of human–robot interaction (HRI). Active learning is an attractive strategy for robot learners because it has the potential to improve the accuracy and the speed of learning, but it can cause issues from an interaction perspective. We present three interaction modes that enable a robot to use active learning queries. The three modes differ in when they make queries: the first makes a query every turn, the second makes a query only under certain conditions, and the third makes a query only when explicitly requested by the teacher. We conduct an experiment in which 24 human subjects teach concepts to our upper-torso humanoid robot, Simon, in each interaction mode, and we compare these modes against a baseline mode using only passive supervised learning. We report results from both a learning and an interaction perspective. The data show that the three modes using active learning are preferable to the mode using passive supervised learning both in terms of performance and human subject preference, but each mode has advantages and disadvantages. Based on our results, we lay out several guidelines that can inform the design of future robotic systems that use active learning in an HRI setting.

M. Cakmak, C. Chao, and A.L. Thomaz, "Designing Interactions for Robot Active Learners." in IEEE Transactions on Autonomous Mental Development, 2010.

M. Cakmak and A.L. Thomaz, "Optimality of Human Teachers for Robot Learners." in Proceedings of the International Conference on Development and Learning (ICDL), 2010.

C. Chao, M. Cakmak, and A.L. Thomaz, "Transparent active learning for robots." In Proceedings of the International Conference on Human-Robot Interaction (HRI), 2010.

Robot Playmates - Biologically Inspired Social Learning Mechanisms
Video: The Robot Playmates
Contributors: M. Cakmak, N. DePalma, R.I. Arriaga, A.L.Thomaz

Social learning in robotics has largely focused on imitation learning. In this work, we take a broader view of social learning and are interested in the multifaceted ways that a social partner can influence the learning process. We implement stimulus enhancement and emulation on a robot, and illustrate the computational benefits of social learning over self exploration. Additionally we characterize the differences between these two social learning strategies, showing that the preferred strategy is dependent on the current behavior of the social partner.

M. Cakmak, N. DePalma, R. Arriaga, and A.L. Thomaz. "Exploiting social partners in robot learning." Autonomous Robots, 2010.

M. Cakmak, N. DePalma, R. Arriaga, and A.L. Thomaz. "Computational benefits of social learning mechanisms: Stimulus enhancement and emulation." In Proceedings of the International Conference on Developmental Learning (ICDL), 2009.

A.L. Thomaz, M. Cakmak, N. DePalma, and R. Arriaga. "Effects of social exploration mechanisms on robot learning." In Proceedings of the 18th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2009.

Learning about Objects from Humans
Video: Junior: Object Affordances
Contributors: M. Cakmak, A.L.Thomaz

A general learning task for a robot in a new environment is to learn about objects and what actions/effects they afford. To approach this, we look at ways that a human partner can intuitively help the robot learn, Socially Guided Machine Learning. We conducted experiments with our robot, Junior, and made six observations characterizing how people approached teaching about objects. We showed that Junior successfully used transparency to mitigate errors. Finally, we present the impact of “social” versus “non-social” data sets when training SVM classifiers.

A.L. Thomaz and M. Cakmak, "Learning about objects with human teachers." In Proceedings of the International Conference on Human-Robot Interaction (HRI), 2009.

Webgames to Advance Interactive Learning Agents
Contributors: P. Zang, A. Irani, P. Zhou, R. Tian, C. Isbell, A.L. Thomaz

We are interested in machines that can learn from everyday people. To study this, we are building a suite of short computer games, with interactive learning agents, to be deployed on the web. These serve as a testbed for experiments with various algorithms and interface techniques, looking at how to allow the average person to successfully teach machine learning agents.

P. Zang, A. Irani, P. Zhou, C. Isbell, and A.L. Thomaz. "Combining function approximation, human teachers, and training regimens for real-world RL.", In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2010.

P. Zang, R. Tian, A.L. Thomaz, and C. Isbell, "Batch versus Interactive Learning by Demonstration." In Proceedings of the International Conference on Development and Learning (ICDL) 2010.

Shimon Listens
Videos:MidEastJam, Jazz, Head-closeup
Contributors: G. Weinberg, G. Hoffman, M. Cakmak, M. Gielniak, A.L. Thomaz

Shimon, is designed to play the marimba. It utilizes more melodic oriented algorithms for music perception and improvisation in comparison to Haile, Georgia Tech’s first robotic drummer. Shimon can also create richer sound and more communicative social cues with its human counterparts. The robot’s head is made to provide fellow musicians social cues that represent the music being played, from beat detection to tonality and spatial interaction. See review of Shimon on Wired - Robot Pass Musical Turing Test.