Research Interests

Scalable Reinforcement Learning (RL)

RL provides an expressive and powerful model for describing and solving many planning and control problems; however, scaling RL to large domains with high dimensional state spaces has posed some difficulty. I am interested in scaling RL to these domains. I believe problem decomposition (eg. Motif-RL), function approximation and in particular, their combination are important to this end. My interest in problem decomposition overlaps with that of concept discovery.

Concept Discovery

Reasoning approaches can solve complex and difficult problems, but there is no free lunch; the cost lies in the engineering of the right representation and the right domain knowledge. Concept discovery is the task of finding the representations and so represented knowledge for the purposes of enabling a decision making system. Not only does the correct concepts improve performance, but it enables transfer of learned skills to new tasks and new domains.

We are particularly interested in concept discovery in the reinforcement learning (RL) context. In particular this translates to learning hierarchical structure. We will be focusing this work in single-agent, markov domains such as flight sims, casual games such as Fort and real time strategy games.

Ensemble Learning

Ensemble techniques are interesting because they allow us to leverage a variety of different learners together. This implies the ability to leverage different models (ie. different parametric models as well as nonparametric techniques), different sets of domain knowledge and different machine learning techniques. Further more, ensemble techniques are powerful. Recent empirical results also show that ensemble techniques consistently out perform any single learner. In MBoost, we demonstrate that boosting different models together is a viable and often better performing alternative to model selection (eg. via cross validation).

Interactive Learning

Interactive learning and related work in active learning is an evergreen research interest for me. Any time there is a scarcity of examples, you must make the most out of those examples that you can get. Active learning is all about optimizing the problem of getting the right examples to maximize learning. I am particularly interested in the case when the source of examples is the user, making it an interactive learning problem. This fits well with my overall philosophy that AIs are, in the end, built for humans and as such human-AI interfacing and interaction is key.

Previous Research

Online Behavior Adaptation

"AI" agents in games are typically no more than a set of static scripts (behaviors). While behavior generation and adaptation is equivalent to automatic programming and undecidable in general, limited local search for "fixing" behaviors is possible. We explore a transformational approach to behavior adaptation and show how it can be applied to behavior sets as a means for achieve online agent adaptation. Preliminary experiments show the system can be effective in improving agent behavior in situations where its original static behaviors fail.

Social Network Analysis

Analyzing social groups and their interactions, particularly to identifying opinion leaders, communication carriers and blockers. To aid in this analysis, we also pursued named entity extraction and particularly, their relationships.

Text Mining

Relatively little has been on automated comparision of multi-collection documents. This is an extremely useful function. For example, the ability to compare and contrast articles on the same event from different news sources. An automated comparator should be able to pick out common themes between two collections of documents and point of biases that one collection might have over another. Our approach is to view the collections as a mixture of language models making the compare/contrast process a clustering problem of finding the original language models.

Papers submitted to many academic conferences must be stripped of authors' names for double-blind peer review. All too often, names are left on and these instances must be detected and flagged for manual removal. This process is complicated by the common use of anonymous or fake names such as "John Doe" in place of stripping the names. Our approach is to combine entity extraction techniques to identify names with name disambiguation using Google's search mechanisms.