AFABL: A Language and Tools for Agent Modeling

My Ph.D. focuses on developing a language and tools for agent modeling. Agent modeling languages are not new; my contribution will be to bring the power of advanced agent modeling to people who are not experts in computer science or artificial intelligence, and to simplify the task for those who are. Like most science, this effort does not begin from nothing. Fortunately, there are towering shoulders on which to stand: Michael Mateas's ABL agent programming language, and Charles Isbell's A2BL machine learning extensions. We will begin from ABL and A2BL in designing our new language and tools, which we call AFABL, for "A Friendly Adaptive Behavior Language."

Three things:

Developing the modeling theory.

Developing modular reinforcement learning algorithms and architectures needed to realize these models in intelligent computational agents.

Three Goals

  1. Better agent models. Natural agents, such as humans, continuously pursue multiple goals in parallel and arbitrate those goals in real-time to effect their minute-to-minute behavior. Artificial agents must be able to do the same if they are to approach any significant level of fidelity to natural agents. Our soution: modular reinforcement learning.

  2. Better agent software engineering. Different agents share many different kinds of goals and resultant behaviors - we will call these packages of goals and resultant behaviors sub-agents. It would be large step forward for agent software engineering if we could write those sub-agents in a modular way that allowed agent authors to compose their agents from libraries of pre-defined sub-agents. However, with current appraches different sub-agents acting within the same agent are coupled by the need to be able to compare the reward and preference signals of each of the sub-agents. To enable the composition of separately-authored sub-agents within a single agent we need to be able to arbitrate between sub-agents in a way that allows the sub-agents to be authored without regard to reward comparability. Our solution: modular reinforcement learning.

  3. Better agent programming usability. A programming language is a user interface. The breadth of the user population of any system is determined by the complexity of its interface. Currently, the most advanced agent modeling systems require an advanced level of programming expertise. To make advanced agent modeling accessible to a greater user population, such as social scientists, we need an agent programming language that enables advanced agent programming without requiring advanced general programming expertise. Our solution: adaptive programming via the partial programming paradigm.

The Current State of the Art: ABL

ABL represents the cutting edge of agent modeling languages. ABL supports sequential and parallel behaviors in a reactive planning framework. Connected to a game or simulation engine, it supports multi-agent play with collaborative and adversarial interactions within a distributed decision-making framework. ABL supports action and goal conflicts as well as the representation of team intentionality that can be used for modeling and reasoning. ABL gives behaviors reflective access to the current state, supporting meta- behaviors that modify other behaviors. Meta-behaviors are useful for modeling intermixed chains of communication that arise in group decision-making and adversarial reasoning. From the foundation of these ABL features, there are many possible directions for AFABL.

Usability

Because "behavior" is a part of the ABL acronym, one might believe that ABL is designed for experts in human behavior, such as psychologists or sociologists. While ABL can support the needs of such designers, ABL is a complex language that exposes many technical details to agent authors, making it suitable mainly for programming experts. So far, mainly senior undergraduate and graduate students in computer science have been productive with ABL. Thus, bringing the power of ABL to non-programming experts is a major research challenge that requires that we answer difficult questions:

Adaptability

One of the promises of machine learning is that it allows designers to specify problems in broad strokes while allowing a machine to do further parameter fine-tuning. Typically, one thinks of building a system or agent for some specific task and then providing it some kind of feedback, allowing it to learn. In this case, the agent is the point of the exercise. A2BL embeds this notion within a programming language itself by extending ABL with adaptive behaviors. The power of such a merger of machine learning and a programming language is that it allows for what has become known as partial programming; that is, it allows a designer to specify what he knows how to express exactly and leave the system to learn how to do the rest. Building or extending a programming language so that it has learning as a primitive operator invites many interesting research questions:

Software Engineering

At the end of the day, a AFABL agent is a kind of computer program running in a run-time environment. Whatever language features AFABL supports, computer programs (or computational agent models) will need to be written and debugged. Given the complexity of individual agents and our desire to support real world-scale multi-agent system modeling, the task of writing AFABL agents and multi-agent systems is likely to be a significant effort, akin to that of a large software engineering project. It is likely, therefore, that the field of software engineering will have much to offer us in the design of AFABL. Several research questions need to be addressed: