Homework 6: Hero Agents

A Multiplayer Online Battle Arena (MOBA) is a form of Real Time Strategy game in which the player controls a powerful agent called the "Hero" in a world populated with simple, weak, fully computer-controlled agents called "Minions." In this assignment, we will implement the decision-making for Minion agents.

A MOBA has two teams. Each team has a base, which is protected by a number of towers. The goal is to destroy the opponent's base. In MOBAs, bases periodically spawn Minion agents, who automatically attack the enemy towers and bases. Towers and bases can defend themselves; they target Minions before targeting Heros. Thus Minions provide cover for Heros, who are much more powerful.

In this version of a MOBA, Heros have a number of special properties that make them more powerful than Minions:

In this assignment, you will implement the AI for a Hero agent in a world similar to a MOBA. However, we will substantially change the rules so we can focus on Hero AI without all the complications of a MOBA. In this game, there are no towers and each team has a maximum of three minions at any given time. The minions wander aimlessly, but shoot at Heroes if they ever get too close. Heros must hunt each other, and the game is scored by how much damage one Hero does to another Hero.

We will build off your previous navigation mesh and path network generation solution from homework 2 and your previous A* implementation from homework 4.

Hero decision-making can be anything, but in a MOBA, typically the Hero focuses on gaining extra powers so it can take out enemy Minions at a faster rate. Eventually, a Hero will be powerful enough to quickly take down towers and bases. Since towers and bases target Minions before targeting Heros, it is beneficial for Heros to protect friendly Minions and use them for cover. By targeting enemy Minions, a Hero can make it harder for the enemy Hero to take cover. Heros may engage each other from time to time to disrupt opponent advantage.

However, in this assignment, we will only focus on the Hero vs. Hero aspects of a MOBA. You will implement a Behavior Tree for Hero agents. You will be provided with a special class for Heroes that knows how to execute a behavior tree. In this assignment, you will write the control logic for behavior trees and complete the code for a number of Hero behaviors.

In Hero vs. Hero combat the best solution is to focus on strategy. Heros have many properties (listed above) that make for interesting trade-offs when deciding what behavior to execute. Examples of strategic decisions include: hunting the enemy Hero, hunting Minions to increase level, using a longer-ranged shooting attack versus a limited-range area effect attack, retreating to the base to heal, hiding from the enemy Hero, hiding from Minions, etc.

The bases will automatically spawn Minion agents, up to a maximum of three. If a Hero dies, it will immediately respawn at the base, but the level will be reset.

Both teams will use the same Minion agent AI, which wanders the map to random places. Minions will shoot at Heros if they are within firing range.

The game score is computed as the cummulative amount of damage one Hero has done to the other Hero. You must implement Hero AI that can result in a higher score than the opponent.

Behavior Trees in a nutshell

A behavior tree is a hierarchically-arranged description of all possible plans of action that an agent can consider, and the conditions under which those plans can be selected or rejected. Behavior trees represent a middle ground between finite state machines, in which all behavior is specified by a designer, and formal planning, in which all decisions are autonomously made. A behavior tree allows designers to specify which plans can be generated and allow the agent to make the final decision on which plan to execute. Since enumerating all possible plans an agent can take may be intractable, behavior trees allow plans to be broken up into sub-plans that are reused in a hierarchical fashion.

Behavior trees are made up of different types of nodes. Internal nodes represent behavior flow control, specifying how to combine sub-trees into plans. Leaf nodes are tasks, specifying behaviors an agent should execute in the world. Tasks also specify the conditions under which tasks can and cannot be performed. The behavior tree acts as a pesudo-mathematical definition of the agent's brain, with tasks performing call-backs to the agent body to act in the world and affect the world state.

Consider the behavior tree below:

At every tick, the agent calls the execute() function on the root of the tree. Each type of node has a different logic for execution. The nodes labeled with question marks are Selector nodes. A Selector node tries to execute each child until one indicates that it can be executed. Thus the behavior tree above first tries to retreat. If it is not appropriate to retreat, it tries something else. The nodes labeled with arrows are Sequence nodes. A Sequence node tries to execute each child in turn until all children have reported that they have executed successfully.

Thus, the logic represented by the above behavior tree is as follows. The agent tries to retreat, returning to the base for healing. If it is not appropriate for the agent to retreat--it has not lost enough health--it then tries to chase the hero and then kill the hero. If that sequence fails for any reason, the agent tries to chase a minion and then kill the minion.

The circles in the diagram are a special type of node called a Daemon. A Daemon checks a condition in the world and decides whether to execute its child or return instant failure. For example, the top most Daemon might check whether the health of the agent is greater than 50%. Thus, the agent can only chase heroes and minions if it has enough health. The lower Daemon might check whether the agent is more powerful than the enemy Hero. Thus, the agent can only execute the sequence of chasing and killing the enemy Hero if it is strong enough to do so. Daemons short-circut the tree, allowing a decision to recurse to be made quickly. They also allow the tree to quickly stop executing on a sub-tree if certain conditions become false.

An agent that implements a behavior tree calls the execute() function of the root node every tick. The purpose of internal nodes (Selectors, Sequences, and Daemons) is to figure out which leaf node (task) should have its execute() function called that tick. Thus think of a behavior tree as a cascade in which control flows from the root to exactly one leaf node. The execute() function of the leaf node is called and any appropriate action taken.

Nodes can return one of three values: success (True), failure (False), or running (None). Success means that the behavior has run to completion and has achieved what it is supposed to achieve. For example, a successful retreat means having the agent's health restored to 100%. Failure means that either the behavior is not applicable in the current world state or that it is no longer able to achieve the success conditions and should terminate. In a non-turn-based game world, some behaviors require many ticks to complete successfully (or to fail). For example, retreating requires navigating back to the home base, which may take many ticks. The running (None) return value means that the node requires more time to determine whether it has succeeded or failed.

Logic for a Selector node:

The execute() function of a Selector node must decide which child should execute. It tries to execute each child in order until one does not return failure. However, only one child can have its execute() function called. Therefore a Selector remembers which child should be tried on the current tick.

  1. If all children have been tried and none have succeeded, then the Selector itself fails and returns False.
  2. Otherwise:
    1. If the current child's execute() returns True, then the Selector itself succeeds and returns True.
    2. If the current child's execute() returns False, then update the current child and return None (indicating that this node should be tried again next tick).
    3. If the current child's execute() returns None, then return None.

Logic for a Sequence node:

The execute() function of a Sequence node must decide which child should execute. It tries to execute each child in order until one returns a failure. However, only one child can have its execute() function called. Therefore a Sequence remembers which child should be tried on the current tick.

  1. If all children have been tried and all have succeeded, then the Sequence itself succeeds and returns True.
  2. Otherwise:
    1. If the current child's execute() returns True, then update the current child and return None (indicating that this nodes whould be tried again next tick).
    2. If the current child's execute() returns False, then the Sequence itself fails and returns False.
    3. If the current child's execute() returns None, then return None.

Leaf nodes:

The execute() functions of a leaf node must do three things. First, it must check applicability conditions and return failure immediately if the behavior is not applicable to run at this time. Second, it must call back to the agent to perform any appropriate actions. Whatever the leaf node does, it shoud not require a lot of computation because execute() is called every tick. Third, it must determine whether the behavior has succeeded. If the behavior is applicable but has not succeeded, it should indicate that it is still running by returning None.

The first time a leaf node executes, an additional enter() function will be called to do any one-time set up for execution. Enter() will only be called once per leaf node. However, if the tree is ever reset, then each leaf node will have its enter() function called again the next time the node is visited.

Daemon nodes:

The execute() function of a Daemon node checks the applicability of an entire sub-tree (as opposed to a single behavior). The execute() function checks the applicability conditions and returns False if the conditions are not met. If the conditions are met, the execute() function of its single child is called and the Daemon returns the child's return value as if it were its own. Daemons assume a single child.

In this assignment, you will implement the logic for Selector and Sequence nodes. You will be given the opportunity to test your implementations before working of Hero agents. You will then be asked to implement the execute() functions for a number of different types of behaviors for MOBA Heroes. We will provide you with several different tree configurations to test your Hero agent with.

What you need to know

Please consult previous homework instructions for background on the Game Engine. In addition to the information about the game engine provided there, the following are new elements you need to know about.

Agent

Three things are newly relevant to this assignment. (1) Agents have hitpoints. (2) Agents can be part of a team. (3) Agents can shoot in the direction they are facing.

Member variables:

Member functions:

Note: To effectively shoot at something, first turn the agent to face the target (or to the point the agent wishes to fire at) with turnToFace() and then call shoot().

BehaviorTree

BehaviorTree is defined in behaviortree.py. A BehaviorTree is a container class that maintains a pointer to the root node (BTNode) of the behavior tree. At every tick, the BehaviorTree will call the execute() function on the root node.

When the root node of the behavior tree returns success or failure, then the tree is reset for another run next tick.

A BehaviorTree object also knows how to build a complete tree from a specification. When the tree is built, each node is instantiated as an object in memory with pointers to its children, but no nodes are actually executed at that time. Once the tree has been built, it is ready for execution during the gameplay loop.

Member variables:

Member functions:

The buildTree() function takes in a specification that tells the BehaviorTree the type of each node in the tree, the child/parent relationships between each node, and any parameters that can be known at build time. The build specification language is a sub-set of the Python language and is as follows:

For example: [(Sequence, 1), [(Sequence, 2), (BTNode, 3), (BTNode, 4)], [(Selector, 5), [(Sequence, 6), (BTNode, 7), (BTNode, 8)], (BTNode, 9), (BTNode, 10), BTNode]] creates the following behavior tree. In this example, we are assuming that the second element in a Tuple is an identification parameter for the node. Note that one node does not have any parameters.

BTNode

BTNode is defined in behaviortree.py. a BTNode is a parent class for nodes in a behavior tree. BTNode assumes that a node has children, but leaf nodes do not use the children. The primary functionality of a BTNode is it's execute() function, which will be called in the course of a tick. The first time a BTNode is visited in the coruse of exection, the enter() function is also called. BTNode assumes a node has children, but that is not necessarily the case for behavior types, which will always be leaf nodes. If a BTNode is an internal node, it keeps track of its current child with an index number.

Member variables:

Member functions:

For exectute() to control the agent, it must make call-backs via the agent member variables. This is node in leaf nodes. For example, if the behavior of a state is to make the agent shoot, the execute() function can call self.agent.shoot().

The execute() function for internal nodes should determine which single child should have its execute() function called. The execute() function for leaf nodes should implement the intended behavior, making call-backs to the agent.

The constructor for the base BTNode class takes a number of arguments, as a list. But it doesn't know what they are meant to be. Constructors for sub-classes can look at the arguments passed in through args and pick out the relevant information and store it or compute with it. For example, TestNode takes the first element in the list and sets the ID.

For example, one might want a Taunt behavior, and the tree will be built to always taunt a particular enemy agent. The tree build specification could be [Sequence, (Taunt, enemyhero), [Selector, Retreat, [Sequence, ChaseHero, KillHero]]] where enemyagent is a pre-computed reference to the enemy hero. Even though the Taunt sub-class is expecting an argument, it will just be passed in to the constructor as args[0]. Use parseArgs(args) to capture the parameter and use it. For example: