Homework 6: Hero Agents

[Updated 3/09/15 -- grading criteria added]

A Multiplayer Online Battle Arena (MOBA) is a form of Real Time Strategy game in which the player controls a powerful agent called the "Hero" in a world populated with simple, weak, fully computer-controlled agents called "Minions." In this assignment, we will implement the decision-making for Minion agents.

A MOBA has two teams. Each team has a base, which is protected by a number of towers. The goal is to destroy the opponent's base. In MOBAs, bases periodically spawn Minion agents, who automatically attack the enemy towers and bases. Towers and bases can defend themselves; they target Minions before targeting Heros. Thus Minions provide cover for Heros, who are much more powerful.

In this version of a MOBA, Heros have a number of special properties that make them more powerful than Minions:

• Heros have twice as many hitpoints as Minions.
• Heros have a more powerful gun.
• Heros have a secondary attack that does damage to all enemies within a certain radius.
• Heros can be healed if they return to the base.
• Heros can dodge bullets by "jumping" a short distance in a given direction (or a random direction).
• Heros can gain level-ups by killing enemy agents. A level is gained for each kill the Hero makes (the last agent to damage an opponent before it dies). The Hero gains one extra point of damage per level and maximum hitpoints incresases by one for each level earned.

In this assignment, you will implement the AI for a Hero agent in a world similar to a MOBA. However, we will substantially change the rules so we can focus on Hero AI without all the complications of a MOBA. In this game, there are no towers and each team has a maximum of three minions at any given time. The minions wander aimlessly, but shoot at Heroes if they ever get too close. Heros must hunt each other, and the game is scored by how much damage one Hero does to another Hero.

We will build off your previous navigation mesh and path network generation solution from homework 2 and your previous A* implementation from homework 4.

Hero decision-making can be anything, but in a MOBA, typically the Hero focuses on gaining extra powers so it can take out enemy Minions at a faster rate. Eventually, a Hero will be powerful enough to quickly take down towers and bases. Since towers and bases target Minions before targeting Heros, it is beneficial for Heros to protect friendly Minions and use them for cover. By targeting enemy Minions, a Hero can make it harder for the enemy Hero to take cover. Heros may engage each other from time to time to disrupt opponent advantage.

However, in this assignment, we will only focus on the Hero vs. Hero aspects of a MOBA. You will implement a Finite State Machine (FSM) for Hero agents. The base Hero class has a state machine built into it and each state is an object that implements the behavior of the agent in that state by telling the agent what to do and where to go. Specifically, states have code in their execute() functions that act as the "brain" of the Hero, instructing it what to do at every tick of the game. The Hero agent class will automatically start off in an Idle state. You must implement at least three other states, and modify the Idle state to immediately transition to one of the other states you have implemented.

In Hero vs. Hero combat, the best solution is to focus on strategy. Heros have many properties (listed above) that make for interesting trade-offs when deciding what behavior to execute. Examples of strategic decisions include: hunting the enemy Hero, hunting Minions to increase level, using a longer-ranged shooting attack versus a limited-range area effect attack, retreating to the base to heal, hiding from the enemy Hero, hiding from Minions, etc.

The bases will automatically spawn Minion agents, up to a maximum of three. If a Hero dies, it will immediately respawn at the base, but the level will be reset.

Both teams will use the same Minion agent AI, which wanders the map to random places. Minions will shoot at Heros if they are within firing range.

The game score is computed as the cummulative amount of damage one Hero has done to the other Hero. You must implement Hero AI that can result in a higher score than the opponent.

What you need to know

Please consult previous homework instructions for background on the Game Engine. In addition to the information about the game engine provided there, the following are new elements you need to know about.

Because the object hierarchy is complex, here is a documentation of how all the object types relate (updated for Homework 6):

Agent

Three things are newly relevant to this assignment. (1) Agents have hitpoints. (2) Agents can be part of a team. (3) Agents can shoot in the direction they are facing.

Member variables:

• hitpoints: the amount of health the agent has. The agent dies when hitpoints reaches zero.
• team: the symbolic name of the team the agent is on.

Member functions:

• getHitpoints(): returns the number of hitpoints.
• getTeam(): returns the symbol of the team the agent is on.
• turnToFace(pos): turn the agent to face a particular point (x, y)
• shoot(): fire the agent's gun in the direction the agent is facing. The agent can only fire after a certain number of ticks have elapsed.

Note: To effectively shoot at something, first turn the agent to face the target (or to the point the agent wishes to fire at) with turnToFace() and then call shoot().

StateMachine

A StateMachine implements the generic functionality of a Finite State Machine. In this assignment, StateMachine is one of the base classes of Minions, thus every Minion is a StateMachine. StateMachines know what the current state is and tell the current state to execute at every update.

Member variables:

• states: A list of states that the machine can be in. States are names of classes that subclass State. Only States that are in this list can be executed.
• state: A pointer to the current state instance.

Member functions:

• getState(): returns the type of the current state. Since it is usually not important to have a pointer to the state object, getState() returns the type of the current state. Thus, if you want to know what the StateMachine is doing, use something like: getState() == Idle
• update(delta): Should be called every tick. This will be called by the engine's game loop. The current state will have its execute() function called.
• changeState(newclass, *args): causes the StateMachine to change state. newclass is the type of the state to run next, which will be instantiated. Sometimes states take arguments, so pass in any arguments through *args. changeState takes an arbitrary number of arguments after the first one. Only the first argument is required.

Note: A StateMachine will not change states to any type of state that is not listed in StateMachine.states.

State

An abstract base class for all states in a StateMachine. States are more than symbols. Each state has an execute() function that gets called every tick of the game and controls the behavior of the agent.

Member variables:

• agent: A reference to the agent that this state is controlling.

Member functions:

• enter(oldstate): this function is called when the state begins execution. A state should do whatever one-time work is necessary to set things up for execution. oldstate is the type of the state that executed just prior to the state change.
• execute(delta): does the work of controlling the agent. Delta is the amount of time elapsed since the last call to execute. This function is part of the game loop update, so should not perform computationally expensive operations.
• exit(): this function is called when the agent is about to transition to another state. Use this function to do any clean up.
• parseArgs(args): you can override this function to take any arguments passed into the state upon creation and do the right thing with them.

For exectute() to control the agent, it must make call-backs via the agent member variables. For example, if the behavior of a state is to make the agent shoot, the execute() function can call self.agent.shoot(). To change states, call back to self.agent.changeState().

When changeState() is called, arguments can be passed to the new state when it is initialized. The constructor for the base State class takes a number of arguments, as a list. But it doesn't know what they are meant to be. Constructors for sub-classes can look at the arguments passed in through args and pick out the relevant information and store it or compute with it. For example, one might want a Taunt state, and pass in an argument for the thing to be taunted. For example: the agent could call self.changeState(Taunt, enemyagent). Even though the Taunt sub-class is expecting an argument, it will just be passed in to the constructor as args[0]. Use parseArgs(args) to capture the parameter and use it. For example:

class Taunt(State):
def parseArgs(self, args):
self.victim = args[0]

def execute(self, delta = 0):
if self.victim is not None:
print "Hey " + str(self.victim) + ", I don't like you!"
self.agent.changeState(Idle)

StateAgent

A StateAgent is a sub-class of Agent and StateMachine. Use getStateType() to determine what state the agent is in.

VisionAgent

A VisionAgent is a sub-class of StateAgent. VisionAgent is given a viewangle, a number of degrees that the agent can see. Every tick, the VisionAgent asks the GameWorld what it can see, based on its view angle, and maintains a list of visible Movers (agents, bullets, towers, and bases). For this assignment, Minions have a view angle of 360 degrees, meaning they can see everything around them irrespective of what direction they are facing.

Member variables:

• viewangle: the number of degrees the agent can see, centered around the front of the agent (i.e., 1/2 viewangle clockwise from the agent's orientation, and 1/2 viewangle counterclockwise from the agent's orientation).
• visible: a list of Movers that is currently visible (re-computed every tick).

Member functions:

• getVisible(): returns a list of visible Movers.
• getVisibleType(type): returns a list of visible Movers of a given class type.

MOBAAgent

A sub-class of VisionAgent (and hence also a StateAgent and a StateMachine), specialized for the MOBA. MOBAAgents do two noteworthy things. First, MOBAAgents die whenever they collide with an Obstacle. Second, they can compute a list of points in navigable space in the even that the agent needs to choose a point to move to without worrying about whether that point is inside an Obstacle (the agent will still have to figure out if it can actually move there).

Additionally, new to this homework, MOBAAgents keep track of the agent that last did damage to them, and have a level, which is used to compute maximum hitpoints and amount of damage they can do.

Member variables:

• maxHitpoints: the maximum number of hitpoints the agent can have.
• lastDamagedBy: a pointer to the last agent to have done damage to this agent.
• level: the level of the agent, which is used to determine maximum hitpoints (one extra hitpoint per level), and amount of damage to do (one extra point of damage per level).

Member functions:

• getMaxHitpoints(): returns the maximum number of hitpoints the agent can have.
• getPossibleDestinations(): returns a list of points that are not in Obstacles.
• getLevel(): return the level, an integer.

Hero

Abstract base class, which is a sub-type of MOBAAgent, and thus is a State Machine.

In this assignment, the Hero can dodge, use an area effect attack, be healed by bases, and level-up whenever it kills another agent of any type.

Member variables:

• canDodge: True when the agent can dodge.
• dodgeTimer: counts the number of ticks until the agent can dodge.
• dodgeRate: the number of ticks until an agent can dodge again.
• canAreaEffect: True when the agent can use the area effect attack.
• areaEffectTimer: counts the number of ticks until the agent can use the area effect attack.
• areaEffectRate: the number of ticks until an agent can use the area effect attack.
• areaEffectDamage: the amount of damage the area effect attack does.

Member functions:

• dodge(angle): causes the agent to jump in the direction of the given angle (0 <= angle <= 360). The angle parameter is option and, if omitted, the agent will dodge in a random direction.
• areaEffect(): use the area effect attack.

MOBABullet

A special Bullet class for the MOBA. MOBABullets differ from regular bullets in that they are range-limited.

There are four sub-classes of MOBABullet: SmallBullet, BigBullet, BaseBullet, and TowerBullet. These bullets are specific to Minions, Heros, Bases, and Towers, respectively and do different amounts of damage.

Base

Each team in a MOBA has a Base. Bases spawn minions at regular intervals as long as the maximum number of minions allowed at any given time has not been reached. Bases cannot be damaged as long as there are towers present on the same team. Bases can heal Heros---if a Hero touches a base, it's hitpoints are restored to maximum value.

Member variables:

• team: the symbol of the team that the base is on.
• hitpoints: the amount of health the base has.

Member functions:

• getTeam(): returns the symbol of the team the base is on.
• getHitpoints(): returns the number of hitpoints.

MOBAWorld

A special type of GameWorld for MOBAs. MOBAWorld is a type of GatedWorld. The MOBAWorld keeps track of bases and towers, in addition to NPCs, Bullets, and the agent.

New to this assignment: the MOBAWorld keeps track of the score for each team.

Member variables:

• score: a dictionary indexed on each team symbol. The score is the amount of damage a Hero does to any enemy Hero.

Member functions:

• getNPCs(): returns a list of NPCs in the game (includes all agents not controlled by the player).
• getNPCsForTeam(team): return a list of NPCs part of the given team.
• getEnemyNPCs(myteam): return a list of NPCs that are not part of the given team.
• getAgent(): returns the player-controlled character.
• getBases(): return a list of all bases.
• getBasesForTeam(team): return a list of all bases on a given team.
• getEnemyBases(team): return a list of all bases not on the given team.
• getTowers(): return a list of all towers.
• getTowersForTeam(team): return a list of all towers on a given team.
• getEnemyTowers(team): return a list of all towers not on the given team.
• getBullets(): return a list of all bullets in the world at that moment.
• getScore(team): return the score for a team.

Instructions

To complete this assignment, you must implement Hero AI. Write at least three State classes, and modify the Idle state class to transition to these customized states.

Use your solution to homework 2 to generate a navigation mesh and a corresponding path network. The instructor can provide you with a default solution if necessary. You may also wish to use any path smoothing operations from homework 3 or homework 4. Use your A* implementation from homework 4. The instructor can provide you with a default solution in necessary.

To run the project code, use runhero.py to run your MyHero against an opponent that also implements the same MyHero:

> python runhero.py

Use runmobacompetition.py to run any two hero agents against each other:

> python runherocompetition.py MyHero MyHero
> python runherocompetition.py MyHero BaselineHero
> python runherocompetition.py BaselineHero BaselineHero
> python runherocompetition.py MyHero BaselineHero2
> python runherocompetition.py MyHero BaselineHero3

BaselineHero is a bare-bones implementation of Hero AI that you can use to test against. BaselineHero agents simply navigate to the nearest Hero and starts shooting. BaselineHero2 will return to the base for healing if it takes too much damage. BaselineHero3 uses a more sophisticated strategy that will not be revealed.

The following steps are required to complete the assignment.

Step 1: Copy your myCreatePathNetwork function from homework 2.

Step 2: Copy your astarnavigator.py from homework 4. Copy mynavigatorhelpers.py functions from homework 3 or homework 4.

Step 3: Implement at least three classes in MyHero.py that sub-class from State. Implement their enter(), exit(), execute(), and constructors as necessary. You must create at least three states (in addition to Idle).

Step 4: Modify the execute() function in the Idle class in MyHero.py. MyHero automatically start in the Idle state. The main purpose of the Idle class is to transition to another state.

Step 5: Modify the constructor of MyHero in MyHero.py to add any new states to self.states. States cannot be executed unless they are in self.states. Make sure you do not remove Idle from the list of states.

Grading

These games have a score. Each team receives one point for every point of damage done by a Hero agent to another Hero agent. Your MyHero agent will be tested against three baseline agents (described above).

• 5 points: Be the first to score 100 points against BaselineHero. Win 2 out of 3 games.
• 3 points: Be the first to score 100 points against BaselineHero2. Win 2 out of 3 games.
• 2 points: Be the first to score 100 points against BaselineHero3. Win 2 out of 3 games.

For pragmatic reasons, games will be limited to 5000 ticks. As a rule of thumb, this should be less than 5 minutes on most computers.

Hints

The player-controlled character is a GhostAgent, which can move through walls without being obstruted. Baseline Heros and Minions will not target GhostAgents. If you change the player-controlled character to a Hero in runhero.py or runherocompetition.py, then you will be able to dodge by pressing 'j' (for jump), and use the area effect attack by pressing 'a'.

Use agent.getVisible() and agent.getVisibleType() to figure out what the agent can shoot at.

Remember, an agent shoots in the direction it is facing, so use agent.turnToFace(enemy) before agent.shoot().

Implement a smart dodging algorithm that figures out which direction to dodge to avoid an incoming bullet (use getVisibleType(Bullet) to see bullets) and not jump into obstacles.

Implement leading shots so you can shoot ahead to where an enemy will be instead of where they are at the time the bullet is released.

If the Hero is hunting an opponent Minion or Hero, consider having the Hero go to where the enemy is also going.

Do not kill the enemy base--there is no strategic advantage in doing so. If you kill the base, the enemy Hero will not respawn and you will not be able to get more points.

Submission

To submit your solution, upload your modified MyHero.py, plus mycreatepathnetwork, mynavigationhelpers.py, and astarnavigator.py if you wish to use your own solutions.

You should not modify any other files in the game engine.

DO NOT upload the entire game engine.