Reminder

Next class (Tuesday) , you should have formed groups and select a topic for presentation among:

(1) Real-Time Strategy games (Luis Villegas, Dulmovits, Alexander J): I: 8.2, 8.3, II: 7.5, 7.6, III: 4.7, 6.7; IV: 4.1, History of RTS games? - informed

(2) Squad tactics (John Formica, Matt Mitchell, David Pennenga ): I: 5.3, 5.4, 5.5. 5.6; II: 3.3, - informed(3) Scripting languages () I: Section 10 (7 chapters)(4) First Person Shooters (Michael Caffrey, Shamus Field, Jon Hardy): I: 8.1, 8.4; II: 7.1, II: 5.2, 5.3, history of

FPS? - informed(5) Racing games () I: 9.1, 9.2, 9.3, 9.4, II: 8.1, 8.2, (6) Role Playing Games (Josh Westbrook, Ethan Harman) I: 8.5, history of RPGs? IV: “Stop Getting Side-Tracked

by Side-Quests”, I: “Level-Of-Detail AI for a Large Role-Playing Game”, I: “A Dynamic Reputation System Based on Event Knowledge” - informed

(7) Story line, drama (): IV: “Implementing Story-Driven Games with the Aid of Dynamical Policy Models”, IV: “Dialogue Managers”,

(8) Sport Games (Daniel Phillips, Dan Ruthrauff) I: 9.7, II: 8.4, 8.5, - informed(9) Individual NPC behavior (Daniel Phang, Sui Ying Teoh): I: 2.4, 8.6, 9.6, II: 3.1, 3.2; IV: 5.6, http

://www.aaai.org/Papers/AIIDE/2008/AIIDE08-025.pdf, - informed(10) Player Modeling () IV 7.3, IV: “Player Modeling for Interactive Storytelling: A Practical Approach”, III:

“Preference-Based Player Modeling”, II: “Player Modeling for Adaptive Games”, http://www.gamegrep.com/top_list/14266-top_15_best_video_game_storylines_ever/, “Dynamic Game Balancing: An Evaluation of User SatisfactionGustavo Andrade, Geber Ramalho, and Alex Sandro Gomes, Universidade Federal de Pernambuco; Vincent Corruble, Université Paris 6”

(11) Map, World, Wall generation  ()

Most Popular Games Genres• Adventure games:

– Solving puzzles– Finding clues

• E.g., through conversations with NPCs– Classic: The Longest Journey (

http://www.youtube.com/watch?v=8XrkGztFr-0)

• Role Playing Games (RPG)– Player assumes fictional roles through Avatars– Avatar acquires skills by performing tasks/actions– “Levels” indicate the progress of the avatar

• Classic: Diablo 2: http://www.youtube.com/watch?v=vl9C8RTu3Oc

Most Popular Games Genres (II)• Strategy games:

– resource gathering, managing economy, technology research

– Real-Time Strategy (RTS)• Simultaneous actions (classic: Command & Conquer)

– Turn-based Games (TBS)• Take turns (classic: Civilization II)

• Sport Games– Simulation of actual sport games (classic: FIFA series)

• Games of chance– Outcome highly influenced by a stochastic environment

• First-person shooter– Controlling avatar in first-person camera view

(classic: Duke Nukem)

Most Popular Games Genres (III)• And many sub-genres

– Combine and/or enhance main genres • Massive Multiplayer Online RPG (MMORPG)

– RPGs with multiple other players in pervasive worlds• Stealth FPS

– FPS emphasizing stealth over shooting• RTS/RPG

– Combine both aspects (Warcraft III)• RTS/FPS

– Combine both aspects (classic: Battlezone)

Finite State Machines in Gameshttp://www.youtube.com/watch?v=3yxoEr5qeEk

Original slides by: Jarret Raim

FSM’s in Theory

• Simple theoretical construct– Set of states (S)– Input signals (I)– Transitional function

(s,i)

• A way of denoting how an NPC can change its state over time.

FSM’s in Practice

• Each state represents some desired behavior.• The transition resides across all states.

– Each state “knows” how to transition to other states.

• Accepting states are considered to be the end of execution for an FSM.– Example of an accepting state in a game?

• Input to the FSM continues as long it has not reached an accepting state.

FSM’s in Games

• Character AI can be modeled as a sequence of mental states.

• World events can force a change in state.

• The mental model is easy to grasp, even for non-programmers– This is important!

Gather Treasure

Flee

Fight

Monster In Sight

No Monster

Monster Dead Cornered

FSM Example

• States– E: enemy in sight– S: hear a sound– D: dead

• Events– E: see an enemy– S: hear a sound– D: die

• Action performed– On each transition– On each update in

some states (e.g. attack)

SpawnD

Inspect

~E

D

AttackE,~D~E

E

E

D

SPatrol

E

~S

S

D

But it gets even easier (event-on-map)

If player walks here thenSpawn Mephisto

Starting place of player

Put 3 orcs here

Even at a Larger Scale

If player cross hereThen declare war

Ok Let Us Construct One Finite State Machine

• Lets program High Priestess Mar'li– Here is she in action

• Step 1: list states and events• Step 2: Construct the Finite State Machine

FSM Implementation - Code

• Simplest method• After an action, the

state might change.• Requires a recompile

for changes• No pluggable AI• Not accessible to non-

programmers• No set structure• Can be a bottleneck.

void RunLogic( int *state ) { switch( *state ) {

case 0: //Wander Wander(); if( SeeEnemy() ) *state = 1; if( Dead() ) *state = 2; break; case 1: //Attack Attack(); *state = 0; if( Dead() ) *state = 2; break; case 3: //Dead SlowlyRot() break; }

FSM Implementation - Macros

• Forces structure• Shallow learning

curve• More readable

– Removes clutter by using macros.

• Easily debugged– Allows focus on

important code.

bool MyStateMachine::States( StateMachineEvent event,

int state );{BeginStateMachine State(0) OnUpdate Wander(); if( SeeEnemy() ) SetState(1); if( Dead() ) SetState(2); State(1) OnUpdate Attack(); SetState(0); if( Dead() ) SetState(2); State(2) OnUpdate RotSlowly();EndStateMachine}

FSM Implementation – Scripts

• Developer creates scripting language to control AI.

• Script is translated to C++ or bytecode.• Requires a vocabulary for interacting with

the game engine.• A ‘glue layer’ must connect scripting

vocabulary to game engine internals.• Allows pluggable AI modules, even after

the game has been released.

FSM Processing

• Interpreted– Simple and easy to debug.– Inefficient since FSM’s are always evaluated.

• Interpreted languages were the “rage” in AI• Event Driven Model

– FSM registers which events it is interested in.– Requires complex Observer model in engine.– Hard to balance granularity of event model.

• Multithreaded– Each FSM assigned its own thread.– Requires thread-safe communication.– Conceptually elegant.– Difficult to debug..

Game Engine Interfacing

• Simple hard coded approach– Allows arbitrary

parameterization– Requires full recompile

• Function pointers– Pointers are stored in a

singleton or global– Implementation in Dynamic-

link library • Allows for pluggable AI.• E.g., The Tielt project

• Data driven– An interface must provide glue

from engine to script engine.

Engine AI

Engine AIDLL

Engine

S. Interface

AI

Compiler

Byte Code

Optimization – Time Management

• Helps manage time spent in processing FSM’s.

• Scheduled Processing– Assigns a priority that decides how often that

particular FSM is evaluated.– Results in uneven (unpredictable) CPU usage

by the AI subsystem.• Can be mitigated using a load balancing algorithm.

• Time Bounded– Places a hard time bound on CPU usage.

• Project # 1 will let you experience this– More complex: interruptible FSM’s

Optimization – Level of Detail

• It’s ok to cut corners if the user won’t notice.

• Each level of detail will require programmer time.

• The selection of which level to execute can be difficult.

• Many decisions cannot be approximated.

FSM Extensions

• Extending States– Adding onEnter() and onExit() states can help

handle state changes gracefully.• Example of when to use it in games?

• Stack Based FSM’s– Allows an AI to switch states, then return to a

previous state.– Gives the AI ‘memory’– More realistic behavior

• Hierarchical FSM’s

FSM Example

• Original version doesn’t remember what the previous state was.

• One solution is to add another state to remember if you heard a sound before attacking.

SpawnD

Inspect

~E

D

AttackE,~D~E

E

E

D

SPatrol

E

~S

S

D

E

Attack-PE,S,~D

~E

~S

S

D

FSM Example

SpawnD

(-E,-S,-L)

Wander-E,-D,-S,-L

E

-SAttack-EE,-D,-S,-L

E

Chase-E,-D,S,-L

S

D

S

D

D

Retreat-EE,-D,-S,L

L

-E

Retreat-S-E,-D,S,L

Wander-L-E,-D,-S,L

Retreat-ESE,-D,S,L

Attack-ESE,-D,S,-L

E

E-E

-L

-S

L

-E E

L-L

-L

-L

L

D

Worst case: Each extra state variable can add 2n extra states

n = number of existing states

Using a stack allows to “remember” without the

extra states.

and remembering N actions into the

pass is not possible!

(finite automata vs. pushdown automata)

Stack FSM – Thief 3

Stack allows AI to move back and forth between

states.

Leads to more realistic behavior

without increasing FSM

complexity.

Hierarchical FSMs

• Expand a state into its own sub-FSM• Some events move you around the same

level in the hierarchy, some move you up a level

• When entering a state, have to choose a state for it’s child in the hierarchy– Set a default, and always go to that– Random choice– Depends on the nature of the behavior

Hierarchical FSM Example

• Note: This is not a complete FSM– All links between top level

states still exist– Need more states for

wander

StartTurn Right

Go-throughDoor

Pick-upPowerup

Wander Attack

Chase

Spawn

~E

E ~S

S

D

~E

Markov Models : An instance of Non-Deterministic FSMs

• Selects transition based on probability distributions

• Have multiple transitions for the same (state,input) pair

• Adds variety to actions• Label each with a

probability that it will be taken

• Probabilities can be learned (Reinforcement Learning)

Attack

Start

Approach

Aim & Jump &Shoot

Aim & Slide Left& Shoot

Aim & Slide Right

& Shoot .3.3

.4

.3.3

.4

More FSM Extensions

• Fuzzy State Machines– Degrees of truth allow

multiple FSM’s to contribute to character actions.

• Multiple FSM’s– High level FSM

coordinates several smaller FSM’s.

• Polymorphic FSM’s– Allows common behavior

to be shared.– Soldier -> German ->

Machine Gunner

Polymorphic FSM Example

Soldier

Rifleman Officer

British Soviet

American German

Machine Gunner

British Soviet

American German

British Soviet

American German

Debugging FSM’s

• Offline Debugging– Logging– Verbosity Levels

• Online Debugging– Graphical

representation is modified based on AI state

– Command line to modify AI behavior on the fly.

Case Study: Robocode

• First determine what states are needed– Attack, Evade, Search,

etc.

• Code up the FSM transition function.– Include an error state

that is noticeable.

• Setup debugging.– Verbosity levels are a

must.

Case Study: Robocode

Attack

DefendSearch

Implement and test each state

separately.A test bot AI

might help test single behaviors. (see Target bot)

Defense and Firing Power

• Enable your bot to dodge incoming fire.

• Every 20 ‘ticks’ reverse direction.

• Adds a circle strafe.• Selects a bullet

power based on our distance away from the target

void doMovement() { if (getTime()%20 == 0) { direction *= -1;

setAhead(direction*300); }

setTurnRightRadians( target.bearing + (PI/2));

}

void doFirePower() { firePower =

400/target.distance;}

Searching

• Reducing the scanner arc allows you to fire faster.

• If a target hasn’t been seen recently, spin.

• Scan where the target is.

• ‘Wobble’ to make sure to find the target.

void doScanner() { double radarOffset; if(getTime() - target.ctime > 4) radarOffset = 360;else radarOffset =

getRadarHeadingRadians() - absbearing(getX(),getY(),target.x,target.y);

if( radarOffset < 0 ) radarOffset -= PI/8; else radarOffset += PI/8;

setTurnRadarLeftRadians(NormaliseBearing(radarOffset));

}

References

• AI Game Programming Wisdom• University of Wisconsin presentation • Robocode Website