1

Professor Charles Rich Computer Science Department rich@wpi.edu

IMGD 4000 (D 17) 1

Basic Game AI

Technical Game Development II

With material from: Millington and Funge, Artificial Intelligence for Games, Morgan Kaufmann 2009. (Chapter 5)

IMGD 4000 (D 17) 2

Definitions?

§  What is artificial intelligence (AI) ? •  subfield of computer science ? •  subfield of cognitive science ?

§  What is “AI for Games” ? •  versus “academic AI” ?

In games, everything (including the AI) is in service of the player’s experience (“fun”)

•  What does it mean for a game AI to “cheat”? Resources: introduction to Buckland, www.gameai.com,

aigamedev.com, www.aiwisdom.com, www.ai4games.org

2

IMGD 4000 (D 17) 3

What’s the AI part of a game?

§  Everything that isn’t graphics (sound) or networking... J

•  or physics (though sometimes lumped in)

•  usually via the non-player characters

•  but sometimes operates more broadly, e.g.,

– Civilization-style games (sophisticated simulations)

–  interactive storytelling (drama control)

IMGD 4000 (D 17) 4

“Levels” of Game AI

§  Basic •  decision-making techniques commonly used in

almost all games

§  Advanced

•  used in practice, but in more sophisticated games

§  Future

•  not yet used, but explored in research

3

IMGD 4000 (D 17) 5

This course

§  Basic game AI •  decision-making techniques commonly used in

almost all games –  basic pathfinding (A*) (IMGD 3000) –  decision trees (today) –  (hierarchical) state machines (today) –  behavior trees (becoming basic) (today)

§  Advanced game AI •  used in practice, but in more sophisticated games

–  advanced pathfinding (last Thursday) –  real-time camera control (last Monday)

IMGD 4000 (D 17) 6

Future Game AI ?

§  Take IMGD 4100 “AI for Interactive Media and Games” •  fuzzy logic •  other special topics

§  Take CS 4341 “Artificial Intelligence” •  machine learning •  planning

4

IMGD 4000 (D 17) 7

Two Fundamental Types of AI Algorithms

§  Non-Search vs. Search •  Non-Search: amount of computation is predictable

–  e.g., decision trees, state machines •  Search: upper bound depends on size of search space

(often large) –  e.g., minimax, planning –  scary for real-time games –  need to otherwise limit computation (e.g., threshold)

§  Where’s the “knowledge”? •  Non-Search: in the code logic (or external tables) •  Search: in state evaluation and search order functions

How about AI Middleware (“AI Engines”)?

§  Recent panel at GDC AI Summit: “Why so wary of AI middleware?”

§  Only one panelist reported completely positive experience •  Steve Gargolinski, Blue Fang (Zoo Tycoon, etc.) •  Used Havok Behavior (with Physics)

§  Most industry AI programmers still mostly write their own AI from scratch (or reuse their own code)

§  So we are going to look at coding details

IMGD 4000 (D 17) 8

5

AI Coding Theme (for Basic AI)

instead of...

§  a tangle of if-then-else statements

use... §  object-oriented paradigm

IMGD 4000 (D 17) 9

First Basic AI Technique: Decision Trees

10 IMGD 4000 (D 17)

See code at:

https://github.com/idmillington/aicore src/dectree.cpp and src/demos/c05-dectree

6

IMGD 4000 (D 17) 11

Decision Trees

§  The most basic of the basic AI techniques

§  Easy to implement

§  Fast execution

§  Simple to understand

IMGD 4000 (D 17) 12

Deciding how to respond to an enemy

if visible? { if close? { attack; } else { if flank? { move; } else { attack; } } } else { if audible? { creep; } }

attack

move attack

creep

yes

visible?

flank?

close? audible? no yes

yes

yes

no

no

no

7

if visible? { if close? { attack; } else { if flank? { move; } else { attack; } } } else { if audible? { creep; }

if visible? { if close? { attack; } else if flank? { move; } else { attack; } } else if audible? { creep; }

IMGD 4000 (D 17) 13

Which would you rather modify?

attack creep

yes

visible?

close? audible? no yes

yes

no

no

move attack

flank?

yes no

??? ???

yes

IMGD 4000 (D 17) 14

O-O Decision Trees (Pseudo-Code)

class Node def decide() //return action

class Boolean : Decision yesNode noNode class MinMax : Boolean minValue maxValue testValue

def getBranch() if maxValue >= testValue >= minValue return yesNode else return noNode

yes

no yes

yes

yes

no

no

no

class Decision : Node

def getBranch() //return node

def decide() return getBranch().decide() class Action : Node def decide() return this

8

IMGD 4000 (D 17) 15

Building an O-O Decision Tree visible = new Boolean... audible = new Boolean... close = new MinMax... flank = new Boolean... attack = new Move... move = new Move... creep = new Move... visible.yesNode = close visible.noNode = audible audible.yesNode = creep close.yesNode = attack close.noNode = flank flank.yesNode = move flank.noNode = attack ...

attack

move attack

creep

yes

visible?

flank?

close? audible? no

yes yes

yes

no

no

no

...or a graphical editor

IMGD 4000 (D 17) 16

Modifying an O-O Decision Tree visible = new Boolean... audible = new Boolean... close = new MinMax... flank = new Boolean... ??? = new Boolean... attack = new Move... move = new Move... creep = new Creep... visible.yesNode = close visible.noNode = audible audible.yesNode = creep close.yesNode = attack close.noNode = ??? ???.yesNode = flank flank.yesNode = move flank.noNode = attack ...

attack creep

yes

visible?

close? audible? no yes

yes

no

no

move attack

flank?

yes no

???

yes

9

IMGD 4000 (D 17) 17

Decision Tree Performance Issues

§  individual node tests (getBranch) typically constant time (and fast)

§  worst case behavior depends on depth of tree •  longest path from root to action

§  roughly “balance” tree (when possible) •  not too deep, not too wide •  make commonly used paths shorter •  put most expensive decisions late

yes

no yes

yes

yes

no

no

no

Second Basic AI Technique: (Hierarchical) State Machines

18 IMGD 4000 (D 17)

10

Finite State Machines §  Consider AI’s as “agents” that sense, think, then act §  But many different rules for behaviors

•  Ex: sensing, thinking and acting when fighting vs. running, vs. exploring…

•  Can be difficult to keep rules consistent!

§  Try Finite State Machine •  Natural correspondence between states and behaviors •  Easy: to diagram, program, debug

§  Formally: •  Set of states •  A starting state •  An input vocabulary •  A transition function that maps inputs and current state to next state

IMGD 4000 (D 17) 20

Game Example: Soldier Agent

on guard

run away

fight small enemy

large enemy losing fight escaped

11

IMGD 4000 (D 17) 21

Hard-Coded Implementation

class Soldier enum State ON_GUARD FIGHT RUN_AWAY currentState

def update() if currentState == ON_GUARD { if small enemy { currentState = FIGHT start Fighting } else if big enemy { currentState = RUN_AWAY start RunningAway } } else if currentState == FIGHT { if losing fight { currentState = RUN_AWAY start RunningAway } } else if currentState == RUN_AWAY { if escaped { currentState = ON_GUARD start Guarding } }

on guard

run away

fight small enemy

large enemy losing fight

escaped

IMGD 4000 (D 17) 22

Hard-Coded State Machines

§  Easy to write (at the start)

§  Very efficient

§  Notoriously hard to maintain (e.g., debug)

12

IMGD 4000 (D 17) 23

Cleaner & More Flexible O-O Implementation class State def getAction() def getEntryAction() def getExitAction() def getTransitions() class Transition def isTriggered() def getTargetState() def getAction()

class StateMachine states initialState currentState = initialState def update() triggeredTransition = null for transition in currentState.getTransitions() { if transition.isTriggered() { triggeredTransition = transition break } } if triggeredTransition != null { targetState = triggeredTransition.getTargetState() actions = currentState.getExitAction() actions += triggeredTransition.getAction() actions += targetState.getEntryAction() currentState = targetState return actions } else return currentState.getAction()

...add tracing

on guard

run away

fight small enemy

large enemy losing

fight escaped

IMGD 4000 (D 17) 24

Combining Decision Trees & State Machines

§  Why? •  to avoid duplicating expensive tests in state

machine:

alert

defend

alarm player in sight AND far

player in sight AND near

13

IMGD 4000 (D 17) 25

Combining Decision Trees & State Machines

alert

defend

alarm

player in sight?

far?

yes

yes

no

no

IMGD 4000 (D 17) 26

Hierarchical State Machines

§  Why?

search

goto disposal

goto trash

see trash

trash disposed

have trash

14

IMGD 4000 (D 17) 27

Interruptions (Alarms), e.g., Recharging?

search

goto disposal

goto trash

see trash

trash disposed

have trash

recharge

low power recharged

recharge

low power recharged

recharge

low power recharged

-search -trash

-disposal 6 - doubled the number of states!

IMGD 4000 (D 17) 28

Add Another Interruption Type?

12 - doubled the number of states again!

hide

battle all clear

-search-recharge

hide

hide

hide

hide hide

15

IMGD 4000 (D 17) 29

Hierarchical State Machine

search

goto disposal

goto trash

see trash

trash disposed

have trash

clean

recharge low power

recharged

•  leave any state in (composite) ‘clean’ state when ‘low power’

•  ‘clean’ remembers internal state and continues when returned to from “recharged’’

IMGD 4000 (D 17) 30

Add Another Interruption Type?

search

goto disposal

goto trash

see trash

trash disposed

have trash

clean

recharge low power

recharged

hide battle

all clear

7 states (including composite) vs. 12

battle all clear

hide -recharge

-clean

16

IMGD 4000 (D 17) 31

Add Another Interruption Type?

search

goto disposal

goto trash

see trash

trash disposed

have trash

clean

recharge low power

recharged

or could add another layer of nesting...

battle all clear

hide

clean-or-recharge

IMGD 4000 (D 17) 32

Cross-Hierarchy Transitions

§  Why? •  suppose we want robot to “top off” battery (even if

it isn’t low) when it doesn’t see any trash

search

goto disposal

goto trash

see trash

trash disposed

have trash

clean

recharge low power

recharged

17

IMGD 4000 (D 17) 33

Cross-Hierarchy Transitions

search

goto disposal

goto trash

see trash

trash disposed

have trash

clean

recharge low power

recharged

no trash and less than 75% power

IMGD 4000 (D 17) 34

HFSM Implementation Sketch

class State // stack of return states def getStates() return [this] // recursive update def update() // rest same as flat machine class Transition // how deep this transition is def getLevel() // rest same as flat machine struct UpdateResult // returned from update transition level actions // same as flat machine

class HierarchicalStateMachine // same state variables as flat machine // complicated recursive algorithm* def update () class SubMachine : HierarchicalStateMachine, State def getStates() push this onto currentState.getStates()

*See full pseudo-code at http://www.cs.wpi.edu/~rich/courses/imgd4000-d17/slides/hsm.pdf Add tracing/debug code!!

18

A “Becoming Basic” AI Technique: Behavior Trees

35 IMGD 4000 (D 17)

What is a Behavior Tree?

§  Drawn as a “tree” like a decision tree

§  Hierarchical like HFSM (hierarchical finite state machine)

§  Real-time version of HTN (hierarchical task network)

§  Builtin to UE4

§  Many plugins for Unity: Behaviour Machine, RAIN, Behave, DecisionFlex, ...

19

What is a Behavior Tree?

§  Specifies a real-time model of task execution

•  Switch between tasks in modular fashion

§  Similar to HFSM, but blocks are tasks (“behavior”) not state

§  Early use for NPCs (Halo, Bioshock, Spore)

§  Tree nodes are root, control flow, execution

Search and grasp plan of two-armed robot

https://upload.wikimedia.org/wikipedia/commons/1/1b/BT_search_and_grasp.png

“Behavior” in Behavior Tree

§  Sense, Think, Act §  Repeat

20

Sense

Think

21

Act

Behavior Tree with Memory

In UE4, the “Memory” is called “Blackboard”

22

C++ Implementation of Behavior Trees

§  See Chapter 9 of IMGD 4100 textbook: Programming Game AI by Example, Mat Buckland, Wordware, 2005

UE4 Behavior Trees vs. “Standard”

§  UE4 Event Driven •  Do not poll for changes, but listen for events that

trigger changes

§  UE4 “conditionals” not at leaf •  Allows easier to distinguish control versus task •  Allows them to be passive (event driven)

§  UE4 simplifies parallel nodes (typically confusing) •  Simple parallel for concurrent tasks •  Services for periodic tasks

https://docs.unrealengine.com/latest/INT/Engine/AI/BehaviorTrees/HowUE4BehaviorTreesDiffer/index.html

23

UE4 Behavior Tree

(Describe each next)

UE4 Behavior Tree - Root

24

UE4 Behavior Tree - Composite

(Describe each next)

Composite - Sequence

“And”

25

Composite - Selector

“Or”

Composite – Simple Parallel

26

Service

Decorators

27

Task

Blackboard (Memory)

28

Project 2 - UE4 Behavior Tree

“The Behavior Tree Quick Start Guide walks you through the process of creating a NavMesh, creating an AI Controller, creating a Character that will be controlled by that AI Controller, and creating all the parts necessary for a simple Behavior

Tree.”

https://youtu.be/q6vTg2roI6k