Chapter 1: Introduction

Playing Games

(Chapter 3)

Generating Content

(Chapter 4)

ModelingPlayers

(Chapter 5)Game AI Panorama

(Chapter 6)

AI Methods(Chapter 2)

Frontier Game AI Research

(Chapter 7)

Chapter 2: AI Methods

Seek for pellets

Chase Ghosts

Evade Ghosts

Ghost on sight

No visible ghost

Power pill eaten

Ghosts flashing

Move(Priority)

Ghost free corridor

Corridor with pellets

Corridor without pellets

Seek for Pellets

Pellet Found Eat next pellet

Until ghost on sight

Spot EnemySelect Weapon

(Probability)

Mini Gun PistolRocket

Launcher

Attack Enemy

Aim Shoot!

Until Health = 0

0.5 0.30.2

10 -1 5 0 2 3 -1

5

0

0

5 MinMax

10 5 3 0

0 0 1 1 1 0 1 0 0 1 00 0 11 1 0 1 00 1 0

0 0 1 1 1 0 1 0 0 1 00 0 0 1 1 0 1 1 0 1 0

0 0 1 1 1 0 1 0 0 1 00 0 11 1 0 1 00 1 0

0 0 1 1 1 0 1 0 0 1 00 0 0 1 0 1 1 1 0 1 0

0 0 1 1 1 0 1 0 0 1 0

0 1 0 1 0 1 1 0 0 0 10 1 0 1 0 1 1 0 0 0 1

0 0 1 1 1 0 1 0 0 1 0

p

0 0 1 1 1 0 1 0 0 1 0

0 1 0 1 0 1 1 0 0 0 10 1 0 1 0 1 1 0 0 0 1

0 0 1 1 1 0 1 0 0 1 0

𝑥2

𝑥1

6

1 4

23

5

<20 >28[20-28]

fairlowyesno high

Age?

Economy Employed? Salary?

City CarNo Car Sports Compact SUV

near not visible

fairnearfarnear far

Ghost

Power pill Pellet

Evade ghosts

Aim for pill

Aim for pellet

Aim for pellet

Aim for fruit

𝑥2

𝑥1

w

Neuron𝑥1

𝑤2

𝑤1

𝑤𝑛

𝑏

𝑥2

𝑥𝑛

x ∙ w + 𝑏

𝑔

𝑔(x ∙ w + 𝑏)

Hidden Layer Output LayerInput Layer

𝑥1

𝑥3

𝑥2

𝑤14𝑤48

𝑤49

𝑤37𝑤79

𝑎8

𝑎9

4

1

2

3

5

6

7

9

8

Selection Expansion Simulation Backpropagation

TreePolicy

DefaultPolicy

Environment (e.g. Maze)

State (s) Action (α)Reward (r)

Agent

G

Fitness value 𝑓2

0 0 1 1 1 0𝑤1 𝑤2 𝑤3 𝑤4 𝑤5 𝑤𝑛

0 0 1 1 1 0𝑤1 𝑤2 𝑤3 𝑤4 𝑤5 𝑤𝑛

0 0 1 1 1 0𝑤1 𝑤2 𝑤3 𝑤4 𝑤5 𝑤𝑛

1

2

P

Convolution Pooling

Reward

Action

Chapter 3: Playing Games

Player Non-Player

Win

MotivationGames as AI testbeds, AI that

challenges players, Simulation-based testing

ExamplesBoard Game AI (TD-Gammon, Chinook, Deep Blue, AlphaGo, Libratus), Jeopardy! (Watson),

StarCraft

MotivationPlaying roles that humans would

not (want to) play, Game balancing

ExamplesRubber banding

Experience

MotivationSimulation-based testing,

Game demonstrations

ExamplesGame Turing Tests (2kBot

Prize/Mario),Persona Modelling

MotivationBelievable and human-like agents

ExamplesAI that: acts as an adversary, provides

assistance, is emotively expressive, tells a story, …

BattleshipScrabble

Poker

Super Mario BrosHalo

StarCraft

Ms Pac-Man

LudoMonopoly

Backgammon

Pac-ManAtari 2600

CheckersChess

Go

Deterministic Non-deterministic Stochasticity

Time Granularity

Turn-Based

Real-Time

ObservabilityPe

rfec

t In

form

atio

nIm

per

fect

Info

rmat

ion

Chapter 4: Generating Content

37%

2%

6%3%

40%

12%

Art Manufacturing Other Debugging Marketing Programming

Ro

leM

eth

od

Co

nte

nt

Stochastic Deterministic

Constructive

ControllableNon-Controllable

Optional Content Necessary ContentContent Type

Determinism

Controllability

Iterativity Generate-and-test

Autonomous Mixed-Initiative

Experience-Agnostic Experience-Driven

Autonomy

Experience

Generation 0

Generation 1Midpoint displacement

Generation 2

Generation 3

Final Generation

Initialize Corner Values

Perform Diamond Step

Perform Square Step

Perform Diamond Step

Perform Square Step

Autonomous Mixed-Initiative

Designer (Initiative)

Player (Experience)

Exp

erie

nce

D

rive

nEx

per

ien

ce

Agn

ost

ic

Super Mario Bros (Pedersen et al., 2010)

Sonancia (Lopes et al., 2015)

Sentient Sketchbook (Liapis et al., 2013)

SpeedTree (IDV, 2002)

StarCraft Maps (Togelius et al., 2013)

Garden of Eden Creation Kit (Bethesda, 2009)

Ropossum (Shaker et al., 2013)

Tanagra (Smith et al., 2010)

Chapter 5: Modeling Players

Input OutputComputational (Player) Model

Gameplay

Objective

Context

Player Profile

Model-Based [Top-Down](Psychology, Cognitive Science, Game

Studies, …)

Model Free [Bottom-Up](Data Science, Machine Learning)

Numerical (Interval)

Nominal (Classes)

Ordinal (Ranks)

No Output

Regression

Classification

Preference Learning

Unsupervised Learning (Clustering, Frequent Pattern Mining)

Free Response vs. Forced Response

First Person vs. Third Person

Discrete vs. Continuous

Time-Discrete vs. Time-Continuous

Pre vs. During vs. Post

Valence

Arousal

Pleasant ( + )Unpleasant ( - )

Excitement

Happiness

TirednessBoredom

Sadness

Anger

Frustration

Fear

Relaxation

Activation ( + )

Deactivation ( - )

Boredom

Anxiety

Skills

Ch

alle

nge

FlowChannel

Nearest Monster

Nearest Treasure

Nearest Portion

Exit SafeNearest Treasure

Safe

Nearest Portion

SafeExit

Hit Points

Monster Treasure Portion Exit SafeTreasure

SafePortion

SafeExit

Chapter 6: Game AI Panorama

Model Players (Behavior)

Model

Generate

Content

Behavior

Designer

Player

AI Researcher

Producer / Publisher

Model Players (Behavior, Experience)

Generate Content (Assisted)

Generate Content (Autonomously)Play Games(Win [NPC], Experience [NPC])

Play Games (Win [PC], Experience [PC])

DO (Process) WHAT (Context) FOR WHO (End User) GAME AI AREA

Model Players

Player

Game

Content NPCs

Interactio

nExperience Behavior

Generate Content

Autonomously

Play Games [as NPC]

Win Experience

Model Players

Experience Behavior

Generate Content

AutonomouslyAssisted

Play Games (as PC or NPC)

To WinFor the (Game)

Experience

Model Players

Experience Behavior

Generate Content

AutonomouslyAssisted

Play Games (as PC or NPC)

To WinFor the (Game)

Experience

Model Players

Experience Behavior

Generate Content

AutonomouslyAssisted

Play Games (as PC or NPC)

To WinFor the (Game)

Experience

Model Players

Experience Behavior

Generate Content

AutonomouslyAssisted

Play Games (as PC or NPC)

To WinFor the (Game)

Experience

Model Players

Experience Behavior

Generate Content

AutonomouslyAssisted

Play Games (as PC or NPC)

To WinFor the (Game)

Experience