Introdution

•We are•-What are we going to talk about

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Analysis

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Analysis

• Elements of winning:– Build Order– Information Gathering– Macro– Micro

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Analysis

• Terran Tactics– Timing Attack– Pushing– Harassment

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Analysis

• Unit analysis– Marine– Vulture– Wraiths

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Baysian N

etw

orks

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Baysian N

etw

ork

• Choice of decision model– Bayesian Networks– Decision Trees

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Baysian N

etw

ork

• Spawn Prediction

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Baysian N

etw

ork

• Threat level prediction – tvt

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Potential Fields

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Potential Field In G

eneral

• Attractive behavior

• Repulsive behavior

• From vector to number

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Our

Potential Field

Function

• Behavior is determined by + or –• due vs de• (2de - due)

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Changed W

hen

Impl

eme

nting Potential Field

• All forces are positive• Forces are learned

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Reinforcem

ent Learning

• Generalization of Q-Learning

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Environm

ent Variab

les

• Distance to Ally• Distance from Current Tile to Ally• Distance to Center of Squad• Distance from Current Tile to Center of Squad• Distance to Enemy• Distance from Current Tile to Enemy• Distance to Cliff or Edge• Distance from Current Tile to Cliff or Edge

A State is defined as the combination of all the following characteristics:

Number of units

Health Lost

Damage Dealt

Number of Units

Time

Weapon’s Cool Down

Shooting Range

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Reinforcem

ent Learning

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Gen

eralization Form

ulas

• Q-Approximation

• Reward

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Updating

Rules

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Algorithm

• Image that i’m still doing

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

TEST

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Buil

ding a

test

Figure of StarEdit – a Starcraft BroodWar map editor

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Base

case

Testing without Reinforcement learning, and potential fields

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Base

case

Testing with potential fields, but not reinforcement learning

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Will

it converge?

Which Alpha and Gamma values? How many iterations is needed?

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Will

it converge?

Running tests with different values

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Will

it converge?

Values Alpha 0,2 and Gamma 0,9 (30852)

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Will

it converge?

Values Alpha 0,4 and Gamma 0,6 (135936)

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Pre

dict spaw

n and build

order

• Click to edit Master text stylesSecond levelThird levelFourth levelFifth level

Conclusion