0

Romain Ducassé Aurélien Lermant Pierre-Alban Simonin 2014

State of the art AI COMBAT

1

TABLE OF CONTENTS

Introduction ............................................................................................................................................................ 2

Inspiration ............................................................................................................................................................... 3

Combat AI in games: FPS .................................................................................................................................... 3

Combat AI in games: RTS .................................................................................................................................. 10

Combat AI in games: MOBA ............................................................................................................................. 13

Combat AI in games: MMORPG ........................................................................................................................ 16

Combat AI in Games: Racing ............................................................................................................................. 17

Combat AI in games: Adventure ....................................................................................................................... 19

Real-life combat tactics .................................................................................................................................... 20

AI contests ........................................................................................................................................................ 22

Combat AI design .............................................................................................................................................. 25

Toolbox.................................................................................................................................................................. 32

Navigation techniques ...................................................................................................................................... 32

Data structures and algorithms for AI .............................................................................................................. 34

Middlewares ..................................................................................................................................................... 40

Other interesting references ................................................................................................................................. 43

2

INTRODUCTION

This document is an attempt to describe the state of the art of combat-oriented artificial intelligence.

Artificial intelligence has always been a part of computer science, and is currently widely used in multiple

domains. In particular, there are some fields where the artificial intelligence is made to “fight”: that is to say,

one or multiple AI-driven entities fighting each other or fighting human players. Some examples include:

Allies/enemies in video games

Real-life simulations for research purposes

Crowd behavior for video games and movies

There are currently a lot of researches being made in the field of combat-oriented AI. At this time (year 2014),

some domains where combat-oriented AI excels are:

Multi-tasking, micro-management

Computation time

Information gathering

On the contrary, AIs still struggle in some specific areas, namely:

Reacting smartly to unpredictable events (like the human player’s actions)

Having a “global goal” based on the current situation, and being able to change or adapt it

Being able to coordinate with allies (humans or other AIs)

In this document, you will find various links to videos and articles about these subjects:

Examples of combat AI in video games

Examples and uses of different AI architectures, middleware and algorithms

Examples and results of some AI contests

Some real-life combat tactics that inspired AI

strategies

…and lots of other AI-related references

Combat formation in the game Overlord Schematic of an AI based on a

state machine

To get started, here are some interesting references:

AI in different types of video games: FPS, RTS, MOBA, MMORPG

Website with news about Artificial Intelligence: http://aigamedev.com

Examples of online AI contests

3

INSPIRATION

COMBAT AI IN GAMES: FPS

HALO 2 AI

http://electronics.howstuffworks.com/halo2-ai5.htm/printable

In this article Robert Valdes report an interview of Chris Butcher of Bungie Studios of the functioning of the

Halo 2 AI. He describes the following aspects of the AI:

The Basics:

o AI have similar senses as the player.

o AI is not omniscient, it has to

perceive the world in which it’s

placed and take the best decision

considering the collected data.

o Lifelike behavior: AI can be surprised,

can make mistakes and make

decisions bases on their perception

AI Perception:

o Simulated senses (vision, hearing and

tactile)

o Simulated memory (number of allies, enemies, type of entities, characteristics, where was

going the enemy when he left the vision field, evaluate situation (combat zone, superiority,

etc.)…)

Decision Making:

o AI quickly take turns “thinking”.

o Decision based on processed data.

o 4 states: Idle, Guard/Patrol, Attack/Defend, and Retreat. Type of AI taken into account

(aggressive, coward, etc.).

o No random chance, predictable actions but unpredictable consequences

Pathfinding:

o AI is about pathfinding

o Finding advantageous positions (points specified by level designer)

o Each point assigned with a value based on what happening (defensive, attacking…)

o Challenge of pathfinding: getting to a “good" point in a way that makes sense

o Define objects volume to know if IA can run through or run around

o Tagging objects: animation points (ex: jump), awareness of changes (ex: cover object

destroyed)

Working Together:

o Built-in responses

o Interaction with group (ex: leader killed -> run away; simple implementation (if/then) seen a

emotional response)

o Different types of team working for each faction (ex: Marines grouped but not on others field

of fire)

o System of request, acceptance/objection between AIs.

o AI teamwork when similar objectives (ex: find the same guy)

o Dialogues

4

Uncanny Valley

o Zombie-like behavior with non-handled situations

o Trick the player

GHOSTBUSTERS

http://www.youtube.com/watch?v=amSBN62MGNU

Presentation of the crowd behavior in the game

Ghostbusters.

At first, hundreds of AIs calmly walk

randomly in the streets

When an enemy or a danger is detected,

they start to run in panic:

o They will try to run away from the

danger/enemy

o While running away, they tend to follow each other and form groups and lines

SNIPER: GHOST WARRIOR

http://www.youtube.com/watch?v=BKtxN99J2Z0

Demonstration of a weird AI behavior in the game

Sniper: Ghost Warrior. It is a big flaw that allows the

human player to easily kill several AI enemies:

The player places himself in a position where

all AI enemies have to walk

Even when AIs get killed one by one, they

keep coming through the same passage

Not only AIs all walk through the same passage, but they don’t even react to being killed one by one at the

same place.

MEDAL OF HONOR WARFIGHTER

http://www.youtube.com/watch?v=uOZxR9gpUKs

In this video, the human player “plays” with the

game’s AI to show some flaws. The main issues are:

The AI doesn’t realize its bullets hit

background elements and not the player

(when the player is really close to a wall)

and will endlessly shoot the same area

The AI will always aim the human player as

if he was in “standing” stance, even if the

5

player is crouched. This leads to absurd situations where the AI is right in front of the player, yet can’t

shoot him because he’s crouched.

COUNTER STRIKE: GLOBAL OFFENSIVE VS ARMA III

https://www.youtube.com/watch?v=75HG18oGLVs

This video shows sequences of gameplay of Counter

Strike: GO and Arma III to show how superior

Counter Strike’s AI is.

In Counter-strike:

AIs detect the human player immediately

AIs keep moving while shooting the player

AIs have good aim and quickly kill the player

In Arma III:

AIs are very ineffective at detecting the human player

AIs don’t react or move when the player starts to shoot them one by one

When AIs finally detect the human player, they try to shoot him through a wall

KILLZONE

http://www.cgf-ai.com/docs/straatman_remco_killzone_ai.pdf

Document about AI techniques used in Killzone.

In this first personal shooter game, the player is

confronted to enemy AIs and must work with ally AIs

as well. To provide a good challenge and game

experience to the player, AIs must take into account

the human player’s actions as much as possible, in

terms of moving, firing, taking cover…

Due to the unpredictability of the player’s actions,

some common AI techniques are not very effective.

For example, AI scripts: they are effective for small,

static terrain; not so much when the AI position is hard

to predict, or if the environment is likely to change.

To provide the most effective and convincing AI as possible, a lot of parameters are taken into account to

dynamically adapt the AIs’ behavior, for example:

Ally positions

Enemy positions

Current line of sight

Current terrain surroundings

6

With this, AIs are very effective at adapting to new situations and reacting to other players’ actions. They can

dynamically choose the best decision in terms of:

Pathfinding

Taking cover

Regrouping or dispersing

Taking advantage of terrain (staying out of ally’s and enemy’s line of fire, taking cover…)

All these techniques make a robust and adaptive AI, both in single-player and multi-player mode.

KILLZONE II

WAYPOINT COVER MAPS AND EFFICIENT RAYCASTS ON PS3 IN KILLZONE 2

http://aigamedev.com/insider/coverage/waypoint-

cover-maps/

This article talks about cover maps, and how they can be

used to optimize raycasts.

The AI in Killzone II is waypoint-based. Each

waypoint has a cubemap (top-left), generated

offline in pre-processing step.

From every waypoint we can see the distance

towards any part of the world, we can how far a wall is or any other piece of geometry, we can do line

of fire checks from any point the AI can reason about. This is basically how the AI sees. They don't use

a lot of raycasts, they use mostly these cover maps and augment them a little bit with raycasts.

These things are really suited to SPU's (System Processing Unit) because you can DMA (Direct Memory

Access) them in batches it up to do lots of reasoning about equal things. They are extremely compute

heavy, and that's the thing that SPUs are really good at because they have lots of processing power as

long as you stay within the local memory.

THREAT PREDICTION AND REASONING IN KILLZONE 2

http://aigamedev.com/insider/coverage/threat-

prediction/

This article talks about the AI behavior.

Threat prediction: Suppose you hide behind

cover and the AI responds to that. They start to

search for you, they do suppressive fire, if you

go into a building, they will fire at a window and

fire at a door to pin you down where you are.

This is all dynamic behavior, the AI doesn't

"cheat"

How does it work: AI remembers time and

waypoint of last contact, and marks waypoints where threat could move to. If waypoints are visible

7

then remove from list. If threat’s predicted position is a small set then attack position or search based

on possible location

This is all dynamic behavior and allows for really interesting AI situations. Whatever you throw at the

situation, the AI is able to inspect a pretty big domain of potential situations and be able to respond to

them dynamically.

SENSIBLE COMBAT BEHAVIORS USING SPU JOBS

http://aigamedev.com/insider/coverage/sensible-line-of-

fire/

This article answers the problem of AI agents which run

into each other’s line-of-fire.

Line of fire annotations: if I'm engaged in combat

I calculate which line of fire I have, and which

waypoints intersect with that line of fire. This is

published as a hint for other people when they're

path planning to avoid those links to stand there.

In battle, this results in flanking behaviors and walking around enemies engaged in battle. This

behavior is "interpreted" by humans as appearing more intelligent.

QUAKE III

http://www.kbs.twi.tudelft.nl/docs/MSc/2001/Waveren_Jean-Paul_van/thesis.pdf

Document about the bots in Quake III Arena.

At first, the document describes the game, then what

are the requirements for the bots in the game. For

example:

Bots must be hard to distinguish from human

players, and they must provide a challenge. As

such, they have to:

o Have an efficient pathfinding algorithm

o Understand the goals and rules of the

game

o Make useful decisions

Bots have to use the in-game chat. They can be asked questions and have a variety of available

expressions and responses, depending on the situation

Multiple bots must be available, each with a specific behavior and play style

They must be resource-efficient in terms of CPU usage

In order to achieve all of these requirements, multiple techniques and optimizations have been made to

provide an effective AI. Some of the techniques used are:

Neural networks and fuzzy logic for decision making

Genetic algorithms, used to optimize various parameters of the bots

8

There are samples of pseudo-code to illustrate various algorithms (like the question-answer system in the

chat).

Possible optimizations: mostly in team-oriented AI, in terms of team goals and coordinated actions.

LEFT 4 DEAD

http://www.valvesoftware.com/publications/2009/ai_

systems_of_l4d_mike_booth.pdf

Left 4 Dead is a cooperation-oriented FPS where the

human player must go through a number of levels

filled with different sorts of zombies. To help him, he

has 3 allies (other human players, or controlled by the

AI). In order to provide replayability, robust and

coherent behaviors, and a real challenge, multiple

fields have been tested and optimized to provide the

best possible AI during the player’s experience. The

most notable points are:

Unpredictability. Multiple units and items will randomly spawn during the game:

o Basic enemies (high frequency)

o Special enemies (medium frequency)

o Bosses (low frequency)

o Items and weapons caches

Coherent and robust allied bots. These bots are supposed to assist the human player as another

human would, so multiple factors are taken into account to make its behavior as efficient as possible:

o Bots will prioritize the human player’s survival over their own.

o Bots will adapt to the human player’s actions (moving, defending…).

o Some “cheats” are used in the game to guarantee that some undesirable events cannot

occur:

Ally bots cannot deal damage to friendly units (with direct fire, or using area-of-

effect damage like Molotovs).

If a bot gets too far away from the rest of the group, it is teleported near the team

when no human is looking at it.

The “AI director”: artificial intelligence in charge of adapting the

game’s pacing to the player’s performance. This means, for

example, spawning more or less enemies.

The pathfinding used is called “Reactive path following”. The

shortest path is computed with an A* on a navmesh, but actors

don’t follow it precisely (this would look a bit “robotic”). The actors

will try to go to intermediate goals on the way to the destination,

while using local obstacle avoidance during the travel. As such, AI

trajectories look fluid and natural. This is used for all actors in the

game.

9

F.E.A.R

http://web.cs.wpi.edu/~rich/courses/imgd4000-d08/lectures/fear.pdf

FPS game released in 2005

Innovative AI that was well-received by

the public. Some examples of what this

AI is capable of:

o Taking cover

o Using the environment cleverly

o Performing specific maneuvers

(flanking, suppression fire…)

o Opening doors

o …

Finite State Machines and Planning are

used together

o Finite State Machines:

Describes “How” to do

something

Usage of States to describe an entity’s actions

o Planning:

Describes “What” to do

Usage of Goals to describe an entity’s action

Layering behaviors:

o Sequences of behaviors in order of importance

o Dodge, take cover, kill enemy…

o AI will decide the importance of these actions based on current context

Dynamic problem solving

o Example: An AI tries to open a door

o If it fails, the AI will try to kick the door

o If it fails again, the AI will try to take another path (through a window…)

10

COMBAT AI IN GAMES: RTS

EMPIRE EARTH

GENERAL: PERSONAL EXPERIENCE

Strengths:

Excellent multi-tasking. AIs will build units and buildings non-stop, and always try to repair and replace

losses. This makes AIs very hard to kill, as they can simply rebuild any building destroyed once the

attack is over. Unless the opponent stays in the area and prevents any construction, they will rebuild

their bases faster than players can destroy them. This is especially true on big maps.

Good adaptability. Unlike many games, AIs will adapt their army composition based on to the

opponent’s current army, and will always build the exact counter for it. This is possible because the AI

“cheats” and gets a full vision of the map. This forces human players to use a balanced mix of units in

their armies (air, ground, various types of weapons) so that it cannot be totally countered.

AIs are very unpredictable. They can suddenly change their whole army composition and they can

attack on any point of the map. They will sometimes send a constant stream of units, and sometimes

wait until they have a huge army to launch a devastating attack.

Weaknesses:

Poor micro-management. AIs will blindly send units (even non-attacking ones) to attack other players,

no matter the losses. Also, they will never try to find the least defended spot to attack.

AIs literally don’t care about their base being attacked. Also, allied AIs won’t particularly protect or

help each other.

Also, AIs in this game “cheat” since they get infinite resources. Due to the surprisingly good adaptability and

huge resource advantage, it is actually very hard for a human player to beat the AI in this game. Giving infinite

resources is a simple but effective way to create a very powerful AI, because this huge advantage can easily

compensate some poor decision-making. These infinite resources can even make some weaknesses look like

strengths. For example, the constant flow of units that AIs will send would be an enormous waste of resources

for a normal player, but it doesn’t matter for an AI and it forces human players to play more defensively. Also,

the lack of coordination between allied AIs makes them very unpredictable, as each one will attack at a

different area at a different time.

EMPIRE EARTH: DISCUSSION FORUM ABOUT AI’S CHEATS

http://ee.heavengames.com/cgi-bin/forums/display.cgi?action=ct&f=7,2392,30,all

This forum is about Empire Earth’s hard AI and how it

cheats.

Most notable points are:

AIs have infinite resources, and can easily

outbuild and outproduce their human enemies

AIs permanently have a full vision of the map,

and are always aware of all players’ positions

and army composition (in addition, they will

11

always build the perfect counter to this army composition)

These cheats makes AIs very hard to beat without AI allies or specific strategies. Since bots cheat and get

infinite resources in this game, the only way to win is to be extremely aggressive and end the game as fast as

possible: as the game goes, bots will eventually overwhelm human players with streams of units. Another

method is to play very defensive (with lots of towers) and gain terrain progressively.

EMPIRE EARTH: 1V1 AGAINST A BOT

http://www.youtube.com/watch?v=xq5VYXjH1Rk

This video shows a human player losing against an AI

on hard mode.

The human player tries to be aggressive in

the early game, quickly sending units to

attack the AI.

Despite his aggression, the AI manages to

severely outproduce the human player. Not

only the human player’s attack is destroyed, but a few minutes after, the human player’s base is

surrounded by hundreds of AI units.

This is a good example of how powerful an AI can be with unlimited resources and perfect micro/macro-

management, compared to a human player.

STARCRAFT 2

GENERAL: PERSONAL EXPERIENCE

Strengths:

Multi-tasking. Bots are perfect at this.

Map awareness. Contrary to human players, bots don’t lose time looking all over the map during the

game.

Weaknesses:

Very predictable. There are very few build orders that bots can choose, and once they do, they will

stick to it, no matter the map or opponent they are facing.

No adaptability. Bots won’t modify their behavior based on the opponent’s actions. This is why some

flaws, even insignificant ones, can be exploited by human players for an easy victory.

12

STARCRAFT 2: 1V4 INSANE AI

http://www.youtube.com/watch?v=Nctrf9PHHrM

This video shows a human player winning alone against

4 bots on the highest difficulty level (insane).

The human player uses a huge flaw in the game’s AI

which makes it very easy to beat them 1vs4:

The AI builds few defenses in its own base

AI’s armies will all retreat when the base is

under attack

Due to these two characteristics, a single fast unit can keep attacking the enemy base to make the

army retreat again and again

The human player manages to keep all 4 armies running back and forth across the map, preserving its

own base.

Eventually, the human player manages to destroy each AI opponent one by one.

STARCRAFT 2: AI VIDEOS PLAYLIST

http://www.youtube.com/watch?v=VWdUGv

o8B2Q&list=PL56B505006C6E09C6&index=1

Playlist of 19 videos related to the game’s AI.

Here are some examples of the topics shown

in the videos:

Demonstrations of custom AIs performing specific tasks, for example:

o Pathfinding

o Micro-management (hit-and-run, spell casting, unit splitting…)

“Sandbox” maps, where any unit can be freely and instantly created to practice micro-management

against the game’s AI.

Tutorials for the Starcraft 2 map editor.

STARCRAFT 2: HOW TO BEAT INSANE AI

http://www.youtube.com/watch?v=lWkVV81XtoE

In this video, a human player manages to beat Starcraft’s

insane AI without exploiting any particular flaw. This is a

proof that it is possible to beat the AI by playing “normally”.

The main requirements to be able to beat the insane AI

“normally” are:

Proper build order

Good micro-management

Being able to adapt its army composition

Using the terrain: chokepoints, high ground…

Knowing when to attack and when to defend

13

COMBAT AI IN GAMES: MOBA

DOTA 2

GENERAL: PERSONAL EXPERIENCE

Strengths:

Amazing micro management. Bots excel at controlling multiple units at a time, and will even use them

to surround enemies.

No reaction time. Bots can target any unit instantly, and they will use their spells faster than any

human could. This is even more amazing when bots are in group, since they will suddenly decide to

attack all at the same time, which will often kill the target in a few seconds.

Weaknesses:

Bots will always buy the same items every game, in the same order, no matter what. If specific items

must be bought (to counter a hero in the enemy team), human players will have to do it.

Bots have a hard time reacting to help near allies in danger. This is especially true concerning human

players, who will often get ignored by their ally bots in battle.

Great coordination is sometimes needed to launch some specific spells or kill an enemy hero: this is

almost impossible to achieve with bots, since you can’t give them orders or show them that you are

about to attack.

DOTA 2: 5V5 AGAINST BOTS

http://www.youtube.com/watch?v=3VKzhhlKz2w

In this game, 5 human players lose a game against 5

bots. The game is pretty balanced until around 10min

into the game, when bots start to make kills and

snowball.

This is a good example of how strong Dota 2’s AI can

be. If they start to get ahead in kills and gold, their

perfect coordination and reflexes makes it very hard

to reverse the course of the game.

DOTA 2: BOTS FAIL COMPILATION

http://www.youtube.com/watch?v=iQACkqIbiio

This video is a compilation of all sorts of fails and

weird behaviors that bots can produce in specific

circumstances. For instance:

They often underestimate enemies’ and

creep’s damage in the early game and do not

retreat efficiently when getting attacked

They will sometimes take awkward paths to

move across the map, often bumping into the enemy team or enemy towers.

14

DOTA 2: POST ABOUT BOTS PROBLEMS

http://dev.dota2.com/showthread.php?t=91166

This post is a big list of various problems that often

occur when facing bots in games. Some examples:

A lot of heroes will behave a lot better when

they’re in the enemy team

Bots always ignore human players’ actions and

position

Some specific heroes are overly effective

against bots (taking advantage of the bots

clumping together, or being unable to counter heroes with unusual gameplay), some others are

almost unplayable (because they require good team coordination, or are effective against isolated

targets)

HEROES OF NEWERTH

HEROES OF NEWERTH: 2 HUMANS AND 3 BOTS VS 5 BOTS

http://www.youtube.com/watch?v=SzzL95YF7EY

This video is a 50min game where the 2 human players

and their 3 ally bots manage to win a very close game

(65 kills to 75). Although human players were able to

get a few kills early game, the bots’ grouping and

pushing allowed them to gain a huge advantage by

getting kills and destroying a lot of buildings. Human

players’ base was in great danger until the very last

seconds of the game.

HEROES OF NEWERTH: BOT CREATION TUTORIAL

http://forums.heroesofnewerth.com/showthread.php?470687-Bot-Creation-Tutorial

This is a tutorial on how to create a basic bot for Heroes of Newerth (written in Lua). All bots have a default basic behavior that includes:

Moving on the map, retreating, pushing

Last-hit and deny

Use consumables items

Some examples of actions and decisions being made by the custom bots are:

Buying items

Leveling skills

Using skills

Using items

15

Custom bots can be submitted on the official Heroes of Newerth website for further testing, some of the best

bots will be included in the game.

16

COMBAT AI IN GAMES: MMORPG

GUILD WARS

GUILD WARS: AI CAPABILITIES AND LIMITATIONS

http://wiki.guildwars.com/wiki/Hero_behavior#AI_capabilities_and_limitations

This article describes the behavior of the artificial intelligence

in combat:

Knowledge

Coordination

Prioritization

It also talks about their skills and how they use them:

Rituals

Enchantments

Healing

Spells

We can see that the AI is not very complex, and does not coordinate its actions very well in group, but behaves

a way that is enough difficult and realistic for the player.

MMORPGS

MMORPGS: 4 WAYS TO IMPROVE AI

http://internetgames.about.com/od/mmorpgs/a/Mmorpgs-Ways-To-Improve-Ai.htm

We can notice that the AI in MMORPGs has not evolved a lot

since many years. Its behavior is very repetitive and predictable,

and attacks without real strategy. The author of this article

proposes several ideas to make it more interesting:

Dynamic spawning : varied spawn locations, emulating

reproduction, rare mobs spawning wherever to

surprise the player -> Ultima Online, Ryzom

Less predictable agro system : greater variety of agro

conditions (sight or sound), a mob which run to seek

help instead of attack alone

AI controlled territory : genuine threat of invasion

-> Tabula Rasa, Rift

Mobs that progress : regular mobs evolve into elite mobs, NPC factions get more threatening over

time

17

COMBAT AI IN GAMES: RACING

ARTIFICIAL INTELLIGENCE IN RACING GAMES

http://www.cs.bham.ac.uk/~ddp/AIP/RacingGames.pdf

Areas of AI in Racing games

o Steering (for background animation)

o Pathfinding

Steering + Racing Lines

o Cars follow or stuck to a predefined

dawn line

o Uses splines (velocity included)

o Advantage:

very easy

o Disadvantages

Very limited

Not realistic

Pathfinding + Tactical AI

o Pathfinding: main popular issue in

gaming industry

o Tactical AI involves decision making

o no one solution is appropriate to every type of pathfinding problem

Neural Network

o Learning and improving performance with previous experience

o Dozen to thousands neurons for applications, 10¹¹ for human brain

o Function giving a unique set of output for a given input

o Uses:

Simplifies coding of FSMs or rule-based systems by relegating decision to the trained

neural network

Adapts as the game is played

Limited use in games

o Limitations:

Great for nonlinear problems handling -> hard to tackle

Cannot predict the behavior

=> Test and debugging very difficult

o Examples of NN executions here and here

18

GAME AI: SIMULATING CAR RACING GAME BY APPLYING PATHFINDING ALGORITHMS

http://www.ijmlc.org/papers/82-A1090.pdf

2 A* algorithms for pathfinding:

o A* with line-of-sight

o Dynamic A* avoiding random obstacles

Problems:

o Waypoints change for each track

o Waypoints setup manually

o Number and location of waypoints

different due to human factor

o Obstacle avoidance

A* algorithm

o Trick of color collision (collision

detection map)

o Check 4 adjacent nodes (no diagonal)

A* with line-of-sight

o Save many CPU cycles

o Pythagoras Theorem for deleting path with obstacles on the way

o Selects farthest “no-collision” node

A* forward direction: only checking 3 nodes (front right/left, directly ahead)

Dynamic A*

o 2 detection points ahead of the car, calculate next position in advance:

If left/right collision: car goes (turn default setting radian) to the right/left

If two collisions: move detection point by 45° -> more robust control of the car

PERSONAL EXPERIENCE IN RACING GAMES

Games: F1 2010/2011/2012/2013 (Codemasters), Forza Motorsport, Gran Turismo

Usually the negative point of the game

Negatives points:

o Doesn’t defend his position properly.

If close behind someone the right thing

to protect his position is veering to the

inside of the corner -> AI don’t do that

o Breaking point: When attack or being

attacked AI keeps the same breaking

point instead of delaying it. It is not the

most optimized for a lap time, but it would help to gain or keep a position, that is at the end

more profitable.

o Doesn’t respond to the user’s actions. AI comparison video between Forza Motorsport 4 and

Gran Turismo 5 showing the problem here

o Lack of strategy (ex: forced strategy changes lead to silly behaviors (ex: tire change due to

weather), like pit stopping at the last lap of the race)

19

COMBAT AI IN GAMES: ADVENTURE

UNCHARTED 2

http://aigamedev.com/open/interview/hive-mind-behaviors/

Alex J. Champandard reports an interview with Christian Gyrling, AI & Animation Programmer on UNCHARTED

2, about hive-mind combat behaviors in Uncharted 2.

“Hive-mind” Behavior tree: ask

information about world points,

visibility, etc. Each behavior

provide series of tactical

positions, gathered and

evaluated (“cost” weighted)

Solutions are evaluated by

higher-level tactical heuristic

with criteria

Voting system with score for

decision making

10 behavior trees at a time

Reevaluation every second or so

AI has one or several behavior trees each

Self-contained behaviors (no dependency)

Evaluations functions separated of behaviors threes

Better behavior but performance and extensibility problems

20

REAL-LIFE COMBAT TACTICS

WIKIPEDIA ARTICLES

LIST OF MILITARY STRATEGIES, CONCEPTS, AND TACTICS

http://en.wikipedia.org/wiki/List_of_military_strategies_and_concepts

http://en.wikipedia.org/wiki/List_of_military_tactics

A summary of common military terms and strategies.

The main types of strategies described in this article are:

General tactics (reconnaissance, night combat…)

Small unit tactics (patrol, ambush…)

Offensive Tactics (blitzkrieg, encirclement, shock and awe…)

Defensive tactics (counter-attack scorched earth…)

Deception (camouflage, disinformation…)

THE ART OF WAR

http://en.wikipedia.org/wiki/The_Art_of_War

Description of the book “The Art of War” by

Chinese general Sun Tzu. In 13 chapters of the

book, Sun Tzu explains various military concepts:

The planning and calculations before the

battle.

Warfare economy: how to limit the cost of

conflicts.

The 5 factors to win a war: Attack, Strategy, Alliances, Army and Cities.

How to maneuver an army.

How to use terrain/positioning.

How and when to use different types of weapons.

Spies and intelligence.

How to take advantage of strong/weak points of its own army or the enemy’s.

21

INFANTRY TACTICS

http://en.wikipedia.org/wiki/Infantry_tactics

Description of all sorts of military tactics and formations used by infantry

throughout history, from Sumerians to modern warfare.

Examples of fields aborted in this article:

Evolution of weapons used by infantry: melee weapons like

swords and spears at first, then ranged weapons: bows,

crossbows, and guns.

How to use different kinds of terrain: mountain warfare, jungle

warfare…

Offensive/defensive tactics and formations

The main periods covered in the article are:

Antiquity

Middle Ages

Renaissance

Napoleonic period

Modern warfare (previously to 1945)

Modern warfare (after 1945)

MILITARY TACTICS

http://www.everycitizenasoldier.org/id42.html

The article explains some real-life military concepts and

strategies. Some of the most notable points are:

In order for a team to work, good coordination is

a must. Fast communication must be available

(for example, hand signs).

Each member of the team must take into account

the other team members’ position to act

efficiently. For example, lines of fire should be

able to cover the largest area possible without

any blind spots.

The enemies’ position ids also crucial, as every team should always try to gain superior positioning

over the enemy. For example, being able to fire at the whole enemy team with no allies in the line of

fire is considered the ideal positioning.

It is always best to attack the enemy team by surprise, using strategies like flanking or ambushes.

Specific weapons and strategies can be used based on the goal and the environment (open ground, urban

environment…).

22

AI CONTESTS

GOOGLE AI CHALLENGE

WEBSITE

http://ants.aichallenge.org/

The AI challenge, sponsored by Google, allows submitting custom AIs

(written in any language). The submitted AIs represent ant colonies

fighting over resources on a map. The goal is to be the last ant colony

alive.

Further description of the simulation:

The game is turn-based and each player’s vision is limited by the

line of sight of its own ants.

Each ant can move, attack and collect food.

Hive will spawn new ants.

There are two types of terrain: normal and water. Water is impassable.

New resources appear randomly on the map during the simulation.

Matches are played constantly and a ranking of all members is updated continuously. The “strategy” section of

the website provides advice and code samples on:

How to explore efficiently.

How to avoid collisions and not block its own ants.

How to plan an attack on enemy hives

GOOGLE AI CHALLENGE WINNER (2011)

http://xathis.com/posts/ai-challenge-2011-ants.html

Link to 2011 AI challenge’s winner, Xathis. His whole strategy is

described, along with samples of code. His bot is turn-based,

meaning all ants and enemy ants detected have a lifespan of one

turn in his algorithm. As such, he does not have a “global” vision

of the game (knowing if he’s winning or losing, for instance).

23

STARCRAFT: AI COMPETITION

WEBSITE

http://webdocs.cs.ualberta.ca/~cdavid/starcraftaicomp/report2013.shtml

Website where Starcraft AIs can be submitted to fight

in a tournament that takes place each year. Matches

take place as 1vs1 fights on Starcraft: Broodwar.

Anybody can freely submit its own bot, and fights will

take place between every bot during the tournament.

An example bot is available on this google code:

https://code.google.com/p/ualbertabot/.

Some specific rules:

Bots must be written in C++.

Some specific bug/tricks are permitted (Mineral walk, drop on mines…), others are forbidden (Stacking

ground units, gas walk…).

The most difficult points for the creation of an effective bot are:

Scouting the map.

Proper build order/army composition.

Adapting to the opponent’s actions, knowing when to attack/defend.

Efficient usage of resources, expanding on the map.

Micro management, usage of spell casters.

Using counters to specific units (air units, invisible units…).

AI COMPETITION MATCHES

https://www.youtube.com/watch?v=jYzSffdvvwo

In this video, we can see examples of matches taking

place between members of the Starcraft AI

Competition tournament. There are examples of

successful/unsuccessful micro-management, build

orders, harassment…

24

OVERMIND BOT

http://overmind.cs.berkeley.edu/

Winner of the 2010 Starcraft AI Competition. On this

website, there are videos explaining specific strategies

used by this bot and a full description of the bot’s

creation before the 2010 competition.

This bot is based entirely on the mutalisk (a

fast flying unit) to try and harass the opponent

as must as possible to keep him from

expanding, until there are enough mutalisks to

destroy him.

The bot uses a dynamic representation of

danger zones with potential fields, in order to know where it is safe to attack (dodging defense turrets

and units).

Micro-management training was made with a genetic algorithm, launching several simulations to try

and improve the mutalisk’s actions and movements.

STARCRAFT: AI TOURNAMENT

WEBSITE

http://www.sscaitournament.com/

Just like the previous reference (starcraft AI

competition), SSCAI is a tournament that takes place

every year, and players can submit their own bot. A

stream is online at all times on the main page, and

features random matches between submitted bots.

Actually, some members of SSCAI (like Tomas Vajda and

Florian Richoux) are also memebers of the Starcraft AI

Competition.

Bots must be written in C++ or Java.

Just like Starcraft: AI Competition, the main issues when creating a bot are usually:

o Scouting the map.

o Proper build order/army composition.

o Adapting to the opponent’s actions, knowing when to attack/defend.

o Efficient usage of resources, expanding on the map.

o Micro management, usage of spell casters.

o Using counters to specific units (air units, invisible units…).

25

COMBAT AI DESIGN

FUNDAMENTAL RULES IN DESIGNING A COMBAT SYSTEM

http://www.gamasutra.com/view/feature/175950/the_fundamental_pillars_of_a_.php

Mentioned games : Call of Duty, Street Fighter II, Spider-

Man: Shattered Dimensions, Halo, Mario

Aim: presentation of the basics of a combat system:

The abilities: Must have a unique function and a

risk vs reward trade-off.

The aim is to create abilities in order to the player

has to evaluate and choose the best option for

each situation.

Each ability has to have a risk/reward coefficient.

Example: A quick attack with little damages vs an

attack with a long time to be activated but with a

lot of damages

The 3 challenges: evaluate the distance (choose

the right ability), evaluate the time (time to perform and duration), anticipate

(counterattack).

Design challenging enemies (clever IA doesn’t matter). They must have different behaviors

(how to attack him, how to defend against him). Examples: sniper, bomber, shield…

Subclasses and variations of enemies: add new abilities.

Enemies must match the player abilities

Large variety of combats => entertaining

DESIGNING ARTIFICIAL INTELLIGENCE FOR GAMES

DESIGN & IMPLEMENTATION

https://software.intel.com/en-us/articles/designing-artificial-intelligence-for-games-part-1/

Mentioned games : FEAR, Pac-Man

AI: emulating behavior of other players

Simulated: more “artificial” than “intelligence”

(simple or complex)

AI goal: simulate intelligent behavior in order to

provide a challenge to the player

AI: not just computer-controlled players in

multiplayer games, but multiple roles like scripted

events

The more complex the system, the more

requirements an AI will have

Complex systems requirements: perceiving AI’s

environment, recording player actions, evaluation success of actions, etc.

Core concept of AI: decision making

AI system: able to affect entities using it

26

“AI push” or “Entity pull” strategy

AI push:

AI system as a separate element

Separate thread(s): calculating best choice depending on game options

Broadcasting choice to involved entities

Good in real-time strategy games (big picture)

Entity pull:

Simple entities

AI system calls on updates

Good with large number of entities which do not think very often (ex: shooters)

Designing challenge: identify key features important to the AI system

Rules-Bases systems:

Most basic form

Preset behaviors

Finite State Machine

FMS: entity has distinct states throughout is life

Implementation: state = checkable variable, or functions pointers (C) and virtual function

(C++, C#, Java…: OOP)

Adaptive AI

Dynamic enemies, able to learn and adapt themselves

Fighting and strategy games

Anticipate opponents’ moves: past-pattern matching, random guess…

Recorded past decisions: evaluating success (and so what do next)

PERCEPTIONS & PATH FINDING

https://software.intel.com/en-us/articles/designing-artificial-intelligence-for-games-part-2/

Mentioned Games: Call of Duty 4, Starcraft

Sight:

Most basic level

Basic game: access to everything you want

More complex (FPS…): IA more selective, sight =

o distance between agent a target

o Angle of the vector

o Obstacle between agent and target?

(ray tracing)

Use multiple ray tracing on key points to know if

the target is completely hidden or not

Sound:

Sound level associated to each action

Same method as sight (distance, ray tracing, …)

+ sound spread/soundproof materials

Some others:

Smell: radius + scent trail left behind (decreasing)

Radar: radius check

Touch: collision

Aim => be able to react to what you percept

Temporal entities: visual effect, do not thinks, visible, information for AI or player

27

Determine Cover: size of the box, ray cast …

Navigation:

Crash and turn: most basic approach, keeping track of previous paths, good for changing maps

Path Finding: A*, multithreading

TACTICAL & STRATEGIC AI

https://software.intel.com/en-us/articles/designing-artificial-intelligence-for-games-part-3/

Mentioned games: Overlord, Enemy Territories Quake Wars, StarCraft

Tactical AI:

Groups of entities (keep track of each and update separately)

Sharing information and support

Each group: main goal, composition and roles (captain, units…)

Grouped path finding:

Less computations

Captain goes to the target,

units follow

Formations:

Move in a nice pattern

Each member is a spot in the

formation

He must keep his position

relative to other members. Here the example of the pyramid formation in Overlord. (Captain

and followers)

Tactics:

Captain: planning and coordinating the team

Examples: Healing support, Scouting, Covering fire, Sacrifice

Leader should know capacities of each member of the team in order know the situations

where the team has the advantage and when retreat

Diversity and numbers of units must be taken into account for optimal decision

Strategic:

Strategic systems roles: resources gathering, researching up tech tree, building army…

Commanders:

o As own system or not physically existing empty entity which thinks and updated

o Guided by hierarchical rules systems

o Explore game map to identify player and key points of interest, build defense, and

analyze other player strategies (defense…)

Use of Decision Maps: 2D array division of the map in regions with information in it => help

to the decision.

Use of Resource Maps: contains information about location of resources. Help to decide:

where to deploy expansions and satellite bases, where will the enemy do it, where are

contested places, which units could I create with it…?

Use of Objective Maps: filled with information about goals of the team (protect place, attack

objectives, avoid defenses…)

Use of Conflicts Maps: More often updated. Keep tracks of where battle occurs (type,

capabilities strength and number of units). Helps to decide effectiveness of defenses (enemy

attack), avoid conflict, deploy countermeasures (type of units, weapons…), etc.

Building maps: scouting continuously. Threading for better performances.

28

MULTI-THREADING

https://software.intel.com/en-us/articles/designing-artificial-intelligence-for-games-part-4/

This is an article about multi-threading in video games. The main points are:

Task parallelization

o Dispatching different, autonomous

tasks to different core to gain time

o The different tasks in a video game

are not entirely independent, but

some can still afford to be separated

o Examples of threadable tasks:

Audio

Rendering

Path finding

Entity update

…

Critical sections

o Some portions of the code must not be executed by several threads at the same time

o The programmer must make sure that only one thread can execute this critical section (with

locks for instance)

Synchronization

o Two different threads may not execute at the same speed

o Synchronization technique example: a time index which tells whether 2 tasks are correctly

synchronized or not: if not, one of the threads may have to sleep to wait the other

When NOT to thread:

o Overly complex systems. If the subsystem is tied in with too many other systems that

constantly cause it to wait or synchronize, it may not be a good idea to try to parallelize it.

o Atomic workload. If the work of the program cannot be broken down into atomic tasks, it

may not be possible to go parallel.

o Costly overhead. Extra work will be performed in the threads, compared to the original

program. If the overhead outweighs the benefits gained from threading, then it might be a

good idea to avoid it.

o Redundant code. It should always be checked before the program is parallelized.

Example of threading tool: OpenMP

DESIGNING AI FOR GAMES

http://www.youtube.com/watch?v=62VUZKI8G2M

Presentation of some concepts about video games AI,

such as why to build AIs in video games, how AIs fit in

the gameplay of a particular game, how some AIs

“cheat” in some games, and what are the most

promising data structures and algorithms for future

AIs (autonomous learning, goal-oriented-planning…).

29

AN INTRO TO VIDEO GAME AI

http://www.youtube.com/watch?v=wsmMOJj6ETo

Presentation of some of the most used data

structures and algorithms used to implement video

game AIs, such as:

Finite state machines

Behavior trees

Planners

Neural networks

ARTIFICIAL INTELLIGENCE IN VIDEO GAMES

http://www.youtube.com/watch?v=Ux0TZqEAiK0

Discussion about current researches being made in

video game AI. There is a demonstration of a game

(Ruin) where the player can use vocal commands

(such as “move”, “fire”…) and the avatar will try to

execute these commands based on the current

environment, enemies, allies…

30

COMBAT ORIENTED AI

http://ai-depot.com/GameAI/Design.html

Mentioned games: Half-Life, Counter-Strike, Wolfenstein, Black & White

3 roles of AI in video games : Combat, Non-

Combat and Analytical

Combat Oriented AI:

Most often implemented form

Major turn with Half-Life release

Understanding & exploitation of

the environment: cover, alternate routes, spots for ambushes. Dynamic bot-environment

interaction (no training) important feature of future video games

Efficient use of teamwork: irregular and self-serving at best. AI as teams would improve

realism and efficiency.

Ability to hunt: player affecting the environment, AI capable of seeing “clues” (footprints…)

Survival instinct: most human like feature. Hiding behind boxes, prevent damages, avoid

combat zones.

Non-Combat oriented AI:

Role playing games

Non playable characters

Help the player and guide him through the game

Weaknesses: dialogue (scripted), NPCs have no real memory

Analytical:

Unimplemented in current games

Enhance game experience without interacting with the player

Examples: traffic lights, controlling AI difficulty, number of opponents, monitoring player’s

health and reactions times…

Efficiency: postpone pre-caching of parts of the map where the player rarely go, more

polygons in player’s vision field, etc.

COVER SYSTEM

http://en.wikipedia.org/wiki/Cover_system

Article about the cover system (being able to hide behind an obstacle)

in video games.

A brief recap of the cover system thorough the video games history:

At first, very little games featured a cover system, and when

they did it was very basic.

Some more complex cover systems were created with

stealth-oriented games (like Metal Gear Solid, 1998). After

that, some other AIs started to use the cover system.

It is now included in almost every third-person shooters and first-person shooter.

Although it is a great way of making combat AIs more realistic, it is quite difficult for an AI to recognize ideal

cover spots and knowing when to use it. That is why a lot of video games actually have a limited number of

“fixed” cover points in a specific map.

31

AI BEHAVIOR IN URBAN SETTIING

https://www.youtube.com/watch?v=U-oJ_xdjaeM

Demonstration of a fight between two AI teams in an

urban environment.

At first, the two teams move in group to try and

find the other team.

When they do, they start to take cover and fire at

each other. They also change stances (stand up,

crouch…) depending on the situation.

When only one team remains, this team wins.

For each individual AI, when can dynamically see its line of fire and its destination.

AI SCRIPTS IN COD 4

http://aigamedev.com/open/article/call-of-duty-4-tools/

In this article Alex J. Champandard describes the structure of the AI logic of Call of Duty 4: Modern Warfare.

Scripted level design

Heavily data-driven approach

Directories laid out as follow: Binary

Tools, Engine Librairies (.dll), Raw

Assets

Raw assets containing .gsc (C-like) and

.csv files

AI types configured by two files each:

*.gsc, *.csv.

csv approach for AI parameters handling

Level scripting: base file (_*.gsc) for each level + associated scripts handling logic, animations, particles

and graphics, and sound (*_code.gsc, *_anim.gsc, *_fx.gsc and *_amb.gsc respectively)

32

TOOLBOX

NAVIGATION TECHNIQUES

NAVMESH

http://udn.epicgames.com/Three/NavigationMeshRefere

nce.html

Presentation of the “Navigation Mesh” technique to

implement efficient path finding for AIs:

The AI’s configuration space is represented by a

connected graph of convex polygons. At each node

(polygon) we know that an AI can get from any point in

that node, to any other point in that node due to its

convexity. Thus the task of path finding through the graph simplifies into path finding along a connected graph

of nodes

Generation process:

Exploration: The map is “flood filled” -> Each segment of the map is examined via raycasts and once verified, added to the mesh. We end up with a high density mesh that resembles a grid.

Mesh simplification : o Square merge to reduce number of polys and speed up the following steps o Merge all polys into concave slabs separated only by differences in slope o Decompose concave slabs into convex shapes

Mesh finalization: The final step is building path-able edges between nodes, and generating the obstacle mesh

Benefits: Since with a mesh we can represent a large area with a single polygon, overall graph density goes down. This is a win for many reasons:

Memory footprint is reduced with the decrease in nodes being stored

Path finding times go down as the density of the graph being searched shrinks

Less nodes means less time fixing up cross-level pathing information

RECAST/DETOUR

http://www.garagegames.com/community/blogs/view/21377

Recast is a library which automatically generates navmeshes from input geometry. Detour is a set of functions which performs pathfinding and navmeshes. Features:

Off-mesh links

Temporary obstacles in the navmesh

33

Different surface flags

Here is a demo video to illustrate this: https://www.youtube.com/watch?v=hi63I87dFg0

FLOCKING

http://www.red3d.com/cwr/boids/

This website is an illustration of the concept of “flocking”, that is to say, using multiple artificial agents to try

and behave like a real flock of animals would. The “boids” project is a 3D example of flocking.

The 3 basic flocking strategies used are the following:

Separation: steer to avoid crowding local flockmates

Alignment: steer towards the average heading of local flockmates

Cohesion: steer to move toward the average position of local flockmates

Separation Alignment Cohesion

34

DATA STRUCTURES AND ALGORITHMS FOR AI

AI ARCHITECTURES

http://intrinsicalgorithm.com/IAonAI/tag/hfsm/

http://www.cc.gatech.edu/~surban6/cs4731/lectures/2013_05_23_DecisionMaking_FSM.pdf

Intrinsic Algorithm has a very good intro on AI architectures. These are the followings:

FINITE STATE MACHINE

Description:

o One state at a time

o Everything the agent needs to know about what it is doing is

contained in the code for the state that it is in

o Stay in current state? Move into another one?

o Trigger mechanism involving

o Each state: what do to while, and when, if what to do next

Core references:

o Wikipedia article here

o Unitygem article (tutorial) here

o Tuts+ article (theory and implementation) here

Pros:

o Easy to understand and build

o Very fast

o Compact data structure

Cons:

o More store, more transitions (a lot)

o Adding a state leads to update all other states’ code to add transitions to the new one

o Bigger it gets, the more opportunity for disaster

o Here an article explaining why FSM are obsoletes

Uses:

o Pac-Man, Thief 3 (Stack FSM)

o Simple sports games

o FPSs

o RTSs

BEHAVIOR TREE

Description:

o Softer approach

o Separates states from decision logic

o Decision logic in stand-alone

architecture

o Much more organized structure

o Can pack much more behaviors

Core references:

o Wikipedia article here

o Alex J. Champandard article here

35

o Björn Knafla 5 parts article here

o Example of implementation in an Android game here

o Libraries (github): BehaviorLibrary, UnityBehaviorLibrary

Pros:

o All decisions logic in a single place

o Can have a lot of states, no worries about how to keep it all synchronized between these

states

o Adding new behavior doesn’t lead to change all states’ code. (code added in one place only)

o Easy editing of the transition logic (one place only)

o More formal method of building behaviors (though tools, templates, structures)

Cons:

o Hard-coded priority of behaviors

Uses: One of the more “go-to” AI architectures in games

o Halo 2 / Halo 3

o Spore

o Action/Combat games

HIERACHICAL FINITE STATE MACHINE

Description:

o Multiple level of states

o Higher-level states: transitioning to

other states on same level

o Lower-level states: transitioning to

levels of the same parent state

Core references:

o Miro Samek article (introduction ) here

o Miro Samek slides (theory and

implementation) here

o Stefan Heinzmann article (C++ implementation) here

Pros:

o Tree-like structure

o Provides little structural organization to flat FSM

o Keep some of the complexity under control

Cons:

o Same as FSM (transitions)

Uses:

o Destroy All Humans 2

o Sports game

36

PLANNING

Description:

o Different way of getting states

o Like behavior tree, reasoning

architecture separated from the code

of states

o Compare situation (current state) to a

collection of individual atomic actions

doable

o Assembles then one or more tasks

into a sequence: the plan (executed

forward)

o Look backwards from its goal

o Isn’t specifically hand-authored

Core references:

o Wikipedia article here

o Jussi Rintaren articles: Planning, Algorithms for Classical Planning

o Dana Nau lecture (17 chapters) here

o Craig Boutilier lecture here

o Example of implementation in F.E.A.R here, by Jeff Orkin

Pros:

o It can often come up with solutions to novel situations that the designer or programmer

didn’t necessarily account for and handle directly in code

o New action can be dropped into to the game and planner architecture know how to use it

o Time up development time

Cons:

o Authorial control diminished

o Scripted predictable actions are exceptions of the system (contrary to previous ones where

creative solutions are exceptions) -> We have to trick/force the system

Uses:

o F.E.A.R. (GOAP: Goal Oriented Action Planning)

o Killzone 2 (HTN: Hierarchical Task Network)

o RTS games

UTILITY-BASED

Description:

o Less structured than FSM or BT

o No pre-determined arrangement of what to do when

o Potential actions considered by weighting some factors and selecting most appropriate

decision

o Similar point with planner: AI gets to choose what is best at a time

o But instead of assembling plan (planner), selects next action

o Appropriate when large number of potentially competing actions the AI can take (often no

obvious “right answer”)

o Offers still deep level of control but in a different way: AI simply looks at its available options

and then decides what is most appropriate.

37

Core references:

o Dave Mark and Kevin Dill Session about improving AI decision through utility theory here.

Slides here

o Alastair Aitchison article here

o Miguel Nieves concrete example here

Pros:

o Like planners and BT, reasoner code: decision decided up -> transition to a state

o Reasoner code at one single place: add decision to system without any changes to the

underlying code

o Building, editing, tuning and tweaking the system much more compartmentalized

o Like planner, adding actions fairly straightforward: add action with appropriate weights ->

automatically taken into account by the AI and being used in relevant situations

Cons:

o No always a good way to intuit what will happen in a given situation (more fuzzy than binary)

o AI not configurable

Uses:

o The Sims

o RPGs

o Complement to other architectures

NEURAL NETWORK

Description:

o Type of learning AI (machine-leaning)

o Need to be trained with test or live performance data

o States wrapped in the system are unknown until they happen

o Cant’ script anything or change results, only solution: retrain the Neural Network

Core references:

o Wikipedia article here

o Ai-junkie.com tutorial here (6 pages, implementation page 6)

o Eyal Reingold lecture here

o Bobby Anguelov C++ implementation here

o Joonatan Mänttäri and Jonas Larsson report of NN applications in games here

Pros:

o able to pile a lot of things into a huge collections of possibilities

o able to learn

o can be set-up quickly

Cons:

o lack of designer control (however its very often a requirement)

o impossible to edit

Uses:

o Racing games, Unreal Tournament

o Lack on use in video games

o Mostly seen in some projects like Othello, Tic Tac Toe (everything involving machine learning)

38

CONCLUSION

There is no “one size fits all” solution to AI architectures

Table from http://intrinsicalgorithm.com/IAonAI/tag/hfsm/

NEURAL NETWORKS AND GENETIC ALGORITHMS EXAMPLES

EVOLUTION OF A SELF-ORGANIZING ROBOT SOCCER TEAM

https://mobile.aau.at/~welmenre/papers/fehervari-2010-

Evolving_Neural_Network_Controllers_for_a_Team_of_Self-

organizing_Robots.pdf

Article about the creation of a neural network to play soccer

matches. Best AIs are chosen using a genetic algorithm.

Multiple simulations are being made: the team’s

performance is evaluated with a fitness function.

The fitness function doesn’t only look at final score

for both teams (scoring goal is far too complex at

first when behavior is still more or less random to expect an improvement from one generation to

another). Instead, multiple parameters are taken into account :

o Distance to the ball

o Number of kicks

o Distance to the opponent’s goal

A video of a match between two of these AIs can be seen here:

http://www.youtube.com/watch?v=cP035M_w82s

39

A GENETIC ALGORITHM LEARNS HOW TO FIGHT

http://www.youtube.com/watch?v=u2t77mQmJiY

Demonstration of fights between AIs driven by a neural

network. AIs will evolve throughout he fights via a genetic

algorithm and become more and more clever and efficient

thorough the simulations.

AIs have 5 possible actions:

o move forward

o turn right

o turn left

o fire

o adapt field of vision (reducing it will

induce better aiming)

At first, AIs don’t move or shoot (or do it very randomly)

After a few simulations, they begin to move on the map and try to shoot toward the enemy

As the simulations go on, some specific strategies appear: dodging enemy bullets, reducing field of

vision to shoot more precisely…

This is a good example of how the combination of neural networks and genetic algorithms can produce smart

behaviors (dodging) from basic actions (moving) without any intervention from the designer/programmer.

POLYWORLD: USING EVOLUTION TO DESIGN ARTIFICIAL INTELL IGENCE

http://www.youtube.com/watch?v=_m97_kL4ox0

Demonstration of a genetic algorithm used in the project

“Polyworld”, were creatures with unique genes must

compete for resources, reproduce, and evolve to adapt.

Each creature have several possible actions:

fighting, mating, feeding, moving…

Each creature has a set of genes that determine its

size, lifespan, speed, mutation rate, vision…

Creatures will mix their genes when mating.

Random mutations can also appear.

No fitness function is used: since the creatures

compete for resources, natural selection alone chooses the best creatures.

Possible improvements:

o More complex environment, resources

o More complex creatures (more actions, sense of touch, smell)

o More complex behaviors: take genetic similarity into account to interact with other creatures

Debate: are these creatures alive? Intelligent?

o Yes and no, it depends on the definitions of “alive” and “intelligence”

This project is written in C++ and source code is freely available: http://www.sf.net/projects/polyworld/

40

MIDDLEWARES

MOST USED IN THE INDUSTRY

AUTODESK GAMEWARE NAVIGATION

http://gameware.autodesk.com/navigation

http://gameware.autodesk.com/navigation/features

Autodesk Gameware Navigation is the successor of

Autodesk Kynapse. It’s an artificial intelligence

middleware that provides automatic NavMesh

generation, hierarchical pathfinding, and path

following in complex game environments. Character

and obstacle avoidance, splines and channels for

animation-driven locomotion, dynamic NavMesh, and

swappable sectors are supported out-of-the-box.

Gameware Navigation is designed to enable developers to create ambitious, more complex AI.

A demo video can be seen here: https://www.youtube.com/watch?v=reGq7G58T1Y

Examples of games using Autodesk: Mass Effect (I and II), Dragon Age II, The Witcher II, Civilization IV

NAVPOWER

http://www.babelflux.com/technology.html

NavPower is a powerful AI pathfinding package chosen

by top publishers and independent studios. NavPower

features a leading-edge build process which quickly and

reliably generates high-quality navmesh data, as well as a

modular runtime environment which provides high-

performance console pathfinding and the scalability

needed for MMO server environments.

Features:

Navgraph Auto-Generation

Dynamic NavGraph Modification

3-D Surface Movement

Flying and Swimming movement

Examples of games using Navpower: Battlefield III, Tomb Raider (2013), Rift, Orcs must Die!

41

PATHENGINE

http://www.pathengine.com/overview

http://www.pathengine.com/demo

PathEngine is a sophisticated middleware tool-

kit for the implementation of intelligent agent

movement, built around an advanced

implementation of points-of-visibility

pathfinding on 3D ground surfaces.

At the heart of the SDK is a well-defined agent 'movement model', with pathfinding and collision both provided

in tight integration against this movement model.

This kind of integrated collision architecture is key to the implementation of fundamentally robust movement.

The toolkit also comes with powerful content processing and ground management functionality, a graphical

testbed and associated tools.

Examples of games using PathEngine: Just Cause II, Stronghold III, Kinect Sports Rivals

HAVOK AI

http://www.havok.com/products/ai

http://www.havok.com/sites/default/files/pdf/Havok_AI_2013.pdf

Havok AI is a platform-optimized SDK focused on

efficient pathfinding and path following in

dynamic game environments.

Features:

Automatic NavMesh Generation

Optimized Runtime

Dynamic Pathfinding

Streaming and Instancing

Character Steering

Some benefits:

Optimized runtime queries allowing NPCs to efficiently make decisions about their environment

Out-of-the-box path following module that controls the motion of individual characters even in highly

congested crowd scenes

Characters are able to predict the motion of others in their vicinity and react accordingly exhibiting

natural queuing and flowing

A demo video can be seen here: https://www.youtube.com/watch?v=rlbjGiP104M

Examples of games using Havok AI: Guild Wars II, Assassin’s Creed III, Skyrim, Far Cry III, Halo IV

42

USED WITH UNITY

MIDDLEWARE/MUDDLEWARE: UNITY-BASED STUFF THAT WE’VE BEEN USING TO MAKE

DEVELOPMENT EASIER AND MORE INTERESTING

http://www.big-robot.com/2012/06/07/middlewaremuddleware-unity-based-stuff-that-weve-been-using-to-

make-development-easier-and-more-interesting/

Interesting middlewares for Unity:

Track IR: head tracking system

Rune Skovbo Johansen’s Locomotive

System: smoothly blended IK

animation for any surface

Ultimate FPS: smoothest controls and

powerful camera

Ezgui : GUI elements in the 3D space

of the gameworld

Strumpy Shader Editor: great visual effects and procedurally created terrain

Tomaszek’s Volume Grass systems: http://www.stobierski.pl/unity/soccer.html

43

OTHER INTERESTING REFERENCES

AI SUMMIT ‘10

http://aigamedev.com/open/coverage/gdc10-slides-highlights/

In this article Alex J. Champandard summarize the 2

first days of the GDC 2010. His work contains

resumes, photos and slides of the conferences.

The different sessions talk about the following

topics:

CASE STUDIES: AI IN RECENT GAMES

AI programmers explain the encountered challenges they faced in Killzone 2, Brütal Legend, Dawn of

War 2

Brütal Legend

o Challenge with the diversity of units

o AI avatar control troops and makes decision for the most powerful unit of each faction

o Final state machine without explicit transition

o Logic for selecting next state based on utility functions (weight for the value of state)

o Slides here

Killzone 2

o Multiplayer bots

o Description of the solution here

o Slides here

o Not handling special cases because of expert systems use at runtime. Solution: creative form

of AI: optimization algorithms and data mining

Dawn of War 2

o Melee implementation: search-based melee

o Allows each unit to find a shortest path that tries to touch the player's bounding circle, but

without overlapping the circles of any other unit

o Limits the number of units attacking in proximity based on the unit's radius

o Slides here

BEHAVIOR TREES: THREE WAYS OF CULTIVATING STRONG AI

Various form of BTs have become the standard in industry

Main strengths: flexibility and customization

More user friendly with scripts

Techniques for keeping AI code decoupled from the game logic (and improve performance)

Slides of the conference here

44

WHY SO WARY OF MIDDLEWARE

“Success stories and tales of horror”

Advices for middleware developers

o Narrow interfaces

o Modular components

o Provide good demos

o …

IMPROVING AI DECISION MODELING THROUGH UTILITY THEORY

Roots in areas such as psychology, economics, sociology, and classical game theory

Improving the modeling of the underlying brain of agents, broaden the potential decision space…

Advantages of utility systems:

o Graphs of various response curves and their corresponding equations usable in utility

functions

o Approach of weight and veto for each factor (simplest approach)

Slides of the conference here

AI AND INTERACTIVE STORYTELLING: HOW WE CAN HELP EACH OTHER

Branching narratives, dynamic adaptive dialog, interactive storytelling, and drama management.

“Interactive storytelling”: analysis of the opportunities for applying AI to various parts of games. (Ex:

Mass Effect, unlocking dialogues/choices based on previous actions/decisions). Slides here

Dynamic stories:

o Leverage traditional techniques (like BTs) to create more reactive stories

o Limits: style stories of "one thing happening after another” quickly boring (The Sims)

o Author modeling as a tool to help craft better stories and experiences

o Slides here

Dialogue trees:

o Problems: NPCs repeat themselves, the author has no control over the scene as a whole, and

no room for great writing beyond single lines.

o Solution: big dialog tree of dependencies (each line spoken opens up new dialog lines)

o Slides here

SUSPENDING DISBELIEF: BRINGING YOUR CHARACTERS TO LIFE WITH BE TTER AI

First-hand experiences with successfully creating AI for a variety of small budget games

Presentation of limitations and useful tips

Simple needs-based AI system, inspired by The Sims: AI has a set of drives, each action can satisfy

these drives (score calculation for each action that depends on a variety of factors)

Action queue for execution

Environments that are easy to setup

Modularity: objects and actions easy to add

If use of FSM: use push-down automata to keep a history of states and an action queue

45

Behavior Trees: Recast/Detour libraries

Slides here

Organic HFSM implementation available here

DECIDING ON AN AI ARCHITECTURE: WHICH TOOL FOR THE JOB?

Most important issues an AI programmer

Quoted examples:

o For a medium-fast pace action combat game with lots of designer control required, behavior

trees seemed like the best choice.

o In an RPG game with a wide variety of weapons and items to use, utility was a good solution

to provide reactive behavior, or planners otherwise.

o For a simple sports game (e.g. baseball), a FSM or HFSM seemed to be the best choice to

satisfy the requirements for an individual on the field.

o In a RTS game that includes a tech-tree, a variety of buildings, and lots of interdependencies,

planners seemed like the obvious choice.

Utility is not an architecture but a complement to other approaches.

MASSIVE PARALLELISM IN AI: THROUGHPUT VS. REALTIME

Approaches to implement parallelism in-game

Evaluating trade-offs

Applies to pathfinding problem

Middleware

Slides of the conference here

AI MASHUPS: INSIGHTS INTO INTERTWINED ARCHITECTURES

In games: AI Architectures mashed together

Examples:

o Event-driven FSM implementation by Steve Rabin here

o Hybrid declarative BT (Sony NBA games): specify group coordination via roles and situations.

System similar to Prolog

o Idea of architecture based on hierarchies of hybrid reasoners: differs from BT, “each layer has

a floating point "activation" and has the ability to check what would happen before actually

executing”

EXPERIMENTAL GAME AI : LIVE DEMOS OF INNOVATION

Interesting AI prototypes

Examples:

o Deontic logic: Behavior of individuals emerging from rules (decided by the user)

o Long-term planning (environment similar to The Sims) prototype: individual working on his

skills to get promoted

o Starcraft group bot: ABL architecture (similar to BT)

46

AI GAME DEV

http://aigamedev.com

News and articles website about Artificial Intelligence. This

website features various articles related to AI in a specific

domain or in a specific game.

AI GAME RESEARCH

http://www.aigameresearch.org/

Website that features several playable AI projects being

developed by researchers.

VALVE PUBLICATIONS

http://www.valvesoftware.com/company/publications.html

Website about the latest developments on Valve’s games. The

news posted are mostly about Valves’s latest games, such as

Portal 2, Dota 2 and Team Fortress 2. There are various

subjects aborded in these articles, for example:

The algorithms used (AI, pathfinding…)

Textures and display techniques

Performance discussion

…

47

TOP 10 MOST INFLUENTIAL AI GAMES

http://aigamedev.com/open/highlights/top-ai-games/

A list of 10 games with impressive/original/efficient AI. Some

examples of AI behavior improvement:

Thousands of AIs displayed interacting in real time

Complex reasoning and behavior for individual AIs (like

using specific strategies, displaying emotions…)

Adaptative AI, taking advantage of the environment

TOP 10 GAMES WITH STUPID AI

http://www.cheatcc.com/extra/top10gameswithstupidai.html

A compilation of 10 video games with

weird/inefficient/frustrating AI. Most common problems are

Awkward pathfinding

Weird decisions from allied AI

Big flaws for enemy AI (like being unable to walk

around an obstacle)

WHATEVER HAPPENED TO VIDEO GAME AI?

http://uk.ign.com/articles/2013/04/08/whatever-happened-to-

video-game-ai

Article about the evolution of AI in video games and how it has

stagnated over the last decade. According to the author, huge

progress was made in the years 1990-2005 in terms of AI. People

expected even more for the next years, but nothing incredible

happened. Progresses were very minimal and game-specific.

48

TWO CHATBOTS TALKING TO EACH OTHER

http://www.youtube.com/watch?v=WnzlbyTZsQY

Two chatbots, who are supposed to talk and chat like

humans would do, are talking to each other. Answers are

very convincing at first, but the conversation slowly

becomes a little weird.

HOW INTELLIGENT IS ARTIFICIAL INTELLIGENCE?

http://www.youtube.com/watch?v=hcoa7OMAmRk

Discussion about the definition of artificial intelligence.

According to the guy in the video, there are two main

types of AIs:

The human-type AI, which would look and

behave like a human. This is the “science-fiction”

AI that we can see in numerous movies and

books.

The computer-type AI, which does not look like a

human at all. However, it is capable of very complex calculations and reasoning.

According to the guy in the video, the real future of AIs is the second type. Little to no success was made in the

first type, while enormous progress is made in computer-type AIs.

FIGHTING GAMES

http://shoryuken.com/srk/FightingGamePrimer.pdf

This is an article about the game Super Street Fighter II, a

1vs1 fighting game. This is not really related to AI, but

rather about the basics of fighting games:

Meaning and usage of the character’s “hitbox”:

the part of the character’s body that is able to

damage the enemy or that can get hit.

Different types of movements: jump, walk,

crouch, block… These movements often allow the

player to perform several types of attacks (normal

attacks, air attacks…).

Special actions or attacks to knockback the opponent or knock him out.

How to “combo” the opponent, how to break an enemy’s combo.

The 3 phases of the attack animation of the character:

49

o Startup (The animations starts)

o Active (The attack is able to damage the enemy during this phase only)

o Recovery (The character goes back to its normal stance)