591

Towards a Science of Security GamesKey Algorithmic Principles Deployed Systems Research Challenges

Milind TambeHelen N and Emmett H Jones Professor in Engineering

University of Southern Californiawith

Currentformer PhD studentspostdocs

Bo An Matthew Brown Francesco Delle Fave Fei

Fang Benjamin Ford William Haskell Manish Jain

Albert Jiang Debarun Kar Chris Kiekintveld Rajiv

Maheswaran Janusz Marecki Praveen Paruchuri

Jonathan PearceJames Pita Thanh Nguyen Yundi

Qian Eric Shieh Jason Tsai Pradeep Varakantham

Haifeng Xu Amulya Yadav Rong Yang Zhengyu

Yin Chao Zhang

Other collaborators Fernando Ordonez (USC amp U Chile) Richard John

(USC) David Kempe (USC) Shaddin Dughmi

(USC) amp

Craig Boutilier (Toronto) Jeff Brantingahm

(UCLA) Vince Conitzer (Duke) Sarit Kraus (BIU

Israel) Andrew Lemieux (NCSR)Kevin Leyton-

Brown (UBC) M Pechoucek (CTU Czech R) Ariel

Procaccia (CMU) Tuomas Sandholm (CMU)

Martin Short (GATech) Y Vorobeychik

(Vanderbilt) hellip

59

Global Challenges for Security

Game Theory for Security Resource Optimization

2

59

Example Model

Stackelberg Security Games

Security allocation

Targets have weights

Adversary surveillance

Target

1

Target

2

Target 1 4 -3 -1 1

Target 2 -5 5 2 -1

Adversary

3

Defender

59

Example Model

Stackelberg Security Games

Security allocation

Targets have weights

Adversary surveillance

Target

1

Target

2

Target 1 4 -3 -1 1

Target 2 -5 5 2 -1

Adversary

4

Defender

59

Example Model

Stackelberg Security Games

Security allocation

Targets have weights

Adversary surveillance

Target

1

Target

2

Target 1 4 -3 -1 1

Target 2 -5 5 2 -1

Adversary

5

Defender

59

Stackelberg Security GamesSecurity Resource Optimization Not 100 Security

Random strategy

Increase costuncertainty to attackers

Stackelberg game

Defender commits to mixed strategy

Adversary conducts surveillance responds

Stackelberg Equilibrium Optimal random

Target

1

Target

2

Target 1 4 -3 -1 1

Target 2 -5 5 2 -1

Adversary

6

Defender

59

Research Contributions

Game Theory for Security

Computational Game Theory in the Field

Computational game

theory

bull Massive games

Behavioral game

theory

bull Exploit human

behavior

models

+ Planning under uncertainty learninghellip

7

59

Example Model

Stackelberg Security Games

Security allocation

Targets have weights

Adversary surveillance

Target

1

Target

2

Target 1 4 -3 -1 1

Target 2 -5 5 2 -1

Adversary

4

Defender

59

Example Model

Stackelberg Security Games

Security allocation

Targets have weights

Adversary surveillance

Target

1

Target

2

Target 1 4 -3 -1 1

Target 2 -5 5 2 -1

Adversary

5

Defender

59

Stackelberg Security GamesSecurity Resource Optimization Not 100 Security

Random strategy

Increase costuncertainty to attackers

Stackelberg game

Defender commits to mixed strategy

Adversary conducts surveillance responds

Stackelberg Equilibrium Optimal random

Target

1

Target

2

Target 1 4 -3 -1 1

Target 2 -5 5 2 -1

Adversary

6

Defender

59

Research Contributions

Game Theory for Security

Computational Game Theory in the Field

Computational game

theory

bull Massive games

Behavioral game

theory

bull Exploit human

behavior

models

+ Planning under uncertainty learninghellip

7

59

Example Model

Stackelberg Security Games

Security allocation

Targets have weights

Adversary surveillance

Target

1

Target

2

Target 1 4 -3 -1 1

Target 2 -5 5 2 -1

Adversary

5

Defender

59

Stackelberg Security GamesSecurity Resource Optimization Not 100 Security

Random strategy

Increase costuncertainty to attackers

Stackelberg game

Defender commits to mixed strategy

Adversary conducts surveillance responds

Stackelberg Equilibrium Optimal random

Target

1

Target

2

Target 1 4 -3 -1 1

Target 2 -5 5 2 -1

Adversary

6

Defender

59

Research Contributions

Game Theory for Security

Computational Game Theory in the Field

Computational game

theory

bull Massive games

Behavioral game

theory

bull Exploit human

behavior

models

+ Planning under uncertainty learninghellip

7

59

Stackelberg Security GamesSecurity Resource Optimization Not 100 Security

Random strategy

Increase costuncertainty to attackers

Stackelberg game

Defender commits to mixed strategy

Adversary conducts surveillance responds

Stackelberg Equilibrium Optimal random

Target

1

Target

2

Target 1 4 -3 -1 1

Target 2 -5 5 2 -1

Adversary

6

Defender

59

Research Contributions

Game Theory for Security

Computational Game Theory in the Field

Computational game

theory

bull Massive games

Behavioral game

theory

bull Exploit human

behavior

models

+ Planning under uncertainty learninghellip

7

59

Research Contributions

Game Theory for Security

Computational Game Theory in the Field

Computational game

theory

bull Massive games

Behavioral game

theory

bull Exploit human

behavior

models

+ Planning under uncertainty learninghellip

7

859

Ports amp port traffic

US Coast Guard

Applications Deployed Security Assistants

Airports access

roads amp flights

TSA

Airport Police

Urban transport

LA SheriffrsquosTSA

Singapore Police

Environment

US Coast Guard

WWF WCShellip

59

Key Lessons Security Games

Decision aids based on computational game theory in daily use

Optimize limited security resources against adversaries

Applications yield research challenges Science of security games

Scale-up Incremental strategy generation amp Marginals

Uncertainty Integrate MDPs Robustness Quantal response

Current applications (wildlife security) Interdisciplinary challenge

Global challenges Merge planninglearning amp security games

9

5910

Outline ldquoSecurity Gamesrdquo Research (2007-Now)

2007 2009 2011 2012 2013 2013-

Airports

Flights

PortsRoads

TrainsEnvironment

Evaluation II Real-world deployments

(Patience)

Evaluation I Scale up Handle uncertainty

Publications

AAMAS AAAI IJCAIhellip

2007 onwards

5910

Outline ldquoSecurity Gamesrdquo Research (2007-Now)

2007 2009 2011 2012 2013 2013-

Airports

Flights

PortsRoads

TrainsEnvironment

Evaluation II Real-world deployments

(Patience)

Evaluation I Scale up Handle uncertainty

Publications

AAMAS AAAI IJCAIhellip

2007 onwards

5911

bull 6 plots against LAX

ARMOR LAX (2007) GUARDS TSA (2011)

Airport Security Mapping to Stackelberg Games

GLASGOW 63007

5912

ARMOR Operation [2007]Generate Detailed Defender Schedule Pita Paruchuri

Mixed Integer Program

Pr(Canine patrol 8 AM Terminals 357) = 033

Pr(Canine patrol 8 AM Terminals 256) = 017

helliphellipCanine Team Schedule July 28Term 1 Term 2 Term 3 Term 4 Term 5 Term 6 Term 7 Term 8

8 AM Team1 Team3 Team5

9 AM Team1 Team2 Team4

10 AM Team3 Team5 Team2

Target 1 Target 2

Defender 1 2 -1 -3 4

Defender 2 -3 3 3 -2

5913

ARMOR MIP [2007]Generate Mixed Strategy for Defender

1 i

ixts

jq

ix

ijR

Xi Qj

max

1Qj

jq

MqxCa ji

Xi

ij )1()(0

Maximize defender

expected utility

Defender mixed

strategy

Adversary best

response

Pita Paruchuri

Target 1 Target 2

Defender 1 2 -1 -3 4

Defender 2 -3 3 3 -2

Adversary response

5914

ARMOR Payoffs [2007]Previous Research Provides Payoffs in Security Game Domains

Target 1 Target 2

Defender 1 2 -1 -3 4

Defender 2 -3 3 3 -2

jq

ix

ijR

Xi Qj

maxMaximize defender

expected utility

5915

ARMOR MIP [2007]Solving for a Single Adversary Type

Term 1 Term 2

Defend1 2 -1 -3 4

Defend2 -3 1 3 -3

1 i

ixts

jq

ix

ijR

Xi Qj

max

1Qj

jq

ljq][ix

MqxCa ji

Xi

ij

1010

)1()(0

Maximize defender

expected utility

Defender strategy

Adversary strategy

Adversary best

response

119909119894

ARMORhellipthrows a digital cloak of invisibilityhellip

59

IRIS Federal Air Marshals Service [2009]Scale Up Number of Defender Strategies

1000 Flights 20 air marshals 1041

combinations

ARMOR out of memory

Strategy 1 Strategy 2 Strategy 3

Strateg

y 1

Strateg

y 2

Strateg

y 3

Strateg

y 4

Strateg

y 5

Strateg

y 6

Not enumerate all combinations

Branch and price Incremental strategy generation

Strategy 1 Strategy 2 Strategy 3

Strategy 1

Strategy 2

Strategy 3

Strategy 4

Strategy 5

Strategy 6

16

59

IRIS Scale Up Number of Defender Strategies [2009]

Small Support Set for Mixed Strategies

Small support set size

bull Most xi variables zero

x124=0239

x123=00

x135=00

Attack

1

Attack

2

Attack

hellip

Attack

1000

123 5-10 4-8 hellip -209

124 5-10 4-8 hellip -209

135 5-10 -95 hellip -209

hellip

hellip1041 rowsx378=0123

10]10[

)1()(0

11

max

jqx

MqxCa

qxts

qxR

i

ji

Xi

ij

Qj

j

i

i

jiij

Xi Qj

qx

1000 flights 20 air marshals

1041combinations

17

59

Target 3 Target 7

hellip hellip

Resource Sink

Best new pure strategy

Minimum cost network flow

IRIS Incremental Strategy Generation

Exploit Small Support

Attack 1 Attack 2 Attack Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

hellip

500 rows

NOT 1041

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

Slave (LP Duality Theory)

Master

Converge

GLOBAL

OPTIMAL

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

18

Jain Kiekintveld

59

IRIS Deployed FAMS (2009-)

ldquohellipin 2011 the Military Operations Research Society selected a University

of Southern California project with FAMS on randomizing flight

schedules for the prestigious Rist Awardhelliprdquo

-R S Bray (TSA)

Transportation Security Subcommittee

US House of Representatives 2012

19

Significant change in operations

59

Networks Mumbai Police Checkpoints[2013]

150 edges 2 Checkpoints

150-choose-2 strategies

With V Conitzer 20

Jain

5912

ARMOR Operation [2007]Generate Detailed Defender Schedule Pita Paruchuri

Mixed Integer Program

Pr(Canine patrol 8 AM Terminals 357) = 033

Pr(Canine patrol 8 AM Terminals 256) = 017

helliphellipCanine Team Schedule July 28Term 1 Term 2 Term 3 Term 4 Term 5 Term 6 Term 7 Term 8

8 AM Team1 Team3 Team5

9 AM Team1 Team2 Team4

10 AM Team3 Team5 Team2

Target 1 Target 2

Defender 1 2 -1 -3 4

Defender 2 -3 3 3 -2

5913

ARMOR MIP [2007]Generate Mixed Strategy for Defender

1 i

ixts

jq

ix

ijR

Xi Qj

max

1Qj

jq

MqxCa ji

Xi

ij )1()(0

Maximize defender

expected utility

Defender mixed

strategy

Adversary best

response

Pita Paruchuri

Target 1 Target 2

Defender 1 2 -1 -3 4

Defender 2 -3 3 3 -2

Adversary response

5914

ARMOR Payoffs [2007]Previous Research Provides Payoffs in Security Game Domains

Target 1 Target 2

Defender 1 2 -1 -3 4

Defender 2 -3 3 3 -2

jq

ix

ijR

Xi Qj

maxMaximize defender

expected utility

5915

ARMOR MIP [2007]Solving for a Single Adversary Type

Term 1 Term 2

Defend1 2 -1 -3 4

Defend2 -3 1 3 -3

1 i

ixts

jq

ix

ijR

Xi Qj

max

1Qj

jq

ljq][ix

MqxCa ji

Xi

ij

1010

)1()(0

Maximize defender

expected utility

Defender strategy

Adversary strategy

Adversary best

response

119909119894

ARMORhellipthrows a digital cloak of invisibilityhellip

59

IRIS Federal Air Marshals Service [2009]Scale Up Number of Defender Strategies

1000 Flights 20 air marshals 1041

combinations

ARMOR out of memory

Strategy 1 Strategy 2 Strategy 3

Strateg

y 1

Strateg

y 2

Strateg

y 3

Strateg

y 4

Strateg

y 5

Strateg

y 6

Not enumerate all combinations

Branch and price Incremental strategy generation

Strategy 1 Strategy 2 Strategy 3

Strategy 1

Strategy 2

Strategy 3

Strategy 4

Strategy 5

Strategy 6

16

59

IRIS Scale Up Number of Defender Strategies [2009]

Small Support Set for Mixed Strategies

Small support set size

bull Most xi variables zero

x124=0239

x123=00

x135=00

Attack

1

Attack

2

Attack

hellip

Attack

1000

123 5-10 4-8 hellip -209

124 5-10 4-8 hellip -209

135 5-10 -95 hellip -209

hellip

hellip1041 rowsx378=0123

10]10[

)1()(0

11

max

jqx

MqxCa

qxts

qxR

i

ji

Xi

ij

Qj

j

i

i

jiij

Xi Qj

qx

1000 flights 20 air marshals

1041combinations

17

59

Target 3 Target 7

hellip hellip

Resource Sink

Best new pure strategy

Minimum cost network flow

IRIS Incremental Strategy Generation

Exploit Small Support

Attack 1 Attack 2 Attack Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

hellip

500 rows

NOT 1041

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

Slave (LP Duality Theory)

Master

Converge

GLOBAL

OPTIMAL

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

18

Jain Kiekintveld

59

IRIS Deployed FAMS (2009-)

ldquohellipin 2011 the Military Operations Research Society selected a University

of Southern California project with FAMS on randomizing flight

schedules for the prestigious Rist Awardhelliprdquo

-R S Bray (TSA)

Transportation Security Subcommittee

US House of Representatives 2012

19

Significant change in operations

59

Networks Mumbai Police Checkpoints[2013]

150 edges 2 Checkpoints

150-choose-2 strategies

With V Conitzer 20

Jain

5913

ARMOR MIP [2007]Generate Mixed Strategy for Defender

1 i

ixts

jq

ix

ijR

Xi Qj

max

1Qj

jq

MqxCa ji

Xi

ij )1()(0

Maximize defender

expected utility

Defender mixed

strategy

Adversary best

response

Pita Paruchuri

Target 1 Target 2

Defender 1 2 -1 -3 4

Defender 2 -3 3 3 -2

Adversary response

5914

ARMOR Payoffs [2007]Previous Research Provides Payoffs in Security Game Domains

Target 1 Target 2

Defender 1 2 -1 -3 4

Defender 2 -3 3 3 -2

jq

ix

ijR

Xi Qj

maxMaximize defender

expected utility

5915

ARMOR MIP [2007]Solving for a Single Adversary Type

Term 1 Term 2

Defend1 2 -1 -3 4

Defend2 -3 1 3 -3

1 i

ixts

jq

ix

ijR

Xi Qj

max

1Qj

jq

ljq][ix

MqxCa ji

Xi

ij

1010

)1()(0

Maximize defender

expected utility

Defender strategy

Adversary strategy

Adversary best

response

119909119894

ARMORhellipthrows a digital cloak of invisibilityhellip

59

IRIS Federal Air Marshals Service [2009]Scale Up Number of Defender Strategies

1000 Flights 20 air marshals 1041

combinations

ARMOR out of memory

Strategy 1 Strategy 2 Strategy 3

Strateg

y 1

Strateg

y 2

Strateg

y 3

Strateg

y 4

Strateg

y 5

Strateg

y 6

Not enumerate all combinations

Branch and price Incremental strategy generation

Strategy 1 Strategy 2 Strategy 3

Strategy 1

Strategy 2

Strategy 3

Strategy 4

Strategy 5

Strategy 6

16

59

IRIS Scale Up Number of Defender Strategies [2009]

Small Support Set for Mixed Strategies

Small support set size

bull Most xi variables zero

x124=0239

x123=00

x135=00

Attack

1

Attack

2

Attack

hellip

Attack

1000

123 5-10 4-8 hellip -209

124 5-10 4-8 hellip -209

135 5-10 -95 hellip -209

hellip

hellip1041 rowsx378=0123

10]10[

)1()(0

11

max

jqx

MqxCa

qxts

qxR

i

ji

Xi

ij

Qj

j

i

i

jiij

Xi Qj

qx

1000 flights 20 air marshals

1041combinations

17

59

Target 3 Target 7

hellip hellip

Resource Sink

Best new pure strategy

Minimum cost network flow

IRIS Incremental Strategy Generation

Exploit Small Support

Attack 1 Attack 2 Attack Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

hellip

500 rows

NOT 1041

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

Slave (LP Duality Theory)

Master

Converge

GLOBAL

OPTIMAL

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

18

Jain Kiekintveld

59

IRIS Deployed FAMS (2009-)

ldquohellipin 2011 the Military Operations Research Society selected a University

of Southern California project with FAMS on randomizing flight

schedules for the prestigious Rist Awardhelliprdquo

-R S Bray (TSA)

Transportation Security Subcommittee

US House of Representatives 2012

19

Significant change in operations

59

Networks Mumbai Police Checkpoints[2013]

150 edges 2 Checkpoints

150-choose-2 strategies

With V Conitzer 20

Jain

5914

ARMOR Payoffs [2007]Previous Research Provides Payoffs in Security Game Domains

Target 1 Target 2

Defender 1 2 -1 -3 4

Defender 2 -3 3 3 -2

jq

ix

ijR

Xi Qj

maxMaximize defender

expected utility

5915

ARMOR MIP [2007]Solving for a Single Adversary Type

Term 1 Term 2

Defend1 2 -1 -3 4

Defend2 -3 1 3 -3

1 i

ixts

jq

ix

ijR

Xi Qj

max

1Qj

jq

ljq][ix

MqxCa ji

Xi

ij

1010

)1()(0

Maximize defender

expected utility

Defender strategy

Adversary strategy

Adversary best

response

119909119894

ARMORhellipthrows a digital cloak of invisibilityhellip

59

IRIS Federal Air Marshals Service [2009]Scale Up Number of Defender Strategies

1000 Flights 20 air marshals 1041

combinations

ARMOR out of memory

Strategy 1 Strategy 2 Strategy 3

Strateg

y 1

Strateg

y 2

Strateg

y 3

Strateg

y 4

Strateg

y 5

Strateg

y 6

Not enumerate all combinations

Branch and price Incremental strategy generation

Strategy 1 Strategy 2 Strategy 3

Strategy 1

Strategy 2

Strategy 3

Strategy 4

Strategy 5

Strategy 6

16

59

IRIS Scale Up Number of Defender Strategies [2009]

Small Support Set for Mixed Strategies

Small support set size

bull Most xi variables zero

x124=0239

x123=00

x135=00

Attack

1

Attack

2

Attack

hellip

Attack

1000

123 5-10 4-8 hellip -209

124 5-10 4-8 hellip -209

135 5-10 -95 hellip -209

hellip

hellip1041 rowsx378=0123

10]10[

)1()(0

11

max

jqx

MqxCa

qxts

qxR

i

ji

Xi

ij

Qj

j

i

i

jiij

Xi Qj

qx

1000 flights 20 air marshals

1041combinations

17

59

Target 3 Target 7

hellip hellip

Resource Sink

Best new pure strategy

Minimum cost network flow

IRIS Incremental Strategy Generation

Exploit Small Support

Attack 1 Attack 2 Attack Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

hellip

500 rows

NOT 1041

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

Slave (LP Duality Theory)

Master

Converge

GLOBAL

OPTIMAL

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

18

Jain Kiekintveld

59

IRIS Deployed FAMS (2009-)

ldquohellipin 2011 the Military Operations Research Society selected a University

of Southern California project with FAMS on randomizing flight

schedules for the prestigious Rist Awardhelliprdquo

-R S Bray (TSA)

Transportation Security Subcommittee

US House of Representatives 2012

19

Significant change in operations

59

Networks Mumbai Police Checkpoints[2013]

150 edges 2 Checkpoints

150-choose-2 strategies

With V Conitzer 20

Jain

5915

ARMOR MIP [2007]Solving for a Single Adversary Type

Term 1 Term 2

Defend1 2 -1 -3 4

Defend2 -3 1 3 -3

1 i

ixts

jq

ix

ijR

Xi Qj

max

1Qj

jq

ljq][ix

MqxCa ji

Xi

ij

1010

)1()(0

Maximize defender

expected utility

Defender strategy

Adversary strategy

Adversary best

response

119909119894

ARMORhellipthrows a digital cloak of invisibilityhellip

59

IRIS Federal Air Marshals Service [2009]Scale Up Number of Defender Strategies

1000 Flights 20 air marshals 1041

combinations

ARMOR out of memory

Strategy 1 Strategy 2 Strategy 3

Strateg

y 1

Strateg

y 2

Strateg

y 3

Strateg

y 4

Strateg

y 5

Strateg

y 6

Not enumerate all combinations

Branch and price Incremental strategy generation

Strategy 1 Strategy 2 Strategy 3

Strategy 1

Strategy 2

Strategy 3

Strategy 4

Strategy 5

Strategy 6

16

59

IRIS Scale Up Number of Defender Strategies [2009]

Small Support Set for Mixed Strategies

Small support set size

bull Most xi variables zero

x124=0239

x123=00

x135=00

Attack

1

Attack

2

Attack

hellip

Attack

1000

123 5-10 4-8 hellip -209

124 5-10 4-8 hellip -209

135 5-10 -95 hellip -209

hellip

hellip1041 rowsx378=0123

10]10[

)1()(0

11

max

jqx

MqxCa

qxts

qxR

i

ji

Xi

ij

Qj

j

i

i

jiij

Xi Qj

qx

1000 flights 20 air marshals

1041combinations

17

59

Target 3 Target 7

hellip hellip

Resource Sink

Best new pure strategy

Minimum cost network flow

IRIS Incremental Strategy Generation

Exploit Small Support

Attack 1 Attack 2 Attack Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

hellip

500 rows

NOT 1041

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

Slave (LP Duality Theory)

Master

Converge

GLOBAL

OPTIMAL

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

18

Jain Kiekintveld

59

IRIS Deployed FAMS (2009-)

ldquohellipin 2011 the Military Operations Research Society selected a University

of Southern California project with FAMS on randomizing flight

schedules for the prestigious Rist Awardhelliprdquo

-R S Bray (TSA)

Transportation Security Subcommittee

US House of Representatives 2012

19

Significant change in operations

59

Networks Mumbai Police Checkpoints[2013]

150 edges 2 Checkpoints

150-choose-2 strategies

With V Conitzer 20

Jain

59

IRIS Federal Air Marshals Service [2009]Scale Up Number of Defender Strategies

1000 Flights 20 air marshals 1041

combinations

ARMOR out of memory

Strategy 1 Strategy 2 Strategy 3

Strateg

y 1

Strateg

y 2

Strateg

y 3

Strateg

y 4

Strateg

y 5

Strateg

y 6

Not enumerate all combinations

Branch and price Incremental strategy generation

Strategy 1 Strategy 2 Strategy 3

Strategy 1

Strategy 2

Strategy 3

Strategy 4

Strategy 5

Strategy 6

16

59

IRIS Scale Up Number of Defender Strategies [2009]

Small Support Set for Mixed Strategies

Small support set size

bull Most xi variables zero

x124=0239

x123=00

x135=00

Attack

1

Attack

2

Attack

hellip

Attack

1000

123 5-10 4-8 hellip -209

124 5-10 4-8 hellip -209

135 5-10 -95 hellip -209

hellip

hellip1041 rowsx378=0123

10]10[

)1()(0

11

max

jqx

MqxCa

qxts

qxR

i

ji

Xi

ij

Qj

j

i

i

jiij

Xi Qj

qx

1000 flights 20 air marshals

1041combinations

17

59

Target 3 Target 7

hellip hellip

Resource Sink

Best new pure strategy

Minimum cost network flow

IRIS Incremental Strategy Generation

Exploit Small Support

Attack 1 Attack 2 Attack Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

hellip

500 rows

NOT 1041

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

Slave (LP Duality Theory)

Master

Converge

GLOBAL

OPTIMAL

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

18

Jain Kiekintveld

59

IRIS Deployed FAMS (2009-)

ldquohellipin 2011 the Military Operations Research Society selected a University

of Southern California project with FAMS on randomizing flight

schedules for the prestigious Rist Awardhelliprdquo

-R S Bray (TSA)

Transportation Security Subcommittee

US House of Representatives 2012

19

Significant change in operations

59

Networks Mumbai Police Checkpoints[2013]

150 edges 2 Checkpoints

150-choose-2 strategies

With V Conitzer 20

Jain

59

IRIS Scale Up Number of Defender Strategies [2009]

Small Support Set for Mixed Strategies

Small support set size

bull Most xi variables zero

x124=0239

x123=00

x135=00

Attack

1

Attack

2

Attack

hellip

Attack

1000

123 5-10 4-8 hellip -209

124 5-10 4-8 hellip -209

135 5-10 -95 hellip -209

hellip

hellip1041 rowsx378=0123

10]10[

)1()(0

11

max

jqx

MqxCa

qxts

qxR

i

ji

Xi

ij

Qj

j

i

i

jiij

Xi Qj

qx

1000 flights 20 air marshals

1041combinations

17

59

Target 3 Target 7

hellip hellip

Resource Sink

Best new pure strategy

Minimum cost network flow

IRIS Incremental Strategy Generation

Exploit Small Support

Attack 1 Attack 2 Attack Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

hellip

500 rows

NOT 1041

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

Slave (LP Duality Theory)

Master

Converge

GLOBAL

OPTIMAL

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

18

Jain Kiekintveld

59

IRIS Deployed FAMS (2009-)

ldquohellipin 2011 the Military Operations Research Society selected a University

of Southern California project with FAMS on randomizing flight

schedules for the prestigious Rist Awardhelliprdquo

-R S Bray (TSA)

Transportation Security Subcommittee

US House of Representatives 2012

19

Significant change in operations

59

Networks Mumbai Police Checkpoints[2013]

150 edges 2 Checkpoints

150-choose-2 strategies

With V Conitzer 20

Jain

59

Target 3 Target 7

hellip hellip

Resource Sink

Best new pure strategy

Minimum cost network flow

IRIS Incremental Strategy Generation

Exploit Small Support

Attack 1 Attack 2 Attack Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

hellip

500 rows

NOT 1041

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

Slave (LP Duality Theory)

Master

Converge

GLOBAL

OPTIMAL

Attack 1 Attack 2 Attackhellip Attack 6

124 5-10 4-8 hellip -209

378 -8 10 -810 hellip -810

18

Jain Kiekintveld

59

IRIS Deployed FAMS (2009-)

ldquohellipin 2011 the Military Operations Research Society selected a University

of Southern California project with FAMS on randomizing flight

schedules for the prestigious Rist Awardhelliprdquo

-R S Bray (TSA)

Transportation Security Subcommittee

US House of Representatives 2012

19

Significant change in operations

59

Networks Mumbai Police Checkpoints[2013]

150 edges 2 Checkpoints

150-choose-2 strategies

With V Conitzer 20

Jain

59

IRIS Deployed FAMS (2009-)

ldquohellipin 2011 the Military Operations Research Society selected a University

of Southern California project with FAMS on randomizing flight

schedules for the prestigious Rist Awardhelliprdquo

-R S Bray (TSA)

Transportation Security Subcommittee

US House of Representatives 2012

19

Significant change in operations

59

Networks Mumbai Police Checkpoints[2013]

150 edges 2 Checkpoints

150-choose-2 strategies

With V Conitzer 20

Jain

59

Networks Mumbai Police Checkpoints[2013]

150 edges 2 Checkpoints

150-choose-2 strategies

With V Conitzer 20

Jain

59

Networks Mumbai Police Checkpoints[2013]

Incremental Strategy Generation

With V Conitzer 21

Jain

Double oracle Converge to a global optimal

Path 1 Path 2

Checkpoint

strategy 15 -5 -1 1

Path 1 Path 2

Checkpoint

strategy 15 -5 -1 1

Checkpoint

strategy 2-5 5 2 -1

Defender oracle

Path 1 Path 2 Path 3

Checkpoint

strategy 15 -5 -1 1 -2 2

Checkpoint

strategy 2-5 5 1 -1 -2 2

Attacker oracle

Path 1 Path 2 Path 3

Checkpoint

strategy 15 -5 -1 1 -2 2

Checkpoint

strategy 2-5 5 1 -1 -2 2

59

Double Oracle[2013]Incremental Strategy Generation Exploit Small Support

22

150 edges 2 Checkpoints

Only six candidate edges

for checkpoints

Jain

5923

Mumbai Police Checkpoints[2013]Results of Scale-up

20416 Roads15 checkpoints

20 min

59

Double Oracle Social Cyber Networks[2013]

Incremental Strategy Generation

24

Social networkseg counter-insurgency

Sources

Intermediate

Nodes

Links

Targets

Cyber networks

Tsai

59

Outline ldquoSecurity Gamesrdquo Research

2007 2009 2011 2012 2013 2013-

Airports

Flights

PortsRoads

TrainsEnvironment

25

59

Double Oracle Social Cyber Networks[2013]

Incremental Strategy Generation

24

Social networkseg counter-insurgency

Sources

Intermediate

Nodes

Links

Targets

Cyber networks

Tsai

59

Outline ldquoSecurity Gamesrdquo Research

2007 2009 2011 2012 2013 2013-

Airports

Flights

PortsRoads

TrainsEnvironment

25

59

Outline ldquoSecurity Gamesrdquo Research

2007 2009 2011 2012 2013 2013-

Airports

Flights

PortsRoads

TrainsEnvironment

25

59

Port Security Threat Scenarios

US Ports $315 trillion economy

USS Cole after suicide attack

French oil tanker hit by small boat

Attack on a ferry

26

59

PROTECT Randomized Patrol Scheduling [2013]

27

Port Protection (Scale-up) and Ferries (Continuous Spacetime)

59

PROTECT Randomized Patrol Scheduling [2013]Port Protection (Scale-up) and Ferries (Continuous Spacetime)

28

59

Ferries Scale-up with Mobile Resources amp Moving TargetsTransition Graph Representation

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

29

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

Ferries Scale-up with Mobile Resources amp Moving TargetsTransition Graph Representation

30

59

PROTECT Randomized Patrol Scheduling [2013]

27

Port Protection (Scale-up) and Ferries (Continuous Spacetime)

59

PROTECT Randomized Patrol Scheduling [2013]Port Protection (Scale-up) and Ferries (Continuous Spacetime)

28

59

Ferries Scale-up with Mobile Resources amp Moving TargetsTransition Graph Representation

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

29

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

Ferries Scale-up with Mobile Resources amp Moving TargetsTransition Graph Representation

30

59

PROTECT Randomized Patrol Scheduling [2013]Port Protection (Scale-up) and Ferries (Continuous Spacetime)

28

59

Ferries Scale-up with Mobile Resources amp Moving TargetsTransition Graph Representation

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

29

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

Ferries Scale-up with Mobile Resources amp Moving TargetsTransition Graph Representation

30

59

Ferries Scale-up with Mobile Resources amp Moving TargetsTransition Graph Representation

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

29

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

Ferries Scale-up with Mobile Resources amp Moving TargetsTransition Graph Representation

30

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

Ferries Scale-up with Mobile Resources amp Moving TargetsTransition Graph Representation

30

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

31

Patrols protect nearby ferry location Solve as done in ARMOR

Ferries Patrol RoutesExponential Numbers of Patrol Routes

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

Patrols protect nearby ferry location Solve as done in ARMOR

Ferries Patrol RoutesExponential Numbers of Patrol Routes

32

Pr([(B5) (C 10) (C15)]) = 017

Pr([(B5) (C10) (B15)]) =013

NT variables

Pr([(A5) (A10) (B15)]) = 007

Pr([(A5) (A10) (A15)]) = 003

Jiang KiekintveldFang

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

Variables NOT routes but probability flow over each segment

Ferries Scale-up Marginal Probabilities Over Segments

33

NT variables

Jiang KiekintveldFang

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

030

017

013

Obtain marginal probabilities over segments

010

007

003

N2T variablesExtract Pr([(B5) (C 10) (C15)]) = 017

Pr([(B5) (C10) (B15)]) =013

Ferries Scale-up with Marginals Over Separable SegmentsSignificant Speedup

34

NT variables

59

5936

Outline ldquoSecurity Gamesrdquo Research (2007-Now)

2007 2009 2011 2012 2013 2013-

Airports

Flights

PortsRoads

TrainsEnvironment

Evaluation II Real-world deployments

(Patience)

Evaluation I Scale up Handle uncertainty

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

Patrols protect nearby ferry location Solve as done in ARMOR

Ferries Patrol RoutesExponential Numbers of Patrol Routes

32

Pr([(B5) (C 10) (C15)]) = 017

Pr([(B5) (C10) (B15)]) =013

NT variables

Pr([(A5) (A10) (B15)]) = 007

Pr([(A5) (A10) (A15)]) = 003

Jiang KiekintveldFang

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

Variables NOT routes but probability flow over each segment

Ferries Scale-up Marginal Probabilities Over Segments

33

NT variables

Jiang KiekintveldFang

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

030

017

013

Obtain marginal probabilities over segments

010

007

003

N2T variablesExtract Pr([(B5) (C 10) (C15)]) = 017

Pr([(B5) (C10) (B15)]) =013

Ferries Scale-up with Marginals Over Separable SegmentsSignificant Speedup

34

NT variables

59

5936

Outline ldquoSecurity Gamesrdquo Research (2007-Now)

2007 2009 2011 2012 2013 2013-

Airports

Flights

PortsRoads

TrainsEnvironment

Evaluation II Real-world deployments

(Patience)

Evaluation I Scale up Handle uncertainty

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

Variables NOT routes but probability flow over each segment

Ferries Scale-up Marginal Probabilities Over Segments

33

NT variables

Jiang KiekintveldFang

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

030

017

013

Obtain marginal probabilities over segments

010

007

003

N2T variablesExtract Pr([(B5) (C 10) (C15)]) = 017

Pr([(B5) (C10) (B15)]) =013

Ferries Scale-up with Marginals Over Separable SegmentsSignificant Speedup

34

NT variables

59

5936

Outline ldquoSecurity Gamesrdquo Research (2007-Now)

2007 2009 2011 2012 2013 2013-

Airports

Flights

PortsRoads

TrainsEnvironment

Evaluation II Real-world deployments

(Patience)

Evaluation I Scale up Handle uncertainty

59

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

Ferry

030

017

013

Obtain marginal probabilities over segments

010

007

003

N2T variablesExtract Pr([(B5) (C 10) (C15)]) = 017

Pr([(B5) (C10) (B15)]) =013

Ferries Scale-up with Marginals Over Separable SegmentsSignificant Speedup

34

NT variables

59

5936

Outline ldquoSecurity Gamesrdquo Research (2007-Now)

2007 2009 2011 2012 2013 2013-

Airports

Flights

PortsRoads

TrainsEnvironment

Evaluation II Real-world deployments

(Patience)

Evaluation I Scale up Handle uncertainty

59

5936

Outline ldquoSecurity Gamesrdquo Research (2007-Now)

2007 2009 2011 2012 2013 2013-

Airports

Flights

PortsRoads

TrainsEnvironment

Evaluation II Real-world deployments

(Patience)

Evaluation I Scale up Handle uncertainty

5936

Outline ldquoSecurity Gamesrdquo Research (2007-Now)

2007 2009 2011 2012 2013 2013-

Airports

Flights

PortsRoads

TrainsEnvironment

Evaluation II Real-world deployments

(Patience)

Evaluation I Scale up Handle uncertainty

59

TRUSTS Frequent adversary interaction games

Patrols Against Fare Evaders

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

030

017

013

010

010

37

59

TRUSTS Frequent adversary interaction

Uncertainty in Defender Action Execution

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

030

015

010

010

007

003

005

38

Jiang Delle Fave

59

Markov Decision Problems in Security games

A 5 min A 10 min A 15 min

B 5 min B 10 min B 15 min

C 5 min C 10 min C 15 min

A

B

C

5 min 10 min 15 min

030

015

010

010

007

003

005

39

TRUSTS Frequent adversary interaction

Uncertainty in Defender Action Execution Jiang Delle Fave

5940

Urban Transportation Security

MDPs amp DEC-MDPs in Security Games

COPS LA Metro System(Against Opportunistic Crime)

STREETS Singapore Roads(Against Reckless Driving)

DEC-MDPs in Security Games Teamwork + Uncertainty

A 5 A 10 A 15

B 5 B 10 B 15

C 5 C 10 C 15

A 5 A 10 A 15

B 5 B 10 B 15

C 5 C 10 C 15

ZhangShieh

59

Uncertainty Space Algorithms

Bayesian and Robust Approaches

Adversary payoff uncertainty

Adversary observation amp

defender execution

uncertainty

Adversary rationality

uncertainty

RECON

BRASS

Monotonic Maximin

(Monotonic adversary)

41

URAC

Payoff interval

Not point

estimateGMC HUNTER

Bayesian

Robust

Yin Nguyen An

0

05

1

15

5 10

Def

end

ers

EU

Targets

ISG RECON URAC

59

Security Games Environmental Crime

amp Bounded Rationality

FisheryGulf of Mexico

Nakai Nam Theun

Forest Area Laos

WildlifeQueen Elizabeth National

ParkUganda

No patrols

Higher

density

Lower densityx

Yang Ford Brown

59

Uncertainty Space Algorithms

Bayesian and Robust Approaches

Adversary payoff uncertainty

Adversary observation amp

defender execution

uncertainty

Adversary rationality

uncertainty

RECON

BRASS

Monotonic Maximin

(Monotonic adversary)

41

URAC

Payoff interval

Not point

estimateGMC HUNTER

Bayesian

Robust

Yin Nguyen An

0

05

1

15

5 10

Def

end

ers

EU

Targets

ISG RECON URAC

59

Security Games Environmental Crime

amp Bounded Rationality

FisheryGulf of Mexico

Nakai Nam Theun

Forest Area Laos

WildlifeQueen Elizabeth National

ParkUganda

No patrols

Higher

density

Lower densityx

Yang Ford Brown

59

Security Games Environmental Crime

amp Bounded Rationality

FisheryGulf of Mexico

Nakai Nam Theun

Forest Area Laos

WildlifeQueen Elizabeth National

ParkUganda

No patrols

Higher

density

Lower densityx

Yang Ford Brown

59

Uncertainty in Adversary Decision Bounded Rationality

Human Subjects as Poachers

59

Uncertainty in Adversary Decision[2009]

Human subjects Anchoring e-Optimality

-4

-3

-2

-1

0

1

Unobserved 5 Observations 20 Observations Unlimited

Average expected reward

DOBSS

Uniform

COBRA

ARMOR

ARMOR Outperforms uniform random

COBRA

MqxCaq

Xxxts

qxR

ji

Xi

ijj

Xi Qj

jiijqx

)1()()1(

|)|1()1(

max

ee

Anchoring

e-optimality

With Sarit Kraus 44

Pita

5945

Quantal Response Stochastic choice better choice more likely

Quantal Response Model of Adversary [2011] Not Maximize Expected Utility

T

j

jxadversary

EU

jxadversary

EU

j

e

eq

1

))((

))((

-3

-25

-2

-15

-1

-05

0

05

1

15

2

Payoff 1 Payoff 2 Payoff 3 Payoff 4

QuantalResponse

COBRA

ARMOR

Adversaryrsquos probability

of choosing target j

Yang

10

)(max1

1

)(

)(

t tt

defendT

jT

j

jxEU

jxEU

x

xKxts

jxEU

e

e

adversary

adversary

59

Uncertainty in Adversary Decision [2012]

Robust vs Modeling Adversaries

Robustness Bound loss to defender Not model attacker via QR

Defeating Robust Learned subjective utility

Robust wins Draw QR wins

42 52 6

Results on 100 games

SU-QR wins Draw Robust

13 8 1

Results on 22 games

MqxCa

qxR

ji

Xi

ij

jiij

Xi Qj

qx

)1()(0

max

β (Adversaryrsquos utility loss if

deviates from optimal) gt=

(Defenderrsquos utility loss due to

adversary deviation)

penaltyattackw

rewardattackw

probcapturewjaSEU

3

2

1)(

M

j

jxSEU

jxSEU

jadversary

adversary

e

eq

1

)(

)(

SU-QR wins Draw Robust

6 13 3

Results against security experts

With Sarit Kraus 46

5947

PAWS Protection Assistant for Wildlife Security[2014]Repeated Stackelberg Game

Simulation

Learn from crime data

Improve model

Defender calculates

mixed strategy

Defender executes

randomized patrols

Poachers attack targets

119890119878119864119880119894(w1w2w3)

119896 119890119878119864119880119896(w1w2w3)

Bayesian SUQR Heterogeneous Poachers

5948

PAWS Test April 2014

Trials in Queen Elizabeth National Park

Andrew Lemieux with rangers

on PAWS patrol in Uganda

59

Security Resource OptimizationEvaluating Deployed Security Systems Not Easy

Game theory Improvement over previous approaches

Previous Human schedulers or ldquosimple randomrdquo

49

Lab

Evaluation

Simulated

adversary

Human subject

adversaries

Field Evaluation

Patrol quality

Unpredictable Cover

Compare real schedules

Scheduling competition

Expert evaluation

Field Evaluation

Tests against adversaries

ldquoMock attackersrdquo

Capture rates of

real adversaries

59

Why Does Game Theory Perform Better

Weaknesses of Previous Methods

Human schedulers

Predictable patterns eg US Coast Guard

Scheduling effort amp cognitive burden

Simple random (eg dice roll)

Wrong weightscoverage eg officers to sparsely crowded terminals

No adversary reactions

Multiple deployments over multiple years without us forcing them

50

59

Lab Evaluation via Simulations

Example from IRIS (FAMS)

-10

-8

-6

-4

-2

0

2

4

6

50 150 250

Defe

nde

r E

xpe

cte

d u

tilit

y

Schedule Size

Uniform Weighted random 1 Weighted random 2 IRIS

51

59

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage

Patrols Before PROTECT Boston Patrols After PROTECT Boston

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Base Patrol Area

52

PROTECT (Coast Guard) 350 increase defender expected utility

59

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage for Less Effort

53

IRIS for FAMS Outperformed expert human over six monthsReportGAO-09-903T

ARMOR at LAX Savings of up to an hour a day in scheduling

59

Field Evaluation Human vs Game Theory Competition

Counter-terrorism Patrol Scheduling

90 officers on LA Metro Trains

Humans required significant effort

Worse schedules than game theory

Observerrsquos report on questions

54

3

35

4

45

5

55

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Secu

rity

Sco

re

Human Game Theory

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

59

Uncertainty in Adversary Decision[2009]

Human subjects Anchoring e-Optimality

-4

-3

-2

-1

0

1

Unobserved 5 Observations 20 Observations Unlimited

Average expected reward

DOBSS

Uniform

COBRA

ARMOR

ARMOR Outperforms uniform random

COBRA

MqxCaq

Xxxts

qxR

ji

Xi

ijj

Xi Qj

jiijqx

)1()()1(

|)|1()1(

max

ee

Anchoring

e-optimality

With Sarit Kraus 44

Pita

5945

Quantal Response Stochastic choice better choice more likely

Quantal Response Model of Adversary [2011] Not Maximize Expected Utility

T

j

jxadversary

EU

jxadversary

EU

j

e

eq

1

))((

))((

-3

-25

-2

-15

-1

-05

0

05

1

15

2

Payoff 1 Payoff 2 Payoff 3 Payoff 4

QuantalResponse

COBRA

ARMOR

Adversaryrsquos probability

of choosing target j

Yang

10

)(max1

1

)(

)(

t tt

defendT

jT

j

jxEU

jxEU

x

xKxts

jxEU

e

e

adversary

adversary

59

Uncertainty in Adversary Decision [2012]

Robust vs Modeling Adversaries

Robustness Bound loss to defender Not model attacker via QR

Defeating Robust Learned subjective utility

Robust wins Draw QR wins

42 52 6

Results on 100 games

SU-QR wins Draw Robust

13 8 1

Results on 22 games

MqxCa

qxR

ji

Xi

ij

jiij

Xi Qj

qx

)1()(0

max

β (Adversaryrsquos utility loss if

deviates from optimal) gt=

(Defenderrsquos utility loss due to

adversary deviation)

penaltyattackw

rewardattackw

probcapturewjaSEU

3

2

1)(

M

j

jxSEU

jxSEU

jadversary

adversary

e

eq

1

)(

)(

SU-QR wins Draw Robust

6 13 3

Results against security experts

With Sarit Kraus 46

5947

PAWS Protection Assistant for Wildlife Security[2014]Repeated Stackelberg Game

Simulation

Learn from crime data

Improve model

Defender calculates

mixed strategy

Defender executes

randomized patrols

Poachers attack targets

119890119878119864119880119894(w1w2w3)

119896 119890119878119864119880119896(w1w2w3)

Bayesian SUQR Heterogeneous Poachers

5948

PAWS Test April 2014

Trials in Queen Elizabeth National Park

Andrew Lemieux with rangers

on PAWS patrol in Uganda

59

Security Resource OptimizationEvaluating Deployed Security Systems Not Easy

Game theory Improvement over previous approaches

Previous Human schedulers or ldquosimple randomrdquo

49

Lab

Evaluation

Simulated

adversary

Human subject

adversaries

Field Evaluation

Patrol quality

Unpredictable Cover

Compare real schedules

Scheduling competition

Expert evaluation

Field Evaluation

Tests against adversaries

ldquoMock attackersrdquo

Capture rates of

real adversaries

59

Why Does Game Theory Perform Better

Weaknesses of Previous Methods

Human schedulers

Predictable patterns eg US Coast Guard

Scheduling effort amp cognitive burden

Simple random (eg dice roll)

Wrong weightscoverage eg officers to sparsely crowded terminals

No adversary reactions

Multiple deployments over multiple years without us forcing them

50

59

Lab Evaluation via Simulations

Example from IRIS (FAMS)

-10

-8

-6

-4

-2

0

2

4

6

50 150 250

Defe

nde

r E

xpe

cte

d u

tilit

y

Schedule Size

Uniform Weighted random 1 Weighted random 2 IRIS

51

59

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage

Patrols Before PROTECT Boston Patrols After PROTECT Boston

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Base Patrol Area

52

PROTECT (Coast Guard) 350 increase defender expected utility

59

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage for Less Effort

53

IRIS for FAMS Outperformed expert human over six monthsReportGAO-09-903T

ARMOR at LAX Savings of up to an hour a day in scheduling

59

Field Evaluation Human vs Game Theory Competition

Counter-terrorism Patrol Scheduling

90 officers on LA Metro Trains

Humans required significant effort

Worse schedules than game theory

Observerrsquos report on questions

54

3

35

4

45

5

55

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Secu

rity

Sco

re

Human Game Theory

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

5945

Quantal Response Stochastic choice better choice more likely

Quantal Response Model of Adversary [2011] Not Maximize Expected Utility

T

j

jxadversary

EU

jxadversary

EU

j

e

eq

1

))((

))((

-3

-25

-2

-15

-1

-05

0

05

1

15

2

Payoff 1 Payoff 2 Payoff 3 Payoff 4

QuantalResponse

COBRA

ARMOR

Adversaryrsquos probability

of choosing target j

Yang

10

)(max1

1

)(

)(

t tt

defendT

jT

j

jxEU

jxEU

x

xKxts

jxEU

e

e

adversary

adversary

59

Uncertainty in Adversary Decision [2012]

Robust vs Modeling Adversaries

Robustness Bound loss to defender Not model attacker via QR

Defeating Robust Learned subjective utility

Robust wins Draw QR wins

42 52 6

Results on 100 games

SU-QR wins Draw Robust

13 8 1

Results on 22 games

MqxCa

qxR

ji

Xi

ij

jiij

Xi Qj

qx

)1()(0

max

β (Adversaryrsquos utility loss if

deviates from optimal) gt=

(Defenderrsquos utility loss due to

adversary deviation)

penaltyattackw

rewardattackw

probcapturewjaSEU

3

2

1)(

M

j

jxSEU

jxSEU

jadversary

adversary

e

eq

1

)(

)(

SU-QR wins Draw Robust

6 13 3

Results against security experts

With Sarit Kraus 46

5947

PAWS Protection Assistant for Wildlife Security[2014]Repeated Stackelberg Game

Simulation

Learn from crime data

Improve model

Defender calculates

mixed strategy

Defender executes

randomized patrols

Poachers attack targets

119890119878119864119880119894(w1w2w3)

119896 119890119878119864119880119896(w1w2w3)

Bayesian SUQR Heterogeneous Poachers

5948

PAWS Test April 2014

Trials in Queen Elizabeth National Park

Andrew Lemieux with rangers

on PAWS patrol in Uganda

59

Security Resource OptimizationEvaluating Deployed Security Systems Not Easy

Game theory Improvement over previous approaches

Previous Human schedulers or ldquosimple randomrdquo

49

Lab

Evaluation

Simulated

adversary

Human subject

adversaries

Field Evaluation

Patrol quality

Unpredictable Cover

Compare real schedules

Scheduling competition

Expert evaluation

Field Evaluation

Tests against adversaries

ldquoMock attackersrdquo

Capture rates of

real adversaries

59

Why Does Game Theory Perform Better

Weaknesses of Previous Methods

Human schedulers

Predictable patterns eg US Coast Guard

Scheduling effort amp cognitive burden

Simple random (eg dice roll)

Wrong weightscoverage eg officers to sparsely crowded terminals

No adversary reactions

Multiple deployments over multiple years without us forcing them

50

59

Lab Evaluation via Simulations

Example from IRIS (FAMS)

-10

-8

-6

-4

-2

0

2

4

6

50 150 250

Defe

nde

r E

xpe

cte

d u

tilit

y

Schedule Size

Uniform Weighted random 1 Weighted random 2 IRIS

51

59

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage

Patrols Before PROTECT Boston Patrols After PROTECT Boston

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Base Patrol Area

52

PROTECT (Coast Guard) 350 increase defender expected utility

59

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage for Less Effort

53

IRIS for FAMS Outperformed expert human over six monthsReportGAO-09-903T

ARMOR at LAX Savings of up to an hour a day in scheduling

59

Field Evaluation Human vs Game Theory Competition

Counter-terrorism Patrol Scheduling

90 officers on LA Metro Trains

Humans required significant effort

Worse schedules than game theory

Observerrsquos report on questions

54

3

35

4

45

5

55

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Secu

rity

Sco

re

Human Game Theory

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

59

Uncertainty in Adversary Decision [2012]

Robust vs Modeling Adversaries

Robustness Bound loss to defender Not model attacker via QR

Defeating Robust Learned subjective utility

Robust wins Draw QR wins

42 52 6

Results on 100 games

SU-QR wins Draw Robust

13 8 1

Results on 22 games

MqxCa

qxR

ji

Xi

ij

jiij

Xi Qj

qx

)1()(0

max

β (Adversaryrsquos utility loss if

deviates from optimal) gt=

(Defenderrsquos utility loss due to

adversary deviation)

penaltyattackw

rewardattackw

probcapturewjaSEU

3

2

1)(

M

j

jxSEU

jxSEU

jadversary

adversary

e

eq

1

)(

)(

SU-QR wins Draw Robust

6 13 3

Results against security experts

With Sarit Kraus 46

5947

PAWS Protection Assistant for Wildlife Security[2014]Repeated Stackelberg Game

Simulation

Learn from crime data

Improve model

Defender calculates

mixed strategy

Defender executes

randomized patrols

Poachers attack targets

119890119878119864119880119894(w1w2w3)

119896 119890119878119864119880119896(w1w2w3)

Bayesian SUQR Heterogeneous Poachers

5948

PAWS Test April 2014

Trials in Queen Elizabeth National Park

Andrew Lemieux with rangers

on PAWS patrol in Uganda

59

Security Resource OptimizationEvaluating Deployed Security Systems Not Easy

Game theory Improvement over previous approaches

Previous Human schedulers or ldquosimple randomrdquo

49

Lab

Evaluation

Simulated

adversary

Human subject

adversaries

Field Evaluation

Patrol quality

Unpredictable Cover

Compare real schedules

Scheduling competition

Expert evaluation

Field Evaluation

Tests against adversaries

ldquoMock attackersrdquo

Capture rates of

real adversaries

59

Why Does Game Theory Perform Better

Weaknesses of Previous Methods

Human schedulers

Predictable patterns eg US Coast Guard

Scheduling effort amp cognitive burden

Simple random (eg dice roll)

Wrong weightscoverage eg officers to sparsely crowded terminals

No adversary reactions

Multiple deployments over multiple years without us forcing them

50

59

Lab Evaluation via Simulations

Example from IRIS (FAMS)

-10

-8

-6

-4

-2

0

2

4

6

50 150 250

Defe

nde

r E

xpe

cte

d u

tilit

y

Schedule Size

Uniform Weighted random 1 Weighted random 2 IRIS

51

59

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage

Patrols Before PROTECT Boston Patrols After PROTECT Boston

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Base Patrol Area

52

PROTECT (Coast Guard) 350 increase defender expected utility

59

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage for Less Effort

53

IRIS for FAMS Outperformed expert human over six monthsReportGAO-09-903T

ARMOR at LAX Savings of up to an hour a day in scheduling

59

Field Evaluation Human vs Game Theory Competition

Counter-terrorism Patrol Scheduling

90 officers on LA Metro Trains

Humans required significant effort

Worse schedules than game theory

Observerrsquos report on questions

54

3

35

4

45

5

55

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Secu

rity

Sco

re

Human Game Theory

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

5947

PAWS Protection Assistant for Wildlife Security[2014]Repeated Stackelberg Game

Simulation

Learn from crime data

Improve model

Defender calculates

mixed strategy

Defender executes

randomized patrols

Poachers attack targets

119890119878119864119880119894(w1w2w3)

119896 119890119878119864119880119896(w1w2w3)

Bayesian SUQR Heterogeneous Poachers

5948

PAWS Test April 2014

Trials in Queen Elizabeth National Park

Andrew Lemieux with rangers

on PAWS patrol in Uganda

59

Security Resource OptimizationEvaluating Deployed Security Systems Not Easy

Game theory Improvement over previous approaches

Previous Human schedulers or ldquosimple randomrdquo

49

Lab

Evaluation

Simulated

adversary

Human subject

adversaries

Field Evaluation

Patrol quality

Unpredictable Cover

Compare real schedules

Scheduling competition

Expert evaluation

Field Evaluation

Tests against adversaries

ldquoMock attackersrdquo

Capture rates of

real adversaries

59

Why Does Game Theory Perform Better

Weaknesses of Previous Methods

Human schedulers

Predictable patterns eg US Coast Guard

Scheduling effort amp cognitive burden

Simple random (eg dice roll)

Wrong weightscoverage eg officers to sparsely crowded terminals

No adversary reactions

Multiple deployments over multiple years without us forcing them

50

59

Lab Evaluation via Simulations

Example from IRIS (FAMS)

-10

-8

-6

-4

-2

0

2

4

6

50 150 250

Defe

nde

r E

xpe

cte

d u

tilit

y

Schedule Size

Uniform Weighted random 1 Weighted random 2 IRIS

51

59

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage

Patrols Before PROTECT Boston Patrols After PROTECT Boston

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Base Patrol Area

52

PROTECT (Coast Guard) 350 increase defender expected utility

59

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage for Less Effort

53

IRIS for FAMS Outperformed expert human over six monthsReportGAO-09-903T

ARMOR at LAX Savings of up to an hour a day in scheduling

59

Field Evaluation Human vs Game Theory Competition

Counter-terrorism Patrol Scheduling

90 officers on LA Metro Trains

Humans required significant effort

Worse schedules than game theory

Observerrsquos report on questions

54

3

35

4

45

5

55

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Secu

rity

Sco

re

Human Game Theory

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

5948

PAWS Test April 2014

Trials in Queen Elizabeth National Park

Andrew Lemieux with rangers

on PAWS patrol in Uganda

59

Security Resource OptimizationEvaluating Deployed Security Systems Not Easy

Game theory Improvement over previous approaches

Previous Human schedulers or ldquosimple randomrdquo

49

Lab

Evaluation

Simulated

adversary

Human subject

adversaries

Field Evaluation

Patrol quality

Unpredictable Cover

Compare real schedules

Scheduling competition

Expert evaluation

Field Evaluation

Tests against adversaries

ldquoMock attackersrdquo

Capture rates of

real adversaries

59

Why Does Game Theory Perform Better

Weaknesses of Previous Methods

Human schedulers

Predictable patterns eg US Coast Guard

Scheduling effort amp cognitive burden

Simple random (eg dice roll)

Wrong weightscoverage eg officers to sparsely crowded terminals

No adversary reactions

Multiple deployments over multiple years without us forcing them

50

59

Lab Evaluation via Simulations

Example from IRIS (FAMS)

-10

-8

-6

-4

-2

0

2

4

6

50 150 250

Defe

nde

r E

xpe

cte

d u

tilit

y

Schedule Size

Uniform Weighted random 1 Weighted random 2 IRIS

51

59

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage

Patrols Before PROTECT Boston Patrols After PROTECT Boston

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Base Patrol Area

52

PROTECT (Coast Guard) 350 increase defender expected utility

59

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage for Less Effort

53

IRIS for FAMS Outperformed expert human over six monthsReportGAO-09-903T

ARMOR at LAX Savings of up to an hour a day in scheduling

59

Field Evaluation Human vs Game Theory Competition

Counter-terrorism Patrol Scheduling

90 officers on LA Metro Trains

Humans required significant effort

Worse schedules than game theory

Observerrsquos report on questions

54

3

35

4

45

5

55

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Secu

rity

Sco

re

Human Game Theory

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

59

Security Resource OptimizationEvaluating Deployed Security Systems Not Easy

Game theory Improvement over previous approaches

Previous Human schedulers or ldquosimple randomrdquo

49

Lab

Evaluation

Simulated

adversary

Human subject

adversaries

Field Evaluation

Patrol quality

Unpredictable Cover

Compare real schedules

Scheduling competition

Expert evaluation

Field Evaluation

Tests against adversaries

ldquoMock attackersrdquo

Capture rates of

real adversaries

59

Why Does Game Theory Perform Better

Weaknesses of Previous Methods

Human schedulers

Predictable patterns eg US Coast Guard

Scheduling effort amp cognitive burden

Simple random (eg dice roll)

Wrong weightscoverage eg officers to sparsely crowded terminals

No adversary reactions

Multiple deployments over multiple years without us forcing them

50

59

Lab Evaluation via Simulations

Example from IRIS (FAMS)

-10

-8

-6

-4

-2

0

2

4

6

50 150 250

Defe

nde

r E

xpe

cte

d u

tilit

y

Schedule Size

Uniform Weighted random 1 Weighted random 2 IRIS

51

59

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage

Patrols Before PROTECT Boston Patrols After PROTECT Boston

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Base Patrol Area

52

PROTECT (Coast Guard) 350 increase defender expected utility

59

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage for Less Effort

53

IRIS for FAMS Outperformed expert human over six monthsReportGAO-09-903T

ARMOR at LAX Savings of up to an hour a day in scheduling

59

Field Evaluation Human vs Game Theory Competition

Counter-terrorism Patrol Scheduling

90 officers on LA Metro Trains

Humans required significant effort

Worse schedules than game theory

Observerrsquos report on questions

54

3

35

4

45

5

55

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Secu

rity

Sco

re

Human Game Theory

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

59

Why Does Game Theory Perform Better

Weaknesses of Previous Methods

Human schedulers

Predictable patterns eg US Coast Guard

Scheduling effort amp cognitive burden

Simple random (eg dice roll)

Wrong weightscoverage eg officers to sparsely crowded terminals

No adversary reactions

Multiple deployments over multiple years without us forcing them

50

59

Lab Evaluation via Simulations

Example from IRIS (FAMS)

-10

-8

-6

-4

-2

0

2

4

6

50 150 250

Defe

nde

r E

xpe

cte

d u

tilit

y

Schedule Size

Uniform Weighted random 1 Weighted random 2 IRIS

51

59

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage

Patrols Before PROTECT Boston Patrols After PROTECT Boston

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Base Patrol Area

52

PROTECT (Coast Guard) 350 increase defender expected utility

59

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage for Less Effort

53

IRIS for FAMS Outperformed expert human over six monthsReportGAO-09-903T

ARMOR at LAX Savings of up to an hour a day in scheduling

59

Field Evaluation Human vs Game Theory Competition

Counter-terrorism Patrol Scheduling

90 officers on LA Metro Trains

Humans required significant effort

Worse schedules than game theory

Observerrsquos report on questions

54

3

35

4

45

5

55

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Secu

rity

Sco

re

Human Game Theory

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

59

Lab Evaluation via Simulations

Example from IRIS (FAMS)

-10

-8

-6

-4

-2

0

2

4

6

50 150 250

Defe

nde

r E

xpe

cte

d u

tilit

y

Schedule Size

Uniform Weighted random 1 Weighted random 2 IRIS

51

59

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage

Patrols Before PROTECT Boston Patrols After PROTECT Boston

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Base Patrol Area

52

PROTECT (Coast Guard) 350 increase defender expected utility

59

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage for Less Effort

53

IRIS for FAMS Outperformed expert human over six monthsReportGAO-09-903T

ARMOR at LAX Savings of up to an hour a day in scheduling

59

Field Evaluation Human vs Game Theory Competition

Counter-terrorism Patrol Scheduling

90 officers on LA Metro Trains

Humans required significant effort

Worse schedules than game theory

Observerrsquos report on questions

54

3

35

4

45

5

55

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Secu

rity

Sco

re

Human Game Theory

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

59

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage

Patrols Before PROTECT Boston Patrols After PROTECT Boston

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Co

un

t

Base Patrol Area

52

PROTECT (Coast Guard) 350 increase defender expected utility

59

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage for Less Effort

53

IRIS for FAMS Outperformed expert human over six monthsReportGAO-09-903T

ARMOR at LAX Savings of up to an hour a day in scheduling

59

Field Evaluation Human vs Game Theory Competition

Counter-terrorism Patrol Scheduling

90 officers on LA Metro Trains

Humans required significant effort

Worse schedules than game theory

Observerrsquos report on questions

54

3

35

4

45

5

55

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Secu

rity

Sco

re

Human Game Theory

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

59

Field Evaluation of Schedule Quality

Improved Patrol Unpredictability amp Coverage for Less Effort

53

IRIS for FAMS Outperformed expert human over six monthsReportGAO-09-903T

ARMOR at LAX Savings of up to an hour a day in scheduling

59

Field Evaluation Human vs Game Theory Competition

Counter-terrorism Patrol Scheduling

90 officers on LA Metro Trains

Humans required significant effort

Worse schedules than game theory

Observerrsquos report on questions

54

3

35

4

45

5

55

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Secu

rity

Sco

re

Human Game Theory

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

59

Field Evaluation Human vs Game Theory Competition

Counter-terrorism Patrol Scheduling

90 officers on LA Metro Trains

Humans required significant effort

Worse schedules than game theory

Observerrsquos report on questions

54

3

35

4

45

5

55

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Secu

rity

Sco

re

Human Game Theory

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

59

Field Test Against Adversaries Mock Attackers

Example from PROTECT

ldquoMock attackerrdquo team deployed in Boston

Comparing PRE- to POST-PROTECT ldquodeterrencerdquo improved

Additional real-world indicators from Boston

Boston boaters questions

ldquohas the Coast Guard recently acquired more boatsrdquo

POST-PROTECT Actual reports of illegal activity

55

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

5956

Field Tests Against Adversaries

Computational Game Theory in the Field

Game theory vs Random

21 days of patrol

Identical conditions

Random + Human

Not controlled

0

20

40

60

80

100 Miscellaneous

Drugs

Firearm Violations

0

5

10

15

20

Captures30 min

Warnings30 min

Violations30 min

Game Theory

Rand+Human

Controlled

59

Expert Evaluation

Example from ARMOR IRIS amp PROTECT

February 2009 Commendations

LAX Police (City of Los Angeles)

July 2011 Operational Excellence

Award (US Coast Guard Boston)

September 2011 Certificate of

Appreciation (Federal Air Marshals)

June 2013 Meritorious Team Commendation

from Commandant (US Coast Guard)

57

59

Summary Security Games

Decision aids based on computational game theory in daily use

Optimize limited security resources against adversaries

Applications yield research challenges Science of security games

Scale-up Incremental strategy generation amp Marginals

Uncertainty Integrate MDPs Robustness Quantal response

Current applications (wildlife security) Interdisciplinar challenge

Global challenges Merge planninglearning amp security games

58

59

Just the Beginning of ldquoSecurity Gamesrdquohellip

Our next steps

59

Thank you

to sponsors

StartupARMORWAY Game theory in the field

Follow on research amp applications

Privacy audits

(Sinha IJCAIrsquo13)

Software testing

(Kukreja ASErsquo13)

Sport event security

(Yin AAAIrsquo14)

Singapore train

(Varakantham IAAIrsquo13)

Exam questions

(Li IJCAIrsquo13)

59

THANK YOU

tambeuscedu

httpteamcoreuscedusecurity

60

59

Just the Beginning of ldquoSecurity Gamesrdquohellip

Our next steps

59

Thank you

to sponsors

StartupARMORWAY Game theory in the field

Follow on research amp applications

Privacy audits

(Sinha IJCAIrsquo13)

Software testing

(Kukreja ASErsquo13)

Sport event security

(Yin AAAIrsquo14)

Singapore train

(Varakantham IAAIrsquo13)

Exam questions

(Li IJCAIrsquo13)

59

THANK YOU

tambeuscedu

httpteamcoreuscedusecurity

60

59

THANK YOU

tambeuscedu

httpteamcoreuscedusecurity

60