mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j...
Transcript of mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j...
![Page 1: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/1.jpg)
MixedStrategies
CMPT882ComputationalGameTheory
SimonFraserUniversitySpring2010
Instructor:OliverSchulte
![Page 2: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/2.jpg)
Motivation
• Some games like Rock, Paper, Scissor don’t have a Nash equilibrium as defined so far.
• Intuitively, the reason is that there is no steady state where players have perfect knowledge of each other’s actions: knowing exactly what the other player will do allows me to achieve an optimal payoff at their expense.
• Von Neumann and Morgenstern observed that this changes if we allow players to be unpredictable by choosing randomized strategies.
![Page 3: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/3.jpg)
Definition
• Consider a 2-player game (A1,A2,u1,u2). • The members of Ai are the pure strategies for
player i. • A mixed strategy is a probability distribution
over pure strategies. • Or, if we have k pure strategies, a mixed strategy
is a k-dimensional vector whose nonnegative entries sum to 1.
• Note: this is the notation from the text. More usual are Greek letters for mixed strategies, e.g. !.
![Page 4: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/4.jpg)
Example:MatchingPennies
Heads: 1/2 Tails: 1/2
Heads: 1/3 1,-1 -1,1
Tails: 2/3 -1,1 1,-1
![Page 5: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/5.jpg)
MixedStrategiesctd.
• Supposethateachplayerchoosesamixedstrategysi.• Theexpectedutilityofpurestrategya1isgivenbyEU1(a1,s2)=Σku1(a1,ak)xs2(ak).whereajrangesoverthestrategiesofplayer2.
• Theexpectedutilityofmixedstrategys1istheexpectationoverthepurestrategies:EU1(s1,s2)=ΣjEU1(aj,s2)xs1(aj)=ΣjΣku1(aj,ak)xs1(aj)xs2(ak).
• Wealsowriteu1(s1,s2)forEU1(s1,s2).Thenwehaveineffectanewgamewhosestrategysetsarethesetsofmixedstrategies,andwhoseutilityfunctionsaretheexpectedpayoffs.
![Page 6: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/6.jpg)
Nashequilibriuminmixedstrategies
• Thedefinitionofbestresponseisasbefore:amixedstrategys1isabestresponsetos2ifandonlyifthereisnoothermixedstrategys’1withu1(s’1,s2)>u1(s1,s2).
• Similarly,twomixedstrategies(s1,s2)areaNashequilibriumifeachisabestresponsetotheother.
![Page 7: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/7.jpg)
ExamplesandExercises
• FindamixedNashequilibriumforthefollowinggames.– MatchingPennies.– CoordinationGame.– Prisoner’sDilemma.
– BattleoftheSexesorChicken.• CanyoufindalltheNashequilibria?
![Page 8: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/8.jpg)
VisualizationofEquilibria
• Ina2x2game,wecangraphplayer2’sutilityasafunctionofp,theprobabilitythat1choosesstrategy1.Similarlyforplayer1’sutility.
![Page 9: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/9.jpg)
StrategiesandTopology
• Considermixedstrategyprofilesask‐dimensionalvectors,wherekisthetotalnumberofpurestrategiesforeachplayer.Thissetisconvexandcompact(why?).
![Page 10: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/10.jpg)
VisualizationofMatchingPennies
![Page 11: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/11.jpg)
VisualizationofM.P.equilibrium
![Page 12: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/12.jpg)
ComputationofEqulibria
• DefinitionThesupportofaprobabilitymeasurepisthesetofallpointsxs.t.p(x)>0.
• Proposition.Amixedstrategypair(s1,s2)isanN.E.ifandonlyifforallpurestrategiesaiinthesupportofsi,thestrategysiisabestreplysos‐i.
• Corollary.If(s1,s2)isanN.E.anda1,b1areinthesupportofs1,thenu1(a1,s2)=u1(b1,s2).Thatis,player1isindifferentbetweena1andb1.Dittofora2,b2inthesupportofs2.
![Page 13: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/13.jpg)
AgeneralNPprocedureforfindingaNashEquilibrium
1. Choosesupportforplayer1,supportforplayer2.
2. CheckifthereisaNashequilibriumwiththosesupports.
![Page 14: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/14.jpg)
ExistenceofNashEquilibrium
• Theorem(Nash1950).Inanyfinitegame(anynumberofplayers)thereexistsaNashequilibrium.
• ShortproofbyNash.
![Page 15: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/15.jpg)
ExistenceProof(1)
• Transformagivenpairofmixedstrategies(s1,s2)asfollows.Foreachpurestrategya1ofplayer1,setc(a1):=max(0,[u1(a1,s2)–u1(s1,s2)]).s’1(a1):=[s1(a1)+c(a1)]/[1+Σbc(b)].Dittoforplayer2.
• Thisdefinesacontinuousoperatorf(s1,s2)=(s’1,s’2)onthespaceofmixedstrategypairs.
• Astrategyaisabestreplyifandonlyifc(a)=0.• Soif(s1,s2)isanN.E.,thenc(a)=0foralla,sof
(s1,s2)=(s1,s2).
![Page 16: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/16.jpg)
ExistenceProof(2)
• ThegeneralizedBrouwerfixedpointtheoremstatesthatifKisconvexandcompact,andf:KKiscontinuous,thenfhasafixedpointf(x)=x.
• If(s1,s2)isafixedpointofthemappingonthepreviousslide,then(s1,s2)isanN.E.
• Proofoutline:Someactiona1withs1(a)>0mustbeabestreplyagainstanys2.Thereforec(a1)=0.Sincewehaveafixedpoint,1+Σbc(b)=1.Thisimpliesthatc(b)=0foreachb.
![Page 17: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/17.jpg)
IllustrationinExcel
• IllustrateconstructioninExcel.• Notethattheupdateoperationcanbeviewedasalocalcomputationmethod,andevenasalearningmethod!
![Page 18: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/18.jpg)
MaxminandMinmax
Considera2‐playergame.• Amaxminstrategyforplayer1solvesmaxs1mins2u1(s1,s2).Dittoforplayer2.
• Interpretation:Conservativelychoosestrategyagainstworst‐caseadversary.
• Thevaluemaxs1mins2u1(s1,s2)iscalledthesecuritylevelofplayer1.
• Aminmaxstrategyforplayer1solvesmins1maxs2u2(s1,s2).Dittoforplayer2.
• Interpreation:Punishtheotherplayerbyminimizingthebestpayoffshecanget.
![Page 19: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/19.jpg)
N.E.inZero‐SumGames:TheMinimaxTheorem(vonNeumann1928).
.Considera2‐playerzero‐sumgame.1. siisamaxminstrategyifandonlyifsiisa
minmaxstrategyfori=1,2.2. Forbothplayers,themaxminvalue=minmax
value.3. Ifs1,s2areeachmaxmin(minmax)strategies,
then(s1,s2)isaNashequilibrium.4. If(s1,s2)isanN.E.,thens1ands2aremaxmin
(minmax)strategies.
![Page 20: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/20.jpg)
InterpretationofmixedN.E.
![Page 21: mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j EU 1 (a j,s 2) x s 1 (a j) = Σ j Σ k u 1 (a j,a k) x s 1 (a j) x s 2 (a k). • We also write u 1 (s 1,s 2) for](https://reader030.fdocuments.in/reader030/viewer/2022041105/5f06ee8f7e708231d41a73f0/html5/thumbnails/21.jpg)
N.E.inPopulationGames