Evolution of cooperation in Stackelberg games

Raimo P. Hämäläinen

Ilkka Leppänen

Systems Analysis Laboratory

Aalto University

Main theme

• To learn about cooperative behavior in repeated interactions

• In a Stackelberg setting, the players are in asymmetric positions. How does this affect cooperation?

• How do people act in these games when they have possibility to cheap talk / cheat / second-play?

• We study cheap talk with a second-play model

Earlier results from repeated settings• Cournot duopoly: random pairs converge to Nash, fixed

pairs collude (Holt 1985) • Sequential prisoner’s dilemma: cooperation decreases

over repetition (Clark and Sefton 2001)• Market games (auctions): convergence to equilibrium

prices (Roth et al. 1991)• Ultimatum games: offers and rejections become lower

over time (Roth et al. 1991, Camerer 2003) • Public goods games: cooperation deteriorates and free-

riding increases, but cooperation is maintained with punishment (Fehr and Schmidt 1999)

Stackelberg game

Basic setting:1) Leader decides: knows both payoffs and takes into

account the best response reaction of the follower2) Follower reacts to the leader’s decision

Second-play Stackelberg setting (Hämäläinen 1981):1) Leader decides and announces his decision to the

follower2) Follower reacts to the leader’s decision3) Leader takes into account follower’s decision and decides

again (re-optimizes / cheats)

Stackelberg game

• Leader has the opportunity to better payoffs because he decides first

• Second-play is possible when follower does not have full information about leader’s payoffs

• Stackelberg games have not received much attention in the experimental games literature

Stackelberg game with second-play

• In the second-play Stackelberg game, the leader’s announcement should not affect the follower’s choice

• In theory, the leader is the follower

• Second-play announcement introduces a possibility of cheap talk and cheating

• Does this affect cooperation, and is this possibility used?

• Cheap talk literature: cheap talk signaling helps cooperation in social dilemma situations (Crawford, 1998)

Experiment of Huck, Müller, and Normann (2001)

• Stackelberg game in a market context• Players acting as quantity-choosing firms• 13 × 13 payoff matrix, pen and paper• 92 subjects• Repeated for 10 rounds, with

– fixed pairs where each player meets the same pair repeatedly– random pairs where each player meets a new randomly

selected pair every round

• Compensated from the payoffs of two randomly selected rounds out of the ten rounds (average DM 15.67)

JO = joint optimum (6,6) L = Stackelberg leader (12,6)LS = second-play Stackelberg leader (9,6) F = Stackelberg follower as leader (6,12)N = Cournot-Nash equilibrium (8,8)

FollowerLe

ader

Our experiment

• Same payoff matrix and market context (similar instructions) as Huck et al. (2001)

• Computerized: SAL experimental game platform• 210 student subjects from engineering faculties• Repeated for 20 to 24 rounds, players did not know the

number of rounds• Compensated from the payoffs of two randomly selected

rounds out of ten final rounds (average 6.77 €)

8 different games, only one for each subject

Game PairsFollower knows payoffs of leader

Number of pairs

Stackelberg fixed Yes (*) 11

fixed No 15

random Yes (*) 10

random No 9

Second-play Stackelberg fixed Yes 14

fixed No 14

random Yes 18

random No 14

(*) Games studied by Huck et al. (2001)Subjects remained in their roles leader/follower for the whole session

SAL experimental game platform

• Web-based to allow independence of location• Our experiments arranged in a regular computer classroom

Evolution of cooperation: Stackelberg fixed pairs

F does not know payoffs of L

F knows payoffs of L

Evolution of cooperation: Stackelberg random pairs

F does not know payoffs of L

F knows payoffs of L

Evolution of cooperation: second-play fixed pairs

F does not know payoffs of L

F knows payoffs of L

Evolution of cooperation: second-play random pairs

F does not know payoffs of L

F knows payoffs of L

Stackelberg games result: gains and losses in cooperation

• Reference outcome is the Stackelberg outcome (72,36), payoff difference 36

• Fixed pairs: joint optimum (72,72), leader’s loss 0, follower’s gain 36

• Random pairs: Cournot-Nash (64,64), leader’s loss 8, follower’s gain 28

Second-play games result: gains and losses in cooperation

• Reference outcome is the second-play Stackelberg outcome (81,54), payoff difference 27

• Fixed pairs: joint optimum (72,72), leader’s loss 9, follower’s gain 18

• Random pairs: Cournot-Nash (64,64), leader’s loss 17, follower’s gain 10

• In second-play games, the evolution to cooperation is driven by the leader

Comparison to Huck et al. (2001)

Fixed pairs Random pairs

Frequencies of leader choices over all rounds, Stackelberg games with complete information

Sta

ckel

berg

Cou

rnot

-Nas

h

Join

t-op

timum

Explanation for the different leader behavior?

Evolution: Stackelberg fixed pairs, F knows payoffs of L

Convergence to the joint optimum

Vertical axis: number of given outcomes

Evolution: Stackelberg fixed pairs, F does not know payoffs of L

Convergence to both the joint optimum and the Cournot-Nash

Vertical axis: number of given outcomes

Evolution: Stackelberg random pairs, F knows payoffs of L

Cournot-Nash

Vertical axis: number of given outcomes

Evolution: Stackelberg random pairs, F does not know payoffs of L

Cournot-Nash

Vertical axis: number of given outcomes

Evolution: Second-play fixed pairs, F knows payoffs of L

Joint optimum

Vertical axis: number of given outcomes

Evolution: Second-play fixed pairs, F does not know payoffs of L

Joint optimum

Vertical axis: number of given outcomes

Evolution: Second-play random pairs, F knows payoffs of L

Convergence to equal payoffs

Vertical axis: number of given outcomes

Evolution: Second-play random pairs, F does not know payoffs of L

Convergence to equal payoffs

Vertical axis: number of given outcomes

Outcomes from last five rounds

GameFollower better off

Equal payoffs

Leader better off

Questionnaire:Leader

cooperative

Questionnaire: Follower signals

cooperation

Stackelberg fixed, F knows 25,5 % 61,8 % 12,7 % 91 % 91 %

Stackelberg fixed, F does not know 10,0 % 42,9 % 47,1 % 60 % 73 %

Stackelberg random, F knows 48,0 % 18,0 % 34,0 % 55 % 60 %

Stackelberg random, F does not know 6,7 % 57,8 % 35,6 % 78 % 67 %

Second-play fixed, F knows 15,7 % 58,6 % 25,7 % 71 % 86 %

Second-play fixed, F does not know 7,1 % 68,6 % 24,3 % 71 % 71 %

Second-play random, F knows 25,3 % 50,5 % 24,2 % 72 % 56 %

Second-play random, F does not know 27,1 % 48,6 % 24,3 % 32 % 43 %

Average 20,7 % 50,8 % 28,5 %

Questionnaires

• Leaders drive cooperation– In stackelberg games, 10 out of 11 leaders answered that they tried

either to maximize the sum of both payoffs or aim for equal payoffs; in second-play, 8/14 leaders

• Followers signal for cooperation, but if leaders do not notice it, cooperation does not result– In stackelberg games, 10 out of 11 followers answered that with their

choices, they tried to signal the intention to get better payoffs for both; in second-play, 12/14 followers

– In pairs that do not converge to the joint optimum, only 1 leader out of 6 has noticed the follower’s signaling; in second-play 2/7 leaders

• Punishments are used only in pairs which are not cooperating

Summary

• Strong other-regarding behavior: fixed pairs converge to joint optimum, random pairs Cournot-Nash, both result in equal payoffs

• On average 51% of outcomes have equal payoffs

• No Stackelberg outcomes

• Leader drives cooperation in second-play games with cheap talk

• Leader has no threat of loss of payoffs (as in the ultimatum game) and still chooses cooperative strategy, even in random pairs

• Frequencies of cooperation:– Our leaders: 35,5%– Huck leaders: 21%– Typical ultimatum game fair offers: 71% (Fehr and Schmidt 1999)

Neural correlates to other regarding behavior?

• Are there differences in the neural areas activated for self regarding and other regarding players?

• Compare to Sanfey et al. (2003) fMRI observations from receiving unfair ultimatum proposals:– Activation in the bilateral anterior insula, ”emotional goal of

resisting unfairness”– Activation in the dorsolateral prefrontal cortex, ”cognitive goal of

accumulating money”– Activation in the anterior cingulate cortex, ”motivational conflict

between fairness and self-interest”

References• Camerer, C. 2003. Behavioral Game Theory, Princeton University Press.

• Crawford, V. 1998. A survey of experiments on communication via cheap talk. Journal of Economic Theory, Vol. 78.

• Fehr, E. and Schmidt, K.M. 1999. A theory of fairness, competition, and cooperation. Quarterly Journal of Economics, Vol. 114. No. 3.

• Holt, C. 1985. An experimental test of the consistent-conjectures hypothesis. The American Economic Review, Vol. 75, No. 3.

• Huck, S., Müller, W., and Normann, H-T. 2001. Stackelberg beats Cournot: on collusion and efficiency in experimental markets. The Economic Journal, Vol. 111.

• Hämäläinen, R. 1981. On the cheating problem in Stackelberg games. Int J Syst Sci, Vol. 12, No. 6.

• Roth, A., Prasnikar, V., Okuno-Fujiwara, M. and Zamir, S. 1991. Bargaining and Market Behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: An Experimental Study. The American Economic Review, Vol. 81, No. 5.

• Sanfey, A.G., Rilling, J.K., Aronson, J.A., Nystrom, L.E., Cohen, J.D. 2003. The neural basis of economic decision-making in the ultimatum game. Science, Vol. 300.

More Huck and Müller et al. references

• Huck, S., Müller, W. and Normann, H-T. 2002. To commit of not to commit: Endogenous timing in experimental duopoly markets. Games and Economic Behavior, Vol. 38, No. 2..

• Huck, S., and Wallace, B. 2002. Reciprocal strategies and aspiration levels in a Cournot-Stackelberg experiment. Economics Bulletin, Vol. 3., No. 3.

• Müller, W. 2006. Allowing for two production periods in the Cournot duopoly: experimental evidence. Journal of Economic Behavior & Organization, Vol. 60, No. 1.

• Müller, W., and Tan, F. 2010. Team versus individual play in a sequential market game. Manuscript, http://www.tilburguniversity.edu/research/institutes-and-research-groups/center/phd_stud/tan/Mueller_Tan.pdf