A Study of Computational and Human Strategies in Revelation Games
description
Transcript of A Study of Computational and Human Strategies in Revelation Games
![Page 1: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/1.jpg)
A Study of Computational and Human Strategies in Revelation Games
1Noam Peled, 2Kobi Gal, 1Sarit Kraus1Bar-Ilan university, Israel.
2Ben-Gurion university, Israel.
1
![Page 2: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/2.jpg)
Outline
● What is this research about? ● An agent that negotiates with people in incomplete
information settings.● People can choose to reveal private information.
● Why is it difficult? ● Uncertainty over people’s preferences. ● People are affected by social/psychological issues.
2
![Page 3: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/3.jpg)
Our approach
● Opponent modeling + machine learning.● Results
● Agent outperformed people.● Learned to exploit how people reveal information. ● People increased their performance when playing
with agent (as compared to other people).
3
![Page 4: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/4.jpg)
Why is it interesting?
4
![Page 5: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/5.jpg)
Revelation games
● Incomplete information over preferences:● Players’ types are unknown.
● Players can reaveal their type before negotiating for a finite number of rounds.
● Revelation is costless.● No discount factor.
5
![Page 6: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/6.jpg)
Our setting
● This is the minimal setting where each of the two players can make one proposal:
6
Revelation phase
1st proposal
Counter proposal
![Page 7: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/7.jpg)
Roadmap
● Probabilistic model of people.● The model explicity represent social features.● We built an agent that uses this model for
negotiation with people.● Extensive empirical evaluation
● over 400 subjects from different countries!
7
![Page 8: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/8.jpg)
Colored Trails
● Open source empirical test-bed for investigating decision making.
● Family of computer board games that involve negotiation over resources.
● Easy to design new games.● Built in functionality for conducting experiments
with people.● Over 30 publications.
8
![Page 9: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/9.jpg)
Revelation game in CT
9
![Page 10: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/10.jpg)
Shortest path to the goal
10
![Page 11: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/11.jpg)
Objective
● The objective is to maximize your score:● Try to get as close as possible to the goal.● Using as few chips as possible.● End up with as many chips as you can.
11
![Page 12: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/12.jpg)
If “me” chooses to reveal…
12
![Page 13: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/13.jpg)
Example proposal of ‘me’ player
● Here, the ‘me’ player is signaling aboutit’s true goal location.
13
![Page 14: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/14.jpg)
The other participant decides whether to accept the proposal
If it accepts the proposal, the game
will end
If it rejects, it will be able to make a counter proposal
14
![Page 15: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/15.jpg)
The end of the game
The ‘me’ player was moved to its goal using one gray and one red
chips
The other participant was moved to his goal
using 2 cyan chips
15
![Page 16: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/16.jpg)
Agent design
● SIGAL: SIGmoid Acceptance Learning.● Models people’s behavior using probability
distributions:● Making/accepting offers.● Whether people reveal their goals.
● Maximizes its expected benefit given the model using backward induction.
16
![Page 17: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/17.jpg)
SIGAL strategy: Round 2
● As a responder: accepts any beneficial proposal in terms of score in the game.
● As a proposer: ● Calculates its expected benefit from each proposal:
● Choose offer that maximizes expected benefit.
),|()|(),|()|(),|( 1111 hrejectpthacceptptthE
17
Revelation phase
1st proposal
Counter proposal
![Page 18: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/18.jpg)
SIGAL strategy: Round 1
● As responder: accept any proposal that gives it more than the expected benefit from round 2.
● As a proposer: ● Estimate its benefit from the other player counter
proposal.● Calculates the expected benefit from each proposal.● Choose the offer that maximizes expected benefit.
18
Revelation phase
1st proposal
Counter proposal
![Page 19: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/19.jpg)
Maximum expected benefit
19
![Page 20: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/20.jpg)
Revelation phase
● Use decision theory: SIGAL Calculates its expected benefit for both scenarios – revealing or not revealing.
● Picks the best option.
20
Revelation phase
1st proposal
Counter proposal
![Page 21: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/21.jpg)
Modeling acceptance of offers
● Use logistic function (Camerer 2001)
Acceptance probability xue
xacceptp 11)|(
Social utility function 21
![Page 22: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/22.jpg)
Social utility function
● People are not fully rational – a proposal is not desirable only for its benefit.
● Weighted sum of social features.
22
![Page 23: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/23.jpg)
Benefit feature
● Benefit = proposed offer score – no agreement score
● As example, the score of the ‘me’ player from the demo game was 135.
● Without reaching and agreement, its score is 30● Its benefit from the proposal is 135-30=105
23
![Page 24: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/24.jpg)
Other social features
● Difference in benefit = proposer’s benefit from the offer – responder’s benefit from the offer
● Revelation decision.● Previous round:
● The first proposal, if rejected, may affect the probability to accept the counter-proposal.
24
![Page 25: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/25.jpg)
Learning
● We used a genetic algorithm to find the optimal weights for people’s social utility function.
● Use density estimation to learn how people make offers
● Cross validation (10-fold).● Over-fitting removal: Stop learning in the
minimum of the generalization error.● Error calculation on held out test set.
25
![Page 26: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/26.jpg)
Fit to data
● The percentages are averages over similarutility ranges (bins)
26
![Page 27: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/27.jpg)
What did SIGAL learn?
● Which proposal to give in each round.● Whether to accept proposals or not in each
round.● Whether to reveal its goal or not.
27
![Page 28: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/28.jpg)
Empirical methodology
● Israeli CS students and users over the web. ● Subjects received an identical tutorial on
revelation, and had to pass a test.● Each participant played two different boards● Compared SIGAL performance with people’s
performance playing other people.
28
![Page 29: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/29.jpg)
Performance
29
![Page 30: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/30.jpg)
Why did SIGAL succeed?
● Learned to ask for more from people when they reveal their goal.
● Learned to make ‘fair’ proposals:● People dislike proposals with high benefits
difference in favor to the proposer.● Learned to exploit generous people:
● If people propose a generous offer in the first round, they are more willing to accept the counter offer.
30
![Page 31: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/31.jpg)
People also benefit from SIGAL!
● People playing with SIGAL got much more than people playing with people!
31
![Page 32: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/32.jpg)
Web users: Amazon Turk
● Web based bulletin board for ‘Human Intelligence Tasks‘.
● Millions of ‘workers’ are exposed to your task.● We got 140 ‘qualified’ participants in 8 hours!
32
![Page 33: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/33.jpg)
Baseline equilibrium agent
● There are lots of equilibria in the game.● We’ve developed an agent based on a pure
equilibrium strategy.
33
![Page 34: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/34.jpg)
Equilibrium agent didn't work
34
![Page 35: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/35.jpg)
Related work
● Did not take a decision theoretic approach.● Repeated negotiation (Oshrat et al. 2009).● Bayesian techniques (Hindriks et al. 2008).● Approximation heuristics (Jonker et al. 2007).
● Did not evaluate a computer agent● Repeated one-shot take-it-or-leave-it games (Gal et
al. 2007).
35
![Page 36: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/36.jpg)
Conclusions
● Revelation games are a new setting to study how people reveal information in a negotiation.
● Using a simple model, an agent can learn to outperform people in revelation games.
● Behavioral studies can actually help agent design.
● Combining decision theory with learning is a good approach for agent-design.
36
![Page 37: A Study of Computational and Human Strategies in Revelation Games](https://reader036.fdocuments.in/reader036/viewer/2022062501/568167b5550346895ddd03fa/html5/thumbnails/37.jpg)
Future work
● Extend the argumentation framework.● More signaling and revelation possibilities.● Develop a model which predict in which extant
the private information should be revealed during the game.
● EEG: Does features in brain waves can improve the prediction model?
37