1

Mechanism Design for Computationally Bounded Agents

Kate Larson

Carnegie Mellon University

Thesis Proposal

April 8, 2002

2

Introduction

• Fundamental problem in building open distributed systems:– There are users and agents acting in their own

self interest.– Designer wishes to guarantee optimal system

wide solutions while being unable to enforce particular behavior on users.

3

Introduction

• Techniques from game theory and mechanism design have been very useful:– Routing in networks– Resource allocation in operating systems– Auction design in electronic markets– Bidding agents for electronic auctions– ….

4

Introduction

• However, perfect rationality is often assumed!

• But in real systems…

Hard problemsCost Time constraints

5

Package Delivery and Vehicle Routing

Chicago to Home

Chicago to Toronto

Home to Chicago

Chicago

Toronto

Home

Depot

2

3 4

5

1

The delivery route can be computed.

Toronto to Home

6

Package Delivery and Vehicle Routing

Chicago to Home

Chicago to Toronto

Home to Chicago

Chicago

Toronto

Home

Depot

Toronto to Home

12

3

4

5

The new package can be easily fit into the delivery route. The cost can be kept low, and thus the bid also.

7

Computing of Valuations

• Agents may need to compute their valuations for

(bundles of) goods

– Vehicle routing problem in transportation exchanges

– Manufacturing scheduling problem in procurement

• Value of a set of items

=value of solution with items - value of solution without

• May involve solving multiple NP-complete problems

– Infeasible to solve optimally

8

Incentives• Game theory handles incentives

– System designer specifies the mechanism (auction rules)

– Each agent picks its own strategy

• Computational issues have to be handled also!– Computing as part of an agent’s strategy

• Interaction between incentives and computing– Equilibrium for rational agents is not always one

for computationally bounded agents!

9

My ApproachResource Bounded Reasoning from AI

Game Theory and Mechanism Design

Normative model of bounded rationality. Mechanism design for computationally bounded agents.

10

Thesis Statement

• It is possible to incorporate agents’ computational actions into a game theoretic model.

• Agents’ computational restrictions have a strong impact on the choice of strategies.

• It is possible to design mechanisms that take into account agents’ computational restrictions.

11

Structure

Model of bounded rationality

Game theoretic formulation

Impact on existing mechanisms

Mechanisms for bounded agents

Experiments

12

Outline• A model of bounded rationality

– A normative model of deliberation control for computationally bounded agents

• Game theory for computationally bounded agents

• The impact of agents’ computation • Mechanism design for computationally

bounded agents• Experiments• Schedule

13

Restricted Computational Resources

• Agents have limited computational resources– Deadlines

• All parcels must be delivered by Friday evening!

• Battery on my laptop only lasts 3 hours

– Cost associated with computation and deliberation

• Sending jobs to rendering farms

14

Anytime Algorithms• Anytime algorithms can be used to

approximate valuations– Anytime algorithms can return a solution at any time– Solution improves over time Allows a trade off between computing time and

solution quality

• Examples– Iterative refinement algorithms: Local search,

simulated annealing– Search algorithms: Depth first search, branch and

bound

15

Performance Profiles

• Performance profiles describe how computation changes the solution– Characterize the quality of an algorithm’s output as a

function of computing time

• Performance profile representation is important– Earlier methods were not normative: They did not

capture all possible ways an agent can control its deliberation

16

Performance Profiles

Computing time

Solution quality

Deterministic performance profile

Solution quality

Variance introduced by different problem instances

Computing time

[Horvitz 87, 89, Dean & Boddy 89]

Optimum

17

Conditioning on solution quality so far [Hansen & Zilberstein 96]

Ignores conditioning on the path

Table-based Representation of Uncertainty in Performance Profiles

Computing time

Solutionquality

[Zilberstein & Russell 91, 96]

.08 .19 .24

.15 .30 .17 .39

.16 .10 .16 .25 .30 .22

.08 .04 .17 .20 .22 .30 .24 .19 .15

.09 .10 .20 .22 .23 .37 .31 .13 .15

.11 .14 .33 .18 .21 .18 .08

.22 .17 .25 .24 .15 .13

.40 .31 .15 .19 .05

.15 .20 .03

.03

18

Performance Profile Tree

0

4

2

4

5

10

7

15

20

A

P(B|A) B

5

CP(C|A)Solution Quality

Value Node

Random Node

P(1)

P(2)

P(3)

[Larson & Sandholm 2001a]

Stores value of solution so far

Captures uncertainty from random algorithms, unknown algorithms

19

Properties of Performance Profile Trees

• Normative - allows conditioning on everything including– Path of solution quality– Path of other solution features– Problem instance features

• Can model uncertainty from– Randomized algorithms– Lack of knowledge about what algorithms other

agents use– Problem instances

20

Outline• A model of bounded rationality• Game theory for computationally bounded agents

– Roles of computation– Strategic computing– Deliberation equilibrium

• Effects of agents’ computation • Mechanism design for computationally bounded

agents• Experiments• Schedule

21

Roles of Computing• Improving:

– Agents compute to improve valuations

– Computation changes the valuation

• Example: Solving TSP’s to find cost of best route

• Refining:– Agents compute to

refine valuations– Computation

changes agents’ beliefs about valuations

• Example: Researching if an art work is a forgery

22

Roles of Computation• For computationally bounded agents,

computing has several strategic roles:1. Improves or refines the solution to the agent’s

own problem

2. Reduces uncertainty as to what future computing steps will bring

3. Improves the agent’s knowledge about others’ valuations

4. Improves the agent’s knowledge about what problems others may have computed on

23

Strategic Computing

• Good estimates of other agents’ valuations can allow an agent to tailor its strategy to achieve higher utility

• Strong strategic computing: Agent uses some of its deliberation resources to compute on others’ problems

• Weak strategic computing: Agent uses information from others’ performance profiles

24

Strategic Computing

• How an agent should allocate its computation can depend on– how others allocate their computation – what actions they all plan on taking

Incorporate computation actions into agents’ strategies. Analyze games for deliberation equilibria

25

Deliberation Equilibria

• A (Nash, dominant, etc.) deliberation equilibrium for computationally bounded agents is a (Nash, dominant, etc.) equilibrium where:1. The agents’ computing strategies form a

(Nash, dominant, etc) equilibrium, and

2. The agents’ noncomputing strategies form a (Nash, dominant, etc) equilibrium.

26

Outline• A model of bounded rationality• Game theory for computationally bounded agents• Effects of agents’ computation

– One-to-One Negotiation (Bargaining)– One-to-Many Negotiation (Auctions)– Many-to-Many Negotiation (Exchanges)

• Mechanism design for computationally bounded agents

• Experiments• Schedule

27

Bargaining

• Bargaining

A

A

B

B

A B

A

B

BA

28

Bargaining[Larson and Sandholm, 2001a]

• Take-it-or-leave-it: One agent makes an offer to the other who can accept or reject.– Determined under what settings agents have

dominant strategies.– Determined under what settings agents will not split

their computation.– Designed algorithms for determining offline, agents

optimal online strategies

29

Bargaining[Larson and Sandholm, 2002]

• Alternating Offers Model: Agents take turns making proposals and counter proposals

• Despite richer strategy space, alternating offers model is similar to take-it-or-leave it model– In (Bayes Nash) deliberation equilibrium, agents behave as

though they were in take-it-or-leave-it setting

30

Future Bargaining Work

• So far have only worked in settings where computation is free but limited– What happens if computation is costly?

Conjecture: Costly computation will not change the results.

31

Auctions and Strategic Computing

yesyesno

Generalized Vickrey

On which agent, bundle pair to allocate next computation step ?

Multiple items

for sale

noEnglish (1st price ascending) yes

yes

no

nonoVickrey (2nd price sealed bid)

yesDutch (1st price descending) yesyes

yesyesyesFirst price sealed-bidSingle item for

sale

Costly computation

Limited computation

Strong strategic computingCounter-speculation by rational agents ?

Auction

mechanism

[Larson and Sandholm 2001b, Larson and Sandholm 2001c]

32

Exchanges• Exchanges are a generalization of

auctions where there are potentially multiple buyers and multiple sellers– Does strategic computing occur in

exchanges?– Does the type of restriction on agents’

computational capabilities determine the occurrence of strategic computation?

Conjecture: Strategic computation can occur with either type of restriction (costly or limited computation).

33

Outline• A model of bounded rationality• Game theory for computationally

bounded agents• Effects of agents’ computation • Mechanism design for computationally

bounded agents• Experiments• Schedule

34

Mechanism Design• So far have focused on standard

mechanisms and determining their strengths and limitations if agents have computational restriction.

• Change focus:1. Mechanisms for computationally

restricted agents.

2. Measures for mechanisms.

35

Issues for Mechanism Design• There are several issues that must be

addressed for computationally restricted agents– What should agents report to the mechanism?

• Valuations once computed?• Valuations agents expect to obtain?• Results of partial computation?

– What is the role of the mechanism?• To specify allocations only?• To specify allocations and computation policies?

36

Issues for Mechanism Design• Does there exist a mechanism that is:

– Strategy proof,– Efficient,– Individually rational, and – Limits wastage of computational resources.

37

Measuring Mechanisms: Miscomputing Ratio

• How does allowing agents free choice of computing policies effect the outcome, o?

• Miscomputing Ratio:

R=SW(o*)/SW(o(NE))

SW(o*)=Social welfare if a global controller enforces computing policies so as to maximize social welfare

SW(o(NE))=Social welfare of worst Nash Eq.

38

Results and Future Work• Results (Vickrey auctions):

– If agents have limited or costly computation then the miscomputing ratio can be unbounded.

– The choice of appropriate cost functions can motivate bidders to choose strategies that maximize social welfare.

• Future Work:– Constraining deadlines or cost functions– Using performance profile trees to predict R– Constraining performance profile trees – Comparing mechanisms using miscomputing ratio, R

39

Outline• A model of bounded rationality• Game theory for computationally

bounded agents• Effects of agents’ computation • Mechanism design for computationally

bounded agents• Experiments• Schedule

40

Experiments

• Goals:– To demonstrate that the performance

profile tree is a feasible model.– To demonstrate that the performance

profile tree model is general.– To determine if strategic computing is an

artifice of analysis or something that occurs in the “real world”.

• Tools for controlling computation and visualization of deliberation and negotiation

41

Status

Model of bounded rationality

Game theoretic formulation

Impact on existing mechanisms

Mechanisms for bounded agents

Experiments

completedpartiallyfuture work

42

Schedule

0 1 2 3 4 5 6 7

Writing

Experiments

Miscomputing Ratio

Mech. Design

Exchanges

Bargaining (Costly)

Sp02 Sm02 F02 Sp03 Sm03 F03

43

Related Publications• Larson and Sandholm, 2001a, Bargaining with Limited

Computation:Deliberation Equilibrium, Artificial Intelligence, 132(2), pp.183-217

• Larson and Sandholm 2001b, Computationally Limited Agents in Auctions, Workshop on Agent based approaches to B2B, Autonomous Agents 2001

• Larson and Sandholm, 2001c, Costly Valuation Computation in Auction, TARK VIII

• Larson and Sandholm, 2002, An Alternating Offers Bargaining Model for Computationally Limited Agents, AAMAS 2002

• Larson and Sandholm, 2002, Miscomputing Ratio: The Social Cost of Selfish Computing, draft.

44

Other Publications

• Larson and Sandholm, 2000, Anytime Coalition Generation: An Average Case Study, Journal of Experimental and Theoretical Artificial Intelligence, 12(2000),23-42.

• Sandholm, Larson, Andersson, Shehory, and Tohme, 1999, Anytime Coalition Structure Generation with Worst Case Guarantees, Artificial Intelligence, (111)1-2, pp209-238.

45

Future Auction Work: Reverse Auctions

• Are they different from standard auctions if agents have bounded computational resources?– Maybe or maybe not!– Reverse auctions differ from auctions in:

• Complexity of winner determination• Communication complexity

– At first look, it does not appear as though bidding strategies are different.

46

Related Work• Parkes 2001, Iterated combinatorial auctions so that

both the computational burden of agents and mechanism are reduced

• Persico 2000, Information acquisition in auctions

• Bergemann and Pesendorfer 2001, Model where the auctioneer controls what information agents have available to them

• Bergemann and Välimäki 2001, Costly information acquisition, but no clear model of how information is obtained

47

Related Work

• Nisan and Ronen 2000: Algorithmic mechanism design. What happens if the mechanism is computationally limited? Investigates ways of introducing approximation algorithms to compute outcomes without loosing incentive compatibility.

• Kfir-Dahav, Monderer, Tennenholtz 2000

48

Experiments• To demonstrate that the performance profile

tree is a feasible model:– Multi-item auction where agents submit bids where

the value is determined by the solution to a TSP problem

– Prebundling to simplify the winner determination problem and to restrict the number of optimization problems agents are faced with

– Focus on having agents use performance profile trees, determining how to make the performance profile tree model feasible for actual problems.

49

Experiments

• To demonstrate that the performance profile tree is general:– Single item auction where agents

valuations of the item is based on schedules they compute.

– Use off the shelf schedulers (Micro-Boss, or a stochastic scheduler from Steve Smith’s group) to show that the techniques I propose are general

50

Experiments• To determine if strategic computation is an

artifice of analysis or something that occurs in the “real world”:– Multi-item reverse auction using data from actual

delivery companies– Agents must solve multiagent vehicle routing

problems– How does one tailor performance profile trees to

actual data without losing their analytical power?– Does strategic computing occur?

51

Game Theory Concepts• Game theory: method of studying systems of

agents in settings where there are strategic interactions

• Game has – Set of agents, A– Set of outcomes, O– Set of actions for each agent

• Strategy: contingency plan that determines what action the agent will take at any point in the game

• Equilibrium is fundamental concept in game theory– Stable point in space of agents’ strategies

52

Game Theory Concepts• Different equilibrium concepts

– Dominant strategy equilibrium• Every agent has a strategy that they are best off playing,

independent of other agents actions

– Nash equilibrium• The strategy one agent plays is optimal given what

strategies other agents are playing

• Mechanism Design: – Study of how to implement good system wide solutions that

involve mulitple self-interested agents

53

Mechanism Design• Mechanism Design:

– Study of how to implement good system wide solutions that involve multiple self-interested agents

• Mechanism: M=(S1,…,Sn,g(.)) is a set of strategies Si available to each agent i, and an outcome function g(s).– Defines strategies available to agents– Describes methods used to determine outcome

based on agents’ strategies

54

Game Theory• Game theory studies phenomena that occur

when decision makers interact– It is assumed that decision makers are strategic.

If he bids $10 then I could bid $11 and win.

If he bids $11 then I could bid $12 and win.

55

Auctions with Individual Bidders

• In auctions where the bidders are individual consumers, valuations are often idiosyncratic

$5 for the bear

$100 for the bear

56

The Two Extremes

Hamlet: “What a piece of work is a man! How noble in reason! How infinite in faculties!”

Hamlet, II.2.319

Puck: “Lord, what fools these mortals be!”

Midsummer Night’s Dream, III.3.116

57

One Solution?

“It is evident that the rational thing to do is to be irrational, where deliberation and estimation cost more than they are worth.”

Frank Knight, Risk, Uncertainty and Profit, 1921

58

Example of Bounded Rationality

According to game theory, chess is a trivial game.

-finite, zero sum, two player game with full information

Then why do we find it so challenging to play and

why is it difficult to get computers to play well?