Raga GopalakrishnanUniversity of Colorado at Boulder

Sean D. Nixon (University of Vermont)Jason R. Marden (University of Colorado at

Boulder)

Stable Utility Design forDistributed Resource Allocation

Resource AllocationAllocate agents to

resources to optimize system-level objective

Wireless Frequency Selection

F1 F2 F3 F1 F2 F3

?

frequency

frequency

Wireless Access Point Assignment

?

Resource AllocationAllocate agents to

resources to optimize system-level objective

?

Sensor Coverage

Resource AllocationAllocate agents to

resources to optimize system-level objective

?

Sensor Coverage

Allocate agents to resources to optimize system-level objective

Distributed Resource Allocation

?

Sensor Coverage

Design local control policies for agents that result in desirable global behavior

(convergence to an allocation optimizing system-level objective)

Distributed Resource Allocation

Design local control policies for agents that result in desirable global behavior

(convergence to an allocation optimizing system-level objective)

Distributed Resource Allocation

Game Theoretic Control• Model agents as players in a non-cooperative

game• Equilibria correspond to stable allocations• Goal is to design the game such that

equilibria• exist (stability)• are efficient• are easy to converge to

UTILITY DESIGN (static)

LEARNING DESIGN (dynamic)

Formal Model• – set of agents• – set of resources• – action set of player • – joint action (allocation) set• – utility function of player

A pure Nash equilibrium (PNE) is an action profile such that for each player ,

Formal Model• – set of agents• – set of resources• – action set of player • – joint action (allocation) set• – utility function of player

DESIGN must be “scalable” independent of specific problem instance (resources, action sets)

A pure Nash equilibrium (PNE) is an action profile such that for each player ,

Formal Model• – set of agents• – set of resources• – action set of player • – joint action (allocation) set• – utility function of player • – global objective function

or “welfare”• Separability: • – local “welfare” generated

at resource

• – local “distribution rule” at resource

set of players

that chose

Formal Model• – set of agents• – set of resources• – action set of player • – joint action (allocation) set• – utility function of player • – global objective function

or “welfare”• Separability: • – local “welfare” generated

at resource

• – local “distribution rule” at resource

set of players

that chose

S1

S2

D1

D2

61 6

1

1

6

1?+?

Example: Network formation

S1

S2

D1

D2

61 6

1

1

6

13+3

A Nash equilibrium

Also optimal!

1+5

Unique Nash

equilibriumSuboptimal

Example: Network formation

Key feature:Distribution rules

outcome

?+?

S1

S2

D1

D2

61 6

1

1

6

1

Example: Network formation

Formal Model• – set of agents• – set of resources• – action set of player • – joint action (allocation) set• – utility function of player • – local “welfare” generated

at resource • – local “distribution rule”

at resource

UTILITY DESIGN DISTRIBUTION RULE DESIGN

Most prior work studies two distribution rules

Marginal Contribution (MC)[ Wolpert and Tumer 1999 ]

average marginal contribution over player

orderings

Shapley Value (SV)[ Shapley 1953 ]

externality experienced by all other players

Extensions: “weighted” versions parameterized by weights

Both guarantee PNE in all games!

Question: Are there other such distribution rules?Prior Work: NO, for any given welfare function.[G., Marden, Wierman 2013]

𝒇 𝒓𝑺𝑽 (𝒊 ,𝑺 )= ∑

𝑻⊆𝑺¿𝒊 }¿ ¿¿ ¿¿ 𝒇 𝒓

𝑴𝑪 (𝒊 ,𝑺 )=𝑾 𝒓 (𝑺 )−𝑾 𝒓 (𝑺¿ {𝒊¿})

Most prior work studies two distribution rules

Marginal Contribution (MC)[ Wolpert and Tumer 1999 ]

Shapley Value (SV)[ Shapley 1953 ]

Both guarantee PNE in all games!

Question: Are there other such distribution rules?Prior Work: NO, for any given welfare function.[G., Marden, Wierman 2013]Observation: Many practical problems involve “single-selection”: agents select a single resource.Question: Are there other such distribution rules if we only require equilibrium existence for all single-selection games?Our Answer: No, not for all welfare functions.

Single-Selection ScenarioPrior Work:• “Proportional share” distribution rule

guarantees PNE for certain types of coverage problems (certain forms of )

[Marden and Wierman 2013]

Our Results (characterizations):• The only linear budget-balanced distribution

rules that guarantee PNE in all single-selection games, for all welfare functions, are weighted Shapley values.

• Given any linear welfare function with no dummy players, the only budget-balanced distribution rules that guarantee PNE in all single-selection games are weighted Shapley values.

• Given any welfare function, the only budget-balanced distribution rules that guarantee PNE in all two-player single-selection games are weighted Shapley values.

[G., Nixon, Marden 2013]

Concluding Remarks

• Consequences of the restriction to weighted Shapley values:• Resulting game is a weighted

potential game for which several learning dynamics converge to PNE.

• It is hard for agents to compute their utilities.

• Open Problems:• Obtaining a tighter characterization of

stable distribution rules for a given welfare function.

• Obtaining the characterization when budget-balance is relaxed.

• Optimizing the “weights” for efficiency.

Ragavendran GopalakrishnanUniversity of Colorado at Boulder

Sean D. Nixon (University of Vermont)Jason R. Marden (University of Colorado at

Boulder)

Stable Utility Design forDistributed Resource Allocation