Computing Shapley Values, Manipulating Value Distribution Schemes, and Checking Core Membership in...

Computing Shapley Values, Manipulating Computing Shapley Values, Manipulating Value Distribution Schemes, and Checking Value Distribution Schemes, and Checking Core Membership in Multi-Issue DomainsCore Membership in Multi-Issue Domains

Vincent Conitzer and Tuomas SandholmVincent Conitzer and Tuomas Sandholm

Coalitional GamesCoalitional Games

Coalition formation is a key part of automated Coalition formation is a key part of automated negotiation between self-interested agentsnegotiation between self-interested agents Several of companies can unite into a virtual Several of companies can unite into a virtual

organization to take more diverse orders and gain organization to take more diverse orders and gain more profitmore profit

Truck delivery companies can share truck space, as Truck delivery companies can share truck space, as the cost is mostly dependant on the distance rather the cost is mostly dependant on the distance rather than on the weight carriedthan on the weight carried

Coalition formation has been studied extensively Coalition formation has been studied extensively in game theory, and solution concepts were in game theory, and solution concepts were adopted in multi agent systemsadopted in multi agent systems

Coalitional Games SolutionsCoalitional Games Solutions

Given a coalitional game we want to find the Given a coalitional game we want to find the distribution of the gains of the coalition between distribution of the gains of the coalition between the agentsthe agents

Different solution concepts have different Different solution concepts have different objectivesobjectives The Core promotes The Core promotes stabilitystability The Shapley value promotes The Shapley value promotes fairnessfairness

Game theory has studied these solution Game theory has studied these solution concepts for quite some time, but the concepts for quite some time, but the computational aspect has received little attentioncomputational aspect has received little attention

Some Questions to Keep in MindSome Questions to Keep in Mind

How much should each of the employees of the How much should each of the employees of the company be paid to make sure a group of them company be paid to make sure a group of them won’t be bought away by another company?won’t be bought away by another company? Get a value division in the coreGet a value division in the core

A few truck delivery companies unite to carry a A few truck delivery companies unite to carry a high load of deliveries. How can the profits be high load of deliveries. How can the profits be divided fairly?divided fairly? The Shapley value divisionThe Shapley value division

Coalitional Games With Side Coalitional Games With Side PaymentsPayments

The game is presented as a characteristic The game is presented as a characteristic functionfunction

Let A be the set of agents (players)Let A be the set of agents (players) Each potential coalition S has a value v(S)Each potential coalition S has a value v(S)

The value is independent of what the non members of The value is independent of what the non members of the coalition dothe coalition do

The characteristic function: The characteristic function: Typically it is increasing: Typically it is increasing:

: 2Av R

1 2 1 2( ) ( )S S V S V S

Super additivity Super additivity

The characteristic function is The characteristic function is super additivesuper additive if for if for all disjoint sets of a S,T we have: all disjoint sets of a S,T we have:

This means every two subsets can do better if This means every two subsets can do better if they unitethey unite

Finally we would get the grand coalition of all the Finally we would get the grand coalition of all the agentsagents

This does not always hold:This does not always hold: Hard optimization problem to decide what to do unitedHard optimization problem to decide what to do united Anti trust lawsAnti trust laws

1 2 1 2( ) ( ) ( )v S v S v S S

Solution ConceptsSolution Concepts

On a super additive game, the grand coalition is On a super additive game, the grand coalition is likely to form, and the coalition gets v(A)likely to form, and the coalition gets v(A)

How much does each agent gets?How much does each agent gets? We want a value division We want a value division We want to divide all the gains: We want to divide all the gains:

:d A R( ) ( )

a A

d a v A

The CoreThe Core

The best known solution conceptThe best known solution concept Proposed by Gillies (1953) and von Neumann & Proposed by Gillies (1953) and von Neumann &

Morgenstein (1947)Morgenstein (1947) A value division is in the core if no sub coalition A value division is in the core if no sub coalition

has an incentive to break awayhas an incentive to break away A value division d is blocked by a sub coalition S A value division d is blocked by a sub coalition S

if if If d is blocked by S, it is not in the coreIf d is blocked by S, it is not in the core Some coalitional games have an empty coreSome coalitional games have an empty core

( ) ( )a S

v S d a

Player TypesPlayer Types

Dummy playersDummy players add nothing to all add nothing to all coalitions: coalitions:

Equivalent playersEquivalent players add the same to any add the same to any coalition that does not contain any of the coalition that does not contain any of the two players:two players:

( { }) ( )v S a v S

1 2 1 1: { , } ( { }) ( { })S S a a v S a v S a

The Shapley Value (Cont.)The Shapley Value (Cont.) A well know value division schemeA well know value division scheme Aims to distribute the gains in a fair mannerAims to distribute the gains in a fair manner A value division that conforms to the set of the A value division that conforms to the set of the

following axioms:following axioms: Dummy players get nothingDummy players get nothing Equivalent players get the sameEquivalent players get the same If a game v can be decomposed into two sub games, If a game v can be decomposed into two sub games,

an agent gets the sum of values in the two games:an agent gets the sum of values in the two games:

Only one such value division scheme existsOnly one such value division scheme exists

: ( ) ( ) ( )

: ( ) ( ) ( )V U W

S v S u S w S

a d a d a d a

The Shapley ValueThe Shapley Value

Given an ordering of the agents in A, we Given an ordering of the agents in A, we define to be the set of define to be the set of agents of A that appear before a inagents of A that appear before a in

The Shapley value is defined as the The Shapley value is defined as the marginal contribution of an agent to its set marginal contribution of an agent to its set of predecessors, averaged on all possible of predecessors, averaged on all possible permutations of the agents:permutations of the agents:

1( , ) ( ( ( , ) ) ( ( , )))

!Sh A a v S a a v S a

A

( , )S a

A Simple Way to Compute The A Simple Way to Compute The Shapley ValueShapley Value

Simply go over all the possible Simply go over all the possible permutations of the agents and get the permutations of the agents and get the marginal contribution of the agent, sum marginal contribution of the agent, sum these up, and divide by |A|!these up, and divide by |A|!

Extremely slow Extremely slow Can we use the fact that a game may be Can we use the fact that a game may be

decomposed to sub games, each decomposed to sub games, each concerning only a few of the agents?concerning only a few of the agents?

Computing the Shapley ValueComputing the Shapley Value

If v can be decomposed to several sub If v can be decomposed to several sub games, we know (from the axioms) thatgames, we know (from the axioms) that

If only concerns then for any If only concerns then for any player a, we haveplayer a, we have

1

( , ) ( , )i

T

v vi

Sh A a Sh A a

iv iC A

1( , ) ( ( ( , ) ( ( , )))

!i

i

v i i i iCi

Sh A a v S C a v S C aC

Computing the Shapley ValueComputing the Shapley Value

We do not really need to sum over all possible We do not really need to sum over all possible orderings, but rather on all possible subsets of orderings, but rather on all possible subsets of agents that arrive before player aagents that arrive before player a

For each such sub set we get the same marginal For each such sub set we get the same marginal contribution of player a. contribution of player a.

If the sub set S has n agents, there are n! If the sub set S has n agents, there are n! ordering on the players inside. There are then (|ordering on the players inside. There are then (|A|-n-1)! ways to complete this ordering to an A|-n-1)! ways to complete this ordering to an ordering on all agents. We get:ordering on all agents. We get:

{ }

1( , ) !( 1)!( ( { }) ( ))

! S A a

Sh A a S A S v S a v SA

Computing the Shapley Value Computing the Shapley Value Quickly in Multi Issue DomainsQuickly in Multi Issue Domains

Compute the Shapley value for each sub Compute the Shapley value for each sub game, using the previous formula, only game, using the previous formula, only taking into account the concerning agents, taking into account the concerning agents, then sum these upthen sum these up

If we assume computation of factorials, If we assume computation of factorials, multiplication and addition in constant time multiplication and addition in constant time we get an time complexity of or we get an time complexity of or less precisely less precisely

1

( 2 )iT

C

i

O

max( 2 )i iCO T

Marginal Contribution Based Value Marginal Contribution Based Value Division SchemesDivision Schemes

A marginal contribution scheme is a scheme that A marginal contribution scheme is a scheme that chooses some ordering of the players, and chooses some ordering of the players, and distributes the gains to the players according to distributes the gains to the players according to their marginal contributiontheir marginal contribution

If on the chosen orderings you add much to the If on the chosen orderings you add much to the value of the coalition of the players before you value of the coalition of the players before you on the ordering, you deserve a nice share of the on the ordering, you deserve a nice share of the profitsprofits

We didn’t go beyond this point in class, but We didn’t go beyond this point in class, but I left the rest of the slides in case someone I left the rest of the slides in case someone was interested.was interested.

Marginal Contribution Based Value Marginal Contribution Based Value Division SchemesDivision Schemes

For the Shapley value we have considered For the Shapley value we have considered an average on all possible ordersan average on all possible orders

If we consider just one of the possible If we consider just one of the possible orderings, the value an agent gets orderings, the value an agent gets depends on it location in the orderingdepends on it location in the ordering Obviously, the agent has a specific location it Obviously, the agent has a specific location it

wants to be inwants to be in If the game is convex (you add to a coalition If the game is convex (you add to a coalition

at least as much as you add to any of its at least as much as you add to any of its subsets), you want to be last in the orderingsubsets), you want to be last in the ordering

Marginal Contribution Based Value Marginal Contribution Based Value Division Schemes (Cont.)Division Schemes (Cont.)

If we randomly choose a permutation the If we randomly choose a permutation the expectancy of the value distribution for an agent expectancy of the value distribution for an agent is its Shapley valueis its Shapley value This requires a trusted source of randomness / This requires a trusted source of randomness /

cryptographycryptography Another way is to show that even if an agent has Another way is to show that even if an agent has

total control on the ordering chosen, it would still total control on the ordering chosen, it would still be be computationally intractablecomputationally intractable for that agent to for that agent to find the optimal ordering for himfind the optimal ordering for him

The computational complexity is used as a The computational complexity is used as a barrier for manipulationbarrier for manipulation

Maximal Marginal ContributionMaximal Marginal Contribution

Let v be a game decomposed as follows:Let v be a game decomposed as follows: and the game only concerns and the game only concerns

We are given an agent a and a number k, and We are given an agent a and a number k, and are asked if there is some are asked if there is some such that such that the value the value

We want to see if we can find a subset of the We want to see if we can find a subset of the agents to which a’s marginal contribution is at agents to which a’s marginal contribution is at least kleast k These would be the agents before a in the ordering a These would be the agents before a in the ordering a

would choosewould choose

1

T

ii

v v

iv iC A

{ }S A a ( { }) ( )v S a v S k

NP-Completeness of Max-NP-Completeness of Max-Marginal-ContributionMarginal-Contribution

Conitzer and Sandholm have shown that Conitzer and Sandholm have shown that Max-Marginal-Contribution is NP-Max-Marginal-Contribution is NP-Complete, even in the case that Complete, even in the case that and all ‘s take values in {0,1,2}and all ‘s take values in {0,1,2}

The problem is in NP since for a given The problem is in NP since for a given subset of agents subset of agents we can simply we can simply calculate the marginal contribution of a to calculate the marginal contribution of a to this subsetthis subset

3iC

iV

{ }S A a


NP-hardness is proven by reducing an arbitrary NP-hardness is proven by reducing an arbitrary MAX2SAT instance to a Max-Marginal-Contribution MAX2SAT instance to a Max-Marginal-Contribution instanceinstance

In MAX2SAT we are given a set V of Boolean variables In MAX2SAT we are given a set V of Boolean variables and a set of clauses C, each with 2 literals, and a target and a set of clauses C, each with 2 literals, and a target number r of satisfied clausesnumber r of satisfied clauses

For each variable v in V there is an agent AvFor each variable v in V there is an agent Av We also have an agent a, whose contribution we want to We also have an agent a, whose contribution we want to

maximizemaximize For every clause c there is a sub game (issue) tc, that For every clause c there is a sub game (issue) tc, that

only concerns the agents a and the agents representing only concerns the agents a and the agents representing the variables in the clause cthe variables in the clause c


The sub game results are as follows:The sub game results are as follows: 1 point for having a in the coalition1 point for having a in the coalition 1 point for having 1 point for having all all the agents representing the the agents representing the

negativenegative literals literals But, But, if you want to get 2 points, you also have to have if you want to get 2 points, you also have to have

one of the agents representing the one of the agents representing the positivepositive literals literals

The marginal contribution we want is k=rThe marginal contribution we want is k=r


If there is a solution to MAX2SAT with r satisfied If there is a solution to MAX2SAT with r satisfied clauses, take V+ - the variables set to trueclauses, take V+ - the variables set to true

What is the marginal contribution of a to this subset?What is the marginal contribution of a to this subset? Hint: you either satisfied the clause by setting one of the Hint: you either satisfied the clause by setting one of the

negative literals to false, or if you didn’t, you’ve set one of the negative literals to false, or if you didn’t, you’ve set one of the positive literals to truepositive literals to true

Given a solution S to max-marginal-contribution, look at Given a solution S to max-marginal-contribution, look at the assignment of true to everything in S, false otherwisethe assignment of true to everything in S, false otherwise

If a sub game tc has increased the value by 1 due to adding a, If a sub game tc has increased the value by 1 due to adding a, what can you say about the clause?what can you say about the clause?

Open question: we have used increasing games here, so Open question: we have used increasing games here, so the problem is NP-Complete even if the game is known the problem is NP-Complete even if the game is known to be increasing. What is the complexity for super to be increasing. What is the complexity for super additive games?additive games?

Checking Core MembershipChecking Core Membership

Let v be a game decomposed as follows:Let v be a game decomposed as follows: and the game only concerns and the game only concerns

We are given a value division We are given a value division that that may not even be feasiblemay not even be feasible If it isn’t we can increase only the value of the If it isn’t we can increase only the value of the

grand coalition to the point where it is (the grand coalition to the point where it is (the help of an outside benefactor for the stability)help of an outside benefactor for the stability)

We are asked if the division is in the core, We are asked if the division is in the core, or if there is no blocking sub coalition for itor if there is no blocking sub coalition for it

1

T

ii

v v

iv iC A

:d A R

NP-Completeness of Checking NP-Completeness of Checking Core MembershipCore Membership

Conitzer and Sandholm have shown that Conitzer and Sandholm have shown that checking core membership (CHECKE-IF-checking core membership (CHECKE-IF-BLOCKED) is NP-CompleteBLOCKED) is NP-Complete

The problem is in NP since for a given The problem is in NP since for a given subset of agents we can simply calculate subset of agents we can simply calculate the sum of their values in the division and the sum of their values in the division and see if it is less than v(S)see if it is less than v(S)


NP-hardness is proven by reducing an arbitrary VERTEX-COVER NP-hardness is proven by reducing an arbitrary VERTEX-COVER instance to a core membership probleminstance to a core membership problem

We have an agent for each vertex, av, and another special agent aWe have an agent for each vertex, av, and another special agent a We have a sub game for each edge, that only concerns agent a and We have a sub game for each edge, that only concerns agent a and

the agents of the edge’s verticesthe agents of the edge’s vertices The value of the sub game is 1 if the coalition contains agent a and The value of the sub game is 1 if the coalition contains agent a and

at least one of the edge’s vertices (we have agent a, and the edge is at least one of the edge’s vertices (we have agent a, and the edge is covered)covered)

The value distribution to check: The value distribution to check: 1

( )21

( )1

2( )2

v

d a E

d ar


If there is a vertex cover with W verticesIf there is a vertex cover with W vertices What is the value of the coalition of these vertices and agent a?What is the value of the coalition of these vertices and agent a? How much do they get according to the value distribution?How much do they get according to the value distribution?

If a set of agents is a blocking coalitionIf a set of agents is a blocking coalition It has to contain agent a (or they get nothing)It has to contain agent a (or they get nothing) Consider the set of vertices of the agents in the blocking Consider the set of vertices of the agents in the blocking

coalition, Wcoalition, W How much do they get according to the value distribution?How much do they get according to the value distribution? Can the number of vertices in W be smaller than r?Can the number of vertices in W be smaller than r? To block, v(S) must be greater than v(a), since a is in the To block, v(S) must be greater than v(a), since a is in the

blocking coalitionblocking coalition But then we have to get 1 for every sub game, so we have But then we have to get 1 for every sub game, so we have

covered all the edges, with r vertices or lesscovered all the edges, with r vertices or less

ConclusionsConclusions Coalitional games important for automated negotiation Coalitional games important for automated negotiation

between agentsbetween agents Such games can be decomposed to sub games (issues) Such games can be decomposed to sub games (issues)

which only concern some of the agentswhich only concern some of the agents We can quickly compute the Shapley value in some of We can quickly compute the Shapley value in some of

these casesthese cases Other marginal contribution value distribution schemes Other marginal contribution value distribution schemes

can be manipulated, but the general case is hard (an can be manipulated, but the general case is hard (an NP-complete problem)NP-complete problem)

So such distribution schemes are acceptable in some cases, So such distribution schemes are acceptable in some cases, even if some of the agents have control on the chosen orderingeven if some of the agents have control on the chosen ordering

Checking if a value distribution is stable (in the core) is Checking if a value distribution is stable (in the core) is hard (and NP-Complete problem in the general case)hard (and NP-Complete problem in the general case)

Open QuestionsOpen Questions

NTU games (no side payments)NTU games (no side payments) Finding value divisions the are even harder to Finding value divisions the are even harder to

manipulate (eg. PSPACE-hard)manipulate (eg. PSPACE-hard) Finding stability concepts that take into account Finding stability concepts that take into account

the complexity of finding a beneficial deviationthe complexity of finding a beneficial deviation The complexity of other (longer term) solution The complexity of other (longer term) solution

conceptsconcepts

Computing Shapley Values, Manipulating Value Distribution Schemes, and Checking Core Membership in...

Documents

Transcript of Computing Shapley Values, Manipulating Value Distribution Schemes, and Checking Core Membership in...