Coalition Formation between Self-Interested Heterogeneous

Actors

Arlette van Wissen

Bart Kamphorst

Virginia Dignum Kobi Gal

Introduction

Allowing people to interact with computer systems in a manner

similar to the way that people interact with each other.

To achieve successful interactions, we must understand the factors involved.

This may provide new insights into:- how decisions are made- how agent’s behaviors are perceived

Types of interactions:- cooperative and helpful interactions- competitive interactions- cooperative interactions in competitive scenarios

Human-Agent Interaction

Introduction

utility functiona real number that represents the level of satisfaction of an actor in that state of the world.

non-cooperative games - self-interested actors who can make non-binding

commitments - basic modeling unit is the individual

cooperative games- players can make binding commitments- communication and negotiation between the players is allowed- groups of players (coalitions) may enforce cooperative behavior

- basic modeling unit is the group

Self-Interested Actors

Introduction

A coalition is a subset S of the set of players N where:

1. agreements take place involving approval by every player in S 2. no agreement between any other members of S and any

member of N – S is permitted

A coalitional game with transferable utility is a pair (N, v) where

1. N is the set of all players 2. v associates with each coalition S ⊆ N a real-valued payoff v(S)

that the coalition’s members can distribute among themselves. We assume that v(∅) = 0.

Coalition Formation

Related Work

To become part of successful coalitions, one has to deal with the following

problems (Gentry, 1987):

1. how to identify the members of a winning coalition 2. how to reach an agreement on the allocation of the payoffs3. how to make a tradeoff between payoffs and non-utilitarian

strategy preferences

Humans are not fully rational when it comes to these formation decisions.

Coalition Formation

Related Work

Ultimatum Game

Allocation of Payoff

Related Work

Ultimatum Game

Humans are not completely rational actors (Kagel & Roth, 1995)

Social factors influence human behavior:• Inequity aversion (Loewenstein, 1989)

• Intention (Ross & Fletcher, 1985)

• Altruism, fairness, reciprocity (Camerer, 2003)

Social factors also influence cooperation between humans and intelligent agents (van Wissen et al., 2009)

Allocation of Payoff

Related WorkHuman Coalition Formation

• Relative payoffs clearly matter in the formation of coalitions. There exists a strong correlation between relative payoffs and rejection behavior. (Bolton & Brosig, 2007)

• In 2- and 3-player coalition games, people respond by acting negatively reciprocally. They punish proposers by rejecting unfair offers. (Okada & Riedl, 2005)

• Less attention paid to human-agent interaction.

• Existing models generally do not consider trust or nature.

Related WorkTrust

For an actor a to be said to trust another actor b with respect to a particular goal g, a must have the following beliefs (castelfranchi, 1998, 2001) :

• Competence Belief: b is useful for achieving g and is able to provide the expected result • Disposition Belief: b is not only capable, but also willing to do what is necessary to achieve g • Dependence Belief: the results and rewards of achieving g depend on the involvement of b • Fulfillment Belief: g will come about due to b’s involvement

Related WorkTrust

experience based vs recommendation based?

Research Question

How do nature and trust influence people’s decisions

in mixed coalition formations?

To what extent do trust and fairness influence team formation in mixed networks?

How does the nature of actors affect the way people relate to their actions?

Do actors develop stable relations over time?

Hypotheses

Hypothesis 1

The nature of the actor affects the cooperation between participants.

Hypothesis 2

A trust relationship between the actors affects the cooperation between them.

Conceptual Design

Assumptions in CF

Groups

Prev work

The members of coalitions are predefined.

Our work

The members of a coalition can be freely chosen.

Interchangeability

Prev work

All potential members bring the same utility to a coalition.

Our work

Some potential members are preferred over others.

Membership

Prev work

Membership to a coalition is equally available to everyone.

Our work

Members have to meet certain requirements for membership to be available to them.

Conflict

Prev work

Conflict is eliminated by making agreements binding.

Our work

No conflict-free environment since agreements are not binding.

Conceptual Design

Examine interactions between self-motivated agents that must adapt their behavior depending on the behavior of other agents in the environment.

Package Delivery Problem

Experimental Design

``A key determinant of CT design was the goal of providing a vehicle for comparing the decision-making strategies people deploy when they interact with other people with those they deploy when computer systems are members of their groups.’’Grosz et al., 2004

Colored Trails

- Board

- Chips

- Proposals

- Phases

Experimental DesignPackage Delivery Problem in CT

Conceptual Design

Hypothesis 1The nature of the actor affects the cooperation between participants.

Deception

Hypothesis 2A trust relationship between the actors affects the cooperation between them.

Repeated interaction

Hypotheses revisited

Transferable payoff &

ability to choose team members

Experimental DesignPackage Delivery Problem in CT

Coalition formation game with:• N = 6

• v(Ni) = 3

v(Ni Nj) = 60

v(Ni Nj Nk) = 180

• superadditivity:A game G = (N,v) is superadditive if v(ST) ≥ v(S) + v(T)

for all S, T ∈ N such that S ∩ T = ∅

• imperfect information:players do not have knowledge of all actions and behavior of the players in the game and have partial visibility

• 5 rounds of t minutes, where 7 ≥ t ≤ 10

Experimental DesignPackage Delivery Problem in CT

- 6 players per game (P)- 3 colors (=P/2)

Experimental DesignPackage Delivery Problem in CT

- 6 players per game (P)- 3 colors (=P/2)

Experimental DesignPackage Delivery Problem in CT

- 6 players per game (P)- 3 colors (=P/2)- 6 large packages (=P) - 12 small packages (=2P)

Experimental Design

movie

Experimental Design

1. Survey

- preferences - strategies

2. Log

- proposals- coalitions- defections

Results

Results

Hypothesis 1The nature of the actor affects the cooperation between

participants. • Do players make more proposals to humans than to agents?• Do players offer more points (more fair splits) to humans than to agents?• Do players join more coalitions with a human as an initiator than with an

agent as initiator?

Hypothesis 2A trust relationship between the actors affects the cooperation

between them.• Do players who defect less obtain a higher score than those who defect

more?• Do initiators who propose more fair splits obtain a higher score than those

who propose less fair splits?• Do players who are more often initiator obtain a higher score than those

who are more often member?• Do players offer less points to those who defect more than to those who

defect less?

Results

What strategy did you use for choosing your team members?

``Whether members were reliable. I could give computers a smaller share without feeling guilty. I did build a good history with one other human player.’’

``I chose the computers mostly, since I thought that they would demand less points.’’

What strategy did you use for proposing a split of points?

``Split evenly with humans, keep more for myself when playing with computers.’’``Offered computer players less points, human players more.’’

How did your preference for humans/computers as a leader reflect in your strategy of joining and defecting from teams?

``I tended to feel worse about defecting from a human.’’

Discussion & Future Work

1. Write BDI agents based on a model of human decision making in this scenario (CTAPL framework).

2. Create a condition where the emphasis is on reciprocity: helping without direct payoff.

3. Have people play longer rounds to enable them to develop better trust relations.

Conclusion

Questions/Suggestions?

Game FlowInitiator

Game FlowInitiator

Game FlowMember

Game FlowMember