The Wisdom of Networked Agentsnama/Top/InvitedTalk/07-05_it.pdfcoupled network Add a link, with...

0 - ETH Colloquium ‘07 (Zurich)

The Wisdom of Networked Agents

Akira NamatameNational Defense Academy of Japan

www.nda.ac.jp/~nama


Low

Low

High

High

Scale

Multi-agentsystems

Socio physics(Complex networks)

M&S in social systems

Game theory

Self-interest seeking Adaptability

RESEARCH MAP

Social atom:Not observe individual behaviorbut observe patterns

Complex adaptive systems

<particle> <agent>


Networked worlds:

Everything is connected!

Two phases in networks(1) Phase with positive network effects

: More people means more benefits

(2) Phase with negative network effects

: More persons begin to decrease the value of a network result from resource limits

(traffic congestion, network service overloads)

Why Do Things Get Worse?


Outline

• Social Interaction with Externalities• Consensus Problems on Complex Networks　: Flock of moving agents• Social Choice as Consensus Formation• Collective Decisions with Externalities

: Sequential Decisions and Cascade: Repeated Games on Networks

• Congestion Control of Networked Agents


• Crowds are often foolishHuman beings loose their rationality in a crowd

Crowd psychology: herding, cascade, group think,…

Madness of Crowds


A large collection of people are smarter than an elite few. J. Surowiecki,(2004) suggests new insights regarding how our social and

economic activities should be organized.: The wisdom of crowds emerges only under the right conditions

(diversity, independence, etc)

The Wisdom of Crowds

Dual Phases in Collective Behaviour

Disconnected agents Connected agents

We study interaction and the underlying network topology that flip between two phases

: Under what mechanism can we improve collective behaviour?: Under what mechanism can we improve collective behaviour?

: The key point is to characterize : The key point is to characterize interactionsinteractions among agentsamong agents


Types of Social Interaction (1)

Type 1: Coordination problems: Consensus problem (control theory): Synchronization (physics/complex networks)

　　: Herding (economics/psychology)　　: Social choice (social sciences)　　: Coordination games (game theory)

Type 2: Dispersion problems: Congestion control (engineering) : Stock markets (economics): Minority games (econophysics)


Mixture of coordination and dispersion problems(1) Economics:Tug-of-war between increasing returns and decreasing returns

(2) Social sciences: Reconcile the tension between centripetal and centrifugal

(population dynamics, city development)

(3) Human behaviors: Mixture of conformists and non-conformists: Tension between rational agents and irrational agents

Types of Social Interaction (2)


Outline

• Social Interaction with Externalities• Consensus Problems on Complex Networks

: Flock of moving agents• Social Choice as Consensus Formation• Collective Decisions with Externalities




Consensus Problems

Consensus means to reach an agreement regarding a certain quantity of interest that depends on the state of all agents. A consensus algorithm is a decision rule that results in the convergence of the states of all network nodes to a common value.

[01]: Olfati-Saber 2007

Source: Olfati-Saber 2007 [C1]

xi = xj = …= xconsensus

Convergence of the states of all agents to a common value


The Role of Network Topology in Consensus Problems

The distributed algorithm for consensus problems

The weighted adjacency matrix G=(wij)

(i) Graph G is connected(ii)G is balanced: symmetric graph ∑∑ ≠≠

=ij jiji ij ww

))()(()()1( txtxwtxtx iNi

jijiii

−+=+ ∑∈

ε

nxxxxi in /)0(...21 ∑====

Thorem: (A. Jadbaie, 2006)The algorithm converges to the average of the initial values ofall agents if the underlying graph G is connected and balanced

Agents adjust the states to those of their neighbors.


Synchronization Problems

“Emergence of flocking behavior”

Vicsek T,.Phys Rev Letter (1995)

““Consensus has connections to synchronization problemsConsensus has connections to synchronization problems””


Synchronization in Globally Connected Networks

Observation:Observation:No matter how large the network is, a globally coupled network will synchronize if its coupling strength is sufficiently strong

Good – if synchronization is useful

G. Ron Chen (2006)G. Ron Chen (2006)


Synchronization in Locally Connected NetworksSynchronization in Locally Connected Networks

Observation:Observation:No matter how strong the coupling strength is, a locally coupled network will not synchronize if its size is sufficiently large

Good - if synchronization is harmful

G. Ron Chen (2006)G. Ron Chen (2006)


SynchronizationSynchronization in Small-World Networks

Start from a nearest neighbor coupled network

Add a link, with probability p, between a pair of nodes

Good news

X.F.Wang and G.R.Chen: Int. J. Bifurcation & Chaos (2001)

: : A small-world network is easy to synchronize!!

small-world networkG. Ron Chen (2006)G. Ron Chen (2006)


Synchronization and Underlying Network Topology

λ1 = 0 is always an eigenvalue of a Laplacian matrixλN/λ2：algebraic connectivity is a good measure of synchronization.

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−

−

nk

kk

}1,0{

}1,0{

2

1

Oλ2 = 0.238 λ2 = 0.925

Laplacian matrix = Degree – Adjacency matrix

Laplacian matrixNetwork A Network B

Connectivity of networks determines synchronization

Fiedler, “Algebraic connectivity of graphs,”

Czechoslovak Mathematical Journal, 1973, 23: 298-305


A B

Close, but not too closefar away

too close

Flock: Mixture of Coordination and Dispersion


Flock with BOID Model

Craig W. Reynolds (1994)

Cohesion Separation Alignment

•Cohesion: Head for the perceived center of mass of the neighbors. •Separation: Don't get too close to any object.•Alignment: Try to match the speed and direction of nearby neighbors.


Vcs

0

0.2

0.4

0.6

0 1 2 3 4 5

D

Emergence of Flock with Self-Control Rules

Self-control by minimizing the implicit potential function

cohesion, separation

VaDs

caisicifi eweDwwFFFF rrrrrr

+⎟⎠⎞

⎜⎝⎛ −=++=

DwDw scfi log−=φ

alignment

c

s

ww

<Force of an agent>

c

sn

jij w

wdDti

→=⇒∞→ ∑r

0→=⇒∞→ ∑in

jijvVt r

∞→tBalance between attractiveforce and repulsive force

Consensus: converge to the same velocity


Constrained Flock (1)

Force of flocking movement:Force of moving for destination: invoked with some probability p:

fFr

dFpr

flocking movement force

totalFr

dFpr

fFr

Flock moving toward a given destination

combined force: dftotal FpFFrrr

+=

force for destination


Constrained Flock (2): Avoiding an Obstacle

with probabilistic invoke: p=0.15

Break Away

without probabilistic invoke: p=1 Dead Lock

destination

dftotal FpFFrrr

+=

critical parameter value: pc=0.15


Phase Transition in Network Topology:Connection Probability

p = 10-4 p = 10-3

p: connection probability to others (remote link)

disconnected agents connected agents

critical parameter value: pc= 10-3


Triplets Phases in Network Topologies

random graph small-world graph

random remote link

local links

densely connected

connected graph

random remote link

disconnected agents agents in flock


p

p

p

d: Average links (degree)

dmax : N-1 (complete graph)

Phase Transition in Flocking: Substrate Networks

complete graph

small-world type

random graph

connection probability (remote link)

no flocking

Flocking is emerged


Outline






The Stock Market as Beauty Contest

　Keynes remarked that the stock market is like a beauty contest. “General theory of Employment Interest and Money”. (1936)

: Keynes had in mind contests that were popular in England at the time, where a newspaper would print 100 photographs, and people would write in which six faces they liked most.

: Everyone who picked the most popular face was automatically entered in a raffle, where they could win a prize.


Beauty Contest as Social Choice•There is the set of alternatives to be ranked.•Each member has his/her preference. •The society have to decide ordering on all alternatives, which is the best, the second best,and so on.

Preference aggregationPreference

aggregation

Individual preferences


Social Choice with Voting

Voting is the archetypical form of making a social decision.

Preference aggregationAggregate individual preferences on the set of alternatives to obtain a “social ordering”.

KENNETH J. ARROW (1951): No voting scheme over three or more alternatives can satisfies all of reasonable and logical conditions.


Paradox of Voting: Condorcet (1785)Binary choice: No problem with votingHow to extend the idea with more than two choices?

➭The problem known as “paradox of voting.”:Three candidates: {a, b, c}, Preference of three voters : {A, B, C} A: B: C:

Group choice:

α

cb ffα acb ff bac ff

acb fffα

a b c


Borda Count (1770)

Each voter submits a complete ranking of all the m candidatesFor each voter that places a candidate first, that candidate receives m-1 points, for each voter that place her 2nd place receives m-2 points, and so forth.The candidate with the highest Borda count wins

Preference of three voters

cba ff

acb ff

bac ff

Borda count:

a: 6, b: 6, c: 6

(paradox of voting)


Forward and Inverse Problems in Social Systems

preference adaptation

Forward problem

Aggregatedpreference


<Inverse problem>

How agents preferences should be adapted for better social choice?

Inverse problem

<Forward problem>

How social choice should be maid by aggregating individual preferences?

Agents specify complete preferencesAggregate all preferences and announce it all agentsAgents modify original preferences


Group Preference and Indexing with Borda Count

Group preference

C 2

C nC 1

preference

O α

Oβ

(Oα is preferred to Oβ)

βα

βα

OOthenn

OC

n

OCif

n

i

in

i

i

f )()(

1

)(

1

)( ∑∑== <

Each agent has an ordered list of alternatives

Compute the rank of the alternative

Rank order the alternative according to the decreasing sum of their ranks

O2

O 1

O4

O3

O5

10000

11000

11010

10100

11001

Preference index

54321 ,,,, ooooo


Consensus Formation of Adaptive Agents

α

: aggregated preference of choice j (borda score)

: preference of agent i on choice j.

: adaptation level (social influence )

)()()1()( ααα OGOCOC jiji +−=

)( jOG

)( ji OC

Adaptive model of Individual preference:

“Agents concern both own preferences and group preference”

Preference modification



)( jOG

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

2

1

3

5

4

5

1

4

3

2

Agen t 1 Agen t 5Agen t 4Agen t 3Agen t 2

Agent preferences (paradox of voting occurs)

Agent1:0.9

Agent5:0.1

Agent4:0.3

Agent3:0.5

Agent2:0.7

1 2 3 4 5

1

2

3

4

5

α of A gent

Initial group preference

derived group preference

Simulation Setting

Agent1:0.1

Agent5:0.1

Agent4:0.1

Agent3:0.1

Agent2:0.11 2 3 4 5

α of A gent Initial group preference

derived group preference

1 2 3 4 5

Group B: adaptation levels are high and different

Consensus was not reached

}5,...,2,1:{ : }5,...,2,1:{ :

====

iOWesalternativFiveiAGagentsFive

i

i

Group A: adaptation level is low and the same

Consensus was reached

(Borda count of each alternative)

(Each score of alternatives are same. =Ordering of alternatives was fail. )

Group A: Consensus was not reached

Group B: Consensus was reached. 43215 ooooo ffff


Fallacies of Composition and Division

: The fallacy of composition and the fallacy of division involve parts and wholes.

: The fallacy of composition occurs when we assume that a whole has a property that it's parts do.

: The fallacy of division runs in the opposite direction. It occurs when we assume that the parts of a thing have the sameproperties as the whole.

: The fallacies arise because wholes often have emergent properties that their parts lack, and parts often have adaptive properties that do not belong to the whole that they constitute.


Outline






Social Networks as Complex Systems

• Interactions are more important than individual behaviour

• How do interactions influence rational behaviour?

<madness of crowds>People follow the herdEasier to follow than to thinkIf one does it, others followFashionsPanic in emergenciesPeer pressureThe snowball effect


Factors that Influence on Social Complex Systems

Social pressureAGENT

Social network changes an agent’s behaviour

(1) Preference heterogeneity(2) Social influence (3) Social network: Degree heterogeneity

Preference

Preference determines an agent’s behaviour

Social pressure influences an agent’s behaviour

Social network


Restaurant A

Restaurant B

Which restaurant should I choose ?

How other peoples have chosen?

Agents make decisions in order

agents who prefer A

agents who prefer B

Sequential Decisions and CascadeDecision externalities


Binary Choice under Social Influence (1)

0

1.2

-6 -4 -2 0 2 4 6

q

UA – UB

SellBuy

Probability to buy:　p1

•Probability to choose A

　　　q = 1 / (1+exp(-(UA -UB)))

•Probability to choose B

1-q = 1 / (1+exp(-(UB- UA)))

Logit Model

P(t+1): the probability to choose A at time t+1

)()()1()1( tSUqtp αα +Δ−=+

S(t):the proportion of the agents who have chosen S1 by t

10 ≤≤ α

Utility UA Utility　UB

α :social influence level

social influenceindividualpreference


0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

α

σ2 /N mean

min

max

agent number：1000initial condition: M(0)=N(0)=1

Collective Behavior of Conformists: Agent Preference vs. Social Pressure

Cascade level

∑=

−=N

ttS

1

22 ))(( μσ

social influence level

S(t)=M(t)/{(M(t)+N(t)}: the ratio of agents who have chosen A by time t

:A half of agents prefer A to B and the rest of agents prefer B to A

5.0=μ

Cascade occurs under strong social influence

)()()1()1( tSUqtp αα +Δ−=+


Agent Heterogeneity and Cascade

Conformist: Prefers the majority choice (a longer queue)

Restaurant A

Restaurant B

Non-conformist: Prefers the minority choice (a shorter queue)

)()()1()1( tSUqtp αα +Δ−=+

))(1()()1()1( tSUqtp −+Δ−=+ αα

A mixed population of conformists and non-conformists

Balance between centripetal (conformist) and centrifugal (non-conformist)

social influence invokes the opposite direction


∑=

−=N

t

tS1

22 ))(( μσ

Collective Decision: A Mixed Population of Conformists and Non-conformists

social influence level

ratio of non-conformist

: Large cascade caused under strong social influence(However strong individual preference mitigates cascade): Non-conformists stabilize collective decision: Cascade does not occur when 20% of agents are non-conformists(Cascade is mitigated by a small fraction of irrational agents)

αθ

Cascade level


Outline


:Flock of Moving Agents• Consensus Formation

: Social Choice of Networked Agents • Collective Decisions with Externalities

: Sequential Decisions and Cascade: Repeated Games on Complex Networks



Beauty Contest Games

<Payoff scheme>

Coordination game: An agent wins if his choice is the same as the majority choice

Dispersion game: An agent wins if his choice is the same as the minority choice

1− θ

θ

An agent bits one unit by splitting 1− θ on S1 and θ on S2

S1

S2

S1 S2

S1 0S2 0

θ−1

θ

S1 S2

S1 0S2 0

θ−1θ

Coordination game Dispersion game

47 - ETH Colloquium ‘07 (Zurich)47

Simulation Setting: Majority GameAgent preference level: θ

Agent heterogeneity: the density of θ: f(θ)

Aggregated choicep

5.0<θ5.0>θ

： prefer S1 ： prefer S2

θ>)(tp

θ<)(tp： choose S1 ： choose S2

An agent’s best-response rule

p: the proportion of agents to choose S1

S1 S2

S1 0S2 0

θ−1

θCoordination game


Collective Dynamics with Hub Agents

Impact of hub agentsPeer influence

symmetric interaction (degree) asymmetric interaction (degree)


Social Interaction ModelsHub Model

Hub Model: Collective decision with opinion leaders (hub agent): Hub agent i adapts to the aggregated information of subgroup i: All other agents adapt to 4 local neighbors as well as the hub agents

Hub agent 1 Hub agent 2

Aggregate Information

Global model:(mean-filed model)

Local model


Global Model vs. Local Model

Initial proportion of S1

◆: Global model▲: Local model (4 neighbors)

proportion of S1

(1) p(0) <0.25, p(0)>0.75:

coexistence of the two opinions

(2) 0.25 < p(0) < 0.75:

0

0.25

0.5

0.75

1

0 0.25 0.5 0.75 1p(0)

p*

●: S1●: S2●: S1⇔S2●: S2⇔S1

Local model

Consensus on one opinion

converges to one opinion

critical point: initial ratio at p(0)=0.5global model

Global model

p*=0.5

Local model

Coexistence different opinions


Frangible Collective Decision with Hub Agents

Hub agents destabilize collective decision

0

0.25

0.5

0.75

1

0 0.25 0.5 0.75 1p(0)

p*

Hub Model

proportion of S1

Case 1:Interact with only a hub agent

●: S1●: S2●: S1⇔S2●: S2⇔S1

Case 2:Interact with both a hub agent and neighbors

Hub１ Hub ２

Group 1 Group ２

p*: 0.5⇔1.0

p*=0.5


Outline






Smart Agents Using ICT(ICT:Information/Communication

Technology)


Congestion control should receive much attention

uncontrolled system

EFF

ICIE

NC

Y

controlled system

PROBLEM CAUSED BY CONGESTION

Loss of efficiency

Unfair allocation of resources among competing usersideal

LOADCritical load

Why Do Things Get Worse (cont.)?

The queuing is one of the greatest invention


Queuing Discipline: FIFO

First-In-First-Out (FIFO)• most widely used discipline• first customer that arrives is the first one to be

serviced (or transmitted)Madness of crowds

Wisdom of crowds


Internet Traffic DynamicsQuestions

: How does the network topology influence the traffic dynamics?

: How does user heterogeneity influence the traffic dynamics?

Heavy users send or down load extremely big files


Performance Evaluation via Simulation

networksimulation

• user heterogeneity(file size distribution)

traffic demand

congestioncontrol

performancemeasures

networktopology

• connectivity• bandwidths

• TCP flow• drop tail, red,..

• throughput• packet drop rate

throughput=file size/time to be sent


Uncontrolled vs. Controlled Particles

agent agent

self-controlled particles

uncontrolled particles ・UDP Flow: open-loop flow

U

in flow current flow

X

H

U X－

＋

・TCP Flow: closed-loop flow

Gcurrent flowin flow


Traffic Dynamics on an Open-Loop System

David Arrowsmith, 2006


Little’s Law


Critical Load of an Open-Loop Traffic System

63 - ETH Colloquium ‘07 (Zurich)Packet rate

ScaleScale--free:free:κ=2κ=2

Random networkRandom network

Del

iver

ed p

acke

ts

ScaleScale--free:free:κ=3κ=3

Random network outperforms scale-free network without congestion control

Network Performance: Uncontrolled Traffic Flow


Active Queue management (AQM): the traffic of each link is controlled by the router

Congestion Analysis of Controlled Flow

AQM: Drop tail, RED, CHOKE


TCP Congestion Control

Window algorithm:send W packets at a time

• increase window size W by one per RTT if no loss, : W <- W+1 each RTT

• decrease window size by half on detection of loss : W <− W/2

sender

receiver

W

AIMD: Additive Increase Multiplicative Decrease

RTT: round trip time


Random networkRandom network

ScaleScale--freefree

Scale-free network outperforms random network with congestion control

low traffic heavy traffic

Traffic Dynamics on a Closed-Loop System


Tiers(96 ’ M.B.Doar)

Transit-Stub model

(96’ Ellen W)

Barabasi-Albert model

Hierarchy: Tree Scale Free

Traffic Dynamics: Substrate Network

WAN,MAN,LAN ISP NetworkAutonomous systems

Modular network


High Performance of Scale-free Network with Congestion Control

Edge betweennessNumber of hops

Complementary CDF(CCDF)

Distribution of hops

Scale free (BA) modelTiers modelTS model

Distributions of hops (average shortest paths) and edge betweenness


1 0-1

1 00

1 01

1 02

1 03

1 04

1 0-6

1 0-5

1 0-4

1 0-3

1 0-2

1 0-1

1 00

x

1-F(

x)

C o m p le m e n ta ry C u m u la tiv e D is tr ib u tio n P lo t

P a re to ( in f in ite v a r ia n c e )E x p o n e n tia l (f in ite v a r ia n c e )

Traffic Dynamics: User Heterogeneity

The file size distribution is scale-free (mixture of heavy users and light users): constant: normal distribution: exponential distribution: power-law distribution

IP Flow Size

file size

Complementary CDF


Tiers

Low traffic:　Average size: 30kbyte

Comparison of Network Performances(Light Traffic Phase)

TS

Scale free

throughput


:constant

:normal distribution

:exponential distribution

:power-law distribution

The network performance depends on theuser heterogeneity(the file size distribution)rather than the network topology

Complementary CDF(CCDF) of throughput


Tiers TS

Complementary CDF(CCDF) of Throughput Heavy traffic:Average size: 300kbyte

Comparison of Network Performances (Heavy Traffic Phase)

throughput

The network performance depends on thethe network topology rather than user heterogeneity(the file size distribution)

Scale free


1

)(1

+=

−=

α

α

α

αx

xPDF

xxCDF

m

m

β

β

β

β

M

M

xx

PDF

xx

CDF

1

)(

−

=

=

1+α1−β

log

PDF

logthroughput

Double Pareto Law

left side right side


Power-law distribution has heavy-tail at the right side

Double Pareto law has heavy-tails at the both sides, left and right.


Double Pareto Law at Congestion Phase (1)

TS Model:

exponential

power law

File size distribution:constant File size distribution

normal distribution

Heavy traffic:average size: 300kbyte


Double Pareto Law at Congestion Phase (2)

Scale free network:

power law

File size distribution: constant

exponential

Heavy traffic:average size: 300kbyte

File size distribution normal distribution


Congestion Control Needs to Receive Much Attention

log

log

low rank

PDF

high rank

allocated resource

The distribution of throughput has heavy-tails at the both sides, left and right.

Double Pareto law: “The richer gets richer and the poorer gets more poor”


Summary: Incentive ControlGeneral problem: given a good social model, determine how to change social system behavior to optimize a system performance.In many social systems, intervention (control) can impact the outcome.Decentralized mechanism design with incentive control(guided self-organization)Typical setting:• Agents act in their own best interest with a partial

global view. • Agents can be given incentives to change behavior

Mixture of economics, game theory, computer science, statistical physics, and complex networks.


Challenging Issue:Why Do Things Get Worse?

Selection maintains stability at a local optimum

Balance phase Variation phaseEvolution

Exploration

Disturbance

Unbalanced system

Pressure towards stability

- modifies components- modifies relationships- modifies external systemsStable system

Dual Phase Evolution in Social Systems: DPE

(David Green, 2007)


　　　　　　　Thank you for listening!!


Agent Heterogeneity and Cascade (1)

p(t+1): Probability to choose A at time t+1

10

M(t): the number of agents who have chosen A by time tN(t): the number of agents who have chosen B by time tS(t)=M(t)/{(M(t)+N(t)}: the ratio of agents who have chosen A

≤≤ α :social influence level

α=0: Logit model

α=1: Pure cascade model

)()()1()1( tSUqtp αα +Δ−=+

Population of agents

Prefer A

B

Prefer B

Aindividualpreference

social influence

The Wisdom of Networked Agentsnama/Top/InvitedTalk/07-05_it.pdfcoupled network Add a link, with...

Documents

Transcript of The Wisdom of Networked Agentsnama/Top/InvitedTalk/07-05_it.pdfcoupled network Add a link, with...