An Approach to the Analysis and Design of Multiagent...

� � � ��

� � ��

An Approach to the Analysis and Design ofMultiagent Systems based on Interaction

Frames

Michael Rovatsos, Gerhard Weiß, Marco Wolf

Department of InformaticsTechnical University of Munich

AAMAS’2002, Bologna, July 15-19, 2002 – p.1/41

� � � ��

� � ��

OverviewMotivationInteraction Frames & FramingInFFrAExampleConclusions


� � � ��

� � ��



� � � ��

� � ��

MotivationOpen systems on the Internet

virtual enterprises

supply chain management

electronic marketplaces

ubiquitous information access

New research issuesSomething amiss?


� � � ��

� � ��


virtual enterprises




New research issues

Something amiss?


� � � ��

� � ��


virtual enterprises




New research issuesSomething amiss?


� � � ��

� � ��

MotivationThree claims:

Socio-empirically rational agents thatrecord interaction processes and

employ this experience strategicallyare needed.

Architectures to design and analyse suchagents are missing.InFFrA is a possible solution!


� � � ��

� � ��



employ this experience strategicallyare needed.Architectures to design and analyse suchagents are missing.

InFFrA is a possible solution!


� � � ��

� � ��



employ this experience strategicallyare needed.Architectures to design and analyse suchagents are missing.InFFrA is a possible solution!


� � � ��

� � ��

MotivationIntroduction to InFFrA

Application to Multiagent Learning SystemDiscussion & Outlook


� � � ��

� � ��

MotivationIntroduction to InFFrAApplication to Multiagent Learning System

Discussion & Outlook


� � � ��

� � ��

MotivationIntroduction to InFFrAApplication to Multiagent Learning SystemDiscussion & Outlook


� � � ��

� � ��



� � � ��

� � ��

Interaction Frames & Framing

(Interaction) Frames capture the regularitiesof interaction processes

roles & relationships,

courses of interaction (trajectories),

contexts & conditions,

beliefs.

Local vs. shared knowledgecommon vs. private attributes

Interactions are not subject to direct agentcontrol!


� � � ��

� � ��


Framing is social reasoning thatinterprets situations in terms of frames,

adapts frame conceptions,

strategically employs frames to guideinteraction behaviour.

Socio-centric view with individualist “touch”

Frame & Framing concepts grounded inGoffman’s sociological theory


� � � ��

� � ��


Framing is social reasoning thatinterprets situations in terms of frames,

adapts frame conceptions,

strategically employs frames to guideinteraction behaviour.

Socio-centric view with individualist “touch”Frame & Framing concepts grounded inGoffman’s sociological theory


� � � ��

� � ��



� � � ��

� � ��

InFFrA – Frames

R3

R1

R1 R2 R3

time

R1

R2

R3

C2

C1

C3

C4

C5C8

C9

C10

preconditions postconditions

conditionsactivation

time

trajectory model

sustainment conditions

C6 C7

deactivationconditions

R2

R3

R4 R5

R1

G2

G1

stat

us

C

B

A

D

E

F G

conceptualbeliefs

causalbeliefs

C

B

A

D

F

E

G

H

R2

A4

A6

A7A2

A1

A5

A3

A8

A7

A9

Frame

context

roles & relationships trajectories

beliefs


� � � ��

� � ��

InFFrA – Frames

R3R1

R1 R2 R3

time

A4

A6

A7A2

A1

A5

A3

A8

A7

A9

trajectories

R2

R3

R4 R5

R1

G2

G1

stat

us

roles & relationships

CB

ADE

F G

conceptualbeliefs

causalbeliefs

CB A

DFE

G

H

R2

beliefs

R1R

2R3

C2

C1

C3

C4C5 C8

C9

C10



timetrajectory model

sustainment conditionsC6C7


context

R1 R2 R3

time

Frame

A4A6

A7A2

A1A5

A3A8

A7

A9

trajectories


� � � ��

� � ��

InFFrA – Frames

R3R1

R1 R2 R3

time

A4

A6

A7A2

A1

A5

A3

A8

A7

A9

trajectories

R2

R3

R4 R5

R1

G2

G1

stat

us


CB

ADE

F G

conceptualbeliefs

causalbeliefs

CB A

DFE

G

H

R2

beliefs

R1R

2R3

C2

C1

C3

C4C5 C8

C9

C10






context

R1 R2 R3

time

Frame

A4A6

A7A2

A1A5

A3A8

A7

A9

trajectories

action

actor role

message


� � � ��

� � ��

InFFrA – Frames

R3R1

R2

R3

R4 R5

R1

G2

G1

stat

us


R1 R2 R3

time

A4A6

A7A2

A1A5

A3A8

A7

A9

trajectories

CB

ADE

F G

conceptualbeliefs

causalbeliefs

CB A

DFE

G

H

R2

beliefs

R1R

2R3

C2

C1

C3

C4C5 C8

C9

C10






context

R2

R3

R4 R5

R1

G2

G1

stat

usFrame



� � � ��

� � ��

InFFrA – Frames

R3R1

R2

R3

R4 R5

R1

G2

G1

stat

us


R1 R2 R3

time

A4A6

A7A2

A1A5

A3A8

A7

A9

trajectories

CB

ADE

F G

conceptualbeliefs

causalbeliefs

CB A

DFE

G

H

R2

beliefs

R1R

2R3

C2

C1

C3

C4C5 C8

C9

C10






context

R2

R3

R4 R5

R1

G2

G1

stat

usFrame


group

relationship

role


� � � ��

� � ��

InFFrA – Frames

R3R1

R1

R2

R3

C2

C1

C3

C4

C5C8

C9

C10



time

trajectory model


C6 C7


contextC

BA

DE

F G

conceptualbeliefs

causalbeliefs

CB A

DFE

G

H

R2

beliefs

R1 R2 R3

time

R1R

2R3

C2

C1

C3

C4C5 C8

C9

C10






R2

R3

R4 R5

R1

G2

G1

stat

usFrame

A4A6

A7A2

A1A5

A3A8

A7

A9

trajectories

context



� � � ��

� � ��

InFFrA – Frames

R3R1

R1

R2

R3

C2

C1

C3

C4

C5C8

C9

C10



time

trajectory model


C6 C7


contextC

BA

DE

F G

conceptualbeliefs

causalbeliefs

CB A

DFE

G

H

R2

beliefs

R1 R2 R3

time

R1R

2R3

C2

C1

C3

C4C5 C8

C9

C10






R2

R3

R4 R5

R1

G2

G1

stat

usFrame

A4A6

A7A2

A1A5

A3A8

A7

A9

trajectories

context

roles & relationships condition trajectory

conditionsenactment



� � � ��

� � ��

InFFrA – Frames

R3

R1

R3R1

C

B

A

D

E

F G

conceptualbeliefs

causalbeliefs

C

B

A

D

FE

G

H

R2

beliefs

R2

R3

R4 R5

R1

G2

G1

stat

us


R1 R2 R3

time

A4A6

A7A2

A1A5

A3A8

A7

A9

trajectories

R1R

2R3

C2

C1

C3

C4C5 C8

C9

C10






CB

ADE

F G

Frame

context beliefsconceptualbeliefs

causalbeliefs

CB A

DFE

G

H

R2


� � � ��

� � ��

InFFrA – Frames

R3

R1

R3R1

C

B

A

D

E

F G

conceptualbeliefs

causalbeliefs

C

B

A

D

FE

G

H

R2

beliefs

R2

R3

R4 R5

R1

G2

G1

stat

us


R1 R2 R3

time

A4A6

A7A2

A1A5

A3A8

A7

A9

trajectories

R1R

2R3

C2

C1

C3

C4C5 C8

C9

C10






CB

ADE

F G

Frame

context beliefsconceptualbeliefs

causalbeliefs

CB A

DFE

G

H

R2

ontology

belief net

role


� � � ��

� � ��

InFFrA – Frames

R3

R1

R1 R2 R3

time

R1

R2

R3

C2

C1

C3

C4

C5C8

C9

C10



time

trajectory model


C6 C7


R2

R3

R4 R5

R1

G2

G1

stat

us

C

B

A

D

E

F G

conceptualbeliefs

causalbeliefs

C

B

A

D

F

E

G

H

R2

A4

A6

A7A2

A1

A5

A3

A8

A7

A9

Frame

context

roles & relationships trajectories

beliefs


� � � ��

� � ��

Further FeaturesPrivate attributes

Status slots for common attributes

Mappings and assessments

Meta-frame attributesLinks (“alternative”, “variant”, etc.)

History (of frame evolution)


� � � ��

� � ��




Meta-frame attributes

Links (“alternative”, “variant”, etc.)



� � � ��

� � ��




Meta-frame attributesLinks (“alternative”, “variant”, etc.)



� � � ��

� � ��

InFFrA – FramingData structures:

Perceived frame

Active frame

Difference model

Trial frame

Frame repository


� � � ��

� � ��

InFFrA – FramingInference steps:

Situation interpretation

Matching

Assessment

Framing decision

Adjustment/re-framing

Enactment

Behaviour generation


� � � ��

� � ��

Framing

deviance

F FF

F2

F

deviate comply

situation interpretationmodule

frame matchingmodule

behaviour generation frame enactment

F3 Fn

prescriptivemodel

framing determine frameadequacy

determine framevalidity

determine framedesirability

module

perceptionupdate

currentmodel

descriptive model

normativemodel

generate

framing decision

reasons

modify

create

switch

frame adjustmentmodule

alternativeframes

trial instantiatemodulemodule

derive commitments

framerepository

compliance/

privategoals/values

sub−social level

perception

action

compliance

activated frame difference modelperceived frame

F1. . . . .

assessment

frameupdates

deviancedata

Framing Architecture

F F

roles

contexts

trajectories

beliefs


� � � ��

� � ��

Interpretation & Matching

deviance



perceptionupdate

currentmodel

descriptive model

normativemodel

generate

perception

compliance


F F

roles

contexts

trajectories

beliefs


� � � ��

� � ��

Assessment: “Comply Case”

deviance

comply

privategoals/values


frame enactment

prescriptivemodel




module

normativemodel

generate

framing decision

module

compliance/

compliance

activated frame difference model

sub−social level

assessment

deviancedata

F

roles

contexts

trajectories

beliefs


� � � ��

� � ��

Assessment: “Deviate Case”

deviance

F FF

F2

F


frame enactment

F3 Fn

prescriptivemodel




module

normativemodel

generate

framing decision

reasons

modify

create

switch


alternativeframes

trial instantiatemodule

framerepository

compliance/

compliance

activated frame difference model

F1

deviate

. . . . .

assessment

frameupdates

deviancedata

F

roles

contexts

trajectories

beliefs


� � � ��

� � ��

Enactment


prescriptivemodel

modulemodule

derive commitments

action

activated frame

F


� � � ��

� � ��

Framing

deviance

F FF

F2

F

deviate comply




F3 Fn

prescriptivemodel




module

perceptionupdate

currentmodel

descriptive model

normativemodel

generate

framing decision

reasons

modify

create

switch


alternativeframes

trial instantiatemodulemodule

derive commitments

framerepository

compliance/

privategoals/values

sub−social level

perception

action

compliance


F1. . . . .

assessment

frameupdates

deviancedata

Framing Architecture

F F

roles

contexts

trajectories

beliefs


� � � ��

� � ��



� � � ��

� � ��

Example – AdHocAdHoc = Adaptive Heuristic for OpponentClassification

AdHoc agents classify opponentsdynamically in iterated multiagent games

Scenario:randomly moving agents on toroidal grid

fixed number of PD games upon encounter

goal: utility maximisation


� � � ��

� � ��

Example – AdHocAdHoc = Adaptive Heuristic for OpponentClassificationAdHoc agents classify opponentsdynamically in iterated multiagent games

Scenario:randomly moving agents on toroidal grid

fixed number of PD games upon encounter

goal: utility maximisation


� � � ��

� � ��

Example – AdHocOpponent class models � consist of:

Deterministic finite automaton describingbehaviour of � [Carmel & Markovitch 96]

Q-table for optimal counter-strategy [Watkins &Dayan 92]

Similarity values � �� Learning samples for �

Similarity = Ratio of encounters withopponent understood by class model


� � � ��

� � ��

Example – FeaturesRe-classification, creation of new classes,deletion of obsolete ones

Adaptive (bounded) number of classesLong-term integration of similar classesConvergence to actual number of opponentclassesProblem: AdHoc vs. AdHoc agentsCan be solved “heuristically”


� � � ��

� � ��

Example – FeaturesRe-classification, creation of new classes,deletion of obsolete onesAdaptive (bounded) number of classes

Long-term integration of similar classesConvergence to actual number of opponentclassesProblem: AdHoc vs. AdHoc agentsCan be solved “heuristically”


� � � ��

� � ��

Example – FeaturesRe-classification, creation of new classes,deletion of obsolete onesAdaptive (bounded) number of classesLong-term integration of similar classes

Convergence to actual number of opponentclassesProblem: AdHoc vs. AdHoc agentsCan be solved “heuristically”


� � � ��

� � ��

Example – FeaturesRe-classification, creation of new classes,deletion of obsolete onesAdaptive (bounded) number of classesLong-term integration of similar classesConvergence to actual number of opponentclasses

Problem: AdHoc vs. AdHoc agentsCan be solved “heuristically”


� � � ��

� � ��

Example – FeaturesRe-classification, creation of new classes,deletion of obsolete onesAdaptive (bounded) number of classesLong-term integration of similar classesConvergence to actual number of opponentclassesProblem: AdHoc vs. AdHoc agents

Can be solved “heuristically”


� � � ��

� � ��

Example – FeaturesRe-classification, creation of new classes,deletion of obsolete onesAdaptive (bounded) number of classesLong-term integration of similar classesConvergence to actual number of opponentclassesProblem: AdHoc vs. AdHoc agentsCan be solved “heuristically”


� � � ��

� � ��

Example – Performance

24

26

28

30

32

0 5000 10000 15000 20000 25000 30000 35000

Rew

ard

per 1

00 G

ames

Interactions (IPD Games)

AdHoc AgentOne class per opponent

One class for all opponents


� � � ��

� � ��


0

5

10

15

20

25

30

35

0 1000 2000 3000 4000 5000 6000

Num

ber o

f opp

onen

t cla

sses

Encounters

k = 80k = 60k = 30k = 20k = 40k = 10


� � � ��

� � ��


24

26

28

30

32

34

36

38

0 2000 4000 6000 8000 10000 12000 14000

Rew

ard

per 1

00 G

ames

Interactions (IPD Games)

Tit For TatRandom Class

Maximum QualityHighest Payoff


� � � ��

� � ��

InFFrA AnalysisFraming = Classification procedure

Frame = Opponent class

Deterministic finite automaton Trajectory

Q-table for strategy learning Context

Similarity values Difference model

Learning samples for class History

Roles, links and beliefs: trivialPerceived frame = current encounter gamesequence


� � � ��

� � ��


Frame = Opponent classDeterministic finite automaton

Trajectory






� � � ��

� � ��


Frame = Opponent classDeterministic finite automaton Trajectory






� � � ��

� � ��



Q-table for strategy learning

Context





� � � ��

� � ��








� � � ��

� � ��




Similarity values

Difference model




� � � ��

� � ��








� � � ��

� � ��





Learning samples for class

History



� � � ��

� � ��








� � � ��

� � ��

InFFrA Analysis – Observations

if opponent is known, no re-framing duringencounterelse, matching after each roundframe matching updates all similarity valuesassessment and re-framing only afterencounterno adequacy and desirability test

restricted flexibility


� � � ��

� � ��

InFFrA Analysis – Observations

frame adjustment very complexheart of AdHoc systemtrial instantiation trivialenactment: use Q-tabletrajectory does not restrict ego’s actions!

reason for AdHoc vs. AdHoc problem


� � � ��

� � ��



� � � ��

� � ��

ConclusionAre we talking about...

... interaction protocols? Not really!Protocols have pre-specified semantics....another AOSE methodology?Yes, but...

we focus on specific type of reasoning

we assume models to be used by agents

...case-based reasoning? Yes and No!Depends on frame construction &combination.


� � � ��

� � ��


... interaction protocols?

Not really!Protocols have pre-specified semantics....another AOSE methodology?Yes, but...





� � � ��

� � ��


... interaction protocols? Not really!

Protocols have pre-specified semantics....another AOSE methodology?Yes, but...





� � � ��

� � ��


... interaction protocols? Not really!Protocols have pre-specified semantics.

...another AOSE methodology?Yes, but...





� � � ��

� � ��


... interaction protocols? Not really!Protocols have pre-specified semantics....another AOSE methodology?

Yes, but...





� � � ��

� � ��







� � � ��

� � ��





...case-based reasoning?

Yes and No!Depends on frame construction &combination.


� � � ��

� � ��





...case-based reasoning? Yes and No!

Depends on frame construction &combination.


� � � ��

� � ��







� � � ��

� � ��


...individualist or socio-centric approach?Something in-between!Data=social, reasoning=local....top-down or bottom-up? Both!InFFrA is a meta-architecture!...a silver bullet for open systems?Certainly not!

too few InFFrA-compliant systems

heavy cognitive assumptions


� � � ��

� � ��


...individualist or socio-centric approach?

Something in-between!Data=social, reasoning=local....top-down or bottom-up? Both!InFFrA is a meta-architecture!...a silver bullet for open systems?Certainly not!




� � � ��

� � ��


...individualist or socio-centric approach?Something in-between!

Data=social, reasoning=local....top-down or bottom-up? Both!InFFrA is a meta-architecture!...a silver bullet for open systems?Certainly not!




� � � ��

� � ��


...individualist or socio-centric approach?Something in-between!Data=social, reasoning=local.

...top-down or bottom-up? Both!InFFrA is a meta-architecture!...a silver bullet for open systems?Certainly not!




� � � ��

� � ��


...individualist or socio-centric approach?Something in-between!Data=social, reasoning=local....top-down or bottom-up?

Both!InFFrA is a meta-architecture!...a silver bullet for open systems?Certainly not!




� � � ��

� � ��


...individualist or socio-centric approach?Something in-between!Data=social, reasoning=local....top-down or bottom-up? Both!

InFFrA is a meta-architecture!...a silver bullet for open systems?Certainly not!




� � � ��

� � ��


...individualist or socio-centric approach?Something in-between!Data=social, reasoning=local....top-down or bottom-up? Both!InFFrA is a meta-architecture!

...a silver bullet for open systems?Certainly not!




� � � ��

� � ��


...individualist or socio-centric approach?Something in-between!Data=social, reasoning=local....top-down or bottom-up? Both!InFFrA is a meta-architecture!...a silver bullet for open systems?

Certainly not!too few InFFrA-compliant systems



� � � ��

� � ��


...individualist or socio-centric approach?Something in-between!Data=social, reasoning=local....top-down or bottom-up? Both!InFFrA is a meta-architecture!...a silver bullet for open systems?Certainly

not!too few InFFrA-compliant systems



� � � ��

� � ��


...individualist or socio-centric approach?Something in-between!Data=social, reasoning=local....top-down or bottom-up? Both!InFFrA is a meta-architecture!...a silver bullet for open systems?Certainly not!




� � � ��

� � ��

OutlookRicher communicative scenarios

Emergence of globally valid frames?Interaction frame calculiOrganisational interaction framesDevelop adaptive InFFrA agents


� � � ��

� � ��

OutlookRicher communicative scenariosEmergence of globally valid frames?

Interaction frame calculiOrganisational interaction framesDevelop adaptive InFFrA agents


� � � ��

� � ��

OutlookRicher communicative scenariosEmergence of globally valid frames?Interaction frame calculi

Organisational interaction framesDevelop adaptive InFFrA agents


� � � ��

� � ��

OutlookRicher communicative scenariosEmergence of globally valid frames?Interaction frame calculiOrganisational interaction frames

Develop adaptive InFFrA agents


� � � ��

� � ��

OutlookRicher communicative scenariosEmergence of globally valid frames?Interaction frame calculiOrganisational interaction framesDevelop adaptive InFFrA agents


� � � ��

� � ��

Things to remember!

Open systems

Socio-empirically rational agentsSociological grounding: frames and framing

InFFrA meta-architectureMulti-perspective applicability


� � � ��

� � ��

Things to remember!

Open systemsSocio-empirically rational agents

Sociological grounding: frames and framing



� � � ��

� � ��

Things to remember!

Open systemsSocio-empirically rational agentsSociological grounding: frames and framing



� � � ��

� � ��

Things to remember!

Open systemsSocio-empirically rational agentsSociological grounding: frames and framingInFFrA meta-architecture

Multi-perspective applicability


� � � ��

� � ��

Things to remember!

Open systemsSocio-empirically rational agentsSociological grounding: frames and framingInFFrA meta-architectureMulti-perspective applicability


� � � ��

� � ��

Thank you for your attention!


An Approach to the Analysis and Design of Multiagent...

Documents

Transcript of An Approach to the Analysis and Design of Multiagent...