1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and...
Transcript of 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and...
![Page 1: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/1.jpg)
1
Cooperative multi-agent systems
• In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions
• Limitations on computational power and bandwidth prohibit the global view of the task structure
• Exchange of complete local views between agents is prohibitive
• Need to find a trade-off between the amount of information exchanged and the negotiation’s contribution to increasing combined utility
![Page 2: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/2.jpg)
2
Contribution
The proposed negotiation protocol:
– Empirically shown to approach the utility of distributed agents’ actions very close to that of their aggregation
– Multi-parameter
– Uses marginal utility gain and cost to drive the negotiation
![Page 3: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/3.jpg)
3
Assumptions
• Agents possess local views of the task structures that define their functionality
• Agents have access to a scheduler that takes a task structure and utility function as input and produces a set of schedules that increase the utility function locally
![Page 4: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/4.jpg)
4
Motivating example: task allocation
The agent needs a certain non-local task to be performed by some other agent.
The negotiation’s goal is to increase the combined utility of actions of both agents by choosing a certain quality and time range of the non-local task.
![Page 5: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/5.jpg)
5
Vocabulary 1
• Contractor – agent that has a non-local task NL that needs to be assigned to another agent
• Contractee – agent that performs this task for the contractor
• Utility – weighted sum of quality, cost, and duration of a schedule
![Page 6: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/6.jpg)
6
Vocabulary 2
• Marginal utility gain (MUG) – local utility increment for the contractor with NL performed
• Marginal utility cost (MUC) – local utility decrement for the contractee with NL performed
![Page 7: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/7.jpg)
7
Motivating example: task structures
TCR
Task1 Task2
M1 M2 M4 M5M3
sum
sum sum
q:10c:10d:9
q:10c:10d:9
q:10c:10d:9
q:15c:0d:0
q:10c:10d:9
deadline:50
enables enables
TCE
Task1 Task2
B1 B2 B3 B4
sum
sum sum
q:10c:10d:9
q:10c:10d:9
q:10c:10d:9
q:10c:10d:9
deadline:47
enables
deadline:11 deadline:21
![Page 8: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/8.jpg)
8
Contractor’s FSM
buildProposal
always
sndMsgProposal
Wait
EvalCounterProposal
Finish
NewProposal
rcvMsgCounterProposal
evalCounterProposal
Reject
sndMsgReject
Accept&GetNext
Accept&HaveEnoughsndMsgFinish
always
generateNewProposalsndMsgProposal
rcvMsgAccept&HaveEnough
sndMsgFinish
rcvMsgAccept&GetNext
Start
![Page 9: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/9.jpg)
9
Contractee’s FSM
Start EvalProposal
Wait
Finish
rcvMsgProposal
evalProposal
Accept
sndMsgAccept
rcvMsgFinishReject
buildCounterProposalsndMsgCounterProposal
rcvMsgReject
![Page 10: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/10.jpg)
10
Binary search approachEach next proposal is the midpoint between the contractor’s
current proposal and contractee’s current counter proposal.Disadvantages:• Finish time of first counter proposal may not be the actual
reasonable finish time• Negotiation is about multiple issues (time range, quality)
while the binary search focuses only on finish time• Binary search requires greater certainty about NL’s time
range• Both agents are likely to cling to their own proposals,
greater the likelihood that some solutions are missed.
![Page 11: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/11.jpg)
11
Proposal/counterproposal information
• EST - Earliest start time of the non-local task NL
• FT - Finish time
• minq - Requested quality
• MUC - Marginal utility cost
• MUG - Marginal utility gain
![Page 12: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/12.jpg)
12
Protocol functions overview
• buildProposal:– Creates the initial contractor’s proposal
including the time range and quality of NL and MUG
– Insists on highest local utility by executing NL according to the contractor’s best local schedule
![Page 13: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/13.jpg)
13
Protocol functions overview
• generateCounterProposal:– Creates the contractee’s counter proposals
including the time range and quality of NL and MUC
– Initially insists on highest local utility by executing NL according to the contractee’s best local schedule
![Page 14: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/14.jpg)
14
Protocol functions overview
• generateCounterProposal:– Tries to increase combined utility by lowering
MUC– Successive counter proposals relax the time
range and quality of NL until a schedule is found s.t. MUG>MUC
![Page 15: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/15.jpg)
15
Protocol functions overview
• generateNewProposal:– Creates the next contractor’s proposal based on
the preceding proposal or counter proposal
![Page 16: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/16.jpg)
16
Protocol functions overview
• generateNewProposal:– Tries to increase combined utility by doing
multi-parameter heuristic-based search in quality and time range space
![Page 17: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/17.jpg)
17
Protocol functions: buildProposal
• buildProposal: – produces the initially requested earliest start
time and finish time of the non-local task according to the contractor’s best schedule
– has limited information about the NL’s quality, cost, and duration
– the greater the time range of NL’s execution is the closer to the high bound the requested quality is
![Page 18: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/18.jpg)
18
Protocol functions: generateCounterProposal
• If no previous Counter Proposal exists:– produces a “what-if” local contractee’s schedule
ignoring the time range and quality of NL in the current proposal, but enforcing NL to be in the schedule, thus delivering the minimum marginal utility cost for the NL
• If there is a previous Counter Proposal :– Alternatively relaxes the NL’s time range and lowers
the requested quality from the current proposal until an acceptable schedule is found (MUG>MUC)
![Page 19: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/19.jpg)
19
Protocol functions: generateNewProposal 1
• If Previous Proposal is acceptable for the contractee:– If the NL is out of initial time range and its quality
below average then increase the quality request and make later the deadline for the NL
– If the NL is in initial time range and its quality is above average then reduce quality request in the same time range thus trying to lower MUC
– Otherwise shift time range later and reduce quality request thus trying to lower MUC
![Page 20: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/20.jpg)
20
Protocol functions: generateNewProposal 2
• If Previous Proposal is not acceptable for the contractee:– If this is first counter proposal:
• If the initial time range is too short then the deadline is moved later and lower quality is requested
• Otherwise make the initial range a bit longer than the current estimate of NL’s execution time and request higher than average quality
![Page 21: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/21.jpg)
21
Protocol functions: generateNewProposal 3
– If this is not first counter proposal• Assume a previous counter proposal from the
contractee as current proposal
• If requested quality above average then shift the time range later
• If requested quality below average then request higher quality and shift the NL’s deadline later
![Page 22: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/22.jpg)
22
Modified task structure
TCE
Task1 Task2
B1 B2 B3 B4
sum
sum sum
q:10c:10d:9
q:10c:10d:9
q:15c:0d:0
q:10c:10d:9
deadline:47
enables
deadline:11 deadline:21
New_TCE
M4
M41 M42 M43
exactly_one
sumTCR
Task1 Task2
M1 M2 M4 M5M3
sum
sum sum
q:10c:10d:9
q:10c:10d:9
q:10c:10d:9
q:15c:0d:0
q:10c:10d:9
deadline:50
enables enables
![Page 23: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/23.jpg)
23
Protocol variants
• Single step: the contractor sends a proposal PC to the contractee which accepts it if MUG(PC)>MUC(PC) or rejects it otherwise
• Multi-step-Multiple-Try: the agents perform the negotiation process until a predefined number of acceptable solutions are found or iteration limits are exceeded
• Multi-step-Limited-Effort: the agents perform the negotiation process until iteration limits are exceeded
![Page 24: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/24.jpg)
24
Experiment task structures
TCR
Task1 Task2
M1 M2 M4 M5M3
sum
sum sum
q:10c:10d:9
q:10c:10d:9
q:15c:0d:0
q:10c:10d:9
deadline:50
enables enables
q:10c:10d:9
enables
est:10 est:24
TCE
Task1 Task2
B1 B2 B3 B4
sum
sum sum
q:10c:10d:9
q:10c:10d:9
q:15c:0d:0
q:10c:10d:9
deadline:47
enables
deadline:11 deadline:21
enablesenables
![Page 25: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/25.jpg)
25
Utility function
• S – schedule
Utility(S) = quality_gain(S)*quality_weight +
cost_gain(S)*cost_weight +
duration_gain(S)*duration_weight
![Page 26: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/26.jpg)
26
Collected data
• Negotiation outcome: success if the combined utility is increased
• Utility gain: MUG - MUC
• Gain percentage: percentage of utility gain relative to the combined utilities without the task allocation
• Solution quality: utility percentage relative to the utility of the combined tasks
• Complexity of task structures: function of the number of interrelationships and temporal constraints in the task structures
• Number of negotiation steps (proposal/counterproposal messages).
![Page 27: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/27.jpg)
27
Protocol comparisonSuccess Average Gain
PercentageANNS Gain per
StepSolution
Quality
Single-step 2850 7.63 1.0 7.63 94.43
Multi-step-One-Try
4088 10.17 1.48 6.87 96.69
Multi-step-Two-Try
4088 11.9 4.69 2.55 98.23
Multi-step-Three-Try
4088 13.4 6.42 2.09 99.51
Multi-step-Limited Effort 4088 13.9 8.15 1.7 99.93
![Page 28: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/28.jpg)
28
Complexity
• ir1 – number of interrelationships in contractor’s task structure
• tr1 – number of temporal constraints in the contractor’s task structure
• ir2 – number of interrelationships in contractee’s task structure
• tr2 – number of temporal constraints in the contractee’s task structure
Complexity of task structure = ir1 + tc1 + ir2 + tc2 + (ir1*tc1 + ir2*tc2 + ir1*ir2 + tc1*tc2 + ir1*tc2 + ir2*tc1)/6
![Page 29: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/29.jpg)
29
Dependence of SQ on complexity
88
90
92
94
96
98
100
102
0 5 10 15 20complexity of task structures
so
luti
on
qu
ali
ty
MultiStepLimitedSearchMultiStepThreeTryMultiStepTwoTryMultiStepOneTrySingleStep
![Page 30: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/30.jpg)
30
Net negotiation gain
Assume that one negotiation step costs 0.5% of the overall utility of the negotiating agent’s schedules
Net negotiation utility gain = negotiation utility gain – negotiation effort
![Page 31: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/31.jpg)
31
Dependence of net gain on complexity
88
90
92
94
96
98
100
102
0 5 10 15 20complexity of task structures
ne
t n
eg
oti
ati
on
ga
in
MultiStepLimitedSearchMultiStepThreeTryMultiStepTwoTryMultiStepOneTrySingleStep
![Page 32: 1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations.](https://reader031.fdocuments.in/reader031/viewer/2022013012/5697bff11a28abf838cbb3bd/html5/thumbnails/32.jpg)
32
Observations
• The Multi-step-One-Try protocol is much better than Single step in almost all situations(except for very simple ones)
• The Multi-step-Two-Try and Multi-step-Three-Try protocols are useful in moderately constrained situations
• The number of constraints can be used to choose a protocol that balances negotiation gain and effort for a certain situation