Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe...
-
Upload
emily-cross -
Category
Documents
-
view
217 -
download
0
description
Transcript of Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe...
![Page 1: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/1.jpg)
Agent-Based Grid Load-Balancing
Daniel P. SpoonerUniversity of Warwick, UK
Junwei CaoNEC Europe Ltd., Germany
![Page 2: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/2.jpg)
Outline Grid Computing & Middleware Agent-Based Methodology Distributed Service Discovery Grid Load-Balancing Experimental Results Conclusions & Future Work
![Page 3: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/3.jpg)
Grid Computing Computational Grids - blueprint for a
new computing infrastructure
Grid/Web Services - distributed system and application integration
Data/Service/Community Grids - large-scale resource sharing in VOs
![Page 4: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/4.jpg)
Grid Middleware Resource Management & Scheduling Information Services (Monitoring & Discovery) Data Management and Access Application Programming Environments Security, Accounting, QoS ……
Condor, LSF, Ninf, Nimrod, ……Globus, Legion, DPSS, ……Java/Jini, CORBA, Web Services, ……
![Page 5: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/5.jpg)
Grid Resource Management
Key challenges: Cross-domain Large-scale Dynamic QoS support
GRM
![Page 6: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/6.jpg)
Agent-Based MethodologyAn agent is: A local grid manager An user agent A broker A service provider A service requestor A matchmaker
AAA A
![Page 7: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/7.jpg)
Local ManagementProcessor 1Processor 2Processor 3Processor 4Processor 5Processor 6Processor 7Processor 8
2n-1
Performance Prediction Task models Hardware models
FIFO AlgorithmGenetic Algorithm Heuristic &
Evolutionary Near-optimal on
makespan, deadlines and idletime.
![Page 8: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/8.jpg)
Service Discovery Pure data-pull No advertisement Full discovery Efficient when service
change more quickly
Pure data-push Full advertisement No discovery Efficient when
requests arrive more frequently
R/A
A U/A1 TaskInfo
2 PullSerInfo
3 PullSerInfo4 Return
R/A
R/A
A U/A3 TaskInfo
1 PushSerInfo
2 PushSerInfo4 Return
R/A
Centralised, not applicable for grid computing!
![Page 9: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/9.jpg)
Optimisation Strategies Configurable data-pull or data-push Agent hierarchy Multi-step advertisement & multi-step discovery Efficient when frequencies of request arrivals
and service changes are almost the same
Distributed!balancing betweenadvertisement anddiscovery!
A
A A
A A
User
![Page 10: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/10.jpg)
Agent Implementation
PerformancePrediction
FIFOScheduling
DatabaseManagement
MatchMaker
Advertisement Discovery
Task Model
Results
Sched. Time User Deadline
To Another Agent
Task Execution
TaskManagement
FIFO/GA Scheduling
Service Info.Provider
Current M
akespanService Info
Hw Model
Task ExecutionResourceMonitoring
Task Execution
![Page 11: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/11.jpg)
Load Balancing Metrics Total makespan Average advance time of task execution
completions (required deadline - actual task completion time)
Average processor utilisation rate (busy time / total makespan)
Load balancing level (1 - mean square deviation of processor utilisation rates / average processor utilisation rate)
Total number of network packages for both advertisement and discovery
![Page 12: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/12.jpg)
Experiment DesignS1
(SGIOrigin2000, 16)
S2(SGIOrigin2000, 16)
S4(SunUltra10,
16)S3(SunUltra10,
16)
S5(SunUltra5,
16)
S6(SunUltra5,
16)
S12(SunSPARCsta
tion2, 16)
S11(SunSPARCsta
tion2, 16)
S8(SunUltra1,
16)
S7(SunUltra5,
16)
S10(SunUltra1,
16)
S9(SunUltra1,
16)
Tasks:sweep3dfftimprocclosurejacobimemsortcpi
![Page 13: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/13.jpg)
Experiment 1 FIFO
FIFO FIFO
FIFO FIFO
![Page 14: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/14.jpg)
Experiment 2 GA
GA GA
GA GA
![Page 15: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/15.jpg)
Experiment 3 GA
GA GA
GA GA
![Page 16: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/16.jpg)
Task Execution
-1200
-1000
-800
-600
-400
-200
0
200
1 2 3
Experiment Number
ε (s
)
S1S2S3S4S5S6S7S8S9S10S11S12Total
Both GA and agents contribute towards the improvement in task executions.
![Page 17: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/17.jpg)
Resource Utilisation
0
10
20
30
40
50
60
70
80
90
1 2 3
Experiment Number
υ (%
)
S1S2S3S4S5S6S7S8S9S10S11S12Total
Less powerful S11 & S12 benefit mainly from the GA.
More powerful S1 & S2 benefit mainly from agents.
![Page 18: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/18.jpg)
Load Balancing
0
10
20
30
40
50
60
70
80
90
100
1 2 3
Experiment Number
β (%
)
S1S2S3S4S5S6S7S8S9S10S11S12Total
The GA contributes more to local grid load balancing.
Agents contribute more to global grid load balancing.
![Page 19: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/19.jpg)
Total Makespan
The centralised pure data-pull can always achieve the best results
Distributed agents with the hierarchical model can also achieve reasonably good results0
20
4060
80100
120140
160
4 6 8 10 12 14 16 18 20
Agent Number
t (m
ins)
CentralisedDistributed
![Page 20: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/20.jpg)
Network Package
The network overhead for the pure data-pull strategy to achieve better results is very high.
Distributed agent-based service advertisement and discovery can scale well.
0
5000
10000
15000
20000
25000
4 6 8 10 12 14 16 18 20
Agent Number
pack
ets
CentralisedDistributed
![Page 21: Agent-Based Grid Load-Balancing Daniel P. Spooner University of Warwick, UK Junwei Cao NEC Europe Ltd.,…](https://reader035.fdocuments.in/reader035/viewer/2022070616/5a4d1bf97f8b9ab0599eac5d/html5/thumbnails/21.jpg)
Conclusions An multi-agent paradigm provides a
clear high-level abstraction of grid resource management system.
Distributed service advertisement and discovery strategies can be used to improve agent performance.
Agent-based framework is scalable, flexible, and extensible for further enhancements.