Scheduler Interfaces in the ASKALON Grid Environment
description
Transcript of Scheduler Interfaces in the ASKALON Grid Environment
![Page 1: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/1.jpg)
Scheduler Interfaces in the ASKALON Grid
Environment
Marek Wieczorek, GGF16 Athens, February 14th
![Page 2: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/2.jpg)
Agenda
•Scheduling in the ASKALON environment
•Scheduler interfaces
• Interesting scenarios
•Summary
![Page 3: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/3.jpg)
ASKALON Grid Environment
EnactmentEngine
ResourceManagement
PerformancePredictor
MonitoringService
UserInterface
AGWL(Workflow Representation)
Grid Environment (Globus Toolkit)
ASKALON
Scheduler
![Page 4: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/4.jpg)
Scheduling Model
• Scientific workflow scheduler
• Separate instance for each workflow submission
• Interacts with other ASKALON services
• Dynamic scheduling (rescheduling enabled)
![Page 5: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/5.jpg)
Scheduler Input
• Workflow specification created by the user
• A workflow composed of activity types and dependencies between them
• ASKALON workflow description language (AGWL)
Use
r In
terf
ace
(TEU
TA
)
Sch
edule
rworkflow specificationreceive: AGWL file (XML)protocol: GT4 WS interface
![Page 6: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/6.jpg)
Interface to the Resource Management (GridARM)
• Resource brokerage based on resource constraints (e.g., CPU count, memory)
• Logical resources considered, i.e. software components (executables Grid/Web services)
• Advance reservation support
• Billing/pricing
Sch
edule
r
Reso
urc
eM
anagem
en
t
resource queryreceive: resource listprotocol: GT4 WS interface
service queryreceive: service descriptionprotocol: GT4 WS interface
reservation requestreceive: reservation offersprotocol: GT4 WS interface
reservation confirmationreceive: reservation ticketprotocol: GT4 WS interface
![Page 7: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/7.jpg)
Interface to the Enactment Engine
• Enactment Engine – advanced Job Manager
• Communication protocol based on the internal workflow representation
• Current implementation based on the Java API
Sch
edule
r
Enact
ment
Eng
ine
workflow submissionsend: full workflow graphprotocol: GT4 WS interface
rescheduling requestreceive: executed workflowsend: rescheduled workflowprotocol: GT4 WS interface
![Page 8: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/8.jpg)
Interface to the Performance Predictor
• Auxiliary service
• Important for proper scheduling and for advance reservations
• Model: workflow-oriented activity predictions
Sch
edule
r
Perf
orm
ance
Pre
dic
tor
prediction requestsend: activity type, resource, input parametersreceive: predictionprotocol: Java API/GT4 WS
![Page 9: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/9.jpg)
Interface to the Monitoring Service
• Not implemented yet
• Monitoring of resources and activities
• Control over many aspects of execution (including scheduling)
• Possible execution problems reported
• May trigger workflow rescheduling
Sch
edule
r
Monit
ori
ng
Serv
ice
event notificationreceive: eventprotocol: GT4 WS interface
scheduling eventsend: scheduling/ reschedulingprotocol: GT4 WS interface
![Page 10: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/10.jpg)
Workflow Processing Lifecycle
Scheduler EnactmentEngine
PerformancePredictor
AGWL workflowspecification
ResourceManagement
resource/servicequery
MonitoringService
prediction request
negotiation/reservation
scheduled workflow
rescheduled workflow
rescheduling request
event notification
![Page 11: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/11.jpg)
Scheduling under Uncertainty
• Full-graph scheduling
• ASKALON workflow model (loops, conditional branches, etc.) allow different possible executions
• Runtime assumption made by the scheduler
• Assumption fulfilment controlled by the enactment engine
Scheduler EnactmentEngine
originalworkflow
predicted DAG
+assumptionviolation=
rescheduling
+assumptions=
rescheduled workflow
![Page 12: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/12.jpg)
Advance Reservation
•Negotiations between the scheduler and the Reservation Management– The scheduler acts on behalf of the user,– Resource Management optimizes the
profit of the Grid (resource usage, fair-sharing)
•Two-stage negotiation protocol:– reservation request,– reservation confirmation
![Page 13: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/13.jpg)
Event-driven Dynamic Scheduling
• Different sources of events
• Scheduling and rescheduling performed in a dynamic way.
• Two types of reasons for rescheduling:– execution model
(assumption violation)– unexpected behiavor
(execution problems)
Scheduler
Enactment Engine
ResourceManagement
MonitoringService
assumption violation,execution problem
SLA violation performancebreakdown
![Page 14: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/14.jpg)
Summary
• Importance of the scheduler in the ASKALON environment.
•Multitude of the scheduler interfaces
•Resource negotiation and event-driven scheduling
![Page 15: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.fdocuments.in/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/15.jpg)
ASKALON
http://dps.uibk.ac.at/projects/http://dps.uibk.ac.at/projects/askalon/askalon/