G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened...
Transcript of G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened...
![Page 1: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/1.jpg)
Page 1
G-lambda and Enlightened Middleware and Control Plane interactions
Tomohiro KudohGrid Technology Research Center
National Institute of Advanced Industrial Science and Technology (AIST)and
Jon MacLarenCenter for Computation & Technology
Louisiana State University
![Page 2: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/2.jpg)
Page 2
LIVE DEMO
Inter-domain advance reservation ofcoordinated network and computing resources
over the PacificA G-lambda & Enlightened collaboration
• Sep.11– 1:00PM-2:00PM– 6:00PM-
• Sep.12– 1:00PM-2:00PM
• Sep.13– 12:30PM-1:30PM
At 11th floor of THIS building(AIST meeting room)
![Page 3: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/3.jpg)
Page 3
What we have done
• “Automated” interoperability between network and computing resources in two countries’ grid computing research testbeds is shown– the first such experiment of this scale between two countries
• Integrated computing and communication technology – Automated simultaneous in-advance reservation of network
bandwidth between the US and Japan, and computing resources in the US and Japan
– World’s first inter-domain coordination of resource mangers for in-advance reservation
• Resource managers have different I/F and are independently developed
![Page 4: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/4.jpg)
Page 4
NW Control PlaneLayer
Three models of inter-domain coordination(1) NW Control Plane Layer inter-working (ex. GMPLS E-NNI)
GlobalResourceCoordinator L.Local Resource ManagerLayer
NW data Plane Layer
NRM1 NRM2 NRM3
RC1 RC2 RC3 RC4
Domain 1 Domain 2 Domain 3
UserProgram
UserProgram
UserProgram
UserProgram
![Page 5: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/5.jpg)
Page 5
NW Control PlaneLayer
Three models of inter-domain coordination(2) Local Resource Manager Layer inter-working
GlobalResourceCoordinator L.Local Resource ManagerLayer
NW data Plane Layer
NRM1 NRM2 NRM3
RC1 RC2 RC3 RC4
Domain 1 Domain 2 Domain 3
UserProgram
UserProgram
UserProgram
UserProgram
![Page 6: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/6.jpg)
Page 6
NW Control PlaneLayer
Three models of inter-domain coordination(3) Global Resource Coordinator Layer inter-working
GlobalResourceCoordinator L.Local Resource ManagerLayer
NW data Plane Layer
NRM1 NRM2 NRM3
RC1 RC2 RC3 RC4
Domain 1 Domain 2 Domain 3
UserProgram
UserProgram
UserProgram
UserProgram
![Page 7: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/7.jpg)
Page 7
Pros and Cons of the three models
1. NW Control Plane Layer inter-working (ex. GMPLS E-NNI)– Pros: User do not have to care about “multiple domains”– Cons: GMPLS is an on-demand protocol and can not support
advance reservation– Cons: Very close relationship between domains is required. May
not be always possible for commercial service.2. Resource Manager Layer inter-working
– Pros: User do not have to care about “multiple domains”.– Cons: Requested NRM may make a reservation which is
advantageous for the domain3. Global Resource Coordinator Layer inter-working
– Pros: User can control combination of domains– Pros: No under-layer interaction is required– Cons: User must have knowledge of inter-domain connection
WE EMPLOYED THIS MODEL FOR INTER-DOMAIN CONNECTION
![Page 8: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/8.jpg)
Page 8
NW Control PlaneLayer
Local Resource ManagerLayer
NW data Plane Layer
RM1 RM2 RM3
Domain 1 Domain 2 Domain 3
ApplicationLayer
GlobalResourceCoordinator Layer
App.GUIGL Application
GRS GRC
App. Launcher
HARCAcceptor
GL Application
![Page 9: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/9.jpg)
Page 9
NW Control PlaneLayer
Local Resource ManagerLayer
NW data Plane Layer
RM1 RM2 RM3
Domain 1 Domain 2 Domain 3
ApplicationLayer
GlobalResourceCoordinator Layer
App.GUIGL Application
GRS GRC
App. Launcher
HARCAcceptor
GL Application
EL→GLwrapper
EL→GLGNS-WSIwrapper
GNS-WSIHARC RM
I/FWrappers
![Page 10: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/10.jpg)
JapanApplication
GLGrid Resource
SchedulerEL→GLwrapper
KDDINRM
CRM
Cluster
CRM
Cluster
CRM
Cluster
NTTNRM
CRM
Cluster
CRM
Cluster
ELNRM
CRM
Cluster
CRM
Cluster
CRM
Cluster
USApplication
GL→ELGNS-WSIwrapper
GNS-WSI
JAPA
N US
GL: G-lambdaEL: Enlightened
GL request
EL request
G-lambda/Enlightened middleware coordination diagram
GL→ELCRM
wrapperHARC
Acceptor
ELGrid Resource
Coordinator
EL App. Launcher
![Page 11: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/11.jpg)
Page 11
Resource map of the demo
X2S
Japan South
Japan North USX2N
X1N X1U
X1S
FUK
KHN
RA1(MCNC)
BT2(LSU)Santaka
KMF
AKB
KAN
TKB
OSA
CH1(SL) VC1
(NCSU)
4G
5G5G
2G
NR3
(UO1)
(UO2)
(UO4)
(UO3)
(UR1)
(UR2)
(UR3)
X1
X2
LA1(Caltech)
BT1(LSU) Pelican
BT3(LSU)VizClientMachine
6509
Back up
0.11a.7
0.11a.6
0.11a.2
10.16a.2
LA Foundry
![Page 12: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/12.jpg)
Page 12
• Joint project of KDDI R&D labs., NTT, NICT and AIST.• G-lambda project has been started in December 2004. • The goal of this project is to establish a standard web
services interface (GNS-WSI) between Grid resource manager and network resource manager provided by network operators.
G- lambda project overview
![Page 13: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/13.jpg)
Page 13
GNS-WSI (Grid Network Service / Web Services Interface)
• Web services interface to reserve bandwidth in advance– Network Resource Manager provides this service
• Polling-based operations– Advance reservation of a path between end points– Modification of reservation (i.e. reservation time or duration)– Query of reservation status– Cancellation of reservation
• GNS-WSI2– WSRF(Web Services Resource Framework) based interface
• GT4 (Globus Toolkit 4) Java WS Corehttp://www.globus.org/toolkit/
– 2-phase commit
![Page 14: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/14.jpg)
Page 14
GNS-WSI (Grid Network Service / Web Services Interface)
• Grid Network Service-Web Services Interface• Interface to realize advance reservation of bandwidth• Based on the Web Services interface technology• Can be used for inter-domain coordination• Polling-based operations
– Advance reservation of a path between end points– Modification of reservation (i.e. reservation time or duration)– Query of reservation status– Cancellation of reservation
• GNS-WSI2– WSRF(Web Services Resource Framework) based interface
• GT4 (Globus Toolkit 4) Java WS Corehttp://www.globus.org/toolkit/
– 2-phase commit
![Page 15: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/15.jpg)
Page 15
Service Parameters
Parameter Usage Value Remarks
Site ID(APoint, ZPoint)
ID to specify A and Z points String Name or ID of sites
latency Latency between end points Positive integer (msec)
Reservation time(startTime, endTime)
Start time and end time of the reservation xsd:dateTime YYYY-MM-
DDTHH:MM:SSZ
localUsername user name of certificate String GT4 GSI
commandStatus status of each command String p. 16
resourceStatus status of network resource String Available / NotAvailable
reservationStatus status of reservation String p. 15
availability Network protection of network resource Integer (-232 � 232-1)
0 = Un-protected1 = Protected
bandwidth Bandwidth of the resource Positive integer (kbit/s)
![Page 16: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/16.jpg)
Page 16
An example XML exchanged through GNS-WSI
<requirements><network
aPoint=“AKB" zPoint=“RA1" startTime="2006-09-07T04:15:00Z" endTime="2006-09-07T06:15:00Z“bandwidth="1000000" latency="1000"/>
</requirements>
![Page 17: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/17.jpg)
Page 17
JapanApplication
GLGrid Resource
SchedulerEL→GLwrapper
KDDINRM
CRM
Cluster
CRM
Cluster
CRM
Cluster
NTTNRM
CRM
Cluster
CRM
Cluster
ELNRM
CRM
Cluster
CRM
Cluster
CRM
Cluster
USApplication
GL→ELGNS-WSIwrapper
GNS-WSI
JAPA
N US
GL: G-lambdaEL: Enlightened
GL request
EL request
G-lambda/Enlightened middleware coordination diagram
GL→ELCRM
wrapperHARC
Acceptor
ELGrid Resource
Coordinator
EL App. Launcher
![Page 18: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/18.jpg)
Coallocation of Compute and Network Resources using HARC
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
![Page 19: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/19.jpg)
HARC – Highly AvailableRobust Coallocator
![Page 20: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/20.jpg)
Robustness/Redundancy• Based on Paxos Commit (Lamport/Gray)• The Classic 2PC Transaction Manager
functionality is replicated in multiple acceptors– Algorithm makes progress provided a majority of
acceptors are working– So the �RMs don’t get stuck in “Prepared” state– Messages can be lost, repeated, arrive in an
arbitrary order (but can’t be tampered with)– If you deploy 7 acceptors, you can get a MTTF of
about 5 years (assuming a MTTF of 48 hours, and MTTR of 1 hour per acceptor)
![Page 21: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/21.jpg)
Coallocation• Can support new types of RM
– Can interface practically any reservation system– Without changing Acceptor code– Without changing/adding protocols
• Current RMs:– Compute resources (Batch queues, e.g. Torque/Moab, etc.)– Network resources (EnLIGHTened testbed) – the HARC
NRM (includes simple scheduler)
• Future RMs:– Diary/Calendars (people/�rooms)– VCL Cluster Reservation System
![Page 22: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/22.jpg)
HARC NRM• Sends GMPLS Commands to the 4 Calient
Diamondwave PXCs via TL1 commands in the EnLIGHTened testbed
• EROs are “computed” inside the NRM and sent to the switches as strict paths
• Contains a simple scheduler which maintains a centralized timetable of Trib/TE links
• No priority/preemption
• Replace lookup with simple computation for Supercomputing ‘06.
• Timetable should probably be distributed
![Page 23: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/23.jpg)
Inter-domain Resource Specification
X2S
Japan South
Japan North USX2N
X1N X1U
X1S
KMFFUK
KHN
RA1(MCNC)
BT2(LSU)Santaka
AKB
KAN
TKB
OSA
CH1(SL) VC1
(NCSU)
4G
5G5G
2G
(KR2)(KR3)
(KO3)
NR3
(NO1)
(NO2)
(KO2) (KO1)
(NO3)
(UO1)
(UO2)
(UO4)
(UO3)
(UR1)
(UR2)
(UR3)
X1
X2
LA1(Caltech)
BT1(LSU) Pelican
BT3(LSU)VizMachine Client
6509
Back up
0.11a.6
0.11a.7
0.11a.2
10.16a.2
LA Foundry
![Page 24: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/24.jpg)
![Page 25: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/25.jpg)
![Page 26: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/26.jpg)
Distributed VisualizationVolume rendering using 3D textures of real component of Ψ4 outgoing wavesOptional Isosurfaces to show event horizon of merging black holesPositive values are blue while negative values appear reddish
![Page 27: G-lambda and Enlightened Middleware and Control Plane ... · Page 1 G-lambda and Enlightened Middleware and Control Plane interactions Tomohiro Kudoh Grid Technology Research Center](https://reader033.fdocuments.in/reader033/viewer/2022041909/5e65f3c9d6900324884c1da5/html5/thumbnails/27.jpg)
Page 27
LIVE DEMO
Inter-domain advance reservation ofcoordinated network and computing resources
over the PacificA G-lambda & Enlightened collaboration
• Sep.11– 1:00PM-2:00PM– 6:00PM-
• Sep.12– 1:00PM-2:00PM
• Sep.13– 12:30PM-1:30PM
At 11th floor of THIS building(AIST meeting room)