GT3 Overview
-
Upload
handsomesundar -
Category
Documents
-
view
222 -
download
0
Transcript of GT3 Overview
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 1/58
GT3 Overview
The Globus Project™Argonne National Laboratory
USC Information Sciences Institute
Copyright (C) 2003 University of Chicago and The University of Southern California. All Rights Reserved. This
presentation is licensed for use under the terms of the Globus Toolkit Public License.
See http://www.globus.Org/toolkit/download/license.html for the full text of this license.
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 2/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 2
A Story of Evolution
q Definition of Grid problem has been stable since
original Globus Project proposal in 1995
– Though we’ve gotten better at articulating it
q
But our approach to its solution has evolved:– From APIs and custom protocols…
– to standard protocols…
– to Grid services (OGSA).
q
Driven by experience implementing and deploying theGlobus Toolkit, and building real applications with it
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 3/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 3
But Along The Way…
q Heterogeneous protocol base was hurting us
q Increasing number of virtual services that
needed to be managed
q Web services (WSDL, SOAP) appeared
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 4/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 4
Web Services
q At the heart of Web services is:
– WSDL: Language for defining abstract
service interfaces
– SOAP (and friends): Binding from WSDL tobytes on the wire
q Web services appears to offer a fighting
chance at ubiquity (unlike CORBA)
q But Web services does not go far enough
to serve a common base for the Grid…
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 5/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 5
Transient Service Instancesq
“Web services” address discovery & invocation of persistent services
– Interface to persistent state of entire enterprise
q In Grids, must also support transient service instances,created/destroyed dynamically
– Interfaces to the states of distributed activities
– E.g. workflow, video conf., dist. data analysis, subscription
q Significant implications for how services are managed,named, discovered, and used
– In fact, much of Grid is concerned with the management of service instances
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 6/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 6
Standard Interfaces & Behaviors:Four Interrelated Concepts
q Naming and bindings
– Every service instance has a unique name, from which candiscover supported bindings
q Lifecycle
– Service instances created by factories
– Destroyed explicitly or via soft state
q Information model
– Service data associated with Grid service instances,operations for accessing this info
– Basis for service introspection, monitoring, discoveryq Notification
– Interfaces for registering existence, and deliveringnotifications of changes to service data
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 7/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 7
OGSI Grid Service Specification
q Defines WSDL conventions and GSDL
extensions
– For describing and structuring services
– Working with W3C WSDL working group todrive GSDL extensions into WSDL
q Defines fundamental interfaces (using WSDL)
and behaviors that define a Grid Service– A unifying framework for interoperability &
establishment of total system properties
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 8/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 8
GT2 Evolution To GT3
q What happened to the GT2 key protocols?
– Security: Adapting X.509 proxy certs to integrate
with emerging WS standards
– GRIP/LDAP: Abstractions integrated into OGSI as
serviceData
– GRAM: ManagedJobFactory and related service
definitions
– GridFTP: Unchanged in 3.0, but will evolve into
OGSI-compliant service in 2004q Also rendering collective services in terms of OGSI:
RFT, RLS, etc.
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 9/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 9
GT Timeline
q GT 1.0: 1998
– GRAM, MDS
q GT 2.0: 2001
– GridFTP, packaging, reliabilityq GT3 Technology Preview: Apr-Dec 2002
– Tracking OGSI definition
q GT3.0 Alpha: Jan 2003
– OGSI Base, GT2 functionalityq GT3.0 Production: July 1, 2003
– Tested, documented, etc.
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 10/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 10
GT-OGSA Grid Service Infrastructure
OGSI Spec Implementation Security Infrastructure
System-Level Services
Base Services
User-Defined Services
Grid Service Container
Hosting Environment
Web Service Engine
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 11/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 11
GT3 Core
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 12/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 12
The Specification Defines how Entities can Create,Discover and Interact with a Grid Service
Servicedata
element
Servicedata
element
Servicedata
element
Service Implementation
GridService(required)
… other interfaces …(optional) Optional:
- Service creation
- Notification- Registration- Service Groups
+ application-specific interfaces
Required:- Introspection
(service data)- Explicit destruction- Soft-state lifetime
GT3 Core: OGSI Specification
Includes 0 or more Grid Service Handles (GSHs)
Includes 0 or more Grid Service References (GSRs)
Service locator
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 13/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 13
GT3 Core: OGSI Implementation
q GT3 includes a set of primitives that implement
the interfaces and behaviors defined in the
latest version of the OGSI Specification
q The implementation supports a declarative
programming model in which GT3 users can
compose OGSI-Compliant grid services by
plugging the desired primitives into theirimplementation
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 14/58
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 15/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 15
GT3 Core: OGSI Specification (cont.)
Factory portType
q Factories create services
q Factories are typically persistent services
q Factory is an optional OGSI interface
(Grid Services can also be instantiated by other
mechanisms)
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 16/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 16
GT3 Core: OGSI Specification (cont.)
Notification portTypes
q A subscription for notification causes the creation
of a NotificationSubscription service
q NotificationSinks are not required to implementthe GridService portType
q Notifications can be set on Service Data Elements
q
Notification portTypes are optional
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 17/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 17
GT3 Core: OGSI Specification (cont.)
Service group portTypesq A ServiceGroup is a grid service that maintains
information about a group of other grid services
q The classic registry model can be implemented with the
ServiceGroup portTypesq A grid service can belong to more than one ServiceGroup
q Members of a ServiceGroup can be heterogenous or
homogenous
q
Each entry in a service group can be represented as itsown service
q Service group portTypes are optional OGSI interfaces
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 18/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 18
GT3 Core: OGSI Specification (cont.)
HandleResolver portType
q Defines a means for resolving a GSH (GridService Handle) to a GSR (Grid Service
Reference)– A GSH points to a Grid Service
(GT3 uses a hostname-based GSH scheme)
– A GSR specifies how to communicate with the
Grid Service(GT3 currently supports SOAP over HTTP, soGSRs are in WSDL format)
q HandleResolver is an optional OGSI interface
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 19/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 19
Client
A ServiceCreation Scenario*
Registry
* The scenarios in this presentation are offered as examples and are not prescriptive
1. From a knownregistry, the client
discovers a
factory
by querying the
Service data of the
registry
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 20/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 20
Client
A ServiceCreation Scenario
Registry
2. The client calls the
createService
operation on thefactory
Factory
1. From a knownregistry, the client
discovers a
factory
by querying the
Service data of the
registry
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 21/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 21
Client
A ServiceCreation Scenario
Registry
1. From a knownregistry, the client
discovers a
factory
by querying the
Service data of the
registry
2. The client calls the
createService
operation on thefactory
Factory
Service
3. The factory
creates a
service
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 22/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 22
Client
A ServiceCreation Scenario
Registry
2. The client calls the
createService
operation on thefactory
Factory
Service
3. The factory
creates a
service
4. The factoryreturns a locator
1. From a knownregistry, the client
discovers a
factory
by querying the
Service data of the
registry
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 23/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 23
Client
A ServiceCreation Scenario
Registry
2. The client calls the
createService
operation on thefactory
Factory
Service
3. The factory
creates a
service
4. The factoryreturns a locator
5. The client and service interact
1. From a known
registry, the client
discovers a
factory
by querying the
Service data of the
registry
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 24/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 24
Notification
Sink
A Notification Scenario
1. NotificationSink calls the
subscribe operation on NotificationSource
Notification
Source
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 25/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 25
Notification
Sink
A Notification Scenario
1. NotificationSink calls the
subscribe operation on NotificationSource
Notification
Source
Notification
Subscription
2.NotificationSource creates
a subscription
service
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 26/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 26
Notification
Sink
A Notification Scenario
1. NotificationSink calls the
subscribe operation on NotificationSource
Notification
Source
Notification
Subscription
2.NotificationSource creates
a subscription
service 3. Notification
Source returns alocator to the subscription
service
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 27/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 27
Notification
Sink
A Notification Scenario
1. NotificationSink calls the
subscribe operation on NotificationSource
Notification
Source
Notification
Subscription
2.NotificationSource creates
a subscription
service 3. Notification
Source returns alocator to the subscription
service
4.b The NotificationSink and
Subscription service interact
to perform lifetime management
4.a deliverNotification
stream continuesfor the lifetime of
NotificationSubscription
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 28/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 28
Notification
Sink
A Notification Scenario
1. NotificationSink calls the
subscribe operation on NotificationSource
Notification
Source
Notification
Subscription
2.NotificationSource creates
a subscription
service 3. Notification
Source returns alocator to the subscription
service
4.b The NotificationSink and
Subscription service interact
to perform lifetime management
4.a deliverNotification
stream continuesfor the lifetime of
NotificationSubscription
The sole mandatedcardinality: 1 to 1
subscribe
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 29/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 29
GT-OGSA Grid Service Infrastructure
OGSI Spec Implementation Security Infrastructure
System-Level Services
Base Services
User-Defined Services
Grid Service Container
Hosting Environment
Web Service Engine
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 30/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 30
q Transport Layer Security/Secure SocketLayer (TLS/SSL)
– To be deprecated
q SOAP Layer Security
– Based on WS-Security, XML Encryption,XML Signature
q GT3 uses X.509 identity certificates forauthentication
q It also uses X.509 Proxy certificates tosupport delegation and single sign-on,updated to conform to latest IETF/GGFdraft
GT3 Core: Security Infrastructure
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 31/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 31
GT-OGSA Grid Service Infrastructure
OGSI Spec Implementation Security Infrastructure
System-Level Services
Base Services
User-Defined Services
Grid Service Container
Hosting Environment
Web Service Engine
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 32/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 32
GT3 Core: System Level Services
q General-purpose services that facilitate the use
of Grid Services in production environments
q The 3.0 distribution includes the following
System-Level services:
– An Administration Service
– A Logging Service
– A Management Service
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 33/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 33
GT-OGSA Grid Service Infrastructure
OGSI Spec Implementation Security Infrastructure
System-Level Services
Base Services
User-Defined Services
Grid Service Container
Hosting Environment
Web Service Engine
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 34/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 34
GT3 Core: Grid Service Container
Includes the OGSI Implementation, security
infrastructure and system-level services, plus:
q Service activation, deactivation, construction,
destruction, etc.
q Service data element placeholders that allowyou to dynamically fetch service data values at
query time
q
Evaluator framework (supporting ByXPath andByName notifications and queries)
q Interceptor/callback framework (allows one to
intercept certain service lifecycle events)
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 35/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 35
GT3 Core: Grid Service Container (cont.)
Interface Layer
Transport Layer
Implementation Layer
Layers in the Web Services Model
OGSI Spec is here
Transport/Binding
Layer (GT3 supports
SOAP over HTTP)
Container is here
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 36/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 36
GT-OGSA Grid Service Infrastructure
OGSI Spec Implementation Security Infrastructure
System-Level Services
Base Services
User-Defined Services
Grid Service Container
Hosting Environment
Web Service Engine
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 37/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 37
GT3 Core: Hosting Environment
GT3 currently offers support for four Java
Hosting Environments:
q Embedded
q Standalone
q Servlet
q EJB
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 38/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 38
GT3 Core: Virtual HostingEnvironment Framework
q Virtual Hosting allows grid services to be
distributed across several remote
containers
q Useful in implementing solutions for
problems common to distributed computing
– Load balancing
– User account sandboxing
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 39/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 39
A Service Creation ScenarioIllustrating Redirection in Virtual Hosting
Client
Registry
Router
HE Starter
1. Froma knownregistry,the clientretrievesa factorylocator
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 40/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 40
A Service Creation ScenarioIllustrating Redirection in Virtual Hosting
Client
Registry
Router
. Fromknown
egistry,he clientetrievesfactory
ocator
HE Starter 2. The routerintercepts thecreateService
call on thefactory
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 41/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 41
A Service Creation ScenarioIllustrating Redirection in Virtual Hosting
Client
Registry
Router
. Fromknown
egistry,he clientetrievesfactory
ocator
2. The routerintercepts thecreateService
call on thefactory
HE Starter
3. The router passes the createServicerequest to the Host Env Starter
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 42/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 42
A Service Creation ScenarioIllustrating Redirection in Virtual Hosting
Client
Registry
Router
Service
. Fromknown
egistry,he clientetrievesfactory
ocator
2. The routerintercepts thecreateService
call on thefactory
HE Starter
3. The router passes the createServicerequest to the Host Env Starter
4.The HEStartercreatesa newHost Envas wellas the
service
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 43/58
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 44/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 44
A Service Creation ScenarioIllustrating Redirection in Virtual Hosting
Client
Registry
Router
Service
. Fromknown
egistry,he clientetrievesfactory
ocator
2. The routerintercepts thecreateService
call on thefactory
HE Starter
6. Using the service locator,the router redirectssubsequent client-serviceinteractions
3. The router passes the createServicerequest to the Host Env Starter
4.The HEStartercreatesa newHost Envas wellas the
service
5. The router returnsa service locator
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 45/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 45
GT3 Base Services
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 46/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 46
GT-OGSA Grid Service Infrastructure
OGSI Spec Implementation Security Infrastructure
System-Level Services
Base Services
User-Defined Services
Grid Service Container
Hosting Environment
Web Service Engine
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 47/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 47
MJS
MJSMJS
GT3 Base: Resource Management
q
GRAM Architecture rendered in OGSAq The MMJFS runs as an unprivileged user, with a
small highly-constrained setuid executable
behind it.
MMJFS
MJS
User 1
User 2
User 3
Master User MJFS
MJSMJFS
MJSMJFS
MMJFS: MasterManaged Job Factory
Service
MJFS: Managed JobFactory Service
MJS: Managed Job
Service
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 48/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 48
Client
GRAM Job Submission Scenario
Index
Service
From an indexvice, the client
ooses an
MJFS
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 49/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 49
Client
GRAM Job Submission Scenario
Index
Service
From an indexvice, the client
ooses an
MJFS
2. The client calls the
createService
operation on the
factory,
supplying
RSL
MMJFS
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 50/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 50
Client
GRAM Job Submission Scenario
Index
Service
From an indexvice, the client
ooses an
MJFS
2. The client calls the
createService
operation on the
factory,
supplying
RSL
MMJFS
MJS
3. The factorycreates a
Managed Job
Service
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 51/58
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 52/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 52
Client
GRAM Job Submission Scenario
Index
Service
From an indexvice, the client
ooses an
MJFS
2. The client calls the
createService
operation on the
factory,
supplying
RSL
MMJFS
MJS
3. The factorycreates a
Managed Job
Service
4. The factoryreturns a locator
5. The client subscribes to
the MJS’ status SDE and
retrieves output
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 53/58
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 54/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 54
GT3 Base: Reliable File Transfer
q Reliably performs a third party transfer between two GridFTP
servers
q OGSI-compliant service exposing GridFTP control channel
functionality
q Recoverable Grid Service
– Automatically restarts interrupted transfers from the lastcheckpoint
q Progress and Restart Monitoring
GridFTP
Server 1
GridFTP
Server 2
RFT
JDBC
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 55/58
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 56/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 56
GT3 User-Defined Services
q GT3 can be viewed as a Grid Service
Development Kit that includes:
– Primitives designed to ease the task of
building OGSI-Compliant Services– Primitives for provisioning security
– Base services that provide an infrastructure
with which to build higher-level services
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 57/58
June 2003 GGF8 Introduction to Globus Toolkit® 3.0 57
GT3 User-Defined Services (cont.)
ANT
User source files
GT3 Build Files
User Build File
GridService
executable
files
(Diagram inspired by
Borja Sotomayor’s
excellent tutorial on GT3)
8/14/2019 GT3 Overview
http://slidepdf.com/reader/full/gt3-overview 58/58
Future Directions of GT
q Standardization of container model
q Development of lightweight container/api
q Adding rich support for queries
q Further refinements of Base Service designs
q Pushing on standardizing at a higher level
than OGSI