Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.
-
Upload
erin-dimery -
Category
Documents
-
view
217 -
download
2
Transcript of Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.
![Page 1: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/1.jpg)
Service Orientation Considerations PoC R1
David Groep, Scaling and Validation Programme
![Page 2: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/2.jpg)
Service Orientation
• Service Oriented Architecture (SOA)
"an application architecture within which all functions are defined as independent services with well defined invocable interfaces which can be called in defined sequences to form business processes".
• Web services is a collection of technologies, including XML, SOAP, WSDL, and UDDI,
• Thus Web services does not equal service-oriented architecture.
Source: UvA Grid Master IGC2005, Adam Belloum
![Page 3: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/3.jpg)
Service Decomposition
Source: UvA Grid Master IGC2005, Adam Belloum
![Page 4: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/4.jpg)
Service Orientation
• All functions are defined as services.
• All services are independent. Operate as "black boxes";
o external components neither know nor care how boxes are executed
o merely that they return the expected result.
• The interfaces are invocable; At an architectural level, it is irrelevant whether
o they are local or remoteo what interconnect scheme or protocol is used to effect
the invocation, o what infrastructure components are required to make
the connection.
Source: UvA Grid Master IGC2005, Adam Belloum
![Page 5: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/5.jpg)
Service Orientation
• Typically within a business environment Service means business functions, business
transactions, and system services.
• The difference in the types of services. Business functions are from the application's
perspective, non-system functions that are effectively atomic.
Services might be low-level or complex high-level (fine-grained or course grained) functions
Source: UvA Grid Master IGC2005, Adam Belloum
![Page 6: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/6.jpg)
Services are not Servers
• Web Servers (Apache, Tomcat, ISS) Talks HTTP(S) Hosts web sites and pages May be dynamic (.php, .asp, .jsp) Seen by people
• Web Services (Axis, GT4, .NET, …) Bind to a protocol (HTTP(S), SMTP, …) May be hosted in a container Have a syntax and semantics definition Are seen and used by programs
![Page 7: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/7.jpg)
Defining and building Services in WS-*
[See Introduction to Web Services Tutorial]
1. Start with the WSDL2. Then, generate your interfaces
• E.g. with GT4 tools or JWSDP• See appropriate tutorial(s): “Technologies for
Building Grids” or “Sotomajor”
3. Fill the stubs with an implementation4. Deploy your services in a container
EGEE NA3 Training Home has plenty of material:http://www.egee.nesc.ac.uk/trgmat/
![Page 8: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/8.jpg)
Part 2: Web ServicesCaveats and Deployment Hints
Hints, caveats and the constraints in Service Deployment in the VL-e PoC Environment
![Page 9: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/9.jpg)
Some general caveats
• Re-use services• Use generic services where possible
‘job submission service’, ‘data location srv’• Don’t over-do it
WS are not suitable for low-latency HPC or bulk file transfer
• We’re not running a course on WS-RF …
• Not all services (nor architectures) are mature jet Be prepared to make compromises … … as long as it’s in-line with our long-term goals
• Test it works with your favourite workflow system!
![Page 10: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/10.jpg)
Web Services hints
• Web Services are just a syntax
Define proper sementics and document that to make your service re-useable
Match up the semantics with related services
Make sure your services are compliant o with relevant standards (like WS-I Basic Profile)o with the chosen hosting system (GT4)
![Page 11: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/11.jpg)
SOA Editor
Cape Clear SOA Editor, Cape Clear 2003WSDL © EGEE Consortium and partners, see www.gLite.org
![Page 12: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/12.jpg)
Style
There are multiple ways to bind a portType• RPC/Encoded
In the WSDL: <soap:binding style="rpc"
transport="http://schemas.xmlsoap.org/soap/http"/>In Java, methods are like: public void method(String in1, BigInteger myNumber)
• Document/Literal (or wrapped)In the WSDL: <soap:binding style="document" ...>In Java, methods are like: public void method(PurchaseOrder po)
• Just because it’s no trouble: name the XML schema element referenced by an operation
(through a message element) the same as the operation itself This is the only requirement of document/wrapped which isn't
enforced by the WS-I requirements. Note that document/wrapped is a valid document/literal definition.
See also specific presentation on the web page on WDSL styles
![Page 13: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/13.jpg)
Services Hosting
• Usually, a service lives in a container
• Container takes care of Translating incoming SOAP message to appropriate
objects (Java objects, C structures, …)
Enforcing access control Invoking methods Packaging the results Destroying the invoked objects
![Page 14: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/14.jpg)
Containers
• Many containers, but not all the same
• Vary in what they support, e.g., Platform: JVM, .NET, Unix Language (Java, C, Python) WSDL bindings (rpc or document/literal) Security mechanisms (none or GSI)
• VL-e supplied container GT4 (Axis 2.0-RC2++) Modified to do WS-RF, -Addressing, -Notification With GSI Security framework Almost like standard Axis 2
![Page 15: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/15.jpg)
Implementation
• Remember that web services are stateless
Model state via Resources Don’t keep running after the service invocation is
complete Use database/memory based storage, not process-
based state retentiono Much more friendly on the CPUo Resilient to machine/VM restarts
The ultimate horror scenario: R-GMA producer & consumer threads, see Concurrency P&E paper on the web
![Page 16: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/16.jpg)
Security model
• We use SOAP over TLS This is the default in GT4 orders of magnitude faster than MLS but cannot do secure message forwarding
![Page 17: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/17.jpg)
PoC Environment
Hardware and Software
![Page 18: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/18.jpg)
RP, the P4CTB and the PoC
Application development
NL-Grid production clusterCentral mass-storage facilities
+SURFnetInitial compute
platform
Stable, reliable, testedCert. releases Grid MW & VL-
software
VL-e Proof of Concept EnvironmentVL-e Rapid Prototyping Environment
DAS-2, local resources
VL-e Certification Environment
NL-Grid FabricResearch Cluster
Test & Cert.Grid MW & VL-software
Compatibility
Flexible, test environment
Environments
Usage
Characteristics
Virtual Lab.rapid prototyping
(interactive simulation)
Flexible, ‘unstable’
Download RepositoryPoC InstallerCluster Tools
Developer CVSNightly builds
Unit testsstable, tested
releases
Integration testsFunctionality testsAdventurous application people
PoC Release n Release Candidaten+1
Developers environment
Tagged Release Candidates
RapidPrototypingEnvironment
![Page 19: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/19.jpg)
PoC: Hardware and Software
• The PoC Distribution is set of software• The PoC Environment is the ensemble of systems
running the PoC Distribution• The PoC Central Facilities (“CF”) are those systems
managed by the P4 partners and designated as such
![Page 20: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/20.jpg)
PoC Software Distribution
OS Red Hat Enterprise Edition R3
PoC installer
Middleware Globus Toolkit version 2.x as supported by VDT
Current grid services from LCG, EGEE
Globus Toolkit 4.0 (WSRF components)
Storage Resource Broker (version 3.3.1)
• Operating system supported for all operations: RHEL3 & compatibles will work on a variety of similar systems
• Middleware: Services & Hosting Environments
![Page 21: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/21.jpg)
PoC Software Distribution
Integrative Three systems Kepler, Taverna, Triana
OGSA-DAI as installed with GT4
IBIS (version 1.2)
Tools & Libraries ParaView (version 2.2.1) VTK (version 4.4) MESA (version 6.4 with GLUT)
ITK (version 1.4)
FSL (version 3.2)
MRICRO (version 1.39)
java SDK 1.4.2.x deployment/install
octave (version 2.1.69) MPItb (v2.1.6x) MatLabMPI (v1.2)
lamMPI (version x.x.x)
LUCENE (version 1.4.3)
Ant (version 1.6.2 )
SWI-Prolog (version 5.4.7-190)
R
Grid Weka (version 3.4.2)
Nimrod client software (v3.0.0)
GAT
Sesame (client)
![Page 22: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/22.jpg)
PoC Central Facility topology
![Page 23: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/23.jpg)
Service Deployment constraints CF
• Farm of GT4 containers provided on the CF Based on the concept of “VO Boxes” For now: login to these boxes via gsissh Not for compute-intensive work
• You cannot run a container continuously on worker nodes No inbound IP connectivity Resource management and utilization
• And you should not want to, because All services in a container content for CPU and (disk)
bandwidth JVM does not insulate services from each other
![Page 24: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/24.jpg)
Other constraints on the CF
• Worker nodes are allocated in a transient way
Jobs run in a one-off scratch directory No shared file system for $HOME
(and if it happens to be there don’t use it!) Jobs will be queued
o Short jobs get to run fastero Small jobs get to run fastero Priority depends on your VO affiliation
You can connect out, but cannot listen Your job is limited in wall time
![Page 25: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/25.jpg)
Central Hosting
• A common hosting environment is offered on a set of VO Boxes Provided: GT4.0 WS (as specified for R1) Visible from the outside world Highly monitored systems Can talk to the ‘inside’
• Requirements for deployment on the CF Appropriate AuthN/AuthZ must be used for all
services(this is available by default in GT4)
Allow for request tracability (e.g. via log files)
![Page 26: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/26.jpg)
High performance/throughput Services
‘I need lots of * for my service’• Parallelize it
Use fine-grained services decomposition Use MPI (e.g. via IBIS) Use dedicated programs for processing Submit via a resource broker
to the compute service
• Submit jobs to the compute service to run dedicated GT4 hosting environments with your service Keep in mind to submit these to the cluster where
your front-end is running on a VO Box
![Page 27: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/27.jpg)
Complex Services: job submission
• Use a job submission service GT4 WS-GRAM not (yet) supported on CF:
only single-cluster, security model not complete without use of the WSS
gLite CE WS (“CREAM”): also single cluster, security=compatible, deployment in 2006Q3
Brokered submissiono Select appropriate resources via GLUE schema
constraints & JDLo Submit directly via RB API code on client sideo Use RB job submission WS by Bart
![Page 28: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/28.jpg)
Brokered Job Submission Service
• Submit brokered jobs in JDLthrough WS-interface to the Matrix Resource Broker
• WSDL specificationhttp://mu7.matrix.sara.nl/SaraGridService/SaraGridServiceLCG.wsdl
• For detailed info: ask Bart Heupers• Future version of the basic Grid middleware will
have a native WMS WS interface
![Page 29: Service Orientation Considerations PoC R1 David Groep, Scaling and Validation Programme.](https://reader030.fdocuments.in/reader030/viewer/2022032516/56649c785503460f9492d5d3/html5/thumbnails/29.jpg)
Questions and Discussion!
http://poc.vl-e.nl/events/poc-ws-induction-20060221/