1

OGSAFramework

for Higher-Level

andInternational Collaboration

UK e-Science Core Programme

Meeting

Malcolm Atkinson

31st January 2005

2

Contents & Story

Standards vitalmiddleware, applications development & deployment

Why standards matterInteroperability, replaceability, portability & direction

Why architecture (OGSA) mattersIntegration, completeness, abstraction & cooperation

UK’s investment in standardsAnnual investment – people and fundsOGSA is making this more effective

Recommendations Collaborate

3

Standards are Vital

What do we want from standardsFoundation for inter-domain operations

My software will work with your softwareAt the level of our business – policies, concepts, data & processes

Foundation for replacementI was using X’s Y-system – I can replace it with Z’s Y-systemEverything else is unchanged

Foundation for PortabilityMy application code works unchanged

on all e-Infrastructures

My (upper) middleware code works unchanged With alternative supporting implementations of the M/W it uses

We get standards only if we work hard

enough

good standards

require sustained

effort, insight &

negotiation

To be useful standards

must match requirements

and bewidely

adopted

4

Standards are Vital

What do we want from standardsFoundation for inter-domain operations

My software will work with your softwareAt the level of our business – policies, concepts, data & processes

Foundation for replacementI was using X’s Y-system – I can replace it with Z’s Y-systemEverything else is unchanged

Foundation for PortabilityMy application code works unchanged

on all e-Infrastructures

My (upper) middleware code works unchanged With alternative supporting implementations of the M/W it uses

E-Science is inter-

domain, international

&multi-

platformE-Science requires

interoperability to function &

inter-manageability to be efficient

5

Standards are Vital

What do we want from standardsFoundation for inter-domain operations

My software will work with your softwareAt the level of our business – policies, concepts, data & processes

Foundation for replacementI was using X’s Y-system – I can replace it with Z’s Y-systemEverything else is unchanged

Foundation for PortabilityMy application code works unchanged

on all e-Infrastructures

My (upper) middleware code works unchanged With alternative supporting implementations of the M/W it uses

Required to improve

while sustaining function &

service

Opportunity for

competition & market

forces

6

Standards are Vital

What do we want from standardsFoundation for inter-domain operations

My software will work with your softwareAt the level of our business – policies, concepts, data & processes

Foundation for replacementI was using X’s Y-system – I can replace it with Z’s Y-systemEverything else is unchanged

Foundation for PortabilityMy application code works unchanged

on all e-Infrastructures

My (upper) middleware code works unchanged With alternative supporting implementations of the M/W it uses

Agreed APIs & semantics

save porting, testing,

maintenance, documentation & training costsRecent

OGSA-DAI experience

shows costs are serious

Lack of portability will drive

developers of scientific code away

7

Importance of Standards - caveats

Waiting for standardsDoesn’t keep pioneering users & projects happy

Competing standardsAlmost as bad as no standards

Partially adopted standards – no enforcementA real nuisance – try SQL – see swissSQL

Industry doesn’t always have the same goalsWaiting for their decisions / endorsement

Backs their business agendaAgainst our collaboration agenda

Commercially supported implementationsMay not happen for every standard we needMay be too lateMay choose different performance trade offs

Open source implementations are neededJoin in improving them – don’t keep starting again!

If you think standards are expensive,try chaosto misquote

Roy Crock, Founder of McDonalds

8

Health Warning

We build e-InfrastructureTo make it easier to conduct researchMulti-purpose infrastructureMulti-domain collaborationsAutonomous and heterogeneous providers

Addressing challenge of distributed systems

Large scaleLong-livedCoordination and policy alignment part of the story

Transfer work & responsibilityFrom client and application developerTo infrastructure & infrastructure deployers

Community will beunforgiving if webase our standards on an unrealistic simplification because the scale of research interaction has not yet exposed well understood features of such distributed systems

9

Why architecture (OGSA) matters

10

Why Invest in OGSArchitecture

IntegrationE-Infrastructure assembled from many standardsThey must work together – to meet requirements

OGSA provides contextTo identify and integrate the requirementsTo assimilate and integrate current experienceTo review, initiate & steer standardsSo they can be integrated to deliver an e-Infrastructure

CompletenessAbstractionCooperation

11

Why Invest in OGSArchitecture 2

Integration

CompletenessAnalysis identifies missing subsystems

Not yet standardised Not yet finished Not yet started Not completely filling the expected niche

Consider Existing requirements Existing standards Existing implementations Existing knowledge of eventual requirements of distributed

systemsAbstractionCooperation

12

Architecture: Status

GRIDCOMPUTING

UTILITYCOMPUTING

DISTRIBUTEDCOMPUTING

Core Services

Base Profile WS-Addressing

Privacy

WS-BaseNotification

CIM/JSIM

WSRF-RAP

WSDM

WS-Security

Naming

OGSA-EMSOGSA Self Mgmt

GFD-C.16

GGF-UR

Data Model

HTTP(S)/SOAP

Discovery

SAML/XACML

WSDL

WSRF-RL

Trust

WS-DAI

VO Management

Information

Distributed query processing

ASP

Data CentreUse Cases &Applications Collaboration Multi MediaPersistent Archive

Data Transport

WSRF-RP

X.509

StandardEvolvingGapHole

Slide from Dave Snelling & Mark Linesch November 2005

13

Why Invest in OGSArchitecture 3

IntegrationCompleteness

AbstractionRequire higher-level abstractions

More easily explained to users Better support for application developers Better isolation from underlying / changing platforms Better opportunity for e-Infrastructure optimisation

Analysis & re-factoring of larger function space

Select recurrent structures Push for higher-level standards

Cooperation

14

Why Invest in OGSArchitecture 4

IntegrationCompletenessAbstraction

CooperationOGSA partitions the e-Infrastructure implementationEncourages independent concurrent & coordinated

Development or evaluation of each part’s standards R&D on implementation of each part

Promises assembly of the partsBasic profile provides context for concurrent R&D

Context for each M/W developer to build against Reduced interdependence – each can deliver if others don’t

Focus effort on reaching

minimum threshold

that makes this work

16

Contributions to Apache All Globus software under an Apache-like licence Contributed over past 2 years

– Many bug fixes and extensions to Axis & Tomcat, etc> E.g. WS-Addressing & WS-Security

– Several GA developers on repository commit team Two new core WS Apache Incubator projects

– Apollo – the WSRF code> Java, C & Perl

– Hermes – the WS-Notification code Test process with these before contributing more

18

UK’s investment in standards

19

Gridnet1 Data – Three years

Active in 42 GGF Groups19 (co) chairs1 Member GFAC5 Members GFSG4 Area DirectorsActive in IETF, OASIS & W3CAuthors on 9 out of 41 GGF documentsMany report high value

Community

+ Platinum sponsor

Much imminent output

OGSA + SAGAJSDL, DFDL, InfoDDAIS…

>6 person-years/yearNot including industryNot GridNet CCLRC, GridPP, …Reference implementation

47 active via GridNet£130K/year GridNet13 universitiesNo academic staff are directly funded for standards work

20

UK Contribution to OGSA

OGSA HistoryFirst OGSA meeting November 2001ATF Recommend UK adoption April 2002

OGSA-DAIProposed December 2001Started February 2002Adopted OGSA modelFollowed OGSA path2000 downloads proto-reference implementation

Extensive consultation with data community

Limited by time availableNeeds more investment

OGSA-WGLeadership Dave SnellingDave Berry data area teamAuthorship

Dave Berry, Abdeslam Djaoui, …

Standards for OGSAJSDLDFDL, DAIS & InfoDNaming scheme, GSM, …

NextGRIDNext architectureLed by Dave SnellingOGSA as basisOGSA base profile as first step

21

Recommendations

22

UK Should Back OGSA more

Invest effort in OGSAInvestigating, evaluating, contributing, commentingImplementing profilesUsing it

It is the ONLY show in townWhich offers

integration, completeness, abstraction A foundation for collaboration

Support UK focus on Data Design TeamSupport new UK efforts in other design teams

EMS, Grid markets, JSDL, GSM, mySpace, …

23

Use OGSA for Collaboration

“All Hands” to Reach OGSA Basic ProfileSufficient platform, context & frameworkFor safely partitioning further R&D

Agree a division of workUpgrading / alternative trade-off componentsNew componentsHigher level facilities

Minimise duplicationMaximise combined efforts to deliver

Function, Stability, Quality & Abstraction

Bury the egos,

project competition & national pride silos

24

Comments & Questions

Reserve Slides

26

What’s This Abouta Globus Company?

Univa was announced December 13, 2004– http://biz.yahoo.com/prnews/041213/nym040_1.html

– http://www.univa.com

Steve Tuecke is CEO Carl Kesselman and Ian Foster are in advisory role

– Remain at USC/ISI and Argonne Basic concept: “Redhat Linux for Globus”

This will NOT affect the GT open source policy– On the contrary, Univa is already contributing resources to open

source software This WILL allow greater industrial involvement and

investment in Grids

27

How to Get Involved

Become a GT4 Friend! Open group of people from various organizations

working with GT4 pre-release code and documents– Reporting problems in code and documents

– Contributing ideas, tests, documentation

– Building GT4-enabled applications Weekly telephone calls Discussion list

– To subscribe to the GT4 friends list, send an email to majordomo@globus.org which contains the words “subscribe gt4-friends” in the message body

Significant input from Terry Harmer and his UK ETF evaluation team

28

29

Java Services in Apache AxisPlus GT Libraries and Handlers

YourJavaService

YourPythonService

YourJavaService R

FT

GR

AM

Del

egat

ion

Inde

x

Trig

ger

Arc

hive

r

pyGlobusWS Core

YourC

Service

C WS Core

RLS

Pre

-WS

MD

S

CA

S

Pre

-WS

GR

AM

Sim

pleC

A

MyP

roxy

OG

SA

-DA

I

GT

CP

Grid

FT

P

C Services using GT Libraries and Handlers

SERVER

CLIENT

InteroperableWS-I-compliant

SOAP messaging

YourJavaClient

YourC

Client

YourPythonClient

YourJavaClient

YourC

Client

YourPythonClient

YourJavaClient

YourC

Client

YourPythonClient

YourJavaClient

YourC

Client

YourPythonClient

X.509 credentials =common authentication

Python hosting, GT Libraries

30

CustomWeb

ServicesWS-Addressing, WSRF,

WS-Notification

CustomWSRF Web

Services

GT4WSRF Web

Services

WSDL, SOAP, WS-Security

User Applications

Reg

istr

yA

dmin

istr

atio

n

GT

4 C

onta

iner