© 2007 Open Grid Forum OGSA and OGF Reference Model OGSA-WG General Session OGF19 OGSA-WG session...

© 2007 Open Grid Forum

OGSA and OGF Reference ModelOGSA-WG General Session

OGF19 OGSA-WG session #2

29 January, 20074-5:30pm

Chapel Hill, NC

2© 2007 Open Grid Forum

OGF IPR Policies Apply

• “I acknowledge that participation in this meeting is subject to the OGF Intellectual Property Policy.”• Intellectual Property Notices Note Well: All statements related to the activities of the OGF and addressed to

the OGF are subject to all provisions of Appendix B of GFD-C.1, which grants to the OGF and its participants certain licenses and rights in such statements. Such statements include verbal statements in OGF meetings, as well as written and electronic communications made at any time or place, which are addressed to:

• the OGF plenary session, • any OGF working group or portion thereof, • the OGF Board of Directors, the GFSG, or any member thereof on behalf of the OGF, • the ADCOM, or any member thereof on behalf of the ADCOM, • any OGF mailing list, including any group list, or any other list functioning under OGF auspices, • the OGF Editor or the document authoring and review process

• Statements made outside of a OGF meeting, mailing list or other function, that are clearly not intended to be input to an OGF activity, group or function, are not subject to these provisions.

• Excerpt from Appendix B of GFD-C.1: ”Where the OGF knows of rights, or claimed rights, the OGF secretariat shall attempt to obtain from the claimant of such rights, a written assurance that upon approval by the GFSG of the relevant OGF document(s), any party will be able to obtain the right to implement, use and distribute the technology or works when implementing, using or distributing technology based upon the specific specification(s) under openly specified, reasonable, non-discriminatory terms. The working group or research group proposing the use of the technology with respect to which the proprietary rights are claimed may assist the OGF secretariat in this effort. The results of this procedure shall not affect advancement of document, except that the GFSG may defer approval where a delay may facilitate the obtaining of such assurances. The results will, however, be recorded by the OGF Secretariat, and made available. The GFSG may also direct that a summary of the results be included in any GFD published containing the specification.”

• OGF Intellectual Property Policies are adapted from the IETF Intellectual Property Policies that support the Internet Standards Process.


Agenda

• OGSA and OGF Reference model (Paul and Hiro, 1 hour)• HPC cluster use case• Glossary unification (Jem)• GWD-I: overview and ToC

• OGSA General session (Hiro, 30 minutes)

• Document status update• Tasks and priority review• March F2F meeting logistics• Telecon schedule


OGSA HPC cluster

• HPC cluster is set of computational servers connected high-speed network

• It is not grid in general sense• Administrator deploy multiple

applications on computing nodes• Users submit computational jobs

to cluster (management host)• Each job runs on one or more

computing nodes exclusively• More than one job can run on

each node in turns• EPS (scheduler) prioritize

submitted jobs based on administers policy

• EPS monitors and logs note usage for accounting and billing

Data Network2.5Gbps

Management Network100Mbps

Computing Node × 16

To external NetworkManagement Host

InfiniBandTM SwitchEthernet HUB


Workload

• Non interactive batch workload• User may submit multiple jobs

• E.g. workload consists of 100 jobs• Each job last several seconds to several days• Single job (run on single node) or parallel job (run

on multiple nodes)

• Each job is “abstract application”: top level entity of Grid Component tree diagram• Job is different form “transactions” of online

shopping application


Physical

VirtualizedOperatingEnvironment

AbstractApplication Definition

HPC Cluster

Index ServerIndex Server

Management Node Query

ServerQuery Server

Query Server

Query Server

Compute Node

Operating System

Operating System

Boot DeviceBoot DeviceServerServer Installation Media

Installation Media

PatchesPatches Cluster SoftwareCluster

Software

CHARMMService

CHARMMService

CHARMMBinary

CHARMMBinary

BLASTServiceBLASTService

BLASTBINARYBLASTBINARY

Paul’sBLAST

Instance

Paul’sBLAST

Instance

Paul’sBLASTData

Paul’sBLASTData

Hiro’s CHARMMInstance

Hiro’s CHARMMInstance

Hiro’s CHARMM

Data

Hiro’s CHARMM

Data

Batch job: grid component mapping

Each node is shared by multiple jobs, even though job runs exclusively


HPC profile, JSDL, and BES only covers here

EGA reference model defines

these usecases

OGSA HPC usecase diagram

Provision node

Provision OS

Deploy Application

Submit Job

Manage job

Undeploy application

Decommission OS

Decommission node

User

Administrator

Provision server pool

Decommission server pool

EGA reference model defines

these usecases

HPC cluster


Manage Service for enterprise apps

• Create • Configure • Start • Update • Notify • Monitor • Control • Stop • Unconfigure • Destroy


Reference model’s general life cycle

Unconfigured

Inactive

Active

Create Destroy

ConfigureUnconfigure

StopStart


BES’s basic state model

Pending Running Finished

Canceled

Failed

TerminateActivity request

System error/failure event

Successful termination of activity


Job Life Cycle

Running

Pending

Activate

Successful Completion

Failed(Give Up)

Defined

Submit

Failed(Retry)

Failed (Resubmit)

Withdraw

Cancel

Reschedule


RunningPending

Submit Activate Complete

Cancel

Failed


Paul’s Comments on BES (1)

1. Why are there 2 client focused interfaces (3 interfaces in total)? It is very natural for there to be 2 interfaces to a service/component, one focused on delivering client functionality and one for manageability. It is also natural to minimize the number of interfaces and the calls across them so as to aid maintainability. It would appear to me that the difference between the two BES client interfaces is that one allows activities to be created and for collections of activities to be monitored and managed. The other is focused on individual activities, but includes very similar functionality. There seems to be a lot of commonality. What are the reasons for them not being combined? Indeed as I read further I note that BES-Activity exposes no operations, so perhaps this interface is being deprecated and the document is in the process of being edited?

2. Why does the state model not end in one terminated state? Are there any real differences between the 3 enumerated end states other than how the activity ended up in the state?

3. Also it seems somewhat counter intuitive that if one is going to have three termination states, one should end up in the one state which reflects both success and failure (Finished) and a separate one for failures that prevent the activity from exiting in a controlled fashion. It seems like the transitions to the terminated state should be something like –

• Completed Successfully• Failed• Cancelled


Paul’s Comments on BES (2)4. Is there any notion of a retry, given that the state machine captures failure? Or

perhaps rescheduling? For example one could imagine additional useful transitions –

• Pending to Pending – i.e. rescheduling• Running to Pending – i.e. failed/retrying

Having these in the base profile or basic model would enable greater flexibility in implementing future profiles where one could request that the BES container implement retries or allow rescheduling. Or are you categorically stating that users will never be able to reschedule or request automatic retries. Again if this is the case, it should probably be explicitly stated. I would actually urge you to make the base specification as broad as possible in terms of the acceptable state changes at this level to avoid unnecessarily constraining yourself down the line. I think this is especially important given what is stated in 4.2.

5. Labeling of the state transitions would be most helpful, together with the adoption of standard UML state chart representation. Included is a state transition diagram which includes a few additional transitions which may, or may not be helpful/appropriate. :o)

6. 4.2 - State Specialization really seems to be indicating that this specification makes no assumptions with respect to sub-states that may be incorporated within an implementation but that such sub-states are acceptable as long as they do not introduce new state transitions between the standard, specified states. It would be very helpful to simply state this before all of the examples.

7. Do you expect to extend the BES model to activities that are not binaries running on operating systems?


CDDLM component Lifecycle Model


OGSA HPC binary status

??

Initialized

Create Destroy

Initialize terminate

????

V02: Separate job and binary

Application binary does not have “running” state.

Instantiated terminated


Reference models vs. services and information models

• The subject is really about the linkage between the services we define in OGSA (of course as guided by the reference model) and the information model that is exposed through those services

• In general this has been addressed in the way that SNIA and DMTF build Profiles. Terminology is a bit overloaded here... • The way we (OGSA) use profiles is in the WS-I sense. • Profiles in the SNIA and DMTF are about refining

information models to instance based models (cardinality restrictions on relationships, required and optional properties, etc) and the tie of those instances to tasks (service interfaces).

• We should consider how we are going to accomplish the same thing in OGSA.

Tom Maguire, 24 Oct., 2006


More Pervasive Service Lifecycle

• Planning• System Integration• Operation

• Maintenance• Feature addition

• Termination


Agenda

• OGSA and Reference model (Paul and Hiro, 1 hour)

• HPC cluster use case• Glossary unification (Jem)• GWD-I: overview and ToC



Agenda


• HPC cluster use case• Glossary unification (Jem)• GWD-I: overview and ToC



GWD-I: overview and ToC

• Title• Reconciling OGSA and EGA Reference Model

• Abstract• OGSA is SoA and web services based instantiation of OGF

Reference Model. Appling OGF Reference Model to current OGSA specs bring forth multiple useful findings including extensive use cases, revealing dependencies among components, and well-defined state transition.

• Proposed schedule• First draft: July 2007• Editor Submission: Oct. 2007• GFD Publication: Dec. 2007

• Editor• Hiro Kishimoto, Paul Strong, and Dave Snelling• OGSA-WG reference-model design-team or emerging reference-

model WG


OGSA and Reference Model ToC

• Background of this work• Interpretation

• OGSA is an instantiation of EGA Reference Model• Example: HPC cluster• Benefits

• extensive use cases• revealing dependencies among components• well-defined state transition• …

• Future plan• Reference model v2.0• Apply reference model to OGSA spec development including

OGSA v2.0• …


Reference Model BOF

• Thursday, February 6:00 pm - 7:30 pm at Azalea

• This session will focus on re-constituting and continuing the efforts of the EGA Reference Model working group within OGF.

• In order to ensure that we share a common language when describing grids, what they are comprised of, how those entities interact and so forth, we need an agreed upon glossary and set of terms. When defining standards, protocols and interfaces for grid computing, the sets of components (both services and resources) that comprise a grid, their relationships and their life-cycles need to be more formally described.

• Providing this formal description and associated terminology is the goal of the Reference Model working group. Whilst extant standards capture aspects of what is required, they are either incomplete or lack a Grid context and so need to be pulled together into a coherent whole. It is not the purpose of this group to replicate the work of established standards that in general meet our needs. Rather the work of this group will be to pragmatically develop the broader model that brings together, references and extends where appropriate.


Agenda



• Document status update• Tasks and priority review• March F2F meeting logistics• Telecon schedule


Document schedule (1)

Document name First draftavailable

Ready for PC

GFDpublication

OGSA Basic Security Profile 1.0 – Core

Sept. 2005Sept. 2005

Feb. 2006Feb. 2006

Dec. 2006Jan. 2007

OGSA Security Profile 1.0 – Secure Channel

Sept. 2005Sept. 2005

Feb. 2006Feb. 2006

Jan. 2007Feb. 2007

OGSA roadmap v1.1 Nov. 2006Nov. 2006

Dec. 2006

Mar. 2007?

March 2007

June 2007

Plan dateActual dateNew plan

completed delayed




Ready for PC

GFDpublication

OGSA EMS Architecture scenarios 1.0

July 2006July 2006

Nov. 2006Jan. 2007

Jan. 2007

April 2007

OGSA EMS Architecture scenarios capability

TBD(3~6 month)

OGSA EMS Architecture Pending


completed delayed




Ready for PC

GFDpublication

Guidelines for Information Modeling for OGSA Entities

(explain necessary process)GFD-I

Jan. 2006Jan. 2006

Feb. 2007March 2007Feb. 2007

April 2007July 2007

April 2007?

container information modelGFD-I

Jan. 2006Jan. 2006

Oct. 2006Jan. 2007Feb. 2007

Feb. 2007March 2007April 2007?

Information Modeling in OGSAXML schema and semantics Best practicesGFD-R.P Also wiki page

Nov. 2006Jan. 2007

Mar. 2007?

Feb. 2007

March 2007?

April 2007

Aug. 2007?

Information model profile document (BES container)

Proof of concept

Dec. 2006

Mar. 2007?

TBD




Ready for PC

GFDpublication

Reconciling OGSA and EGA Reference Model

Mar. 2007

July 2007

June 2007

Oct. 2007

Sept. 2007

Dec. 2007


completed delayed


Tasks and priority

• Security• David Snelling (and Alan or Andrew)

• Resource Management• Ellen Stokes and Fred Maciel

• OGSA EMS• Steven Newhouse and Andreas Savva

• Reference Model• Paul Strong, Hiro Kishimoto, and Jem Treadwell

• OGSA workflow• Andrew Grimshaw

• Roadmap• Chris Jordan


March F2F MeetingMarch 12 (Mon) noon to 14 (Wed) 5pm March 13 (Tue) 9am to 15 (Thu) 5pm March 14 (Wed) 9am to 16 (Fri) noon March 27 (Tue) 9am to 29 (Thu) 5pm

date venueSan Diego Supercomputer Center Oracle Conference Center in Redwood City

Most preferable


Agreed teleconferences restructuring

• The start times, frequency (twice a week) and overall duration (2 hours each) of the calls stay the same.

• We subdivide each two hour call into 1 hour slots, for a total of 4 slots a week.

• Of these 4 one-hour slots (at most) 1 slot a week will be for logistics/administration with the remaining slots for technical discussion. • The logistics slot will alternate between Monday and Thursday

each week. • Only a single technical topic may be scheduled to a slot. • A topic may not exceed its slot duration.

• If discussion on a topic finishes early or is canceled then the next topic does not start until its scheduled slot time.

• In other words the slots are independent of each other.


Telecon Topics

• Monday • EGA reference model (Hiro, Paul)• Roadmap (Chris)• Information modeling (Ellen, Fred)

• Thursday• Security (Alan)• Workflow (Andrew) 2nd half• EMS Arch scenarios (Andreas)


Telecon Schedule (1)

2007/2/5 Mon 4-6pm CT No-call

2007/2/8 Thur 7-9am No-call

2007/2/12 Mon 4-6pm CT EGA reference model

Logistics

2007/2/15 Thur 7-9amSecurity

Workflow (Meciel)

2007/2/19 Mon 4-6pmRoadmap

Information modeling

2007/2/22 Thur 7-9amEMS Arch scenarios

Logistics


Telecon Schedule (2)

2007/2/26 Mon 4-6pmEGA reference model

Logistics

2007/3/1 Thur 7-9amEMS Arch scenariosWorkflow (Steven)

2007/3/5 Mon 4-6pmRoadmap

Information modeling

2007/3/8 Thur 7-9amSecurity (OGSA-AuthZ joint)

Logistics


OGF19 OGSA sessions

# Day Time Title

1 Mon 3-3:30pm OGSA security session

2 Mon 4-5:30pm OGSA and EGA reference model

General session3 Tue 9-10:30am OGSA EMS session

4 Tue 11am-12:30pm

OGSA Information & data modeling

5 Tue 2-3:30pm OGSA workflow

Location: Bellflower


Full Copyright Notice

Copyright (C) Open Grid Forum (2007). All Rights Reserved.

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works.

The limited permissions granted above are perpetual and will not be revoked by the OGF or its successors or assignees.

© 2007 Open Grid Forum OGSA and OGF Reference Model OGSA-WG General Session OGF19 OGSA-WG session...

Documents

Transcript of © 2007 Open Grid Forum OGSA and OGF Reference Model OGSA-WG General Session OGF19 OGSA-WG session...