Software Solutions & group Copyright © 2007 Intel Corporation Ralf Ratering, Hans–Christian Hoppe...

SoftwareSolutions&

groupCopyright © 2007 Intel Corporation

Ralf Ratering,Hans–Christian Hoppe

Intel Software&Solutions Group

GPE4UNICOREGrid Programming Environment for UNICORE

Copyright © 2007 Intel Corporation

How aboutother Grid Software?

GPE services run in the Globus Toolkit 4 container

GPE client framework works with UNICORE6 and ChinaGrid

What isGPE good

for?

GPE turns a collection of distributed computers/data centers into a Grid

Why doesIntel develop

GPE?

GPE is an Open Source technology demonstrator for customers and ISVs

Influence standardsDrive new technologies into

market

What doesGPE provide?

A full Grid Software stack including graphical user interfaces and programming libraries

?


GPE Current Status

• GPE included in UNICORE6 alpha6 release at www.unicore.eu/download/unicore6 – Application Client and GridBeans– Expert Client– BPEL workflow service– UNICORE-specific source code available

from CVS

•Main GPE development under http://gpe4gtk.sourceforge.net– Complete GPE sources available under BSD license– Server implementation uses Globus Toolkit 4

container


GPE EcosystemGPE4GTK as standalone solution

– Intel software testing– Proof of concepts at customers– Research and Education

GPE user interfaces and GridBeans– UNICORE 6– China Education Grid (Ministry of Education)– South Korea (K-Grid)

Support for virtual machine technology

– Cooperation with Globus team– Cooperation with CERN– Strong interest in industry


Why do we engage in UNICORE?

•Major Grid platform in European HPC community

•UNICORE is great for “HPC on demand”

•UNICORE6 implementation will have great performance and scalability

• Join forces in standards bodies


GPE Components

Globus Toolkit 4 Container

TSI

Atomic Services

ServiceRegistry

ResourceBroker

BPELWorkflowService

OSRepository

Service

ApplicationClient

PortalClient

ExpertClient

AdminClient

UNICORE6 container

Atomic Services

ServiceRegistry

ResourceBroker

BPELWorkflowService

TSI

GPE Client API

GridBeans GridBeans GridBeans File Manager


Atomic Services: Standard Grid Interfaces using WSRF

TargetSystem Factory

TargetSystemService

JobManagement

Service

StorageManagement

Service

FileTransferService

Add target system to Grid

TargetSystem

Resource

Hardware and software properties,

workload, etc.

Use target system

JobResource

Job status, description, owner, etc.

Managejobs

StorageResource

Available file space,

mount point, etc.

Managefile

transfers

Accessstorages

FileTransferResource

Transferred bytes, used port numbers,

etc.

Pro

pert

ies

Reso

urc

es

Serv

ices

Op

era

tions


How do GridBeans work?

Applications

Services GridBean

Service

ApplicationGridBean

download

Apps ClientExpert Client Portal Client

ApplicationGridBean

App2GB

App3 GB

App1GB

• GridBeans encapsulate application logic

• GridBeans provide Swing interfaces for Java clients and JSR-168 portlets for web portals

• GridBeans create JSDL descriptions from user input


Data Center 1

GPE: Running a Workflow

...

Atomic Services

Resource Broker

BPEL WorkflowService

send job description including workflow

Other web services orchestrate

Implements Target System Service Interface

Applications

Services

Utility


A GPE-BPEL Workflow

<invoke inputVariable="submitJobRequest" operation="Submit" outputVariable="submitJobResponse" partnerLink="targetSystem" portType="tss:TargetSystem" />

<assign> <copy> <from part="SubmitResponse" query="/tss:SubmitResponse/tss:JobReference" variable="submitJobResponse" /> <to partnerLink="jobManagement" /> </copy></assign>

<while condition="bpws:getVariableData('getJob1StateResponse', 'GetResourcePropertyResponse')/child::text()[1]!='READY' and bpws:getVariableData('getJob1StateResponse', 'GetResourcePropertyResponse')/child::text()[1]!='FAILED'"> <invoke inputVariable="getJobStateRequest" operation="GetResourceProperty" outputVariable="getJobStateResponse" partnerLink="jobManagement“ portType="u-rp:ResourcePropertiesPortType" /></while>

<invoke inputVariable="startJobRequest" operation="Start" outputVariable="startJobResponse" partnerLink="jobManagement" portType="jms:JobManagement" />

<while condition="bpws:getVariableData('getJobStateResponse', 'GetResourcePropertyResponse')/child::text()[1]!= 'SUCCESSFUL' and bpws:getVariableData('getJobStateResponse', 'GetResourcePropertyResponse')/child::text()[1]!='FAILED'"> <invoke inputVariable="getJobStateRequest" operation="GetResourceProperty" outputVariable="getJobStateResponse" partnerLink="jobManagement" portType="u-rp:ResourcePropertiesPortType" /></while>

Wait until job status is SUCCESSFUL or FAILED

Start the job: Start request is stored in startJobRequest

variable

Wait until the

job status is

READY or

FAILED

Retrieve the job EPR from the SubmitResponse

through jobManagement partnerlink

JSDLSubmit JSDL to target system

service


Implementing and Influencing Standards with GPE• Open Grid Forum (OGF)

– JSDL (Job Submission Description Language)• GPE-BPEL extension

– OGSA (Open Grid Services Architecture)• BES (Basic Execution Services)• DMI (Data Movement Interface)

– ByteIO (File Transfer and Streaming)

• OASIS– WSRF (Web Services Resource Framework)– WSN (Web Services Notification)– WSS (Web Services Security)– WSBPEL (WS-Business Process Execution Language)

• W3C– WS-Addressing, SOAP, WSDL, XML, etc.


Expert Client DEMO


Outlook: Using Virtual Machine Technology on the Grid

• Reliability: Partitions remain unaffected if other partitions crash

• Security: Protect sensitive user data in VM instance

• Flexibility: VMs can checkpoint and migrate during run-time– I want to move to a faster system, when it becomes available– My job has high priority, please free the fastest machine for me

• Configurability: Dynamically deployed OS images and applications on user request– I need two cores on a fast CPU with 1Gb of memory and RedHat Linux 9

• Management: Maintenance of systems, software and users in centralized services


GPE Virtualization

• Globus virtual workspaces integrated in GPE– Collaboration with ANL team led by Kate Keahey– Virtual target system factory invokes Globus

workspace service to create XEN virtual machines– OS Images are fetched from GPE OS Image

Repository Service– Available since GPE 1.3

• Create show cases with CERN in OpenLab collaboration– Use virtual machines for gLite testing– Implement prototypes for virtual machines hosting

gLite computing elements

• Create show cases with industrial customers


GPE Plans for 2007• Further exploit Intel technology

– Use virtualization, management and security capabilities on new Intel platforms

– Leverage virtualization technology in the Grid

• Move towards Enterprise SOA– Run proof of concepts at Intel customers and partners– Identify concrete business needs

• Harden HPC-centric implementation for production environments– Scalability, Robustness, Performance

• Implement prototypes of upcoming OGF specifications– participate in “interop feasts”


Backup


History: From UNICORE to GPE• Pallas:

– German HPC software company implemented UNICORE client software

– Commercial version UNICOREpro in 2003

– Other products included Vampir and VampirTrace (now Intel Trace Collector and Trace Analyzer)

• After Pallas’ acquisition by Intel in 2003, the new Open Source project GPE started– Idea: Support Globus and other Grid

Middleware implementations, too– Keep the good concepts from UNICORE

• Graphical User Interfaces• Resource Abstraction• Client Plug-in Concept• Ease of use


Data Center

Running a Simple Job

Target SystemService

ApplicationGridBean

send job description

TSI

JobMgmt. Service

File Transfer Service

transferinput/output

controljob

TSI TSI

GridBean creates JSDL description from user interface

Applications

Services

Utility

Software Solutions & group Copyright © 2007 Intel Corporation Ralf Ratering, Hans–Christian Hoppe...

Documents

Transcript of Software Solutions & group Copyright © 2007 Intel Corporation Ralf Ratering, Hans–Christian Hoppe...