PROGRESS: ICWS'2003

Web Services Communication within the

PROGRESSGrid-Portal Environment

Michał Kosiedowski

PROGRESS: ICWS'2003

PROGRESS (1)

• Duration: December 2001 – May 2003 (R&D)• Budget: ~4,0 MEuro• Project Partners

– SUN Microsystems Poland– PSNC IBCh Poznań– Cyfronet AMM, Kraków– Technical University Łódź

• Co-funded by The State Committee for Scientific Research (KBN) and SUN Microsystems Poland

PROGRESS: ICWS'2003

PROGRESS (2)

• Cluster of 80 processors

• Networked Storage of 1,3 TB

• Software: ORACLE, HPC Cluster Tools, Sun ONE, Sun Grid Engine

PROGRESS: ICWS'2003

PROGRESS - architecture

PROGRESS: ICWS'2003

PORTAL

HPCRESOURCES

GRIDMANAGEMENT

SYSTEM

GRID SERVICEPROVIDER

4-tier new4-tier newgrid-portal environmentgrid-portal environment

Grid-Portal Environment

PORTAL

HPCRESOURCES

GRIDMANAGEMENT

SYSTEM

3-tier classical3-tier classicalgrid-portal grid-portal

environmentenvironment

PROGRESS: ICWS'2003

PROGRESS GPE

Co

nte

nt P

rovid

er

We

bse

rvice

Se

ssion

Be

an

En

tity Be

an

s

PROGRESS: ICWS'2003

Grid Service Provider (1)

• Job Submission Service– create, modify, delete and submit computing

jobs (single task and "workflowed"), copy jobs– manage job tasks, including: setting

application parameters, adding references to input and output files, setting task resources requirements

– monitoring the execution of jobs– change task status (used by the grid resource

broker)

PROGRESS: ICWS'2003

Grid Service Provider (2)

• Application management service– add and remove applications, modify

application definitions– manage virtual applications

PROGRESS: ICWS'2003

Grid Service Provider (3)

• Provider management service– add and delete services, modify their

descriptors– add and delete instances of services

– service descriptors include URL at which the service is available, the service namespace reference (URN) and the service WSDL reference

PROGRESS: ICWS'2003

Grid Service Provider (4)

• Short news service– add, modify, delete and read news– add and delete own instances

(informational services are examples of multiple instance services)

PROGRESS: ICWS'2003

Grid Resource Broker

• submit jobs for execution in the grid

• retrieve a list of jobs• get the status of a task• suspend, resume and cancel jobs• register for notifications

PROGRESS: ICWS'2003

Data Management System

• directory mgmt.: add, remove and rename directories, retrieve root and current path, change path, list contents

• file mgmt.: add, remove and rename files, add, remove and retrieve physical location, add and remove archives, add and remove symbolic links

• metadata mgmt.: metadata scheme mgmt., retrieve list of schemes and attributes, assign schemes to files and edit values and attributes, search metadata repository

PROGRESS: ICWS'2003

DMS: architectureINTERNET

DBFile System

Data Broker

Data Storage

Mirror & Proxy

Data Storage Data Storage

MetadataRepository

SRS

UniTree

WS

GASSFTPGrid FTP(...)

Clients

PortalPortal

Grid brokerGrid broker

Migrating desktop

Migrating desktop

PROGRESS: ICWS'2003

DMS: internal structure (1)

• Metadata repository– create and remove files– add, delete and manage the meta-

information connected with files– add and delete meta-schemes

PROGRESS: ICWS'2003

DMS: internal structure (2)

• Data storage– create and delete files (physical data)– reserve container space– enable/disable access to file through available

data tranfer protocols (GASS, GridFTP, FTP)– lock and unlock files– retrieve information about the state of the file

(BARE, PERMANENT, IN TRANSFER)– retrieve information about container free space

PROGRESS: ICWS'2003

DMS: internal structure (3)

• Mirror & proxy module– serves as a read-only data storage

module for external data– manage replicas of external

databases

PROGRESS: ICWS'2003

Web Services Communication

HPC PortalGrid

Service Provider

DataManagement System

GridResource Broker

saveJob()getApplications()getTemplates()saveTaskOfJob()saveStdOfTask()submitJob()getUserJobs()getJobStatus()

listUserDirectory()addUserFile()

getUserFileLocation()

submitJob() changeTaskStatus()

PROGRESS: ICWS'2003

Authorization scheme

Porta

l GRID SERVICEPROVIDER

Identity serverRAD based

authorizationsystem

Logon

Authentication

Request

Method invocation

Token validation

Resource access

authorization

PROGRESS: ICWS'2003

PROGRESS HPC Portal

PROGRESS: ICWS'2003

SC2002 Demo

PROGRESS: ICWS'2003

PROGRESS future

• R&D finished within the project, test and deployment phase now

• PROGRESS HPC Portal is a bioinformatic thematic portal: other thematic scientific portals possible to deploy

• The grid service provider may be equipped with means of communication with multiple grids

• PROGRESS HPC Portal contents: adding new applications, adding new visualization applets, bioinformatic society building

PROGRESS: ICWS'2003

PROGRESS

http://progress.psnc.pl/

http://progress.psnc.pl/portal/

kat@man.poznan.pl