Data ManagementThe GSM-WG Perspective

Background

• SRM is the Storage Resource Manager

• A Control protocol for Mass Storage Systems

• Standard protocol: OGF GSM-WG– Chairs: A Shoshani (LBNL), J Jensen (STFC

RAL)

Who Is Using It?

• EGEE, OSG,…

• WLCG

• …

• Not only

academic

HPC

TIER 0

Country or ROC

Storage Element Implementations

• dCache (DESY+FNAL)• DPM (CERN)• CASTOR (CERN+RAL)• BestMan (LBNL)• Jasmine (JLAB)• StoRM (INFN/ICTP-EGRID)

1

1

1

1

11

What is a “Storage Element”?

Storage Element

Disk storage

Tape storage

THE (gLite) GRID

•Computing •Data Mgmt Metadata

Disk storageDisk storage

Disk Pools

What is a “Storage Element”

• Interface to mass storage systems

• Part of a loosely coupled grid

• Provide uniform (control) interface

• Protect the storage system against the grid

• Enable grid to efficiently use storage

What is a Storage Element?

DataTransfer

ControlInterface

(SRM)

InformationSystem

StorageElement

Used by EGEE, OSG,…

Six-seven different implementations

An SRM is a “Storage Element” but not vice versaGIN: Use SRB as Storage Element

Tech

• ~6 interoperating implementations– 362 storage elements deployed– 226 are SRM1 implementations– 6 SRM2 implementations (can’t be quite right)

• Some Open Source, some not

• Using SOAP over GSI sockets– Delegation needed for srmCopy– No WS-Anything

Where is the File

GUID

SURL SURL SURL

LFN LFN LFN

PhysicalFile

TURL

PhysicalFile

StorageElements

PhysicalFilePhysical

File

Information View (simplified)

StorageArea

StorageElement

VOInfo

ControlProtocol

Data TransferProtocol

Space?Resource

Accounting

Owner/Access

How is the file?

Capa-bilities

ExpirationMode

RetentionPolicy

AccessLatency

Replica Output CustodialReleaseWhen

expired

WarnWhen

expired

Perma-nent

Online Nearline(Offline)(Lost)(…)

How I reach the file?

File AccessParameters

OverwriteMode

AccessPattern

Connection

Yes NoWhen

DifferentTransfer Processing WAN LAN

Transfer Protocols

• WAN protocols– Must always support GridFTP– Suggestion to support HTTP/HTTPS

• GSI issue though - see tech slide• dCache supports http for reading

• Support LAN protocols– Secure vs insecure

LCG: GFAL

• LCG (will) decide to use GFAL – the “Grid File Access Library”

• It was decided to interface to EDG SE using SRM 1 interface

• For now using EDG RM

POSIX interface

SRM 1 client

EDG 2.1 Storage Element

Mass Storage

EDG SRM

EGEE: DICOM server supportThe GridThe Grid

Storage Element

WP10 DM2

DICOMServer

Metadata

Encrypt, anonymise

Metadata

Store keyStore patient metadata

Access control on metadata required; different ACLs for different types of metadata

Challenges

Services

Capabilities

Scale,Performance

Economy,Sustainability

Middleware

State of the Art

Users

Challenges

Users

Applications

Culture,History

Customermgmt

Usability

Users

Services

Trust

Availability

Accounting

Discovery

Services

State of the Art

WebServices

Virtualisation

Media

State of the Art

Middleware

Stability

Applications

MaintenanceSupport

Ease of installAnd Config

Middleware

Scale, Performance

Staging

Transfer rates Size of files

Number of files

Volume

Scale,Performance

Sustainability, Economy

Scale

Trust Dynamic

Agreement

Cost Model

Economy

Capabilities

Content

Access

Curation

SECURITY

Capabilities