A Proposal ofA Proposal ofCapacity and Performance Assured StorageCapacity and Performance Assured Storage

in The PRAGMA Grid Testbedin The PRAGMA Grid Testbed

Yusuke Tanimura1) ， Hidetaka Koie1,2) ， Tomohiro Kudoh1) Isao Kojima1) ， and Yoshio Tanaka1)

1) National Institute of AIST, Japan

2) SURIGIKEN Co., Ltd.

2

Background• Support of data-intensive scientific applications is one of

the challenges of the PRAGMA grid testbed work.– Avian Flu Grid applications– Geo sciences applications– GLEON/CLEON applications

• Gfarm provides ...– Global file access using a single name space

• POSIX-like

– Efficient file replication among sites• Competitive to grid-ftp

– Excellent performance with data access locality• Each process just accesses a local disk drive.

• However, there is a missing part of the storage resource management from the view point of resource sharing.

3

Problem 1• Most PRAGMA sites are multitenant and a shared

storage tends to be a performance bottleneck.– Total performance is not satisfied.– Access conflict occurs at some of storage servers.

Application A (parallel job)

Compute servers of a cluster

A shared storage for the cluster (NFS, PVFS, Lustre, etc.)

Application B (striping I/O)

Each PRAGMA site

4

Problem 2• Using a remote storage over the Internet is required in

some use cases but ...– The remote storage is not fast enough, the performance is

unpredictable, or the disk space is not enough.– On the other hand, high-bandwidth or bandwidth-guaranteed

dynamic network (Ex. lambda paths) is available.

xx

xx

xLambda path network

Site C

x

Storage

x

x

Client

Performance?

Disk space?

Site BSite A

5

Our proposed storage (Papio)• Allow users to reserve performance in advance

– Specify date, time, and read/write throughput• For write access, disk space can be reserved, too.

• During reserved time, the storage servers are dedicated to the user or the user’s access is prioritized (SLA).– Use existing I/O control techniques for prioritization.

• Disk I/O scheduling– Expect stable disk throughput or performance prediction when using

flush disk (Ex. SSD).

• Flow control of I/O path

• Reserve buffer cache on storage servers

• Reservation interface– Provide a special command and a Web-services based

interface.• Collocation with network resources is also supported.

6

Our proposed storage (Papio)

File metadata service

Reservation management service

Storage server

Reserve request (by command)

Client node

Management server

Global ResourceCoordinator

Network ResourceManager

Storage ResourceManager (SRM)

Collocation

Web services based protocol

Storage server

Storage server

Flow control (by PSPacer)

Disk I/O control (by dm-ioband)

Administrate I/O controls according to the reservation

Reserve

Reserve request (by Web services based protocol, GNS-WSI3)

Proposed storage (deployed in a single site)

Application

7

Resource allocation• Papio allocates the storage resources (Disk space, I/O pat

h, etc.) to each application according to reservations.

Storage server Storage server Storage server Storage server Storage server Storage server Storage server

200MB/s 200MB/s 200MB/s 200MB/s 100MB/s 100MB/s 100MB/s

Application A

150MB/s

150MB/s

Application C Application C Application C Application C

60MB/sec 60MB/sec 60MB/sec 60MB/sec

60MB/s for each process

MPI-IO application, virtual clusters, etc.

Application B

420MB/s

140MB/s

140MB/s

140MB/s

Require high-throughput

8

Current status and future work• A prototype implementation is planned to be completed

by this Summer.– Not for production but for tests and demonstration

• Performance guarantee is challenging.– We first support,

• Dedicated use and then try to support prioritized use.

• Sequential read throughput (MB/sec) reservation– Write access is more complicated.– Need to study how much performance granularity can be guaranteed.

• If someone is interested in this, we can deploy the software on the AIST cluster for experimental use around PRAGMA 19.

9

Acknowledgement• A part of this research is supported by the Special

Coordination Funds for Promoting Science and Technology of Ministry of Education, Culture, Sports, Science and Technology (MEXT) in Japan.