CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the...
-
Upload
norma-holland -
Category
Documents
-
view
217 -
download
0
Transcript of CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the...
![Page 1: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/1.jpg)
CGW 04, Stripped replication for the grid environment as a web service 1
Stripped replication for the Grid environment as a web service
Marek Ciglan, Ondrej Habala, Ladislav HluchýInstitute of informatics Slovak Academy of Sciences
![Page 2: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/2.jpg)
CGW 04, Stripped replication for the grid environment as a web service
2
Stripped Replication for Grids
Overview
• Replication in Grid environment• Principles of stripped replication (SR)
method• Optimization of stripped replication• Prototype Implementation as a Web
Service• Experimental Results• Future Work
![Page 3: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/3.jpg)
CGW 04, Stripped replication for the grid environment as a web service
3
Stripped Replication for Grids
Replication in Grid environment
• Creation of multiple copies of single data source across Grid infrastructure
• Replication increases data availability• RLS - Replica Location Service• Grid monitoring services – network
monitoring
![Page 4: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/4.jpg)
CGW 04, Stripped replication for the grid environment as a web service
4
Stripped Replication for Grids
Replication in Grid environment
Storage Element 1
File 1
Storage Element 2
Storage Element 3
![Page 5: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/5.jpg)
CGW 04, Stripped replication for the grid environment as a web service
5
Stripped Replication for Grids
Replication in Grid environment
Storage Element 1
File 1
Storage Element 2
Storage Element 3
File 1
![Page 6: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/6.jpg)
CGW 04, Stripped replication for the grid environment as a web service
6
Stripped Replication for Grids
Replication in Grid environment
Storage Element 1
File 1
Storage Element 2
Storage Element 3
File 1
![Page 7: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/7.jpg)
CGW 04, Stripped replication for the grid environment as a web service
7
Stripped Replication for Grids
Replication in Grid environment
Storage Element 1
File 1
Storage Element 2
Storage Element 3
File 1
File 1
![Page 8: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/8.jpg)
CGW 04, Stripped replication for the grid environment as a web service
8
Stripped Replication for Grids
Stripped Replication - Principles
• Transfer from multiple Grid sites, in parallel• Transfer only a portion of file from each
Storage Element (SE)• Different file portions (stripes) are obtained
from different SEs• Parallel transfer increases replication speed• If SR is not managed properly, process
could be time consuming• Optimization of SR management is required
![Page 9: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/9.jpg)
CGW 04, Stripped replication for the grid environment as a web service
9
Stripped Replication for Grids
Stripped Replication - Optimization
Replicated data source
Replica 1
Replica 2 Replica 3
![Page 10: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/10.jpg)
CGW 04, Stripped replication for the grid environment as a web service
10
Stripped Replication for Grids
Stripped Replication - Optimization
Replicated data source
Replica 1
Replica 2 Replica 3
Replica 1
Replica 2 Replica 3
![Page 11: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/11.jpg)
CGW 04, Stripped replication for the grid environment as a web service
11
Stripped Replication for Grids
Stripped Replication - Optimization
Replica 1
Replica 2 Replica 3
![Page 12: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/12.jpg)
CGW 04, Stripped replication for the grid environment as a web service
12
Stripped Replication for Grids
Stripped Replication - Optimization
Replica 1
Replica 2 Replica 3
Replica 1
Replica 2 Replica 3
![Page 13: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/13.jpg)
CGW 04, Stripped replication for the grid environment as a web service
13
Stripped Replication for Grids
Stripped Replication - Optimization
Replica 1
Replica 2 Replica 3
Replica 1
Replica 2 Replica 3
Replica 1
Replica 2 Replica 3
![Page 14: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/14.jpg)
CGW 04, Stripped replication for the grid environment as a web service
14
Stripped Replication for Grids
SR Prototype Implementation
• Java programming language• CoG 1.2 API (GridFTP interface)• Integrated with EDG Replica Location
Service• EDG RLS API (RLS interface)• File Chunks – basic data units for transfer• Implemented as a Web Service
( motivation :OGSA, WSRF)
![Page 15: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/15.jpg)
CGW 04, Stripped replication for the grid environment as a web service
15
Stripped Replication for Grids
Service WorkflowStripped Replication Service
LFNGetGUID
![Page 16: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/16.jpg)
CGW 04, Stripped replication for the grid environment as a web service
16
Stripped Replication for Grids
Service Workflow
LFNGetGUID
ReplicaMetadataCatalog
Stripped Replication Service
![Page 17: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/17.jpg)
CGW 04, Stripped replication for the grid environment as a web service
17
Stripped Replication for Grids
Service Workflow
LFNGetGUID
ReplicaMetadataCatalog
Stripped Replication Service
GetPFNs
![Page 18: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/18.jpg)
CGW 04, Stripped replication for the grid environment as a web service
18
Stripped Replication for Grids
Service Workflow
LFNGetGUID
ReplicaMetadataCatalog
GetPFNs
LocalReplicaCatalog
Stripped Replication Service
![Page 19: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/19.jpg)
CGW 04, Stripped replication for the grid environment as a web service
19
Stripped Replication for Grids
Service Workflow
LFNGetGUID
ReplicaMetadataCatalog
GetPFNs
LocalReplicaCatalog
StrippedReplicationAlgorithm
Stripped Replication Service
![Page 20: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/20.jpg)
CGW 04, Stripped replication for the grid environment as a web service
20
Stripped Replication for Grids
Service Workflow
LFNGetGUID
ReplicaMetadataCatalog
GetPFNs
LocalReplicaCatalog
StrippedReplicationAlgorithm
GridFTPSite 1
GridFTPSite N
. .
.
. .
.
Stripped Replication Service
![Page 21: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/21.jpg)
CGW 04, Stripped replication for the grid environment as a web service
21
Stripped Replication for Grids
Service Workflow
LFNGetGUID
ReplicaMetadataCatalog
GetPFNs
LocalReplicaCatalog
StrippedReplicationAlgorithm
GridFTPSite 1
GridFTPSite N
. .
.
. .
.
Register Replica
Stripped Replication Service
![Page 22: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/22.jpg)
CGW 04, Stripped replication for the grid environment as a web service
22
Stripped Replication for Grids
Properties of Stripped Replication
• Parallel transfer from multiple sites increases replication process speed
• Proposed optimization does not use network monitoring services
• SR adapts to varying nature of network load
• SR optimally distributes network load
![Page 23: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/23.jpg)
CGW 04, Stripped replication for the grid environment as a web service
23
Stripped Replication for Grids
Experimental Results
• Motivation test case– File size 223.9Mb – Best replica transfer with standard replication tool
(EDG rm) - 713 sec– Stripped replication (2 replicas) – 405 sec (43 %)– Stripped replication (3 replicas) – 209 sec (71 %)
• Average time saving– 2 replicas – 37% time saving– 3 replica – 55% time saving
![Page 24: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/24.jpg)
CGW 04, Stripped replication for the grid environment as a web service
24
Stripped Replication for Grids
Future Work
• Implementation refinement– Add logging functionality– Refine error states handling
• Evaluation of SR integration in Grid projects
![Page 25: CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.](https://reader031.fdocuments.in/reader031/viewer/2022032611/56649f435503460f94c6432b/html5/thumbnails/25.jpg)
CGW 04, Stripped replication for the grid environment as a web service
25
Stripped Replication for Grids
Thank you for your attention !