Stripped replication for the Grid environment as a web service
description
Transcript of Stripped replication for the Grid environment as a web service
![Page 1: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/1.jpg)
CGW 04, Stripped replication for the grid environment as a web service 1
Stripped replication for the Grid environment as a web service
Marek Ciglan, Ondrej Habala, Ladislav HluchýInstitute of informatics Slovak Academy of Sciences
![Page 2: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/2.jpg)
CGW 04, Stripped replication for the grid environment as a web service
2
Stripped Replication for Grids
Overview
• Replication in Grid environment• Principles of stripped replication (SR)
method• Optimization of stripped replication• Prototype Implementation as a Web
Service• Experimental Results• Future Work
![Page 3: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/3.jpg)
CGW 04, Stripped replication for the grid environment as a web service
3
Stripped Replication for Grids
Replication in Grid environment
• Creation of multiple copies of single data source across Grid infrastructure
• Replication increases data availability• RLS - Replica Location Service• Grid monitoring services – network
monitoring
![Page 4: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/4.jpg)
CGW 04, Stripped replication for the grid environment as a web service
4
Stripped Replication for Grids
Replication in Grid environment
Storage Element 1
File 1
Storage Element 2
Storage Element 3
![Page 5: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/5.jpg)
CGW 04, Stripped replication for the grid environment as a web service
5
Stripped Replication for Grids
Replication in Grid environment
Storage Element 1
File 1
Storage Element 2
Storage Element 3
File 1
![Page 6: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/6.jpg)
CGW 04, Stripped replication for the grid environment as a web service
6
Stripped Replication for Grids
Replication in Grid environment
Storage Element 1
File 1
Storage Element 2
Storage Element 3
File 1
![Page 7: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/7.jpg)
CGW 04, Stripped replication for the grid environment as a web service
7
Stripped Replication for Grids
Replication in Grid environment
Storage Element 1
File 1
Storage Element 2
Storage Element 3
File 1
File 1
![Page 8: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/8.jpg)
CGW 04, Stripped replication for the grid environment as a web service
8
Stripped Replication for Grids
Stripped Replication - Principles
• Transfer from multiple Grid sites, in parallel• Transfer only a portion of file from each
Storage Element (SE)• Different file portions (stripes) are obtained
from different SEs• Parallel transfer increases replication speed• If SR is not managed properly, process
could be time consuming• Optimization of SR management is required
![Page 9: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/9.jpg)
CGW 04, Stripped replication for the grid environment as a web service
9
Stripped Replication for Grids
Stripped Replication - Optimization
Replicated data source
Replica 1
Replica 2 Replica 3
![Page 10: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/10.jpg)
CGW 04, Stripped replication for the grid environment as a web service
10
Stripped Replication for Grids
Stripped Replication - Optimization
Replicated data source
Replica 1
Replica 2 Replica 3
Replica 1
Replica 2 Replica 3
![Page 11: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/11.jpg)
CGW 04, Stripped replication for the grid environment as a web service
11
Stripped Replication for Grids
Stripped Replication - Optimization
Replica 1
Replica 2 Replica 3
![Page 12: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/12.jpg)
CGW 04, Stripped replication for the grid environment as a web service
12
Stripped Replication for Grids
Stripped Replication - Optimization
Replica 1
Replica 2 Replica 3
Replica 1
Replica 2 Replica 3
![Page 13: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/13.jpg)
CGW 04, Stripped replication for the grid environment as a web service
13
Stripped Replication for Grids
Stripped Replication - Optimization
Replica 1
Replica 2 Replica 3
Replica 1
Replica 2 Replica 3
Replica 1
Replica 2 Replica 3
![Page 14: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/14.jpg)
CGW 04, Stripped replication for the grid environment as a web service
14
Stripped Replication for Grids
SR Prototype Implementation
• Java programming language• CoG 1.2 API (GridFTP interface)• Integrated with EDG Replica Location
Service• EDG RLS API (RLS interface)• File Chunks – basic data units for transfer• Implemented as a Web Service
( motivation :OGSA, WSRF)
![Page 15: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/15.jpg)
CGW 04, Stripped replication for the grid environment as a web service
15
Stripped Replication for Grids
Service WorkflowStripped Replication Service
LFNGetGUID
![Page 16: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/16.jpg)
CGW 04, Stripped replication for the grid environment as a web service
16
Stripped Replication for Grids
Service Workflow
LFNGetGUID
ReplicaMetadataCatalog
Stripped Replication Service
![Page 17: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/17.jpg)
CGW 04, Stripped replication for the grid environment as a web service
17
Stripped Replication for Grids
Service Workflow
LFNGetGUID
ReplicaMetadataCatalog
Stripped Replication Service
GetPFNs
![Page 18: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/18.jpg)
CGW 04, Stripped replication for the grid environment as a web service
18
Stripped Replication for Grids
Service Workflow
LFNGetGUID
ReplicaMetadataCatalog
GetPFNs
LocalReplicaCatalog
Stripped Replication Service
![Page 19: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/19.jpg)
CGW 04, Stripped replication for the grid environment as a web service
19
Stripped Replication for Grids
Service Workflow
LFNGetGUID
ReplicaMetadataCatalog
GetPFNs
LocalReplicaCatalog
StrippedReplicationAlgorithm
Stripped Replication Service
![Page 20: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/20.jpg)
CGW 04, Stripped replication for the grid environment as a web service
20
Stripped Replication for Grids
Service Workflow
LFNGetGUID
ReplicaMetadataCatalog
GetPFNs
LocalReplicaCatalog
StrippedReplicationAlgorithm
GridFTPSite 1
GridFTPSite N
. .
.
. .
.
Stripped Replication Service
![Page 21: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/21.jpg)
CGW 04, Stripped replication for the grid environment as a web service
21
Stripped Replication for Grids
Service Workflow
LFNGetGUID
ReplicaMetadataCatalog
GetPFNs
LocalReplicaCatalog
StrippedReplicationAlgorithm
GridFTPSite 1
GridFTPSite N
. .
.
. .
.
Register Replica
Stripped Replication Service
![Page 22: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/22.jpg)
CGW 04, Stripped replication for the grid environment as a web service
22
Stripped Replication for Grids
Properties of Stripped Replication
• Parallel transfer from multiple sites increases replication process speed
• Proposed optimization does not use network monitoring services
• SR adapts to varying nature of network load
• SR optimally distributes network load
![Page 23: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/23.jpg)
CGW 04, Stripped replication for the grid environment as a web service
23
Stripped Replication for Grids
Experimental Results
• Motivation test case– File size 223.9Mb – Best replica transfer with standard replication tool
(EDG rm) - 713 sec– Stripped replication (2 replicas) – 405 sec (43 %)– Stripped replication (3 replicas) – 209 sec (71 %)
• Average time saving– 2 replicas – 37% time saving– 3 replica – 55% time saving
![Page 24: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/24.jpg)
CGW 04, Stripped replication for the grid environment as a web service
24
Stripped Replication for Grids
Future Work
• Implementation refinement– Add logging functionality– Refine error states handling
• Evaluation of SR integration in Grid projects
![Page 25: Stripped replication for the Grid environment as a web service](https://reader035.fdocuments.in/reader035/viewer/2022062802/5681460c550346895db31aae/html5/thumbnails/25.jpg)
CGW 04, Stripped replication for the grid environment as a web service
25
Stripped Replication for Grids
Thank you for your attention !