University of California, San Diego February 17, 2004
description
Transcript of University of California, San Diego February 17, 2004
![Page 1: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/1.jpg)
www.nbirn.net
University of California, San Diego
February 17, 2004
BIRN Data GridRoman Olschanowsky
![Page 2: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/2.jpg)
Introduction
BIRN-CC’s purpose is to provide it’s consortium of neuroscience laboratories the ability to share, compute, and collaborate.
SRB provides the ability to transparently share data across remote sites.
![Page 3: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/3.jpg)
Outline
Introduction to SRB– Discussion on SRB basics– SRB Clients
Overview of BIRN Data Grid– Infrastructure– Topology– A BIRN site– SRB Stability– SRB Statistics
![Page 4: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/4.jpg)
Introduction to SRB
A distributed file system (Data Grid)– Client-server architecture.
– Abstracts physical
![Page 5: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/5.jpg)
What we are familiar with
![Page 6: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/6.jpg)
What we are not familiar with, yet
![Page 7: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/7.jpg)
How do they differ?
Logical Abstraction– Folders are NOT physical– Files do NOT inherit physical location– Everything is potentially distributed
Access Control– Permissions are NOT rwxrwxrwx– Permissions ARE on a object by object basis– Groups and permissions ARE more similar to NTFS
Domains– Provides geographical / logical grouping of users– Provides namespace scalability: john@harvard john@mit– Also doubles as groups
![Page 8: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/8.jpg)
mySRB
![Page 9: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/9.jpg)
Common Scommands
Sinit Senv Spwd Sls -l Sget Sget (Sget –m –b) Sbunload Sput Sput (Sput –m –b) Sbload
Scp Smv (logical) Sphymove (physical) Srm Scd Smkdir Srmdir Serror Schmod
Don’t forget to Sexit!
![Page 10: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/10.jpg)
BIRN Portal
![Page 11: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/11.jpg)
Grid Computation
Globus– lddmm jobs: JHU utilizes both TeraGrid and BIRN for the
calculation of shape differences between normal and diseased brain structures. Producing terabytes of resultant data stored back into BIRN data grid.
SRB Proxy Operations– Image conversions and manipulations via ImageMagick
operations. (convert)– Analyze Volume Slice Extractor (anslex)– Data is not transferred, result of operation is streamed to
user.
![Page 12: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/12.jpg)
The BIRN Data Grid
![Page 13: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/13.jpg)
The BIRN Data Grid
![Page 14: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/14.jpg)
The grid is in the details
![Page 15: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/15.jpg)
File Replication
Sls/home/Demo/SRB-Tutorial/files-2: Doc.txt
Sls -l/home/Demo/SRB-Tutorial/files-2: romanoly 0 z-ucsd-ncmir-nas1 15 2003-07-09-05.15 Doc.txt romanoly 1 z-jhu-cis-nas0 15 2003-07-09-05.16 Doc.txt romanoly 2 z-stanford-lucas-nas 15 2003-07-09-05.16 Doc.txt romanoly 3 z-umn-cmrr-nas0 15 2003-07-09-05.16 Doc.txt romanoly 4 z-uci-bic-nas0 15 2003-07-09-05.17 Doc.txt
![Page 16: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/16.jpg)
Components of SRB Space
DR
DR
MC
DR - Data RepositoryMC - Meta CatalogSC - SRB Client
SRB
SRB
SRB
SRB
SRB
SRB
SRB
SRB
SRB
SRB
SCSC
SC
SC
SC
DR
DR
DR
DR
DR
DR
DR
DR
![Page 17: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/17.jpg)
Physical RackBIRN 1.0 Rack BIRN 1.5 Rack
![Page 18: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/18.jpg)
Physical Rack Layout
DL380 NAS 0.5 to 2.0 TB
Cisco 4506
DL380 Grid POP
• Gigabit/10/100 Network Switch – Cisco 4506
• Cisco 3002 3DES Encrypted VPN for secure statistics and packet tracing
•Gigabit Ethernet Network Probe
• DL 380 NetTools Network Statistics System
• DL 380 General purpose compute platform
• DL 380 Grid POP
• SRB, Globus
• Dual Processor Linux with1GB memory
• Network Attached Storage – Gigabit Ethernet
• 0.5 TB to 2 TB
• UPS for Rack (208VAC/40Amp supply)
• Direct Power for redundancy (208VAC/30Amp)
GigE Net Probe
UPS
DL380 General
DL380 NetTools
Fiber Tap
LCD PopUp Screen/Keyboard
Cisco 3002
![Page 19: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/19.jpg)
SRB “Location” or “Slave Server”
SRB
SRB
“Location”
“Physical Resources”
z-jhu-cis-nas0
“jhu-cis-nas”
DR
z-jhu-cis-nas1z-jhu-cis-nas2
![Page 20: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/20.jpg)
Logical / Compound Resources
SRB
SRB“My-Resource”
“instant replication”
“fast archival”
“resource pooling”
![Page 21: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/21.jpg)
Monitoring Grid Status
0.7 TB
5.2 TB
0 TB
1.6 TB
0.8 TB
0.8 TB
3.2 TB
0.8 TB
2.4 TB
0.8 TB
0.8 TB
2.4 TB
1.6 TB
0.8 TB
5.0 TB
0.78 TB
0.08 TB
![Page 22: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/22.jpg)
Stability
BIRN Grid is up and available approximately 98%
Example of typical site uptime:
![Page 23: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/23.jpg)
4.2 Million Files
![Page 24: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/24.jpg)
6.6 Terabytes Used
![Page 25: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/25.jpg)
Logical Resources
![Page 26: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/26.jpg)
More Domains than Resources
![Page 27: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/27.jpg)
Collaboration Reports
![Page 28: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/28.jpg)
Collaboration Reports
![Page 29: University of California, San Diego February 17, 2004](https://reader036.fdocuments.in/reader036/viewer/2022070416/56815003550346895dbdcf3e/html5/thumbnails/29.jpg)
Conclusion
SRB provides BIRN’s neuroscience laboratories with the ability to share and collaborate.
Questions?
www.nbirn.net
www.sdsc.edu/srb