The Mass Storage System at JLAB - Today and Tomorrow
description
Transcript of The Mass Storage System at JLAB - Today and Tomorrow
![Page 1: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/1.jpg)
The Mass Storage System at JLAB - Today and Tomorrow
Andy Kowalski
![Page 2: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/2.jpg)
Jefferson LabFarm and Mass Storage Systems
2000
FCAL (100MByte)
100 mbit1000 mbit
SCSI
From Hall A,C DAQFrom CLAS DAQ
Work File ServersWork File Servers
Cache File ServersCache File Servers
DST/Cache File ServersDST/Cache File ServersTape ServersTape Servers
Batch and Interactive FarmBatch and Interactive Farm
DB ServerDB Server
![Page 3: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/3.jpg)
Farm
Total Batch Farm 6172.6 SPECint95 125 - Dual Processor Linux Systems4 - Dual Processor Solaris Systems
5 Interactive Farm Systems1 - Quad Processor Linux System2 – Dual Processor Linux Systems 2 – Quad Processor Solaris Systems
![Page 4: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/4.jpg)
Disk Storage
MetaStor (Work) File Servers – Qty. 2 256MB Memory 100mbit Ethernet Ultra Wide SCSI RAID Systems RAID-5 Disk Space – 4TB
FS3/FS4 - 1TB FS5/FS6 - 3TB
Serving NFS User Managed Disk Space
![Page 5: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/5.jpg)
Disk Storage cont.
Linux (Cache) File Servers – Qty. 9 Dual 650/700MHz Systems 512MB ECC Memory Gigabit Ethernet Mylex PCI RAID controller RAID-0 Disk Space – 5.8TB
Qty. 4 with 400GB Qty. 5 with 876GB
Linux (Red Hat 6.2) - 2.2.18pre11-va1.1smp Serving NFS Automatically Managed Disk Space
![Page 6: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/6.jpg)
Tape Storage
StorageTek PowderHorn 9310 350 tape mounts per hour Holds 6000 tapes
Redwood Tape Drives – Qty. 8 50GB tape capacity 10MB/second
9840 Tape Drives – Qty. 10 20GB tape capacity 10MB/second
![Page 7: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/7.jpg)
Tape Storage cont.
Open Storage Manager Computer Associates dropped support January
2000 Not distributed Two installations – mss1 and mss2
TapeServer Front-End Written in Java Makes two OSM servers act as one Stages data to/from disk from/to tape A user interface
![Page 8: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/8.jpg)
Jefferson LabFarm and Mass Storage Systems
2001
FCAL (100MByte)
100 mbit1000 mbit
SCSI
From Hall A,C DAQFrom CLAS DAQ
Work File ServersWork File Servers
Cache File ServersCache File Servers
DST/Cache File ServersDST/Cache File Servers
Batch and Interactive FarmBatch and Interactive Farm
DB ServerDB Server
Tape ServersTape Servers
![Page 9: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/9.jpg)
Projects
JASMineReplacement for OSMDistributed Data Movers and Cache
ManagersScalable to the needs of the experimentsSmart schedulingOff-site cache or ftp servers for data
exporting
![Page 10: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/10.jpg)
Projects cont.
File ExportsJASMine Cache Software
GRID Aware - PPDGFTPBBFTPWEBTape
![Page 11: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/11.jpg)
JASMine Logical Storage Organization
Store A logical entity made up of libraries, servers, data movers, and a
database.
Storage Group An object that belongs to a store and is itself a collection of
storage groups or volume sets.
Volume Set An object that belongs to a storage group and is itself a collection
of volumes.
Volume A unit of storage media.
Bitfile The copy of a file that has been copied into a store.
![Page 12: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/12.jpg)
StorageGroup 2
Store
StorageGroup 1
VolumeSet 1
VolumeSet 2
StorageGroup 3
StorageGroup 4
StorageGroup 5
Vol
VolumeSet 3
VolumeSet 4
VolumeSet 5
Vol Vol
Vol Vol
Bitfiles Bitfiles Bitfiles
Bitfiles Bitfiles
![Page 13: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/13.jpg)
JASMine Physical Storage Organization
Store A logical entity made up of libraries, servers, data
movers, and a database.
Library A set of volumes and drives.
Drive A media reader or writer.
Volume A unit of storage media.
![Page 14: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/14.jpg)
JASMine Services
Request Manager Handles user requests and queries.
Scheduler Prioritizes user requests for tape access.
priority = share / ( .01 + (num_a * ACTIVE_WEIGHT) + (num_c * COMPLETED_WEIGHT) )
Log Manager Writes out log and error files and databases. Sends out notices for failures.
Library Manager Mount and dismounts tapes as well as other library
related tasks.
![Page 15: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/15.jpg)
JASMine Services 2
Data Mover Dispatcher
Keeps track of available local resources and starts requests the local system can work on.
Cache Manager Manages a disk or disks for pre-staging data to and from
tape. Sends and receives data to and from clients.
Volume Manager Manages tapes for availability.
Drive Manager Manages tape drives for usage.
![Page 16: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/16.jpg)
Dispatcher
CacheManager
DriveManager
DriveDisk
LibraryManager
VolumeManager
Client
RequestManager
Scheduler
Data Mover
LogManager
LibraryManager
Database
RequestManager
![Page 17: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/17.jpg)
What to Expect in 2001
Work File ServersAdditional 5 TB Linux File Servers??
Cache File ServersAdditional 5-10TB
![Page 18: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/18.jpg)
2001 cont.
Tape StorageJASMine Installed5-10 Data Movers10 Additional Tape Drives (9840 or 9940)
![Page 19: The Mass Storage System at JLAB - Today and Tomorrow](https://reader034.fdocuments.in/reader034/viewer/2022051517/56815911550346895dc63ea5/html5/thumbnails/19.jpg)
The End