Space allocations in NeST
description
Transcript of Space allocations in NeST
Joseph StanleyComputer Sciences DepartmentUniversity of Wisconsin-Madison
[email protected]://www.cs.wisc.edu/condor
Space allocations in NeST
www.cs.wisc.edu/condor
Outline
› Overview of NeST
› Space allocations
› Implementation
› Big picture
› Measurements and Future Work
www.cs.wisc.edu/condor
Overview of NeST
› Network storage server designed for the Grid
› Flexibility Unprivileged, user-level software Multiple protocols
• Chirp, GridFTP, FTP, HTTP, NFS Multiple concurrency models
• Processes, threads, non-blocking Portable
www.cs.wisc.edu/condor
Overview of NeST
› Functionality Exports Unix based file system Space allocation ( Lot )
› Manageability User management – Dynamic
users Security – GSI authentication
www.cs.wisc.edu/condor
Dispatcher
Transfer MgrConcurrencies
Control flow
Data flow
Chirp FTP Grid FTP NFS
Common protocol layer
HTTP
Physical network layer
Storage Mgr
Lot Mgr
Physical storage layer
NeST structure
www.cs.wisc.edu/condor
Why space allocations ?
› Data generation and storage Data intensive applications Need to allocate space for temporary files
› Data migration Many simultaneous data flows
• Competition for storage space• Possible that all flows end in partial transfers
Need to ensure at least one succeeds
www.cs.wisc.edu/condor
Space allocations in NeST
› Lot – abstraction for space allocation
› Create lot for a specified size and duration
› User and group lots
› Guaranteed and best-effort lots
› Hierarchical lots
www.cs.wisc.edu/condor
Motivation for hierarchical lot
› SRMs on top of NeSTs
› Client requests streamlined through SRMs
› SRMs create a large lot
› SRMs distribute sub-lots among clients carved from the large lot
www.cs.wisc.edu/condor
Lot operations
› Create, Delete, Update› MoveFile
Moves files across lots
› AddUser, RemoveUser Lot level access control List of users allowed to request sub-lots
› Attach / Detach Associates a path to a lot
www.cs.wisc.edu/condor
Implementation - two choices
› Rely on the underlying quota system Kernel-assisted lot
› Provide the space guarantee within NeST User-level lot
› A third choice? Modify kernel
www.cs.wisc.edu/condor
Dispatcher
Transfer MgrConcurrencies
Control flow
Data flow
Chirp FTP Grid FTP NFS
Common protocol layer
HTTP
Physical network layer
Storage Mgr
Lot Mgr
Physical storage layer
Kernel-assisted lot
Quota system
www.cs.wisc.edu/condor
Kernel-assisted lot› Quota system provides the guarantee› Chirp-like protocols pose no challenge
Aware of transfer size
› GridFTP-like protocols ? Transfer size may be unknown Don’t worry at NeST level Quota system watches disk writes
www.cs.wisc.edu/condor
Dispatcher
Concurrencies
Control flow
Data flow
Chirp FTP Grid FTP NFS
Common protocol layer
HTTP
Physical network layer
Physical storage layer
Storage Mgr
Lot Mgr
Transfer Mgr
User-level lot
www.cs.wisc.edu/condor
User-level lot
› NeST provides the guarantee
› Chirp-like protocols pose no challenge
› GridFTP-like protocols ? Iterative data transfer Transfer manager repeatedly
queries lot manager
www.cs.wisc.edu/condor
So which one ?
› Kernel-assisted Pros
Less complexPOSIX access
ConsNeeds root privilegesDepends on quota system – Less portabilityCannot distinguish among different allocations for
the same userDoes not have a hierarchical organization
www.cs.wisc.edu/condor
So which one ?
› User-level Pros
Flexible, PortableHierarchical organization is easyEasy to differentiate among lots for the
same user Cons
Difficult to provide POSIX accessComplexity in the transfer and lot
managers
www.cs.wisc.edu/condor
Big picture
NeST
Job
DAGMan
SRM
NeST
NeST
Remote site
GridFTP/DiskRouter transfer
StorkRequest allocation
Chirp allocations
Activate 3rd party put
Chirp input
Chirp output
www.cs.wisc.edu/condor
Where are the overheads ?
› Lot operations
› Puts
› Recovery
www.cs.wisc.edu/condor
Lot operations
www.cs.wisc.edu/condor
Puts using Chirp
›With increase in transfer size overhead is amortized
www.cs.wisc.edu/condor
Puts using GridFTP
›With increase in transfer size, overheads become comparable
www.cs.wisc.edu/condor
Recovery time
›Compaction reduces lot overhead to a manageable level
www.cs.wisc.edu/condor
Future Work
› Advance reservations
› Combo - Space and disk bandwidth reservation
› Recovery management Transaction based lot operations
www.cs.wisc.edu/condor
Questions ?
› Demo on Wednesday Room 3351, CS building, 9am – noon
› More information available at www.cs.wisc.edu/condor/nest
› People John Bent, Joseph Stanley, Nick LeRoy, Alain
Roy Miron Livny, Andrea Arpaci-Dusseau, Remzi
Arpaci-Dusseau, Arie Shoshani