Economical Big Local Storage Tom Klingler, Kent State University

Economical Big Local StorageTom Klingler, Kent State University

CNI Spring 2013 Membership Meeting

April 4-5, 2013

San Antonio, TX

2Kent State University

Economical Big Local Storage

Storage Pods (present):

Storage Pod: $5,000.00 each x 3 pods = …$15,000.00

Minimum Storage of 36 TB (starting with 15 3-TB drives PER POD): $130.00 each x 15 drives x 3 pods =..$5,850.00

Present total cost: ………...………...……….$20,850.00

Present cost per (triple-redundant) gigabyte:

$20,850.00 / 36,000 gigabytes=……..….$0.58 with triple redundancy. (Actual gigabytes online: 108,000 )

3


Storage Pods (full): If we max out the storage pods, we can triple the storagespace without needing to buy additional pods: Storage Pod: $5,000.00 each x 3 pods = ……….$15,000.00 Maximum Storage of 108 TB: $130.00 each x 45 drives x 3 pods =………..….. $17,550.00 Cost per (triple-redundant) gigabyte:

($15,000.00 + $17,550.00) / 108,000 = ………....……$0.30 with triple redundancy. (Actual gigabytes online: 324,000 )

4


Storage Pod Solution

1. medium-term storage of digital objects

IS not DAM

IS dark archive

IS Staff storage tool

5



2. objects are known as “items” a.items can be: digital assets, images, movies, documents,

limited to basic 41 file types b. batch upload, zip upload, drag/drop upload c. items --> expiration date --> based on retention period

--> based on retention group d. expiration date warning email and expiration report e. items can be labeled as “significant” and then reviewed for potential long-term storage

6



3. Organization:

Workgroup ---Project --Item

--Item metadata

7

Economical Big Local StorageStorage Pod Solution

4. Metadata

a. metadata assigned by the system, such as file type and file size

b. users may provide other metadata about the item, such as title, description, creator, creation date, and retention group c. metadata for items in a project can be set automatically by uploading a ZIP Archive file d. certain metadata, such as creator, can be assigned to every item in a project at one time by editing the project information

•system will also harvest IPTC data from image files if it is present, including title, creator, creation date, and location

8



5. Users

a.can be members of any number of workgroups, b.and create any number of projects,

c. which can contain any number of items.

9



6. Hardware / Software a. 3 servers distributed throughout campus

b.36TB storage each, 15 drives each server (case holds 45 drives)

c. CentOS (Linux) OS on separate system boot drive

d. RAID6 (allows loss of 2 drives with no data loss)

e. single CPU

f. no 24/365 uptime expectation

g. biggest concern is data preservation10

Economical Big Local StorageStorage Pod Solution

6. Hardware / Software h.Interface for staff only; no public interface

i. master storage of filesNo need to store derivative files. They can be regenerated.

•daily sync between three pods if all checksums are valid fixity check (bitwise preservation)

checksum computed on server at file uploadchecksum file integrity check each file each nightadmin notification of checksum problem

11


12


13


14


15


16


17


18


19


20


21


22


23


24


25


26


27


28


29


30


31


32


33


34


35


36


37


38


39


40


41


42


43


44


45


46


47


48


49


50


Vendors

Case Construction:

http://www.protocase.com/products/index.php?e=Backblaze

http://www.45drives.com/

Hardware System Design by BackBlaze:

http://www.backblaze.com/

Digital Storage System Application:

http://carrickenterprises.com/

[email protected]

51

http://www.protocase.com/products/index.php?e=Backblaze

http://www.45drives.com/

http://www.backblaze.com/

http://carrickenterprises.com/

mailto:[email protected]


52


Thank you.

Tom Klingler

[email protected]

CNI Spring 2013 Membership Meeting

April 4-5, 2013San Antonio, TX

53

Economical Big Local Storage Tom Klingler, Kent State University

Documents

Transcript of Economical Big Local Storage Tom Klingler, Kent State University