SAM-QFS: The Core for Building Solutions to Data Management … · 2007. 7. 17. · Sun Proprietary...

SAM-QFS: The Core for Building Solutions to Data Management Problems

Michael SelwaySr. Consultant EngineerArchive Software [email protected]

Sun Proprietary Page 2

SAM-QFS is the Core

• Hardware today is Open (“the Same”)> Servers> Disk Storage> Tape Storage

• Decision to put parts together based> More on Cost> Less on Difference

• Solving Real Data Storage Problem by> Connecting the hardware with software> Creative use of software


Matching Data Value to Storage TierPr

obab

ility o

f Re-

use

1 903 7 16 6030Days Since Creation

Fibre ChannelStorage

SATAStorage

Performance Tape

Capacity Tape/

Deletion

Low

High

$$$/GB

$/GB



obab

ility o

f Re-

use


Low

High

$$$/GB

$$/GB$/GB


SATAStorage

Performance Tape

Capacity Tape/

Deletion


SAM-QFS is the Core

• SAM-QFS is “Key Software” to linking Hardware into a “Basic Solution”

• SAM's File System Interface is Key to linking a Basic Solution into a “Problem Solution” Problem Solution


• Archiving long term and beyond

• Backup without the waste> Point in Time copies / versions> Restore in 1/20th the time> Building test systems> Part of DR/BC plan

• Tiered Storage Management balancing> Access> Value of Data> Cost of Maintaining Data

What SAM-QFS Can Create on it's Own


• Central Data Storage Management to simplify administration> Bringing Data Under Single Strategy> Transparent Technology Refresh> Modular Transparent Scaling> Protecting the Data of “Other” Systems'

Storage> Offer WORM file protection

What SAM-QFS Can Create on it's Own


Data Management Problems to Solve• 100s of Millions of Files

> Deep Archiving Single Entry Point> Deep Archiving Multiple Entry Points> High Performance Metadata Management

• Managing Stove Pipes of Storage> Multiple stand-alone NAS devices> Solaris and “Other” Operating Systems> Migrating between Technologies

• CIS/2 (Infrastructure Solution Platform)> Appliance for specific “local” solution> Small / Medium / Large

100s of Millions of Files

SAM-QFS: The Core for Building

Solutions to Data Management Problems


Key Points for Single Server Access

• File System appears “complete”• Access is transparent regardless of what

tier file system• Primary system remains “controlled size”• Files in Secondary system remain

unchanged and not backed up again• Disk space for Secondary systems is

relatively small – metadata “fragments”• Recovery is no “longer all or nothing”


Managing 100’s of Millions of files through Cascading SAM File Systems

SAM-FS Library

/sam1

Types of Organization

Date/TimeLocation

Source of GenerationProjects

Primary



SAM-FS Library

/sam1

/sam1a /sam1b /sam1c

Primary

Secondary


SAM-FS Library

/sam1

/sam1a /sam1b /sam1c


Secondary

Primary


Establish Architecture that Aligns to Modularity (One View: Date/Time)

Root

Title-n

Title-1

Date-n

Date-1

SubCat-n

SubCat-1

Category-n

Category-1Category Type (


Establish Architecture that Aligns to Modularity (One View: Date/Time)

Root

Title-n

Title-1

2004

SubCat-n

SubCat-1

Category-n

Category-1

2005 Symlink

2004 Symlink

Title-n

Title-1

2005

Separate, smallstatic SAMFile Systems

2006 Active

Patient-n

Patient-1

No physical FS is ever larger than one year's worth of data


Summary• Deep Archived file systems allows almost

infinite scale of data for large-scale archives> “Primary” view: Everything is visible – no application

change> Improved performance – largely reduced number of inodes

– Prime FS size limited to granularity of the break point– Metadata not fragmented

> Backup/recovery/media migration/manageability all improved> “Secondary” view:

> Smaller, granular FS> Easy to migrate to new media> Fast – No metadata fragmentation> Backup/recovery/restore on a fast, granular basis


Key Points for Multi-Server Access• All servers have access to all file systems• Each SAM-QFS file system has a percentage of the actual

data, but the exported directories in each system looks the same

• Server/Disk caches are sized as “incremental” increases to system capacity

• No Single Point of Failure > Loss of any server, data is accessible from other

servers> Loss of any file system, data is accessible from the

other file systems• Performance scales horizontally as file system structures

scale horizontally


/Root/Mercury

/Data/Data1/Data2

/Venus→/Earth→/Mars→

/Root/Mercury→/Venus

/Data/Data1/Data2

/Earth→/Mars→

/Root/Mercury→/Venus→/Earth

/Data/Data1/Data2

/Mars→

/Root/Mercury→/Venus→/Earth→/Mars

/Data/Data1/Data2

Mercury Venus Earth Mars

Access Portal(s)

Server1

Server4

Server2

Server3

Server“n”


Mercury Venus Earth Mars

Access Portal(s)

Server1

Server4

Server2

Server3

Server“n”

/Root/Mercury

/Data/Data

1/Data

2/Venus→/Earth→/Mars→/Jupiter →

/Root/Mercury→/Venus

/Data/Data

1/Data

2/Earth→/Mars→/Jupiter →

/Root/Mercury→/Venus→/Earth

/Data/Data

1/Data

2/Mars→/Jupiter→

/Root/Mercury→/Venus→/Earth→/Mars

/Data/Data

1/Data

2/Jupiter→

Jupiter/Root/Mercury→/Venus→/Earth→/Mars →/Jupiter

/Data/Data

1/Data

2

Managing “Other” Data




Provides Backup

5320

EMC

NetApp Filer

• up to 4 copies

SAM

NFSMigrator

NTMigrator

Pull


Provides Migration

5320

EMC

NetApp Filer

SAM-QFS

NFSMigrator

NTMigrator

Pull5320

EMC

NetApp FilerServer

Server

Push


Provides Disaster Recovery Copy(s)

5320

EMC

NetApp Filer

SAM-QFS

NFSMigrator

NTMigrator

Pull5320

EMC

NetApp FilerServer

Server

Push

SAM-QFS

NFSMigrator

NTMigrator


Provides Working Copies

5320

EMC

NetApp Filer

SAM-QFS

NFSMigrator

NTMigrator

Pull

Push

5320EMC

NetApp FilerServer

5320EMC

NetApp FilerServer

Server

Server


Migrator Possible Architectures

5320

EMC

NetApp Filer

NFS Migrator

SAM

•Shadow•Pshadow•Migrate

Win Client(Galaxy)

Windows

Windows Client

SAM

NFSMigrator

NTMigrator

Win Client(Galaxy)

5320 NAS

Win Client(Galaxy)

Active/Passive(just log onto other system)

Win Client(Galaxy)

CIS/2:Tiered Storage Appliance




What is a CIS/2?

• Complete “Tiered Storage in a Box”solution> Software

> SAM-QFS / SAM-Remote / WORM-FS> “Site Tools”> Migration Tools (Optional)

> Servers (Single/Dual)> Disk Storage (SATA (and FC) disk)> Tape Storage (LTO / L500 tape)

• “Plug” it in, dial in an IP address and Go!


CIS/2 Site Tools

• Monitor key SAM-FS & system functions. Generate alerts if user thresholds are exceeded:> File system capacity> Tape “pick” queue> Complete System Health check> Log snapshot and dump> Trend analysis > Media “sniffer”


Sun Content Infrastructure SystemEasy to Use, Secure, Flexible and Reliable Solution

ClientsClientsClients


Multi-Purpose System• Content Archiver

> Retention and media management for long term storage• Content Server

> Shared file system outperforms competition• Consolidated Backup for NAS

> Non-disruptive > Standard network interfaces

• Consolidated Backup for WinTel> Migrator allows Explorer-like interface for control of file

backup/restore and recall• Scalable Email Retention

> Embedded SAM/QFS allows for 100,000s of mailbox users and PB's of storage at very low cost and incredible low TCO

Sun Proprietary/Confidential: Internal Use Only

CIS/2 ConfigurationsSmall: (15-30 TB Archive)

Single T2000 (4 core)L500 Library (50 slot)16 TB SATA

Medium: (30-60 TB Archive)Dual T2000 (4 Core)L500 Library (100 slot)4.5 TB FC Disk Cache32-64 TB SATA Archive

Large: (30-128+ TB Archive)Dual T2000 (8 Core)1-2 L500 Library (100 slot)4.5 TB FC Disk Cache32-128 TB SATA Archive

2X Capacity2X Bandwidthcompared to a Small Unit

2X Capacity2X Bandwidthcompared to a Medium Unit

Open Discussion



SAM-QFS: The Core for Building Solutions to Data Management Problems

Michael SelwaySr. Consultant EngineerArchive Software [email protected]

SAM-QFS: �The Core for Building �Solutions to Data Management ProblemsSAM-QFS is the CoreMatching Data Value to Storage TierMatching Data Value to Storage TierMatching Data Value to Storage TierSAM-QFS is the CoreWhat SAM-QFS Can Create on it's OwnWhat SAM-QFS Can Create on it's OwnData Management Problems to SolveKey Points for Single Server AccessManaging 100’s of Millions of files �through Cascading SAM File SystemsManaging 100’s of Millions of files �through Cascading SAM File SystemsManaging 100’s of Millions of files �through Cascading SAM File SystemsEstablish Architecture that Aligns to Modularity (One View: Date/Time)Establish Architecture that Aligns to Modularity (One View: Date/Time)SummaryKey Points for Multi-Server AccessProvides BackupProvides MigrationProvides Disaster Recovery Copy(s)Provides Working CopiesMigrator Possible ArchitecturesWhat is a CIS/2?CIS/2 Site ToolsSun Content Infrastructure SystemMulti-Purpose SystemCIS/2 ConfigurationsSAM-QFS: �The Core for Building �Solutions to Data Management Problems

SAM-QFS: The Core for Building Solutions to Data Management … · 2007. 7. 17. · Sun Proprietary...

Documents

Transcript of SAM-QFS: The Core for Building Solutions to Data Management … · 2007. 7. 17. · Sun Proprietary...