Tiered File System without Tiers

Tiered File System without Tiers

Nick KirschDirector of Product Management, Isilon Systems

Tiered File System without Tiers© 2010 Storage Networking Industry Association. All Rights Reserved. 22

SNIA Legal Notice

The material contained in this tutorial is copyrighted by the SNIA. Member companies and individual members may use this material in presentations and literature under the following conditions:

Any slide or slides used must be reproduced in their entirety without modificationThe SNIA must be acknowledged as the source of any material used in the body of any document containing material from these presentations.

This presentation is a project of the SNIA Education Committee.Neither the author nor the presenter is an attorney and nothing in this presentation is intended to be, or should be construed as legal advice or an opinion of counsel. If you need legal advice or a legal opinion please contact your attorney.The information presented herein represents the author's personal opinion and current understanding of the relevant issues involved. The author, the presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information.NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK.


Abstract

Tiered File System - without TiersFile systems have evolved considerably, yet the vast majority of them are still a simple organizational layer on top of a block device. As the sheer amount of data scales in an organization, it becomes absolutely critical that file systems evolve as well. This presentation focuses on a modern file system which include native tiering capabilities, native per-file performance and protection capabilities, and advanced scalability.

Tiered File System without Tiers© 2010 Storage Networking Industry Association. All Rights Reserved.

You Face a Deluge of Data Growth

Worldwide File And Block Disk Storage Systems, 2005-2012*

(PB

)

File Based: 80% CAGR Block Based: 31% CAGR

* Source: IDC


Is Most of That Data Valuable?

5

The Long Tail – Chris Anderson (2006)

Business Critical

Cost and Value

Age and Relevance

Corporate Assets

Organizational Importance


What are the challenges?

CIO/Director of ITBudget – Storage Cost VS Value of DataBusiness Compliance and Security Requirements

Storage AdministratorPolicies for Managing DataPolicies for Managing User InteractionStorage System Management

End-UserForced to Manage DataForced to Understand Storage Landscape

6


In a Perfect World

Scalability with Flat Operating Costs

Average Cost Tracks Cost/Commodity Curve

Automated and Granular Data Classification

Automated and Granular Data Movement

Transparency to Application and End-User

Simple, Easy-To-Manage, Scales “Infinitely”

7


Data Management Options - 1

Turtle Strategy – Slow GrowthData-growth is significantly below Moore’s LawProvision Additional Storage When NecessaryMigrate All Data to New StorageLittle Impact on User Data Management

8



Pod Strategy – Silos of StorageData-growth matches Moore’s LawProvision Additional Storage On Regular CycleNew Data to New StorageUsers Manage Data Locations

9



Tiering/Hierarchical StorageTiering means establishing a hierarchy of storage systems based on service requirements such as performance, business continuity, security, protection, retention, compliance, and cost.

10


Today’s Tiering Implementations

“Stubbing”Copy/Delete SchemeLeaves References

ProsHelps End-User

ConsTechnical ChallengesIncreased ManagementSilos of StorageAffects Backup, DR, etc.

Software-LayerClient SoftwareAsset Management

ProsHelps Administrator

ConsUnfamiliar for Users3rd Party DependencySilos of StorageAffects Backup, DR, etc.

11


Tiering Decisions

12

Protection

Cost

Performance

Security


The File System – The Key

An Abstraction on Top of Storage MediaOrganizes Data into Files, Databases, Indexes

Network Attached System (NAS)File System is Understood by the Storage SystemClients and Storage System Share a Higher-Level Protocol

Storage Area Networking (SAN)File System is Understood by the Client (s)Clients and Storage System Share a Lower-Level Protocol

13


The File System – The Challenges

Tiering Multiple File Systems Introduces ComplexityTiering Requires Different Storage MediaEffective Scaling Requires Networking

File Systems are Media and Network AgnosticFile Systems interact with Volumes and LUNsFile Systems are ignorant of the underlying mediaFile Systems are built for single systems

Hardware Abstraction is No Longer NecessaryIt complicates scaling, management, and increases costs

14


Single File System

What Can You Do with a Single File System?

That spans multiple hardware tiers of storage …That can scale seamlessly from GBs to EBs …That allows old hardware to be removed …That allows new hardware to be added …That performs protection and security at a file-level …That can optimize for specific hardware and media …That can optimize for specific types of files over time …

You can implement Tiered Storage …

15


Building a Tiered File System

Critical Design PrinciplesThe File System Must Be Aware of All MediaThe File System Must Be ScalableThe File System Must Be Easy To Manage

ConsequencesFile System Provides Data ProtectionFile System Addresses Different Storage MediaFile System Accesses Networked ComponentsFile System Allows for Easy Addition and RemovalThe File System Must Provide Granular Policies

16


Implement Tiering with Scale-Out

Hardware Building Blocks – “Nodes”

Single File System and VolumeHigh Performance Network InterconnectIntegrated Multiple Tiers of StorageEasy-to-Use, Cost-EffectiveSAN or NAS

17

Scale-Out Storage System


Tiered Storage with Scale-Out

Different Hardware NodesGlobally distributed DRAM cacheBlazing Fast Tier 0/1 – SSD or SAS nodesMid-Range Tier 2 – performance SATACost-Optimized Tier 3 – dense SATA

Performance Nodes – All CPU, DRAM, no diskMeta-Data Acceleration

Per-File/LUN GranularitySingle File System, Single Volume, Single Namespace

18


Tiered Storage - Example 1

Executive Staff Files (and Exchange Mail Store)Optimize for Random Access and Meta-Data LookupsData Protection: +3 (survive three simultaneous failures)Use Tier-1 Storage Nodes (SSD or SAS)Automatically Migrate to Tier-2 (SATA) after 3 daysAutomatically Migrate to Tier-3 (dense SATA) after 7 daysStore Snapshots on Tier-3 (dense SATA) immediately

Optimizes for Immediate PerformanceMinimizes High-End Performance CostContinual Protection At All TimesCost Reduce “Old” Data Very Quickly

19


Tiered Storage – Example 2

VMware Images (both LUNs and files)Optimize for Random AccessDefault Protection: 2x (survive two simultaneous failures)Use Tier-1 Storage Nodes (SAS)Store Snapshots on Tier-2 (mid-range SATA)

Optimizes for Application PerformanceDifferent Protection Scheme for PerformanceCost Reduce as Value Declines

20


Tiered Storage – Example 3

Archive Policy – For Everything ElseDefault Protection: +2 (survive two simultaneous failures)Use Tier-3 Storage Nodes (dense SATA)Move Data to Tier-2 If Activity Increases

Optimize Cost Across the File SystemNo Manual InterventionNo User Visibility

21


Scale-Out is Perfect for Tiering

Scalability with Flat Operating CostsSingle File System, Single Point of Management

Average Cost Tracks Cost/Commodity CurveAutomatically Drive Data to the Most Cost Effective TierConstantly Minimize the Use of the Performance Tier

Automated/Granular Data Classification/MovementPolicy-Based, Per-File/LUN ClassificationProtection, Performance, Security, and Cost

Transparency to Application and End-UserSingle Name Space, No Name Space Tricks, No Stubs

Simple, Easy-To-Manage, Scales “Infinitely”22


Q&A / Feedback

Please send any questions or comments on this presentation to SNIA: [email protected]

Many thanks to the following individuals for their contributions to this tutorial.

- SNIA Education Committee

Nicholas Kirsch

mailto:[email protected]�

Tiered File System without Tiers

Documents

Transcript of Tiered File System without Tiers