Challenges in Storage Systems - dtc.umn.edu in Storage Systems: ... Disk-to-disk backup with tape...
Transcript of Challenges in Storage Systems - dtc.umn.edu in Storage Systems: ... Disk-to-disk backup with tape...
Tag line, tag line
Challenges in Storage Systems: A NetApp perspective
Deepak Kenchammana-HosekoteAdvanced Technology GroupNetApp
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Agenda
�What we do (context)�What we see happening (trends)�What we are doing about it (initiatives)�How you can help us.
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
NetApp Fact Sheet
� Founded in 1992 � Fastest growing storage company� 4 consecutive years of 30%+ growth� #6 Best Company To Work For� Headquartered in Sunnyvale
– Other engineering sites in: Pittsburgh, RTP, Boston, Bangalore India
� 75,000+ systems installed worldwide� 111 Petabytes shipped Q1 FY08� #1 in NAS PB shipped� #1 in NAS market share
'03
'04
'05
'06
'03
'04
'05
'06
FY07 $2.8 billion‘07
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Key NetApp Ideas
� Purpose-built appliance; does one thing well– One management model & one system to learn
� Simplicity– Simple on the outside ≠ simple on the inside– Easy to use, easy to set up
� Combination of then new research ideas– RAID [Patterson88] – Log-structured file system [Rosenblum92]
� All built from commodity components– Off-the-shelf x86 CPUs, memory, etc– NVRAM is main exception
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08) 5
Scale from 1 TB to >6,000 TB
NetApp DataFort™Data encryption for audio, video, and image storage
Data Security
Information Server IS1200Information classification and management for unstructured data
Information Lifecycle Mgmt
Data ONTAP®Operating System (FC SAN, IP SAN, NAS)
Virtual Tape Library
NearStore VTLDisk-to-disk backup with tape library emulation
Media Center Storage
FAS6000 seriesFAS3000 seriesModular enterprise-class storage; massive scalability
Storage Virtualization
V-SeriesDynamic virtualization for heterogeneous storage
StoreVault™Entry-level nearline archives under $3K
Entry NAS Direct Attach/Small Archive NAS
FAS2000 SeriesExpandable NAS for small to medium nearline applications
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Data ONTAP® from 30,000 feet…
� StorageRAIDWAFL
Pro
toco
ls
NetworkStack
Client
Client
Client
Client
Client NVRAM Disks
Net
wor
k Filer
…looks a lot like an operating system, but…– Optimized for efficient data movement
� Specialized interfaces between components
– Robust HW error recovery
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
WAFL file system
volinfo
Inodefile
File A File B
� No fsck. Ever! � Writes buffered in NVRAM
� Data never overwritten� Metadata stored in files
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Writes
IA IB
A1 A2 B1 B2
Superblock
Inode file
File blocks
I0
Contains inodesfor regular files and dirs
File A File B
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Writes (2)
IA IB
A1 A2 B1 B2 B2’
I0
Superblock
Inode file
File blocks
IB’
I0’
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Snapshots
IA IB
A1 A2 B1 B2’
IB’
I0 I0’
S0
B2
Inode file
File Blocks
Snapshotsuperblock
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Snapshots™ enable many features
� Efficient application recovery (SnapDrive)– MS Exchange Server & Oracle RDBMS
� Consistent image for backups (SnapMirror)– Only changed data need be mirrored remotely
� Compliant data repository (SnapLock)– Read-only online data
� Efficient creation of clones– Copy-on-write snapshot; cloning DBs very useful
� Quickly tell what’s changed (SnapDiff API)– Compare inode files of two snapshots
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Beyond a Single Controller
� Scaling up– What if one controller is not big enough?– Want a namespace that spans multiple controllers
� Scaling out– Add capacity & performance with more controllers– “pay as you go” model
� But what happens to manageability?– Filers should only scale in performance and capacity;
management should be just the same“ One filer is easy to use. 100 filers are a pain to use.”
Answer: Clustered ONTAP
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Clustered ONTAP Architecture
D-blade
N-blade
SpinNP
Protocol
DistributedVolume Location
Database
Clients
D-bladeD-blade
N-blade
…
NFS, CIFSiSCSI, FC
N-blade
WAFL
RAID
Storage
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Global Namespace in a Cluster
Namespace Root
BA C
R
C2
D
D1 D2
Clustered ONTAP® System
C D1 D2 C2 D C1
B A
C1
R
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Advanced Technology Group
� Under office of the CTO; 3yrs old– 25 Members and growing; 4 sites worldwide
� Pursue long term projects– Explore technology driving our strategic direction– Create opportunities beyond current product horizon
� University research investments– Coordination of research funding– Leverage investments through hiring, internships
Goal : Investigate new technologies, influence and create products
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
The Advanced Technology Group (2)
� University collaborations– Wisconsin-Madison, Carnegie Mellon, MIT, UCSC,
UCSD, Harvard, UIUC, Waterloo, Duke, Berkeley,…� Actively publish
– SIGMETRICS 2007 Best student paper– 6 papers with NetApp authors in FAST 2008 incl. Best
student paper– 2 papers in USENIX 2008
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Latent Sector Errors Study [Sigmetrics07]
� How individual sector errors affect data integrity� Examined >70,000 systems with over 1.3 million disks� Are our defense mechanisms good enough?
� Findings: 3.45% of 1.53 million disks with ≥ 1 LSE � 8.5% of SATA vs. 1.9% of FC disks
SATA
On average 77% of Latent Sector Errors discovered b y VERIFY
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
A Comprehensive Study of Failure Characteristics [FAST 2008]
How to best improve resiliency to HW failures? – Ex: RAID group layout
Shelf enclosure model has strong impact on failures
Failures are not independent
The AFR for disks and storage subsystems does not increase with disk size.
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Agenda
�What we do (context)�What we see happening (trends)�What we are doing about it (initiatives)�How you can help us.
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Three Categories of Innovation
� New invention– Create an entirely new technology– Launch new industries
� Monotonic improvement– Technologies that ratchet ever upward
� Exponential improvement– Monotonic improvement that follows an exponential curve
� E.g. DRAM, Processor Mips, Flash Memory, Disk size– Observation 1: The steep part of the curve can be as enabling
as a new invention– Observation 2: The steep part of the curve can break old
solutions� E.g. disk size : disk IOPs ratio
– Observation 3: Suddenly becomes disruptive when applied to new areas, displace existing technology� E.g. tape archive replaced by disk archive� E.g. primary disk storage replaced by flash memory
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Technology Drivers
� Performance� Capacity
� Security� Power
� Management and Complexity� New applications and value-add
� Global regulatory requirements
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Enabling New Applications and Invention
� Server virtualization� Ubiquitous computing
– Network computing– PDA’s as clients– Globally distributed data service
� Increasing modularity and scale– Reducing visible complexity– Self-diagnosing, self-repairing servers– Federating services across administrative and
organizational boundaries– End-to-end integrity and security
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
The Storage Business
� Primarily about providing “containers”– LUNs– Files– Directories– Volumes
� Value-add primarily in:– Container virtualization– Container management– Container reliability– Container access performance– Archive, backup– Data security and integrity
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Key Initiatives
� Building a better container� Scaleout
� Data Management� Data Protection and Retention
� The expanded vision…
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Building a better container
� Increased virtualization– Fine-grained container hierarchy– Further disassociation of logical from physical– Dynamic, fine-grained data selection
� Alternative and parallel hierarchies of containers
� Simplified and improved manageability– Manage relatively few large-scale “datasets” composed
of many fine-grained logical containers
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Building a better container (2)
� Make copies smart: – Leverage archive, backup as active assets.
� Compression and Dedup– Improve storage utilization
� In-place data encryption– Robust and secure key management
� Challenge is to combine these
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Scaleout
� Multi-controllers– Tightly integrated hardware– Embedded clustering– Incrementally expandable– Scales up and down
� Increasing modularity in the core software– Separate key components of the data path from each
other and from software infrastructure– Exploit user space for value-added software
� Increase absolute numbers of containers of all types� Striping file system
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Scaleout (2)
� Exploit scaleout technology to provide– Caching– Heterogeneous storage
� Hybrid storage hierarchies� Heterogeneous federations� Heterogeneous storage backends
� Storage and System resiliency– Self-diagnosing and self-repairing subsystems– Policy-driven fault recovery
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Data Management
� Powerful data management within the single system image cluster
� Layered on-box and off-box data and storage management tools– Spans multiple clustered and unclustered systems– Heterogeneous participants
� Abstract data set model simplifies management– Pushes complexity into the underlying software layers
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Data Protection and Retention
� Low (to zero) Recovery Point Objective (RPO), Recovery Time Objective (RTO) for mirrors
� Smart copies– Multi-use mirrors
� Advanced server and virtual server level DR� Integration with virtualized computing environments
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
The Expanded Vision
� Global Data Service: Federations– Many systems all interoperating to provide
� DR� Remote caching and Vserver presence� Archive and backup� A hierarchy of flexible virtualized containers independent of
the underlying hardware
– Connected across wide geographies� Weak interconnects� Different administration and trust zones
– Ultimately multi-vendor solutions� Requires standards
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
The Database Business
� Primarily about adding value to the “contents” of containers– Indexing– Query processing– Imposes structure on data to facilitate analysis by
reusable tools
� Storage systems do not generally impose any discernable structure aside from block or extent size– Containers are blobs
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Containers and Contents
� Add value to stored content– Extracting structure and information from “unstructured”
blobs of data– Indexing– Query processing– Extended Attributes and Metadata
� E.g. “file provenance”
� Tighter integration of database and storage systems� Add value beyond providing containers
– Produce information from data
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
Tooling to meet challenges� Software engineering challenge is enormous
– Not unique to NetApp– How can we build ever larger systems quickly enough to meet
accelerating market needs?– How can we make all this software reliable?– How can we prove its reliability?
� Big challenges in:– Software tools– System test and validation– Support infrastructure, diagnosis and resolution– Globally distributed development
� Are the programming languages and tools up to the task?– Facing a lot of inertia– MP is now the norm– Moving from C to C++ is not the answer
� We have many of the same problems we help our customers solve
© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)
How you can help us
� We are seeking collaboration and leverage in the academic community– Equipment donation– Funded research– Internships– Full-time hires
� We would like to share real world data– System logs, traces, …
� Hope to energize you to look at some of these very interesting problems