Personal Storage to Enterprise Storage System - A brief journey through Storage Wonderland

Soumen Sarkarco-founder, Idean

2SoumenSarkar

Goal and audienceThe presentation aims to provide

a bird’s eye view on digital storage systems landscape at present

It presents generic details and is not intended to be any vendor-specific

If you are looking for an overview of storage-system in a data centre this presentation could be useful to you

3SoumenSarkar

Types of Storage Systems

Directly Attached StorageNetwork Attached storageStorage Area NetworkCloud storage - can be any

combination of above three but provides RESTful interface for storage access

4SoumenSarkar

Directly Attached Storage (DAS)

Server

• DAS - Storage box containing one or more hard disks connected directly to the host or server• DAS ranges from simple hard disk attached to your laptop to more complex enterprise-class server

disks

LAN

Enterprise DAS

5SoumenSarkar

Network Attached Storage (NAS)

• All users’ request served by the application servers and storage need from application servers are provided by NAS box(es). • Storage Capacity planning becomes simpler and centralized

LAN

NAS Server

Web Server Mail

Server DB Server

Server for Data Backup

File Access is provided by

NFS [for unix] or CIFS [for windows] protocol

File Server [unix/windows]

Server network

Users PC

6SoumenSarkar

Storage Area Network (SAN)

LAN

Fibre-Channel / IP SAN

Web Server

MailServer

Disk Array

Disk Array

DBServer

• SAN provides block storage to the application server

• Servers are connected through SAN switches (not shown in diagram)

• Each Appl server has its own Data organization /File System

• Each appl server can plan storage need independently

FileServer

Users PC

7SoumenSarkar

A simplified stack comparison forDAS, NAS, SAN

Application

File System

RAID

SATA FCSAS

DAS

Application

File System

RAID

SATA FCSAS

NAS

Application

File System

RAID

SATA FCSAS

SAN

Network

Network

NFS, CIFS

iSCSI, FC, FCoE

• In DAS, applications directly call File System calls for storage needs • In NAS, applications [in different box] talk to NAS server over NFS or CIFS protocols for all storage needs• In SAN, applications behave like DAS but File System in the server access storage in disk-array over iSCSI, or FC protocol

8SoumenSarkar

SATA,SAS,FC Disk Drive - A comparative view

SATA [Serial ATA] SAS [Serial Attached SCSI]

FC [Fibre-channel]

Performance Half-duplex,1.5 GB/s introduced 3.0 GB/s in 2005

Full-duplex3.0 GB /s introduced, 6.0 GB/s in 2007

Full-duplex2.0 GB/s introduced4.0 GB in 2004

Connectivity 1 device 1 metre cable SATA only

128 devices6 metre cableSAS & SATA

127 devices 15 metre cable FC only

Redundancy Single-port HDD Dual-port HDD Dual-port HDD

Availability 8x5 duty cycle6M hrs MTBF

24x7 duty cycle12M hrs MTBF

24x7 duty cycle12M hrs MTBF

Typical Cost [see zdnet]

Less than $1/GB $3-$4/GB $4-$5/GB

SATA is the cheapest, slowest and less reliable and FC is costliest but is faster and is more reliable and is predominantly seen in SAN, SAS is relatively new but provides best of the both

9

SCHEMATIC VIEW : A

JOURNEY FROM GIGABYTES OF

PERSONAL DATA TO

PETABYTES OF ENTERPRISE

DATA

10SoumenSarkar

Personal Storage System

Directly Attached Storage (DAS)

Public Cloud [Google Drive,

Microsoft SkyDrive

etc.]

• One’s personal data typically resides inside DAS and in public cloud storage [ Gmail, Yahoo-mail, flickr, youtube, Google Drive, SkyDrive, iTune]• Data back up is taken in another standalone DAS• Size ranges from few GB to few TB at present • Further data growth will see movement of data to Cloud

11SoumenSarkar

Enterprise Data also is growing fast

In the third quarter of 2012 (3Q12), total disk storage systems capacity shipped reach 7,104 PetaBytes or 7 Exabytes, growing 24.4% year-over-year - IDC

Storage capacity growth projection: (V stands for Virtualized) source

12SoumenSarkar

Enterprise Storage: a schematic view

DAS associated with Individual PC/Laptop

Data CentreNetwork attached

Storage [NAS]

Storage Area Network [SAN]

Storage Backup Infrastructure

• Enterprise manages data in the order of PetaBytes and can have either NAS or SAN or both• Data growth, frequency of data access are important consideration for planning storage infrastructure • Data Replication [synchronous or asynchronous] and Data backup policy are two major aspects of data centre strategy

13SoumenSarkar

Enterprise Storage II: Multiple Data Centre helps faster disaster recovery

DAS associated with Individual PC/Laptop

Data Centre

Network

attached

Storage [NAS]

Storage Area

Network [SAN]

Storage Backup Infrastructure

DAS associated with Individual PC/Laptop

Data Centre

Network

attached

Storage

[NAS]

Storage Area Networ

k [SAN]

Storage Backup Infrastructure

Site 1 Site 2

• Multi-site Data centre helps mitigate risk of single site failure• Typically NAS of one site provides for standby NAS for the other site• Site 1 and Site 2 are connected over FC or GigEthernet• Data almost always gets synchronously replicated across sites

14SoumenSarkar

An emerging trend

Infographics source

15SoumenSarkar

Enterprise Storage System III: connecting to Public Cloud

DAS associated with Individual PC/Laptop

Data Centre

Network attache

d Storage [NAS]

Storage Area

Network [SAN]

Storage Backup Infrastructure

DAS associated with Individual PC/Laptop

Data Centre

Network attached Storage [NAS]

Storage Area

Network [SAN]

Storage Backup Infrastructure

NAS gateway to Cloud

SAN gateway to Cloud

Public Cloud Storage [Amazon

EC3, Microsoft Azure, Rackspace]

• To address rack-space limits in captive data centre enterprises often opt to move some data to public cloud• Cloud gateways like Nasuni /Twinstrata/Nirvanix/Ctera appliances connect enterprise NAS or SAN seamlessly to Cloud • They usually provide a RESTful API version for storage protocols like NFS, CIFS or iSCSI

16SoumenSarkar

Enterprise Storage System III-b: connecting to Public Cloud Securely

DAS associated with Individual PC/Laptop

Data Centre

Network

attached

Storage [NAS]

Storage Area Network [SAN]

Storage Backup Infrastructure

DAS associated with Individual PC/Laptop

Data Centre

Network

attached

Storage [NAS]

Storage Area Network [SAN]

Storage Backup InfrastructureNAS gateway

to CloudSAN gateway

to Cloud

Public Cloud Storage

[Amazon EC3, Microsoft Azure,

Rackspace]

Secure gateway to Cloud

• Storage infrastructure majorly deals with data in plaintext and that can be risk while dealing with public cloud • Using VPN only makes the communication channel between cloud and data centre secure but does not protect the enterprise from intentional breach to data inside public cloud• Secure gateway that encrypts data on the fly using AES 256 cryptography addressed the need. One may use services like Ciphercloud or NetApp Datafort appliance to secure its data

17SoumenSarkar

Extrapolating to Future: advent of DCAS Many leading storage vendors are actively

advocating for something called ‘private cloud’ ◦ A large part of that constitutes server and OS virtualization

◦ And storage virtualization i.e. logical provisioning of storage for the servers

However analysts predict this to be a transient phase and eventually enterprises will move towards public clouds [Gartner]

DCAS or Data Centre As Service will be dominant service model to be offered by the cloud vendors to enterprises◦ Application Server, storage, security and retention of enterprise

data will be provided as a single solution

◦ With enterprise users already adopting public cloud for their private data, it will only be consistent that their enterprise data also move to public cloud

So the most likely scenario for future enterprise data centre appear to be a combination of captive data centre and DCAS. But Does that mean that future storage innovations will be dominated by software?

18

.. And the journey continues…

More storage commentary: Industry Bytes

It may be when we no longer know what to do, we have come to our real work, and that when we no longer know which way to go, we have begun our real journey. - Wendell Berry