Choosing the Right Data Storage Solution

Post on 08-Sep-2014

580 views 1 download

Tags:

description

With AWS, you can choose the right storage service for the right use case. Given the myriad of choices, from object storage to block storage, this session will profile details and examples of some of the choices available to you, with details on real world deployments from customers who are using Amazon Simple Storage Service (Amazon S3), Amazon Elastic Block Store (Amazon EBS), Amazon Glacier, and AWS Storage Gateway.

Transcript of Choosing the Right Data Storage Solution

© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

Storage & Archiving: Choosing the

Right Data Storage SolutionTimothy K. Harder, Global Business Development

harder@amazon.com July 10, 2014

Agenda

• Common Data Storage Terms

• Introduction to AWS Storage Platforms

• Use Cases Drive Platform Selection

• TCO

• What to do next

Goals for this session

• 101 Level Introduction to AWS Data Storage – Survey of AWS Data Storage Platforms

– Learn how AWS Cloud Storage compares to classic models

– Cloud Storage Use Cases

– How to get started

AWS Storage Options

• Simple Storage

• General Purpose Storage

• Archive Storage

• Storage Gateway

We are constantly producing more data

From all types of industries

Classic and Contemporary

Applications

File Systems

Database(s)

User Generated Content

Machine Generated Content

Messaging, Collaboration

Backup, Archive, DR

First, let’s visit some storage basics

Storage Basics - Block vs. File vs. Object

• Block Storage : SAN : EBS– Data organized as an array of unrelated blocks

– Host File System places data on disk: Microsoft NTFS or Unix ZFS

– Structured data is predicted to grow at 34% CAGR until 2018

• File Storage : NAS :EBS – Unrelated data blocks managed by a file (serving) system

– Native file system places data on disk: EXT or Filers

– Unstructured data is predicted to grow at 45.6% CAGR until 2018

• Object Storage :S3:RRS:Glacier– A new data access, data storage, and data management model

• API access to data vs. traditional block or file system access

• Metadata Driven, Policy-based, Self Managing storage

• No host overhead for storage functions

– A system that stores virtual containers that encapsulate the data, data attributes, metadata, and Object IDs

Storage Basics – SAN, NAS , Object

SERVERS

Object

NAS

SANDatacenter Proximity

Ethernet / Optical

Datacenter Proximity

Ethernet

Cloud Proximity

API / HTTP(S)

FILESYSTEMS

APPLICATIONShttps://

nfs / cifs

FC / iSCSI / FCoE

• SAN and NAS are integrated with clients via operating system• Cloud Storage is an application level integration

No Up-Front

Capital ExpensePay Only for

What You Use

Self-Service

InfrastructureEasily Scale

Up and Down

Improve Agility &

Time-to-Market

Low Cost

Amazon Storage Service Benefits

AWS Global Infrastructure

10 Regions

25 Availability Zones

51 Edge Locations

AWS Regions and Availability Zones

You decide where your data resides

Storage Services

Amazon EBS

EBS

Block storage for use

with Amazon EC2

Amazon S3

Images

Videos

Files

Binaries

Snapshots

Internet scale

storage via API

AWS Storage Gateway

S3,

Glacier

Integrates on-premises

IT and AWS storage

Amazon Glacier

Images

Videos

Files

Binaries

Snapshots

Storage for archiving

and backup

#1Object Storage

●○○○○

AMAZON S3SIMPLE STORAGE SERVICE

99.999999999%Durability

TrillionsOf Unique Customer Objects

Q4 2006

Q1 2007

Q2 2007

Q3 2007

Q4 2007

Q1 2008

Q2 2008

Q3 2008

Q4 2008

Q1 2009

Q2 2009

Q3 2009

Q4 2009

Q1 2010

Q2 2010

Q3 2010

Q4 2010

Q1 2011

Q2 2011

Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Q1 2013

Q2 2013

Q3 2013

1.5 Million+Peak Transactions Per Second

Storage Tiers: Buckets + Unlimited Objects

Reduced Redundancy Option99.99% saves 20-30%

Amazon S3 Website: Content

Use Amazon S3 When You Need

• Application level connectivity

• Unlimited storage capacity

• Extremely High durability

• Storage for user or machine generated content

• Content Depots and Data Lakes

• Gold copy, Backup and Data Life Cycle

#2Block Storage for

General Computing●●○○○

AMAZON EBSELASTIC BLOCK STORAGE

General Purpose (SSD) Block Storage

• New EBS Volume type

• SSD Backed / Solid State Drive – Designed to deliver 99.999% Availability

• Flat, Per-Gigabyte Rate. No I/O Charges– Simple and Predictive Pricing

• New Default Volume Type - TCO Advantaged– 10x performance improvement over EBS Standard

– High level of base IOPS. Burst Capable.

EBS Magnetic

• Cold Storage

General Purpose SSD

• Default

PIOPS SSD

• Prescriptive Performance

Price Performance

Latency Business Impact

Applications

Snapshot

Backup and Archive

How do we bring this together?

10GB1TB

IOPSProvisioned

4000

IOPS

Amazon EBSSnapshots

Use Amazon EBS When You Need

• Persistent storage

• Data changes frequently

• Block storage for your databases –

• Filesystem enablment NTFS, ExtFS, RAID, LVM…

• Access to raw, unformatted block-level storage

AMAZON GLACIERLOW-COST ARCHIVING SERVICE

$0.01Per GB / Month

$120Per TB / Year

99.999999999%Durability

3-5 HoursData Retrieval

Use Amazon Glacier When You Need

• Inexpensive, Long-term archiving. Cold Use.

• Unlimited storage capacity

• Eliminated Tape Museums

• Eliminate Tech Refresh

• High durability

Amazon S3 / Amazon Glacier IntegrationPOLICY-BASED ARCHIVING SERVICE

Lifecycle Rule

Archive Recovery Process with Tape

+ Days or Weeks

Archive Recovery Process with AWS &

Amazon Glacier

$$

Hours

Glacier S3 EC2

/HPC

CloudFront Generating

Business

Value

Use Amazon S3 and Amazon Glacier When You Need

• Backup and Archive in the cloud

• Archive data from Amazon S3/RRS to Amazon Glacier

by policy

• Delete data from Amazon Glacier by policy

• Tiering Strategy to support Information Lifecycle

Management

AWS STORAGE GATEWAY

The Good News is that Cloud isn’t an ‘All or Nothing’

Corporate

Data

Centers

On-Premises

Resources

Cloud

ResourcesIntegration

Active Directory

Network Configuration

Encryption

Backup Appliances

Your On-Premises

AppsCorporate

Data Centers

Users & Access Rules (IAM)

Your Private Network (VPC)

Encryption (S3, RDS, HSM)

Backups (Storage Gateway)

Your Cloud Apps

AWS Direct Connect

Integrating AWS with your existing On-Premises

Infrastructure

AWS Storage Gateway

What is AWS Storage Gateway?

• Integrates on-prem IT environments with Cloud storage for departmental and remote office backup and DR

• Utilizes a virtual appliance that sits in your datacenter

• Exposes storage and backup interface on front end

• Stores primary data on-AWS in Amazon S3 or on-premise with data backed-up to Amazon S3 as Amazon EBS snapshots or Glacier

IT’S ALL ABOUT

CHOICEPERFORMANCE-ORIENTED

COST-Optimized

Trusted by Enterprises Around the World

Used by Government Agencies & Educational Institutions

Worldwide

2014 Premier Tier Partners

TCO

• AWS Storage Platforms Provide Lowest TCO by

13-44% as compared to classic deployments.

We take cost out for customers in every

engagement.

• When is your next capacity upgrade or refresh?

Goals for this session

• 101 Level Introduction to AWS Data Storage – Survey of AWS Data Storage Platforms

– Learn how AWS Cloud Storage compares to classic models

– Cloud Storage Use Cases

• How to get started– AWS.AMAZON.COM/FREE

– AWS.AMAZON.COM/BACKUP-STORAGE

© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

Storage & Archiving: Choosing the

Right Data Storage SolutionTimothy K. Harder, Global Business Development

harder@amazon.com July 10, 2014