Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION...

34
1 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved. Advanced Data Services for Physical and Virtual Datacenters Ionut Lungu Sr. Systems Engineer @ EMC [email protected]

Transcript of Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION...

Page 1: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

1 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

Advanced Data Services for Physical and Virtual Datacenters

Ionut Lungu Sr. Systems Engineer @ EMC [email protected]

Page 2: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

2 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

Source: EMC Digital Universe with Research and Analysis by IDC, The Digital Universe of

Opportunities: Rich Data and the Increasing Value of the Internet of Things, April 2014.

2020

4.4 ZETTABYTES

44 ZETTABYTES

10x MORE

Digital Universe 2014

2013

ZETTABYTE = 1,000,000,000,000,000,000,000 bytes

Page 3: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

3 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

EMC’s Exabyte Journey

Page 4: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

4 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

Page 5: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

5 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

A CAR

Page 6: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

6 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

A THERMOSTAT

Page 7: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

7 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

INFRASTRUCTURE

I think the pic is maybe more of a hybrid cloud thing

(maybe label it as well)

HYBRID CLOUD

Page 8: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

8 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

INVEST MANAGE COST

New… & Existing Applications

Page 9: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

9 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

EMC Bridges The Gap

Page 10: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

10 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

Workloads Have Different Requirements

ADVANCED DATA SERVICES

SIMPLE DATA SERVICES

Traditional Sweet Spot For

2nd Platform Applications

Traditional Sweet Spot For

3rd Platform Applications

Page 11: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

11 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

Workloads Have Different Requirements

ADVANCED DATA SERVICES

SIMPLE DATA SERVICES

Transaction Processing

Online Archive

VM Infrastructure

Decision Support

Full Motion Video

High Performance Computing

Web Content

Content Management

IT Applications

Storage As-A-Service

Home Directories

Email

Application Development

Tech Apps

File Sync & Share

Online Backup

Data Analytics

Page 12: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

12 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

The Right Tool, For The Right Job.

SIMPLE DATA SERVICES

ADVANCED DATA SERVICES

VMAX

VNX

Isilon

Atmos Data Domain,

Avamar, & NetWorker

Page 13: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

13 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

EMC Is Committed To Best Of Breed

LEADER SCALE-OUT NAS STORAGE

LEADER HIGH-END STORAGE

LEADER MIDRANGE STORAGE

LEADER OBJECT STORAGE

VMAX

VNX

Isilon

Atmos Data Domain,

Avamar, & NetWorker

LEADER ENTERPRISE DATA PROTECTION

Page 14: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

14 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

WHY IS THIS SO HARD?

PLATFORM SPECIFIC

Different SCSI Presentations

Different Fabric Presentation

Incompatible Data Services

No Common Copy Engine

No cross-array visibility or orchestration

RED APPLE

GREEN APPLE

ORANGE PEAR GRAPE FRUIT

Page 15: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

15 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

PLATFORM SPECIFIC FEDERATED STORAGE POOLS

HOW TO MAKE IT SIMPLE AUTOMATION ENGINE

EMC provided, 3rd party, or build your own

COPY/SYNCH ENGINE

EMC provided, 3rd party, or build your own

UNIFORM DATA TRANSFER API

Open API, array and vendor agnostic

STANDARD PATH MANAGER

ALUA, PID=FAU drivers, etc

SCSI PRESENTATION

Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…

RED APPLE

GREEN APPLE

ORANGE PEAR GRAPE FRUIT

Page 16: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

16 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

FEDERATION STORAGE POOLS

SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…)

UNIFORM DATA MOVER API (Open API, array and vendor agnostic)

COPY/SYNCH ENGINE (EMC provided, 3rd party, or build your own)

STANDARD PATH MANAGER (ALUA, PID=FAU drivers, etc)

AUTOMATION ENGINE (EMC provided, 3rd party, or build your own)

FEDERATED STORAGE

POOLS

Page 17: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

17 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

SIDEWAYS FAST

Page 18: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

18 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

FAST SIDEWAYS

ARRAY to ARRAY COPY ARRAY to ARRAY VERIFICATION

PATH FAIL-OVER

ZERO APP DISRUPTION

© Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

FEDERATED STORAGE

POOL

Page 19: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

19 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

0.5ms

1ms

1.5ms

HYBRID ARRAY – FLASH-OPTIMIZED

“GREAT” “WOW”

1.5X MORE IOPS

VMAX 40K 8 ENGINE 8K RRM

1.25M 750K 1.0M 500K

AVAILABLE FOR VMAX

Page 20: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

20 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

HYBRID ARRAY – FLASH-OPTIMIZED

0.5ms

1ms

1.5ms

“OMG!”

1.25M 1.75M 2.0M 2.25M 750K 1.0M 1.5M 500K

“GREAT” “WOW”

VMAX 40K 8 ENGINE 8K RRH

Page 21: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

21 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

SIMPLE DATA SERVICES

ADVANCED DATA SERVICES

Our Flash Strategy: Flash Everywhere

250μS

3mS

1mS

Server Flash

Hybrid Arrays

All Flash Array

Page 22: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

22 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

Our Flash Strategy: Flash Everywhere

SIMPLE DATA SERVICES

ADVANCED DATA SERVICES

250μS

3mS

1mS

Hybrid Arrays

All Flash Array

50μS

Page 23: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

23 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

XtremIO Linear Scaling

• ALWAYS BALANCED

• ALWAYS THIN

• ALWAYS

DEDUPLICATED

• ALWAYS PERFORMS

250K IOPS / 20TB

500K IOPS / 40TB

1M IOPS / 80TB

Page 24: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

24 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

Page 25: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

25 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

What If You Could Save 10¢ Per Gigabyte ?

Page 26: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

26 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

1 Terabyte 1 Petabyte 1 Exabyte

Page 27: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

27 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

Cloud-Scale Is Driving A New Architecture

Storage Software

Commodity Hardware

Page 28: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

28 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

SOFTWARE-DEFINED STORAGE A CLOUD-SCALE PLATFORM FOR ENTERPRISES AND SERVICE PROVIDERS

Page 29: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

29 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

Data Services On Commodity Hardware

Commodity Hardware

ViPR Data Services

Page 30: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

30 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

Data Services On Almost Any Hardware

Commodity Hardware

ViPR Data Services

Storage Arrays

VNX Isilon 3rd Party

VMAX

Commodity Hardware

Page 31: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

31 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

Expanded Support : Native & OpenStack

ORACLE ZFS

NetApp E-Series

Solidfire

IBM XIV

IBM SVC

IBM DS8000

NATIVE INTEGRATION

Isilon

VMAX

Hitachi VSP

(Block)

Hitachi HUS VM

NetApp FAS

EMC ScaleIO

VNX VBLOCK HP 3PAR HP

Left-Hand Dell

Equal-Logic

XtremIO

VIPR Controller

Page 32: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

32 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

Software-Defined Storage, By EMC

Commodity Hardware Storage Arrays

VNX Isilon XIO 3rd Party

VMAX

VIPR Controller

ViPR Data Services

Page 33: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

33 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

EMC Elastic Cloud Storage Configurations

Mixed

Block

24K IOPS 12K IOPS

Object & HDFS

12K IOPS 12K IOPS

Page 34: Advanced Data Services for Physical and Virtual …...FEDERATION STORAGE POOLS SCSI PRESENTATION (Fed Vol ID, VID/PID = FAU/FAU, new uniform SCSI profile…) UNIFORM DATA MOVER API

34 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.

CORE BUSINESS APPS NEXT GEN CLOUD APPS

TRUST SIMPLE