vSphere Virtual Volumes - CMG · vSphere Virtual Volumes 4 Management & Integration Framework for...

vSphere Virtual VolumesTechnical Overview

Josh AtwellAndy Banta

Special Thanks to Rawlinson Rivera and Cormac Hogan

Presenters

2

Josh AtwellSolutions Architect, SolidFire

Andy BantaStorage Janitor, SolidFire

App-centric delivery of storage objects

3

App-centric Automation

• Dynamic delivery of storage service levels when needed

• Fine control of data services at the VM level

• Common management across heterogeneous devices

✓ Rapid provisioning✓ No overprovisioning of

resources

✓ QoS automation✓ Simple change management

Today

• Static pre-allocation of shared storage container (Volume)

• Data services tied to storage container

• Vendor specific management

Today’s Infrastructure-centric Model

vSphere Virtual Volumes

4

Management & Integration Framework for External Storage

vSphere Virtual

Volumes

The Basics

• Virtual disks are natively represented on arrays

• Enables VM granular storage operations using array-based data services

• Storage Policy-Based Management enables automated consumption at scale

• Supports existing storage I/O protocols (Block or NFS)

• Industry-wide initiative supported by major storage vendors

• Included with vSphere

vSphere Virtual Volumes Architecture

vSphereVirtual Volumes

5

SAN / NAS

Vendor Provider (VASA)

Control Path

Control Path

Storage Policies

Access

Capacity

Published Capabilities

Snapshot

Replication

Deduplication

QoS

Virtual Datastore

Storage Admin

vSphere Admin

VVOLs

DataPathProtocol Endpoint PE

VVols Components

6

● Objects○ Virtual Volumes○ Storage Containers○ Protocol Endpoints○ vStorage/vSphere API for Storage Awareness (VASA) Provider

● Concepts○ Out of Band Configuration○ Bindings○ Policy-Based Management

● VVols Capabilities

Vvols – Just Stuff

7


• Virtual Volumes – Virtual machine objects stored natively on the

array.– No Filesystem on-disk formatting required

• There are five different types of recognized Virtual Volumes:– CONFIG – vmx, logs, nvram, log files, etc

– DATA – VMDKs

– MEM – Memory Snapshots

– SWAP – Swap files

– Other – Vendor solution specific

vSphere Web Client View

vvol

Storage Containers – Where Stuff Lives

9

VM

VM

VM

VM

Storage ContainerStorage Containers

• Logical storage constructs for grouping of virtual volumes.

• Defined and setup by storage administrators on the array.

– Capacity is based on physical storage capacity

– Logically partition or isolate VMs with diverse storage needs and requirement

– Storage policy settings based on data service capabilities

– Minimum one storage container per array

– Maximum depends on the array


SAN / NAS

Storage Containers

10

Differences between Storage Containers and LUs

• Size based on array capacity

• Max number of SCs depend only on the array ability

• Size of SC can be extended

• Can distinguish heterogeneous capabilities for different VMs (Virtual Volumes) provisioned in that SC

• Simply an attribute of the VVols for SF

• Fixed size mandates more number of LUs

• Needs a FileSystem

• Can only apply homogeneous capability on all VMs (VMDKs) provisioned in that LU.

• Managed by In-band FileSystem commands

Storage Containers

LUN

Storage Container (SC)

vSphere Web Client

Storage Management UI

Datastore

Storage Container

• What do the Admins see?• How are the storage containers setup?• What does the vSphere Admins see?• Why are we still creating datastores in this

new model?

12

Virtual Volumes

VM objects view from a storage container on an arrayVM objects view from a datastore

vSphere Web Client Storage Management UIvSphere Admin View Storage Admin View

Common Questions

•Are Vvols required for vSphere 6?•How does this help with performance/capacity management?•Do I lose anything with virtual volumes?

14

Protocol Endpoints – How You Get the Stuff

15

Protocol EndpointsProtocol Endpoints• Access points that enables communication

between ESXi hosts and storage array systems.– Part of the physical storage fabric– Created by Storage administrators

Scope of Protocol Endpoints• Compatible with all SAN and NAS Protocols:

- iSCSI- NFS v3 - FC- FCoE

• Existing multi-path policies can be applied to the PE

SAN / NAS

Virtual Datastore



Storage Container

16

Protocol Endpoints• Today, there are different types of logical management

constructs to store VMDKs/objects:– NFS Mount Points– IP or block based datastores

• Datastores serve two purposes today:– Endpoints – receive SCSI reads, write commands– Storage Container – for large number of VMs

metadata and data files

– Differences between Protocol Endpoints and Datastores:– PEs no longer stores VMDKs but it only becomes

the access point. – Now you wont need as many datastores as before

– Certain offloading operation will be done via VASA and other will be done using the standard protocol commands

vSphere

storage fabric

PEprotocol endpointSCSI: proxy LUN

datastore = storage container

storage system

1 VVol =VMDK

One entity on the fabric

Discovery Procedures – Protocol Endpoint

Protocol Endpoint discovery process• SCSI PEs are discovered during an ESX

rescan

• ESX will identify PEs and maintain all discovered PEs in a database.

SAN / NAS

Virtual Datastore



Storage Container

18

VASA Provider

19

VASA Provider (VP)

• Software component developed by Storage Array Vendors

• ESX and vCenter Server connect to VASA Provider

• Provides Storage awareness services

• Single VASA Provider can manage multiple arrays

• Supports VASA APIs exported by ESX

• VASA Provider is implemented within the cluster’s management server or firmware

• Responsible for creating Virtual Volumes

SAN / NAS

Virtual Datastore



Storage Container

Vendor Provider (VASA)

Control Path

Control Path

Discovery Procedures – Storage Container

Virtual Volumes

Snapshot Replication

vCenter

VASA Provider

Storage Container Discovery Process• Storage admin sets up Storage Container

• Capabilities are applied to the Storage Containers

• VASA Provider discovers Storage Container and reports to vCenter

• Any new VMs that are created will subsequently be provisioned in the Storage Container

vvol

DATA

vvol

CONF

vvol

SWAP

vvol

DATA

vvol

CONF

vvol

SWAP

Provisioning Workflow

storage arrays

PE

vSphere Admin1. Create Virtual Machines2. Assign a VM Storage Policy3. Choose a suitable Datastore

Under the Covers

• Provisioning operations are translated into VASA API calls in order to create the individual virtual volumes.

Under the Covers

• Provisioning operations are offloaded to the array for the creation of virtual volumes on the storage container that match the capabilities defined in the VM Storage Policies

offload to array

Virtual Datastore

Storage Container


VVOL/SPBM Provisioning Workflow

Virtual Disk

Storage Container(s)

Storage policies

Virtual Volumes

Virtual Machines

VASAProvider

Virtual VolumeStorage

VASA Provider

Storage presents

capabilities of the storage container to

VASA Provider

Capabilities are used to build

storage policies

User selects available storage

policy at provisioning

Virtual Volume is provisioned

through VASA provider

Virtual disk

policy

Virtual disk resides on Virtual Volume aligned

with storage policy

Administrators

compliance

Administrators can monitor

policy compliance

11

22

33

44 55

66

Storage Capabilities and VM Storage Policies

• Storage Capabilities – are array based features and data services specifications that capture storage requirements that can be satisfied by a storage arrays advertised as capabilities.

• Storage capabilities define what an array can offer to storage containers as opposed to what the VM requires.

• Arrays Storage Capabilities are advertised to vSphere through the Vendor Provider and VASA APIs

• In vSphere Storage Capabilities are consumed via VM Storage Policy constructs.

• VM Storage Policies is a component of the vSphere Storage Policy-based management framework (SPBM)

SPBM

object manager

virtual disk

Datastore ProfileVM Storage Policy

vSphere VM Storage Policy Management Framework

Storage Capabilities for Storage Array

Access

Capacity

Published CapabilitiesSnapshot

Replication

Deduplication

QoS

Virtual Datastore

Storage Policy Based Management (SPBM) – VM Policies

Storage Policy Based Management (SPBM) – Array Capabilities

Virtual Volumes APIs

Storage Policy-Based Mgmt.

CV

CV

CV

Storage admin

Publish Capabilities • Array based features and

data services• Defines what an Array can

offer • Advertised to ESX through

VASA APIs

26

Storage Policy Based Management (SPBM)

SolidFire Implementation

28

PE PE PECM

VP PEMGRTS TS TS

VVOL VVOL VVOL VVOL

vSphereData pathControl pathRPC calls

Bindings

29

Truck A

Truck B

Truck C

Binding Operations

• Bindings are data path coordinating mechanisms that occurs between VASA providers and ESXi hosts for accessing virtual volume.

• Different Binding Mechanism:• Binding – allows array create I/O channels for

a virtual volume• Unbind – destroys the I/O channel for a virtual

volume to a given ESXi host• Rebind – provides the ability to change the I/O

channel (PE) for a given virtual volumes run time using events.

create bind

unbinddelete

VM Creation VM power-onopen (2)

I/O read (2)

I/O write (2)

VM power-offclose (2)

VM destroy

VP rebalance

REBINDI/O

Virtual Volume Lifecycle

Binding Mechanism Workflow• Bindings mechanism workflow

– ESXi hosts reference virtual volumes via a particular binding ID and communicates with the VASA providers via API calls with the given virtual volume ID.

– VASA provider return communication with a protocol endpoint ID as well as a secondary ID.

– A single virtual volume can be bound to multiple PE protocols simultaneously:

• FC• NFS• FCoE• iSCSI

– VASA Provider may choose to unbind a virtual volumes to a PE for several reasons in order to provide vendor specific capabilities.

create bind

unbinddelete

VM Creation VM power-onopen (2)

I/O read (2)

I/O write (2)

VM power-offclose (2)

VM destroy

VP rebalance

REBINDI/O

Virtual Volume Lifecycle

Bind Requests

• IO path is established through a VASA Bind request

• VASA Provider does two things upon receiving a Bind request

• Returns a PE ID to which the VVol is bound• A unique secondary ID to be used for IO

between the bound VVol and PE

• For SCSI, the secondary ID is the secondary LUN ID

• M:M Relationship between VVOL and PE

• For SolidFire, VVOL is bound to the PE on the node of the primary slice.

Virtual Volumes

VASA Provider

VVol

PE

VVol

Bind

32

Virtual Volumes – Other Bind Scenarios

UNBIND

Virtual Volumes

VASA Provider

VVol

PE

VVol

UNBIND

• Delete binding of a VVol to the PE

• VASA Provider may choose to delete a binding for several reasons

REBIND

Virtual Volumes

VASA Provider

VVol

PE

VVol

PE

REBIND

• Move VVols across different PE

• VASA Provider may choose to issue a rebind for several reasons Eg., Slice migration

Backup Information

34

Migration Scenario: with VAAI vs Virtual Volumes

VASA API Compatible Array

2

vSphere

VMFS VVOLs

1

vSphere Admins

Migrate VM from VMFS toVVOL datastore

software data mover implementation

36

vSphere Virtual Volumes Offload Operations

vSphere

VMFS VVOLs

vSphere Admins

offload to array

offload to array

Virtual Machine Operation Offloaded• Virtual Machine provisioning

• Virtual Machine deletes

• Virtual Machine full clones

• Virtual Machine Linked Clones

• Virtual Machine Snapshots

• Storage vMotion

Migration Scenario: with VAAI vs Virtual Volumes

VAAI & VASA API Compatible Array

2

vSphere

VMFS VVOLs

1

vSphere Admins

Migrate VM from VMFS toVVOL datastore

vmkernel data mover use VAAI

APIs XCOPY

/WRITE_SAME primitive for

cloning operation

VAAI vs VVol• VAAI Block and NAS

how they apply on a VVol

• If you have it, what are the use cases:

SC-A

cloning operation


offload to array

clone VM onto sameVVol container

vSphere Admins

vendor native clone

utilized

1

vSphere

2

3


clone VM from VMFSto VVol

container

vSphere Admins

vmkernel data mover use VAAI

APIs XCOPY

/WRITE_SAME primitive for

cloning operation

2

vSphere

VMFS VVol

1

clone VM from VVol container to

different VVol container

vSphere Admins

vSphere

1

2

offload to array

3

Fully VAAI & VASA APIs Compatible Array

SC-A SC-B

always attempt defaultoperation with VASA API

primitives

If default operations fails VAAI API primitives are used

clone VM from VVol container to

different VVol container

vSphere Admins

vmkernel data mover uses

VAAI primitives for cloning operation

vSphere

1

2

offload to array

3

Fully VAAI & VASA APIs Compatible Array

SC-A SC-B

default 1fail back 2

vendor native clone utilized with VASA primitives

cloning operation

VAAI vs VVol

Snapshots• Snapshots are a point in time copy on write

image of a Virtual Volume with a different ID from the original.

• Virtual Volumes snapshots are useful in the contexts of creating:– a quiesced copy for backup or archival

purposes, creating a test and rollback environment for applications, instantly provisioning application images, and so on.

• Two type of snapshots supported:– Managed Snapshot – Managed by ESX.

• A maximum of 32 snapshot are supported for fast clones

– Unmanaged Snapshot – Manage by the storage array.• Maximum snapshot dictated by the storage array

Managed Snapshot - vSphere

Unmanaged Snapshot - Array

vSphere Virtual Volumes Supported FeaturesSupported vSphere Features

• SPBM

• Thin Provisioning

• Linked Clones

• Native Snapshots

• Protocols: NFS3, iSCSI, FC, FCoE

• View Storage Accelerator (CBRC)

• vMotion

• SvMotion

• DRS

• XvMotion

• vSphere SDK (VC APIs)

• VDPA/VDP

• View

• vRealize Operations

• vRealize Automation

• Stateless / Host Profiles

Published Capabilities

Snapshot

Replication

Deduplication

Encryption

vSphereStorage Policy-Based Mgmt.

Virtual Volumes

Storage PolicyCapacity

Availability

Performance

Data Protection

Security

PEVASA Provider

PE

41

Overview•VVols Motivations

–Separation of VMware and storage management–Make use of existing storage protocols–Separate concepts of “datastore” and logical collection of storage–Better use of available storage–Less contention between ESX hosts–Better use of storage network bandwidth–The upshot: better scalability and control

•Configuration Examples–Memory overprovisioning–Multi-tenant use of VVols–Disaster Recovery configurations

Motivations•Ease of management•Built on top of existing protocols•Datastore scalability

–Less wasted storage space–No filesystem contention–Logical Storage Container concept as a replacement

•Scalability through less SAN bandwidth–No vSphere data mover for normal operations–Less Storage vMotion–Less need for SDRS and SIOC

43

vSphere Virtual Volumes Is An Industry-wide Initiative

Multiple Ready at

GA

Unique capabilities

44

And Many More…

29 Partners in the

Program

Common Questions

•How do I configure the VASA provider and protocol endpoints?•What happens if the VASA provider is unavailable?•Is there anything I can start doing TODAY that will help with the transition to virtual volumes?

45

vSphere Virtual Volumes - CMG · vSphere Virtual Volumes 4 Management & Integration Framework for...

Documents

Transcript of vSphere Virtual Volumes - CMG · vSphere Virtual Volumes 4 Management & Integration Framework for...