Turning OpenStack Swift into a VM storage platform

OpenStack Meetup: Turning OpenStack Swift into a VM storage platform

Wim Provoost (@wimpers_be)Open vStorage (@openvstorage)

CloudFounders

• CloudFounders (2007)– 120 people in the US, Belgium, Romania and India

• Products– Open vStorage (open source under an Apache 2.0 license)

• Ultra reliable, high performance, open storage layer for virtual machines.

• Compatible with all major hypervisors.• Supports OpenStack

– vRun = Converged Infrastructure• Software that transforms server hardware to converged infrastructure

with unparalleled performance & flexibility.• Software can run on any x86 server or as a virtual machine

(VMware/OpenStack)• Uses Open vStorage as basis• Competes with Nutanix, Simplivity and upcoming EVO:RAIL

Object storage: the future of scale out storage

Object storage is the future for storing massive amounts of data:•PRO

– Build scale-out storage by adding more disk/servers when needed– High reliability by distributing content across nodes– Easy to manage (no knowledge needed about RAID or volumes)– Standard API as all solutions support S3 API, no vendor lock-in– Cost-effictive: use low cost, large capacity SATA disks, increase performance

by adding spindles/gateways

•CON– NO support for Virtual Machine Disk Storage (see next slide)

Object storage challenges as VM storage

• Eventual consistency (the CAP Theorem)

• Latency & performance– VMs require low latency and high performance– Object stores are developed to contain lots of data

(large disks, low performance)– Additional latency as Object Store is on the Local LAN instead of attached to

the host like DAS

• Different Management Paradigms– Object Stores understand Objects <> Hypervisors understand blocks, files

What is needed is a technology whereby Virtual Machines can use object stores instead of a SAN and get the benefits of the low cost

and scale-out capabilities of object stores

What is needed is a technology whereby Virtual Machines can use object stores instead of a SAN and get the benefits of the low cost

and scale-out capabilities of object stores

Traditional OpenStack Setup

NovaInstance

Management

SwiftObject Storage

CinderBlock Storage

GlanceImage store

VM

Provides volume for

Provisions

Stores image in

Stores backups in

Provides image for

Ceph, SAN, NAS, ...

Provides disk space

2 storage platforms?! 2 storage platforms?!

The answer: Open vStorage

NovaInstance

Management

SwiftObject Storage

GlanceImage store

VM

Provides volume for

Provisions

Stores image in

Stores backups in

Provides image for

Open vStorage

CinderBlock Storage

Provides disk space

Converts Object storage into Block storage

What is Open vStorage

Open vStorage is an open source “Storage Router“ and is installed on a host or a cluster of hosts to create a VM-centric, clustered, reliable,

scale out and high performance storage system for virtual machines.

Open vStorage Intro

• +40.000 IOPS per Hypervisor• Made for virtual machines• Unified Namespace• Ultra Reliable• Unlimited Snapshots• Endless Scalable• Lowest Management Cost In Market

SAN / NASSAN / NASS3 compatibleObject Based

Storage

S3 compatibleObject Based

Storage

Hypervisor

Open v Storage

Hypervisor

Open v StorageOpen v Storage

Hypervisor


Hypervisor


Distributed File Systems

Distributed File Systems

Ultra Scalable Storage For Virtual Machines

KVM

Scale-outVM VM

VM VM

SSDSSDSSDSSD

OpenvStorageOpen

vStorage

KVM

VM VM

VM VM

SSDSSDSSDSSD

OpenvStorageOpen

vStorage

KVM

VM VM

VM VM

SSDSSDSSDSSD

OpenvStorageOpen

vStorage

Virtual File System interface

Solving Eventual Consistency using time based approach

SSD or PCI FlashLBA 1: LBA 1: 4k block 1 4k block 1

LBA 2: LBA 2: 4k block 24k block 2









LBA 8: LBA 8: 4k block 11 4k block 11




New writes

SCO 1

4k block 1 4k block 1

4k block 24k block 2






SCO 2












SCO 3

New writes

SCO1

SCO2 Transfer SCOs once they are full (4MB)to the Storage Backend at slow paceTransfer SCOs once they are full (4MB)to the Storage Backend at slow pace

Each write is appended

to the current Storage

Container Object (SCO)

Each write is appended

to the current Storage

Container Object (SCO)

Open vStorage <> distributed file system

VSA 1 VSA 2 VSA 3

Arakoon – (config params, metadata, ...)

vDisk1

vDisk1

vDisk2

vDisk2

InternalBucket

vDisk3

vDisk3

VFS2 VFS3

xml

VOLDRV

VM

VOLDRV

Object Router

FILEDRV

FILEDRV

VOLDRV

Object Router

FILEDRV

KVM1 KVM2 KVM3

VFS1

Object Router

Live Motion – In depth (Phase 1)

VSA 1 VSA 2 VSA 3


vDisk1

vDisk1

vDisk2

vDisk2

InternalBucket

vDisk3

vDisk3

VFS3

vmx

VOLDRV

VOLDRV

Object Router

FILEDRV

FILEDRV

VM

VOLDRV

Object Router

FILEDRV

KVM1 KVM2 KVM3

VFS1

VMLive Motion

Object Router

VFS2

Live Motion – In depth (Phase 2)

VSA 1 VSA 2 VSA 3


vDisk1

vDisk1

vDisk2

vDisk2

InternalBucket

vDisk3

vDisk3

VFS2 VFS3

xml

VOLDRV

VOLDRV

FILEDRV

FILEDRV

VM

VOLDRV

Object Router

FILEDRV

KVM1 KVM2 KVM3

VFS1

VMLive Motion

Object Router Handover Object Router

How does Open vStorage solve the problem

• Open vStorage is a middleware layer in between the hypervisor and the object store. (Converts object storage into block storage)– On the host: location based storage (block storage).– On the backend: time based storage (ideal for objects stores).– Open vStorage turns a volume into a single bucket.

• OpenStack Cinder Plugin for easy integration (snapshots, ...).• Distributed file systems don’t work! Open vStorage is not a distributed

file sysem!– All hosts ‘think’ they see the same virtual file systems.– Volume is ‘live’ on 1 host instead of all hosts.– Only the virtual file system metadata is distributed.

• Caching inside the host fixes impedance mismatch between slow, high latency backend and fast, low latency requirement of Virtual Machines.

The benefits

• Single distributed storage platform (Swift) to maintain. This platform is very performant and almost infinitely scalable.

• Better performance due to caching inside the host where the Nova instance runs.

• VM-centric management on storage level.– Thin-cloning, zero-copy snapshots, ...

• Add storage which isn’t compatible with Cinder.

Supported OpenStack actions

• Volumes– Add / Remove– Create / Delete snapshot– Clone from snapshot– Clone volume from volume– Import image to volume– Export volume to image

• Instances– Live Migration– Evacuate

Deployments

• Our internal environments are now running Open vStorage + swift

• Open vStorage is available in beta– Certified OpenStack Cinder plugin is expected with

Kilo release– Feel free to use in test environments and provide

feedback– Supported production release Q1

Next steps

• The code: https://bitbucket.org/openvstorage/openvstorage/• Download the packages and deploy in your OpenStack

environment• Documentation: http://doc.openvstorage.com• Support:

https://groups.google.com/forum/#!forum/open-vstorage• Follow us on Twitter (@OpenvStorage) or read our blog

(http://blog.openvstorage.com)

OpenStack Paris

We will be there! Look out for the “Cloudfounders” booth.

November, 3-7

Turning OpenStack Swift into a VM storage platform

Technology

Transcript of Turning OpenStack Swift into a VM storage platform