UniﬁedVirtualMachine MigraonoverLongDistance · XvMo%on: UniﬁedVirtualMachine...

XvMo%on: Unified Virtual Machine

Migra%on over Long Distance Ali José Mash,zadeh, Min Cai, Gabriel Tarasuk-‐Levin,

Ricardo Koller, Tal Garfinkel, Sreekanth SeBy Stanford University – VMware, Inc.

Live Migra%on

Virtual Machine

Source Host Des,na,on Host Cloud

Migra%on Benefits

• Test to produc,on migra,ons • Disaster Preparedness Tes,ng • Cross-‐Datacenter Load Balancing • Shared-‐Nothing Architecture

Virtual Machine

Migra%on in Prac%ce

Virtual Machine

Source Host Des,na,on Host

Requires: Shared Storage Fast Networking Single Subnet

This is not what we really want

• Migra,ons are limited to machines that: With shared storage, fast networks, and same LAN

•  Technological Changes: •  Shared nothing applica,on architectures •  Network mobility possible: LISP, OTV, VXlan, OpenFlow (SDN)

•  Very large virtualized datacenters

• No reason for these limita,ons any longer • Customers have new use cases

XvMo%on

• XvMo,on: First commercially viable WAN migra,on

• Achieve Low Down,me AND Atomic Switchover • Uses Asynchronous IO Mirroring

• Principle: Flow control mechanisms for memory and disk

Customer Scenario

LAN WAN Bandwidth 10 Gbps

(some,mes 40 Gbps) 1 Gbps or less

Latency <1 ms ~100 ms Typical Network Dedicated NIC(s) Shared connec,on

between sites

Example Workload

Applica,on Highly Available Database

Client

HA Timeouts several seconds TCP Timeouts 120 seconds

Down%me (Switchover Time)

Virtual Machine

Running Stopped Running

Goal: Less than 1 Second of Down,me


Virtual Machine

Atomicity

Virtual Machine


Goal: Atomic switchover

XvMo%on

• Unifies Memory Migra,on and Storage Migra,on •  Tolerates wide area network bandwidth/latency and reliability

•  Tolerates heterogeneous storage performance •  Down,mes and workload impact comparable to local migra,on

•  Atomic switchover

• Deployed in customer metro area networks • Cross con,nent migra,on e.g. Palo Alto to India is prac,cal

Overview

• Architecture Overview • Wide Area Memory Migra,on • Wide Area Storage Migra,on • Evalua,on

Virtual Machine

Unified Live Migra%on

Virtual Machine


XvMo%on Architecture

IO Mirror

Virtual Machine IO

IO Buffer

Live Migra,on

Stream

s

Stream

s

Source Des4na4on

IO Buffer/Writer

Live Migra,on

TCP

Overview

• Architecture Overview • Wide Area Memory Migra4on • Wide Area Storage Migra,on • Evalua,on

XvMo%on Architecture: Live Migra%on

IO Mirror

Virtual Machine IO

IO Buffer

Live Migra,on

Stream

s

Stream

s

Source Des4na4on

IO Buffer/Writer

Live Migra,on

TCP

Live Migra%on: Total Time vs Down%me

Memory

Source Des,na,on

Live Migra%on: Total Time vs Down%me

•  Itera,ve copy hopefully reduces the working set each itera,on • Depends on Network being faster than Dirty rate

Memory

Source Des,na,on

Current Solu%on: Early Resume/Post-‐copy • Problem: Applica,ons can change memory faster than network bandwidth

•  Solu,on: •  Stop migra,on copy everything •  Resume early if down,me is high

• Des,na,on remote page faults on missing pages

Stun During Page Send (SDPS)

• Problems with Early Resume: •  Remote page faults very expensive for high latency networks

•  Not atomic: Unsafe for WANs where network hiccups are more likely

•  Solu,on: ThroBle VM just enough to keep dirty rate < network rate

SDPS Down%me Results

0

0.5

1

1.5

2

2.5

24-‐users 48-‐users

Down4

me (s)

SDPS OFF SDPS ON

Overview

• Architecture Overview • Wide Area Memory Migra,on • Wide Area Storage Migra4on • Evalua,on

XvMo%on Architecture: IO Mirroring

IO Mirror

Virtual Machine IO

IO Buffer

Live Migra,on

Stream

s

Stream

s

Source Des4na4on

IO Buffer/Writer

Live Migra,on

TCP

Problem: Synchronous mirroring over the WAN slows guest workload Goal: Hide network latency from VM

VM IO

IO

XvMo%on Buffering: Asynchronous IO

TCP Streams Transport Framework

Source Des4na4on

Bulk Disk Copy

Mirror IO

IO Buffer

Streams Transport Framework IO

XvMo%on Buffering: Asynchronous IO

VM TCP

Streams Transport Framework

Source Des4na4on

IO Buffer


IO IO IO IO

Problem: Workload may be too fast on source for the des;na;on Goal: ThroBle copy process and workload as necessary

IO

XvMo%on Buffering: Conges%on Control

VM TCP


Source Des4na4on

IO Buffer


IO

IO IO IO IO IO IO IO IO Back Pressure

Limit per-‐disk OIOs/buffer on des,na,on

Send free buffer space

Slow Disk

Overview

• Architecture Overview • Wide Area Memory Migra,on • Wide Area Storage Migra,on • Evalua4on

Evalua%on

• Migra,on Time: Total end-‐to-‐end ,me • Down,me: Time machine execu,on is suspended for final switchover • Workload Penalty: Average percentage penalty to the VM workload

• OLTP IO Workload (data disk only) •  10 GB system/12 GB data

XvMo%on Down%mes

0

0.2

0.4

0.6

0.8

1

1.2

0 50 100 150 200

Down,

me (s)

Latency (ms)

Take Away: ~1 second down,me independent of distance

XvMo%on Migra%on Time

0

50

100

150

200

250

300

0 50 100 150 200

Total Tim

e (s)

Latency (ms)

Latency

Take Away: small linear ,me increase with distance

California to India Migra%on

•  1 Gbps network with 214 ms RTT

• OLTP: 68 MB/s disk copy – 89 MB/s memory copy •  ~11% Workload Penalty from ThroBling

Summary

•  XvMo,on frees migra,on from the need for shared storage and fast local networks •  Tolerates wide area network bandwidth/latency and reliability

•  Tolerates heterogeneous storage performance •  Down,mes and workload penalty comparable to local migra,on

•  Atomic Switchover

•  Enables new use cases – e.g. disaster preparedness, cluster upgrade, shared nothing

•  On the path to deployment: •  Deployed in customer metro area networks •  Cross con,nent migra,on e.g. Palo Alto to India is prac,cal

Ques%ons?

UniﬁedVirtualMachine MigraonoverLongDistance · XvMo%on: UniﬁedVirtualMachine...

Documents

Transcript of UniﬁedVirtualMachine MigraonoverLongDistance · XvMo%on: UniﬁedVirtualMachine...