1

Virtual Switching Without a Hypervisor for a More Secure Cloud

Xin JinPrinceton University

Joint work with Eric Keller(UPenn)and Jennifer Rexford(Princeton)

2

Public Cloud Infrastructure

• Cloud providers offer computing resources on demand to multiple “tenants”

• Benefits:– Public (any one can use)– Economies of scale (lower cost)– Flexibility (pay-as-you-go)

3

Hardware

Server Virtualization

• Multiple VMs run on the same server• Benefits– Efficient use of server resources– Backward compatibility

• Examples– Xen– KVM– VMware

VM VM VM

Hypervisor

4

Network Virtualization

• Software switches– Run in the hypervisor or the control VM (Dom0)

• Benefits: Flexible control at the “edge”– Access control– Resource and name space isolation– Efficient communication between co-located VMs

• Examples– Open vSwitch– VMware’s vSwitch– Cisco’s Nexus 1000v Switch Hardware

VM VM SoftwareSwitch

Hypervisor

5

Security: a major impediment for moving to the cloud!

Let’s take a look at where the vulnerabilities are…

6

Guest VM 2 Guest VM 3

Hardware

Hypervisor

Guest VM 1

Vulnerabilities in Server Virtualization

• The hypervisor is quite complex• Large amount of code ―> Bugs (NIST’s National

Vulnerability Database)

7

Guest VM 2 Guest VM 3

Hardware

Hypervisor

Guest VM 1

Vulnerabilities in Server Virtualization

• The hypervisor is an attack surface (bugs, vulnerable) ―> Malicious customers attack the hypervisor

8

Vulnerabilities in Network Virtualization

• Software switch in control VM (Dom0)• Hypervisor is involved in communication

Hardware

Hypervisor

Dom0

SoftwareSwitch

Physical NIC

Guest VM 2Guest VM 1

VirtualInterface

VirtualInterface

9

Hardware

Hypervisor

Dom0

SoftwareSwitch

Physical NIC

Guest VM 2Guest VM 1

VirtualInterface

VirtualInterface

• Software switch is coupled with the control VM ―> e.g., software switch crash can lead to a complete

system crash

Vulnerabilities in Network Virtualization

10

Dom0 Disaggregation [e.g., SOSP’11]

• Disaggregate control VM (Dom0) into smaller, single-purpose and independent components

• Malicious customer can still attack hypervisor!

Hardware

Hypervisor

Guest VM 1 Dom0 Guest VM 2

ServiceVM

ServiceVM

ServiceVM

ServiceVM

11

NoHype [ISCA’10, CCS’11]

• Eliminate the hypervisor attack surface• What if I want to use a software switch?

Hypervisor

Dom0

Emulate,Manage

HardwareVirtualized Physical NIC

Physical Device Driver

Physical Device Driver

• Pre-allocating memory and cores

• Using hardware virtualized I/O devices

• Hypervisor is only used to boot up and shut down guest VMs.

Guest VM 1 Guest VM 2

12

Software Switching in NoHype

• Bouncing packets through the physical NIC• Consumes excessive bandwidth on PCI bus

and the physical NIC!

Guest VM 1 Guest VM 3

Hypervisor

HardwareVirtualized Physical NIC

Physical Device Driver

Physical Device Driver

Guest VM 2

SoftwareSwitch

Dom0

Emulate,Manage

13

Our Solution Overview

• Eliminate the hypervisor attack surface• Enable software switching in an efficient way

Hardware

Guest VM 1 DomS Guest VM 2

VirtualInterface

SoftwareSwitch

Physical NIC

VirtualInterface

Hypervisor

Dom0

Emulate,Manage

14

Hypervisor

Eliminate the Hypervisor-Guest Interaction

• Shared memory– Two FIFO buffers for communication

• Polling only– Do not use event channel; no hypervisor involvement

FIFO

FIFO

PollingGuest VM 1

VirtualInterface

Dom0

SoftwareSwitch

15

Limit Damage From a Compromised Switch

• Decouple software switch from Dom0– Introduce a Switch Domain (DomS)

• Decouple software switch from the hypervisor– Eliminate the hypervisor attack surface

Hypervisor

FIFO

FIFO

PollingGuest VM 1

VirtualInterface

DomS

SoftwareSwitch

Dom0

FIFO

FIFO

PollingGuest VM 1

VirtualInterface

DomS

SoftwareSwitch

Dom0

HypervisorHypervisor

FIFO

FIFO

PollingGuest VM 1

VirtualInterface

Dom0

SoftwareSwitch

16

Preliminary Prototype

• Prototype based on – Xen 4.1: used to boot up/shut down VMs– Linux 3.1: kernel module to implement polling/FIFO– Open vSwitch 1.3

Hypervisor

Dom0

SoftwareSwitch

Guest VM 1

VirtualInterface

Native Xen

Hardware

FIFO

FIFO

PollingGuest VM 1

VirtualInterface

DomS

SoftwareSwitch

Our Solution

Hardware

Hypervisor

Dom0

17

Preliminary Evaluation

• Evaluate the throughput between DomS and a guest VM, compared with native Xen

• Traffic measurement: Netperf• Configuration: each VM has 1 core and 1GB of RAM

Hypervisor

Dom0

SoftwareSwitch

Guest VM 1

VirtualInterface

Native Xen

Hardware

FIFO

FIFO

PollingGuest VM 1

VirtualInterface

DomS

SoftwareSwitch

Our Solution

Hardware

Hypervisor

Dom0

18

Evaluation on Throughput

• FIFO Size– Polling period is fixed to 1ms– Reach high throughput with

just 256 FIFO pages (Only 1MB)

• Polling Period– Shorter polling period,

higher throughput– CPU resource consumption?

―> Future work

4 8 16 32 64 128 256 5120

300

600

900

1200

1500

1800

FIFO Pages (1 page = 4 KB)

Thro

ughp

ut (M

bps)

0.050.25 0.5 0.75 1 1.25 1.5 1.75 20

5001000150020002500300035004000

Polling Period (ms)

Thro

ughp

ut (M

bps)

19

Comparison with Native Xen

• Outperforms native Xen when message size is smaller than 8 KB.

• Future work: incorporate more optimization

64 128 256 512 1K 2K 4K 8K 16K0

2000400060008000

10000

Our Solution Native Xen

Message Size(Bytes)

Thro

ughp

ut (M

bps)

20

Conclusion and Future Work

• Trend towards software switching in the cloud• Security in hypervisor and Dom0 is a big concern• Improve security by enabling software switching

without hypervisor involvement

• Future work– Detection and remediation of DomS compromise

21

Thanks!

Q&A