Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking...
-
Upload
donavan-burritt -
Category
Documents
-
view
225 -
download
5
Transcript of Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking...
![Page 1: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/1.jpg)
![Page 2: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/2.jpg)
Virtual Machine Queue Architecture Review
Ali DabaghArchitectWindows Core Networking
Don StanwyckSr. Program ManagerNDIS [email protected]
![Page 3: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/3.jpg)
Agenda
• Introduction• Network device sharing in virtualization environment• Performance bottlenecks• Virtual Machine Queue (VMQ) Architecture• NIC embedded switch• Coexistence with offload technologies• Call to action• Q&A
![Page 4: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/4.jpg)
Why VMQ?
• Virtualization is now
• Networking optimization supports virtualization
• Virtualization impacts performance – VMQ helps overcome those effects
• VMQ supports Live Migration
• VMQ coexists with other optimizations
• Broad industry support
![Page 5: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/5.jpg)
Terminology
• Virtual Service Provider (VSP)• A driver running in the parent partition• Provides a specific functionality to Virtual Machines• Network VSP (NVSP) implemented as an NDIS intermediate driver
• Virtual Service Client (VSC)• A driver running in the child partition• Usually provides a device class functionality in child partitions• Communicates with its VSP counterpart in the parent partition• Network VSC (NVSC) implemented as an NDIS virtual miniport driver
• Virtual Machine Bus (VMBus) • A communication bus for passing control and data messages between child
and parent partition
• VMBus Channel• A communication channel created between a VSC in child partition and VSP
in parent partition
![Page 6: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/6.jpg)
Synthetic Device Data Path
Parent Partition VM1 VM2
Ethernet
VM BUS
TCP/IP TCP/IP
VM NIC 1 VM NIC 2
Network Virtual Service Provider
NIC
MiniportDriver
RoutingVLAN Filtering
Data Copy Port 1Port 2
![Page 7: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/7.jpg)
VMQ Concepts: Receive Queues
• Receive Queues• Are allocated by Network Virtual Service Provider• Collect network packets that pass filter tests
• Properties of a receive queue• Unique queue ID per miniport adapter• Interrupt processor affinity• Filters• Receive buffers
• Default Queue• Always exists• Collects packets that are not collected by any other queue
![Page 8: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/8.jpg)
VMQ Concepts: Receive Filters
• Receive filters are set on receive queues• One or more “network header field tests”• Unique filter ID per miniport adapter
• Specifying network headers in a filter• VMQ uses MAC header
• Specifying fields within the headers• VMQ uses destination MAC address and VLAN tag
• Setting multiple tests in one receive filter• A packet must pass all the network header field tests
• Setting multiple filters on the same queue• Packet is accepted if it passes any of those filters
• Specifying test operation and test value• VMQ uses “Equal” test operation
![Page 9: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/9.jpg)
Receive Filters and Receive Queues
![Page 10: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/10.jpg)
Packet Filtering Data Flow
![Page 11: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/11.jpg)
Virtual Machine Queue Overview
• Classifying received packets• Parsing the packets in hardware• Setting received packet queue ID
• Grouping received packets• All packets in a receive indication call are for the same VM NIC
• Applying VLAN filtering in hardware• Packets with invalid VLAN IDs are dropped by the NIC
• Concurrent receive processing for multiple VMs• Every queue is serviced by a different processor
• Avoiding copy from NIC receive buffers to VM address space• Using switch on the NIC to do route lookup on transmit
![Page 12: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/12.jpg)
Synthetic Device Data Paths With VMQ
Parent Partition VM1 VM2
Ethernet
VM BUS
TCP/IP TCP/IP
VM NIC 1 VM NIC 2
Network Virtual Service Provider
MiniportDriver
Switch/Routing unit
Q1 DefaultQueueQ2
RoutingVLAN filtering
Data Copy Port 1Port 2
NIC
![Page 13: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/13.jpg)
NIC
Queue 1
Using VM Address Space for Receive Buffers
VM1 VM2
D
D
D
D
D
Parent Partition
H
H
H
H
H
Queue 2D
D
D
D
D
H
H
H
H
HAd
dres
s Sp
ace
Addr
ess
Spac
e
Addr
ess
Spac
e
![Page 14: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/14.jpg)
NIC Embedded Switch
• Receive queues are paired with transmit queues• Each queue pair is a switch port
• No MAC address learning is required• The MAC address of each port is the one set on the receive queue
• Switch inspects destination MAC address +VLAN ID of transmit packets• If they pass the filter set on a receive queue, DMA to that queue• Otherwise send the packet on the wire
• Advantages in VM to VM communication• Avoids route lookup in software• Avoids a packet copy• Offload support in hardware
• Advantages in VM to wire communication• Avoids route lookup in software
![Page 15: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/15.jpg)
Coexistence with Other Offload Technologies
• Transmit and receive checksum• Still needs to be calculated and verified• Should work for both VM-to-VM and VM-to-wire data flow
• Large send offload• Should work for both VM-to-VM and VM-to-wire data flow
• Jumbo frame support• Maximum packet size • Scatter/gather support for received packets
• Chimney offload• If NIC supports both VMQ and Chimney, VMQ is used
![Page 16: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/16.jpg)
Summary
• VMQ improves network throughput• Distributes processing of network traffic for multiple VMs
among multiple processors
• VMQ reduces CPU utilization by: • Offloading packet classification to hardware• Avoiding network data copy• Avoiding route lookup on transmit path
• VMQ is compatible with most task offloads• VMQ is secure• VMQ supports live migration
![Page 17: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/17.jpg)
Call to Action
Call to Action
• Virtualization is NOW
• Network optimizations are crucial for virtualization support
• These vendors are among those committed to supporting VMQ in their NICs:• Alacritech• Broadcom• Intel• Neterion• ServerEngines• Solarflare
• Build/update all your drivers to be NDIS 6.20 compliant
![Page 18: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/18.jpg)
Resources
• Windows 7 WDK will be available as of WinHEC • Windows Logo Program Web site:
http://www.microsoft.com/whdc/winlogo/default.mspx • NDIS 6 Feedback alias:
![Page 19: Virtual Machine Queue Architecture Review Ali Dabagh Architect Windows Core Networking alid@microsoft.com Don Stanwyck Sr. Program Manager NDIS Virtualization.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551b0edb5503465e7d8b5f3d/html5/thumbnails/19.jpg)
Related Sessions
Session Day / Time
Virtual Machine Queue Driver Development Tues. 4-5
Developing TCP Chimney Drivers for Windows 7 Tues. 11-12
Driver Scalability Mon. 11-12 andTues. 11-12
NDIS 6.20 Overview Mon. 9:45-10:45 andTues. 9:45-10:45
Windows Logo Program Tests for NDIS Mon. 11-12 andWed. 9:45-10:45
Discussion: Porting Drivers to NDIS 6.20 Wed. 8:30-9:30
Lab: How to Debug an NDIS 6.20 Driver Mon. 2:45-3:45 andWed. 9:45-10:45