Network performance analysis in virtualized environments · network activities in virtual...
Transcript of Network performance analysis in virtualized environments · network activities in virtual...
Network performance analysis in virtualized environments
Adel Belkhiri Michel DagenaisDecember 6, 2018
Polytechnique Montréal
Laboratoire DORSAL
POLYTECHNIQUE MONTREAL – Adel Belkhiri
Introduction– Previously on vGPU analysis
– Network virtualization analysis
Literature review
Preliminary investigations
Conclusion
2
Agenda
POLYTECHNIQUE MONTREAL – Adel Belkhiri 3
Previously on vGPU analysis
● Gvt-g is a full GPU virtualization product for Intel GPUs (Haswell architecture and up) :
● Gvt-g : Graphics Virtualization Technology- grid
● KvmGT is a version of Gvt-g for KVM
● Performance analysis of I/O operations in virtual machines.
Tracing framework for the performance analysis of vGPU-enabled VMs created with KvmGT
POLYTECHNIQUE MONTREAL – Adel Belkhiri 4
Control Flow view of GPU requests
Per-engine GPU occupancy
Previously on vGPU analysis
virtual GPU occupancy
Per-process GPU occupancy
POLYTECHNIQUE MONTREAL – Adel Belkhiri
● Traditional network monitoring tools (such as Tcpdump) are incapable of monitoring network activities in virtual environments (cloud computing) :
● Incapacity to detect data exchanged between collocated VMs
● Incompatibility with some properties of cloud computing (live migration, etc.)
● Performance metrics not adapted to virtual networks specificities.
5
Network virtualization analysis
Context of the current topic :
POLYTECHNIQUE MONTREAL – Adel Belkhiri
1) Profile different technologies used in networking virtualization : Virtio-net, auto-virtualized network cards, Open vSwitch, etc.
2) Analyze the performance of virtualized networks based on performance metrics relevant to virtualized environments.
3) Understand the latencies caused by packet flow processing, where one of the two endpoints is a VM, and identify bottlenecks.
Motivation :
6
Network virtualization analysis
POLYTECHNIQUE MONTREAL – Adel Belkhiri
Introduction
Literature review– NIC virtualization
– Switch/router virtualization
Preliminary investigations
Conclusion
7
Agenda
POLYTECHNIQUE MONTREAL – Adel Belkhiri
Technologies used in NIC virtualization
NIC Virtualization
8
POLYTECHNIQUE MONTREAL – Adel Belkhiri
● Para-virtualization
● Virtio is a standard interface through which a module of the guest OS could exchange data with a module of the host OS using shared memory (ring buffer).
● Virtio-net uses the Virtio framework to create a paravirtualized network interface, at the user-space level (QEMU).
● Virtio-net limitations : Poor performance due to frequent context switches between host OS, QEMU and guest OS.
NIC Virtualization
9
POLYTECHNIQUE MONTREAL – Adel Belkhiri
virtio-net vs. vhost-net
Kernel
QEMU
Virtual Machine
virtio buffers
Hardware
Tap
virtio net
1
2
3Kernel
QEMU
Hardware
Tap
2
vhost net
Virtual Machine
virtio buffers
virtio-net vhost-net
NIC Virtualization
10
POLYTECHNIQUE MONTREAL – Adel Belkhiri
● Hardware-assisted Virtualization
● VMDq (Virtual Machine Device Queue)
► Sort packets and save them in waiting queues implemented at the hardware level.
► Each waiting queue is associated with a different interruption to allow the hypervisor to distinguish between recipient VMs.
● SR-IOV (Single Root – Input Output Virtualization)
► Reduce the interventions of the hypervisor by copying packets directly in the memory space of the VM.
► High performance.
NIC Virtualization
11
POLYTECHNIQUE MONTREAL – Adel Belkhiri
● Hardware-assisted Virtualization
VMDq vs. SR-IOV
VM1
VMM
NIC
VM2 VM3
RX TX
Layer 2 : virtual switch
VM1
VMM
NIC
VM2 VM3
Layer 2 : virtual switch
VM1
VMM
VM2 VM3
RX TX RX TX RX TX
Virtual Bridge and classifier
VF VF VF
NIC
Layer 2 : classifier
VMDq SR-IOVOrdinary NIC
NIC Virtualization
12
POLYTECHNIQUE MONTREAL – Adel Belkhiri
● SDN (Software-Defined Networking)
● A new networking architecture which separates, at the hardware level, the plan of network control from the plan of forwarding packets.
● OVS (OpenvSwitch), implements the OpenFlow standard and emulates a switch in many virtualized environments.
Switches virtualization
Réf : https://www.commsbusiness.co.uk/features/software-defined-networking-sdn-explained/
13
POLYTECHNIQUE MONTREAL – Adel Belkhiri
Introduction
Literature review
Preliminary investigations– Packets receiving path
– What about tracing ?
Conclusion
Agenda
14
POLYTECHNIQUE MONTREAL – Adel Belkhiri
NIC
CPU
ring buffer
RAM
Packets
SKB (socket buffer)
1. received by NIC
2. copy via DMA
3. run IRQ handler
4. suspend IRQ
Driver
Packets receiving path
15
NIC
CPU
ring buffer
RAM
Packets
SKB (Socket Buffers)
napi_gro_receive
netif_receive_skb GRO List
net_rx_action
1. received by NIC2. copy via DMA
3. run IRQ handler 7. trying to combine packets via GRO
8. packets are held and combined
9. forward packet to TCP/IP stack
6. poll () : harvest packets
...
5. run Soft_IRQ handler
4. suspend IRQ
Driver
poll structure
POLYTECHNIQUE MONTREAL – Adel Belkhiri
Packets receiving path
16
POLYTECHNIQUE MONTREAL – Adel Belkhiri
Packet Rx path
softirq_entry
netif_receive_skb
skb_copy_datagram_iovec
napi_poll
softirq_exit
skb_consume
skb_free
napi_gro_receive
Packet Tx path
softirq_entry
net_dev_start_xmit
net_dev_xmit
softirq_exit
net_dev_queue
What about tracing ?
17
POLYTECHNIQUE MONTREAL – Adel Belkhiri
Trace Compass State System
Ongoing work ...
18
POLYTECHNIQUE MONTREAL – Adel Belkhiri
Views already developped (in Trace Compass) :
View of TCP sockets based on a pcap trace (tcpdump)
Views that we want to develop :
Ongoing work ...
19
POLYTECHNIQUE MONTREAL – Adel Belkhiri
Introduction
Literature review
Preliminary investigations
Conclusion
Agenda
20
POLYTECHNIQUE MONTREAL – Adel Belkhiri
● There is a real need for efficient analyses to diagnose problems in virtual networks and identify the root causes of their latencies.
● Tracing tools (such as LTTng) are great to collect the low-level data needed for these analyses.
● Our current challenge is to develop a tracing framework capable of analyzing the performance of traditional networks.
21
Conclusion
POLYTECHNIQUE MONTREAL – Adel Belkhiri