VM Analysis – Episode 4 - Polytechnique...
Transcript of VM Analysis – Episode 4 - Polytechnique...
![Page 1: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/1.jpg)
VM Analysis – Episode 4
Wait analysis of virtualized environments using host kernel tracing
Hani Nemati
May 5, 2017
Polytechnique Montréal
Laboratoire DORSAL
![Page 2: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/2.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
Agenda
Introduction● Research update and research motivation
New Investigations● Wait analysis of virtualized environments using host kernel tracing
● Sate of the art
● Proposed Algorithm
● Demo
● KVM-Tool for eBPF
Conclusion and in-progress
![Page 3: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/3.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
Available Trace-Points in different layers
HardwareHardware
Host Kernel Host KernelKVM.KOKVM.KO
QemuQemu
GuestKernel
VM 1
QemuQemu
GuestKernel
VM 2
QemuQemu
GuestKernel
VM n
Guest Kernel TraceGuest Kernel Trace
Qemu TraceQemu Trace
KVM TraceKVM Trace
Host Kernel TraceHost Kernel Trace
Hardware PMCHardware PMC
Previously on “VM Analysis”
![Page 4: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/4.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
Available Trace-Points in different layers
HardwareHardware
Host Kernel Host KernelKVM.KOKVM.KO
QemuQemu
GuestKernel
VM 1
QemuQemu
GuestKernel
VM 2
QemuQemu
GuestKernel
VM n
KVM TraceKVM Trace
Host Kernel TraceHost Kernel Trace
Previously on “VM Analysis”
![Page 5: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/5.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
Resource View for VM without tracing the VM
Previously on “VM Analysis”
![Page 6: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/6.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
Previously on “VM Analysis”
VirtFlow: Execution Flow Analysis of Virtual Machine
![Page 7: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/7.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
Two Nested VMs and One VM are preempting each other
Previously on “VM Analysis”
![Page 8: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/8.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
MotivationWhy the VM is waiting?
![Page 9: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/9.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
MotivationLet's use the Critical Flow view of Trace Compass?
![Page 10: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/10.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
InvestigationsMethodology Vec from kvm_inj_virq
CR3 from vcpu_enter_guest
Vec from kvm_inj_virqCR3 from vcpu_enter_guestCR3 from vcpu_enter_guest
If (Vec == (Block I/O irq)) {Block State = Block I/O State
} else if (Vec == (network irq)) {Block State = Network State
}
![Page 11: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/11.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
InvestigationsMethodology
If (Vec == 239) {Block State = Timer
} else if (Vec == 251) {Block State = Task
}
Vec from kvm_inj_virqCR3 from vcpu_enter_guest
CR3 from vcpu_enter_guest
Vec from kvm_inj_virqCR3 from vcpu_enter_guest
![Page 12: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/12.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
Investigations
Demo
![Page 13: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/13.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
InvestigationsWhat do you need to test this project?
● Access to Host only
● Run LTTng on Host with my new added tracepoint (vcpu_enter_guest)
● Clone TraceCompass from my github (virtFlow)● https://github.com/Nemati
● Open Resource View of TraceCompass
![Page 14: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/14.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
Investigations
One More Thing ...
![Page 15: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/15.jpg)
KVM-ToolsFor
eBPF
POLYTECHNIQUE MONTREAL – Hani Nemati
![Page 16: VM Analysis – Episode 4 - Polytechnique Montréalhsdm.dorsal.polymtl.ca/system/files/05May2017.pdf · VM Analysis – Episode 4 Wait analysis of virtualized environments using host](https://reader034.fdocuments.in/reader034/viewer/2022042317/5f067d0f7e708231d4183e57/html5/thumbnails/16.jpg)
POLYTECHNIQUE MONTREAL – Hani Nemati
Conclusion and in-progress
Inferences
● Wait Analyzing of process inside VM● A process is waiting for
● A Block request to finish● A network packet to receive ● Another process● A timer to fire
What you will see in Episode 5
● Wait Analyzing of process inside Nested VM