Secure In-VM Monitoring Using Hardware Virtualization

Monirul Sharif, Wenke Lee, Weidong Cui, and Andrea Lanzi

Presented by Tyler Bletsch

2

Introduction

• Problem: Kernel level rootkits can kill security monitoring software

• Solution: Virtualization allows the monitor to live outside the VM

• Problem: Out-of-VM monitoring can be very expensive– Why?

3

Out-of-VM monitoring

Inspection

H HandlerCM Monitor codeDM Monitor dataR Response

DP Program data

CP Program codeK HookDK Hook dataA Adversary program

4

Problem with out-of-VM monitoring• Significant performance impact

– VM context switch is expensive– Introspection is hypercall-driven: expensive

• We want a monitor without these problems

• Performance guarantees:– (P1) Fast invocation:

No privilege change for handlers– (P2) Data read/write at native speed:

No penalty to access DM or DP

5

Solution: Secure In-VM Monitoring

H HandlerCM Monitor codeDM Monitor dataR Response

DP Program data

CP Program codeK HookDK Hook dataA Adversary program

6

Security guarantees• Run the monitor inside the VM context,

but provide out-of-VM-style security guarantees:

– (S1) Isolation of monitor code CM and data DM:Adversary can’t access the monitor at all

– (S2) Designated point for switching into CM:There’s only point entry point (tiny attack

surface)

– (S3) The handler is called iff the hook is triggered:

The hook calls the handler; nobody else can

– (S4) Behavior of M is not maliciously alterable:No dependency on monitored data or code

7

How to achieve this?

• Separate address space with a one-way page table for the monitor

• Small entry/exit gates into this address space– Invocation checker verifies that the call is

legit

8

The SIM address space

SIM Data/Code• The monitor itself• Visible only within SIM address

space

Invocation checker• Verifies call chain is legit• Visible only in SIM space

Entry/exit gates• Visible in both• Writable only in SIM space• Tiny, well crafted

Kernel code/data• Not executable in SIM space

(can't accidentally run insecure code)

9

Changing the address space

• CR3 register indicates page table directory– Hypervisor controlled

• To avoid hypercall, use Intel CPU's "CR3_TARGET_LIST" to freely switch between:– P_SHADOW: Process's normal space– SIM_SHADOW: Restricted SIM space

10

Entry/exit gates• Only point of entry for SIM address space

• Entry:– Disable interrupts– Save CPU state to the stack– Switch address space– Re-disable interrupts– Switch stack to a SIM-restricted one– Run invocation checker

• Exit:– Restore stack, page table, CPU state– Re-enable interrupts– Jump to return point (usually the point of origin)

11

Invocation checker

• Obviously can't rely on stack• Use CPU debug registers to track jump

history• Lock these registers from modification

by VM

12

Security monitor

• Can't rely on untrusted code– No kernel functions, dynamic libraries, etc.

• Need to traverse kernel data structuresjust like the out-of-VM technique (VMI)

• Hardware access granted by hypervisor API

13

Performance requirements met? (P1) Fast invocation:

Yes: No privilege change needed;

just a page table switch

(P2) Data read/write at native speed:

Yes: Pages are mapped, accessed natively

14

Security requirements met? (S1) Isolation of monitor code CM and data DM:

Yes: Monitor memory isn't mapped to the process space

(S2) Designated point for switching into CM:Yes: Entry/exit points are the only way in

(S3) The handler is called iff the hook is triggered:Yes: Hook calls handler; calls pass the invocation

checker

(S4) Behavior of M is not maliciously alterable:Yes: Untrusted code is non-executable in SIM space

15

Ah, but what if they...

• Jump into the entry gate after the interrupt disable instruction (CLI)?– It gets re-disabled again after the page table

switch

• Switch to SIM_SHADOW directly?– The next instruction is no longer executable– But what if they do it right before a SIM

page?• Any page after an unallocated address is made

non-executable

16

Implementation• Linux KVM host, Windows XP guest• To initialize from a trusted boot state, a driver:

– Reserves virtual address space for entry/exit gates– Has the hypervisor construct a SIM page table– Loads the security monitor into kernel space and

has the hypervisor remap it into SIM space– Informs the hypervisor of each hook and handler;

hypervisor generates entry/exit routines accordingly• Memory protection:

– Don't check every time CR3 is changed,check new page table entries as they are created

17

Evaluation: Monitor invocation

• Time to enter and exit a no-op handler– SIM: 0.469 μs– Out-of-VM: 5.055 μs

• Factor of 10.7 improvement

18

Evaluation: Process creation monitor• Hook process creation• Traverse kernel data to find process name• Process must be on whitelist to proceed

– If not, skip and return failure

• Average time for process creation:– Traditional In-VM approach: 3.487 μs– SIM approach: 3.967 μs (+13.7%)– Out-of-VM approach: 28.039 μs

(+690.5%)

19

Evaluation: System call tracing

• Make note of every system call, emit trace every 10,000 calls

• Macrobenchmarks from PCMark 05 suite

Benchmark Bare Out-of-VM

overhead

SIM overhe

ad

Memory Latency 10.42 MAcc/s 84.58% 7.97%

HTML Render 1.12 pg/s 52.42% 5.83%

File Compress 3.4 MB/s 3.97% 0.59%

File Encrypt 20.56 MB/s 7.85% 0.89%

File Decrypt 78.21 MB/s 2.53% 0.45%

HDD 15.29 MB/s 41.68% 3.74%

Text Edit 82.73 pg/s 128.84%

9.64%

Average - 46.10% 4.15%

20

Conclusion

• SIM: The best of both worlds– Security of out-of-VM solution– Performance of traditional in-VM

monitoring

21

Concerns

• Very difficult to develop monitors– No libraries– No system calls– Porting a full-fledge anti-malware

package?

• Interrupt race condition in entry code?CLIPUSHAMOV EAX, SIM_SHADOWMOV CR3, EAX

CLIMOV [P_ESP] ESPMOV ESP, [SIM_ESP]JMP INVOKATION_CHECK

Address space switch

Normal entry

Attacker entry

Interrupt now?

22

Discussion