ARMv8 port of the Jailhouse hypervisor · Huawei Technologies Duesseldorf GmbH ARMv8 port of the...

Security Level:

Huawei Technologies Duesseldorf GmbH

ARMv8 port of the Jailhouse hypervisor

Antonios Motakis [email protected]

Version: V1.0(20160321)

mailto:[email protected]

Huawei Technologies Duesseldorf GmbH 2

Acknowledgements

Jan Kiszka, SIEMENS (Upstream project)

Jean-Philippe Brucker, ARM (ARM32 port)

Huawei ERC Munich team


Introduction

Why a new hypervisor?

Why Jailhouse on ARM64 / ARMv8?


Modern Virtualization

Hypervisor

A portable abstraction of a machine

VM

CPU CPU CPU CPU CPU CPU CPU CPU

vCPUv vCPU

Hypervisor



Rich in features

CPU CPU CPU CPU

Hypervisor

CPU CPU CPU CPU

VM

vCPUv vCPU

Hypervisor



…can be quite complex


Jailhouse: the hypervisor for safety

Partitioning of hardware resources

Certifiable

Safe and secure

Simple

Less than 10k lines of codeCPU CPU CPU CPU

Cell Cell Cell

RTOS Bare metal


Jail… what?

Cell => guest in other hypervisors

Root cell => host in KVM, Dom0 in Xen

Inmate => guest software


Advantages

Real time

Safety and isolation

Low overheads (close to bare metal)


Safety critical applications

Industrial control

Mixed criticality

Automotive


Beyond safety critical systems

Linux based system + bare metal data plane

Secure

Predictable latencies

Low overhead

Scalability concerns in large multi-core systems


Beyond safety critical systems

Data plane / control plane

separation


Why Jailhouse on ARM64

64 bit instruction set for ARM

Core count keeps increasing


Partitioning a system 101

Our building blocks

Linux module

Jailhouse firmware

Root cell configuration

Inmate cells configuration

+ inmate binaries



Our building blocks

Linux module

Jailhouse firmware



+ inmate binaries

Load Jailhouse

Interface with Jailhouse



Our building blocks

Linux module

Jailhouse firmware



+ inmate binaries

Higher privilege level than Linux

All the interesting stuff



Our building blocks

Linux driver

Jailhouse firmware



+ inmate binaries

All hardware resources (initially)

assigned to the host system



.cell binary built from C

Hypervisor configuration

struct {

struct jailhouse_system header;

...

} __attribute__((packed)) config = {

.header = {

.signature = JAILHOUSE_SYSTEM_SIGNATURE,

.hypervisor_memory = {

.phys_start = 0x82fc000000,

.size = 0x4000000,

},

...

.root_cell = {

.name = "amd-seattle",

...



Hardware resources

.mem_regions = {

/* gpio */ {

.phys_start = 0xe0030000,

.virt_start = 0xe0030000,

.size = 0x1000,

.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |

JAILHOUSE_MEM_IO,

},

/* gpio */ {

.phys_start = 0xe0080000,

.virt_start = 0xe0080000,

...



On x86 can be automatically generated

On ARM: write it yourself

Device tree information

/proc/iomem

Provided configurations

ARMv8 Foundation model (simulation)

A real hardware target



Our building blocks

Linux module

Jailhouse firmware



+ inmate binaries

Resources assigned to a cell

Provided examples

Binary built to be run from within a

cell


Video demo

CPU CPU CPU CPU

Root cell Cell Linux cell

Demoapp

CPUCPU CPU CPU


How to port a Hypervisor

…to ARM 64 bit processors

Hardware virtualization support

Portability of Jailhouse


How to port a Hypervisor

Currently supported

Any ARM 64 bit core with virtualization extensions

GICv2 interrupt controller


ARM64 Privilege Levels

EL0 UserApplications

EL1 KernelRoot cell (Linux)

Inmate cells

EL2 HypervisorJailhouse

EL3 Monitor


Hypervisor initialization

static int jailhouse_cmd_enable(struct jailhouse_system __user *arg)

{

...

on_each_cpu(enter_hypervisor, header, 0);

/* Entry point for Linux loader module on JAILHOUSE_ENABLE */

.text

.globl arch_entry

arch_entry:

...

hypervisor/arch/arm64/entry.S



/* Entry point for Linux loader module on JAILHOUSE_ENABLE */

.text

.globl arch_entry

arch_entry:

...

hypervisor/arch/arm64/entry.S

Initialize a stack

Call generic entry() function


Initialization overview

hypervisor

setup.c

control.c

paging.c

…

arm

paging.c

…

arm64

entry.S

setup.c

control.c

…

arch_entry and friends



hypervisor

setup.c

control.c

paging.c

…

arm

paging.c

…

arm64

entry.S

setup.c

control.c

…

entry

init_early



hypervisor

setup.c

control.c

paging.c

…

arm

paging.c

…

arm64

entry.S

setup.c

control.c

…

Page table generating

code

page_alloc and friends

for the hypervisor

Shared infrastructure

with ARM32



One challenge!

Jailhouse is a

statically linked

binary!

arch_entry• Entry in

Linux context!

entry• …still in

Linux context

early_init

paging_init• Init

page tables


Jailhouse entry (x86, ARM32)Linux VA

address space

Kernel VA

range

JailhouseJAILHOUSE_BASE

Jailhouse VA address space

Jailhouse

0x0

Kernel base1. kernel driver

loads Jailhouse

firmware

2. Jailhouse

initialization

starts in Linux

VA space

3. Jailhouse

switches to own VA

space during init

0x0

JAILHOUSE_BASE


…Jailhouse piggybacks on the Linux page tables during

initialization!!!


Jailhouse entry on ARM64?Linux VA

address space

Kernel VA

range

JailhouseJAILHOUSE_BASE

Jailhouse VA address space

Jailhouse

0x0(TTBR0_EL1)

Kernel base0xffff000000000000

(TTBR1_EL1) 1. kernel driver

loads Jailhouse

firmware

2. Jailhouse

initialization

starts in Linux

VA space3. Ooops! The

same VA range is

not mapable in EL2

0x0(TTBR0_EL2)

JAILHOUSE_BASE


Main challenge summary

Jailhouse on other platforms

Linux loads Jailhouse at JAILHOUSE_BASE

Jailhouse linked to run JAILHOUSE_BASE

Early init relies on this; shared MMU context

Jailhouse on ARM64

Linux loads Jailhouse anywhere

Jailhouse linked to run from JAILHOUSE_BASE (!= anywhere)


Possible solutions

Start with the MMU off

No unaligned memory accesses

Caches are being bypassed

Position independent binary

Need a linker during arch_entry…

Generate early bootstrap page tables


Solutions

Start with the MMU off

No unaligned memory accesses

Caches are being bypassed

Position independent binary

Need a linker during arch_entry…

Generate early bootstrap page tables


Initialization (finally)

hypervisor

setup.c

control.c

paging.c

…

arm

paging.c

…

arm64

entry.S

setup.c

control.c

…

Initialize the hypervisor

Restore the host, as a

root cell


Lifetime control of Jailhouse

hypervisor

setup.c

control.c

paging.c

…

arm

paging.c

…

arm64

entry.S

setup.c

control.c

…

Receive hypercalls from

the root cell

Create, destroy cells


Shared with ARM32 port

MMU / page tables generation

Extended for up to 48 bit address space, 4 level page tables



GIC (ARM Generic Interrupt Controller) handling



PSCI implementation (SMP support)

Low level PSCI operations in assembly


Jailhouse cell demos

Bare metal demo applications using the GIC, timers, and UART

Ported from ARM32


Shared components with ARM32

Extended for AArch64 support• 48 bit support• 4 level page tables• 64 bit PSCI operations

Shared drivers• GICv2• PL011 UART


Linux as an inmate

Allows for fully working Linux inmate, alongside the root cell

On the floor demo we can demonstrate a Linux inmate using

the second NIC of the platform


Floor demo

Only one UART port

No SMMU support (yet)

For demonstration, assign the second NIC using identity mapping

However, we lose the security guarantees offered by the SMMU


Conclusion

ARM64 processor core count increasing

Ideal for safety & real time

Addresses scalability concerns

Upstreaming in progress

https://github.com/siemens/jailhouse branch wip/arm64

https://github.com/siemens/jailhouse


Future directions

Upstreaming of the ARM64 port

More hardware support

GICv3

Targets with uncommon interrupt controller or SMMU

ARM SMMU support

Test and benchmark under more real world scenarios

Copyright©2014 Huawei Technologies Duesseldorf GmbH. All Rights Reserved.

The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating

results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially

from those expressed or implied in the predictive statements. Therefore, such information is provided for reference purpose only and constitutes neither

an offer nor an acceptance. Huawei may change the information at any time without notice.


Huawei is hiring in Munich!

Come talk to us!

ARMv8 port of the Jailhouse hypervisor · Huawei Technologies Duesseldorf GmbH ARMv8 port of the...

Documents

Transcript of ARMv8 port of the Jailhouse hypervisor · Huawei Technologies Duesseldorf GmbH ARMv8 port of the...