Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs...

16
Trusted Docker Containers and Trusted VMs in OpenStack Raghu Yeluri Abhishek Gupta

Transcript of Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs...

Page 1: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

Trusted Docker Containers and Trusted VMs in OpenStack

Raghu Yeluri

Abhishek Gupta

Page 2: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

o Context: Docker Security – Top Customer Asks

o Intel’s Focus: Trusted Docker Containers

o Who Verifies Trust ?

o Reference Architecture with OpenStack

o Demo

o Availability

o Call to Action

Outline

Page 3: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

Lightweight, open source engine for creating, deploying containers

Provides work flow for running, building and containerizing apps.

Separates apps from where they run.; Enables Micro-services; scale by composition.

Underlying building blocks: Linux kernel's namespaces (isolation) + cgroups(resource control) + ..

Components of DockerDocker Engine – Runtime for running, building Docker containers.

Docker Repositories(Hub) - SaaS service for sharing/managing images

Docker Images (layers)Images hold Apps. Shareable snapshot of software. Container is a running instance of image.

Orchestration: OpenStack, Docker Swarm, Kubernetes, Mesos, Fleet, Project

Atomic, Lattice…

Docker Overview in a Slide.. Docker Hub

Docker Layers

Page 4: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

1. How do you know that the Docker Host Integrity is there?o Do you trust the Docker daemon?

o Do you trust the Docker host has booted with Integrity?

2. How do you verify Docker Container Integrityo Who wrote the Docker image? Do you trust the image? Did the right Image get launched?

3. Runtime Protection of Docker Engine & Enhanced Isolationo How can Intel help with runtime Integrity?

4. Enterprise Security Features – Compliance, Manageability, Identity authentication.. Etc.

5. OpenStack as a single Control Plane for Trusted VMs and Trusted Docker Containers..

Docker Security – 5 key Customer Asks

Intel’s Focus: Enable Hardware-based Integrity Assurance for Docker Containers – Trusted Docker Containers

Page 5: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

o Launch Integrity of Docker Host

o Runtime Integrity of Docker Host

o Integrity of Docker Images

Trusted Docker Containers – 3 focus areas

Today’s Focus: Integrity of Docker Host, and how to use it in OpenStack.

Page 6: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

o Launch VMs on Servers that have demonstrated Boot Integrity – Platform Trusto Measured Launch of Boot Process/Components

with Intel TXT.o Trust Chain: HW->FW->BIOS->OS/VMMo What is measured at launch:

o Current: F/W, Core BIOS, OS/VMM Kernel, Initrdo Ext measurements: An7OS/FS modules

o Schedulers/Orchestrators Policy Manager use Trust to launch/create/Migrate VMs.

o Extend Chain of Trust to VMs.o Measure & Attest VM Images prior to Launch.o Encrypt VM Images and decrypt based on Platform Trust

(Tenant-Controls the Keys)

o Boundary Control of VMs– Control where your Trusted VMs are launching and migrating.

Trusted VMs - Summary

Measurements done at the time of boot

(Server boot and VM Launch)

Host OS/Hypervisor Kernel, Initrd++

HW w/ Intel TXT/TPM

AppApp

VM-1 VM-2

vRTM

Tboot

Measurements match! System &

VMs Trusted

Measurements done at the

time of Server boot)

Host OS/Hypervisor Kernel, Initrd

HW w/ Intel TXT/TPM

Apps

vFW VM-2

Tboot

Measurements match! System

trustedTrust Boundary

Trust Boundary

Will enable the same model and use-cases for Trusted Docker Containers

Page 7: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

Ensure Docker Containers are launched on Trusted Docker Hosts

o Boot-time integrity of the Docker Host o Measured Launch of Boot Process &

components with Intel TXT.

o Docker daemon and associated component added to TCB and Measured.

o Chain of Trust: H/w->FW->BIOS->OS->Docker Engine

o Remote attestation using an Attestation Authority*

Trusted Docker Containers - 1

Host OS

HW w/ Intel TXT

Docker Daemon

Container B

e.g.

Apache v2

Container C

e.g. Nginx

container A

e.g. Apache

TPMTBOOT

Docker Host Platform Integrity

Shared Bin/Libs

Page 8: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

Ensure that Docker Images are not tampered prior to Launch -

o Launch time integrity of Docker Imageso Chain of Trust: H/w->FW->BIOS->OS->Docker Engine -> Docker container

layers (apache, Ubuntu14.04, ubuntu14,…, base)o Docker daemon modification: prior to container launch, measure and verify

Docker image (and parent layer graph recursively)

Boundary Control/Geo-Tagging applies equally to Docker Containers as well - Compliance Needs.

o Orchestrator determines location/boundary for launching Docker Images.

Exploring: Docker Image encryption & Trust-based Retrieval of Keys – Sensitive Container Images (VNFs, PCI-DSS/HIPPA Containers.. etc)

Trusted Docker Containers - 2

Host OS

HW w/ Intel TXT

Docker Daemon

Container

B

e.g.

Apache v2

Container

C

e.g. Nginx

container

A

e.g.

Apache

TPM

TBOOT

Agents

Docker Host & Container Launch Integrity

Shared Bin/Libs

}

Page 9: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

What is measured for Trusted Docker Containers

Bootloader, Tboot and OS Kernel

Initrd++ (includes a measurement agent)

Docker Daemon• container management engine (e.g. Docker engine) • Measurement Agents

Trusted launch of containerized application

Ch

ain

of

Tru

st e

xte

nd

ed

to

ap

pli

cati

on

la

un

ch

Intel® TXT + TPM

Bios

ACM signed by manufacturer

Apache Patch v2

Apache Patch v1

Apache

Ubuntu14.04

Ubuntu

Containerized application layers (e.g. Docker image layers)

Page 10: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

What is measured – the details

X

System PowerON

UCodeValidates,Measures BIOS ACM

ACM Validates,MeasuresBIOS Init

Code

Init TXT & Mem,

Load SMM

PCR0

Non-Critical Code

LockTXT &

Memory

Config

Measure SMM & other

Trusted Code

PCR0+ ENTERACCS:LockConfig SENTER

LoadSINIT &

OS code

SINIT MeasuresTBOOT

PCR17

uCodeValidates

SINIT

PCR18

Option ROMs & other non-critical modules

PCR0 + SINIT Hash + …

BIOS

PCR0 PCR19

Source: Intel

SINIT MeasuresOS KernelInitrd++

Tboot-xmMeasuresDocker

Engine, other

PCR19+

LaunchOS

OS

Measurement Phase 1 (H/W + BIOS)• uCode evals BIOS ACM• BIOS ACM (evals BIOS init code)• BIOS• BIOS Option ROMs

Measurement Phase II (TBOOT, OS, Docker Engine…)

• Boot loader• uCode (evals SINIT ACM) • SINIT ACM (measures OS Kernel, initrd•Tboot-xm(agent in initrd) measures DockerEngine, other components

Page 11: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

Principles Of Operation

o Cluster Manager determines best hosts in the cluster, based on utilization, type, location compliance.. etc.

o (for this host list) Cluster Manager verifies Host Integrity with the Attestation Authority.

o Attestation Authority responds with Attestation Reports for the Hosts

o Cluster Manager picks best Server that has the Integrity and instantiates Containers.

Who Verifies the Docker Host Trust?

ImageRegistry

Scheduler/Cluster Manager

Attestation

Authority

Trusted Host

Trust Not Verified.

Trusted Host

Trust

Filter

Examples• OpenStack• Docker

Swarm• Kubernetes• Mesos• Fleet

OS/initrd+

Docker Engine

Agents

TPM v1.2

OS,Initrd+

Docker Engine

Agents

TPM v1.2

OS, Initrd++

Docker Engine

Agents

TPM v1.2

AttestationTraffic

RemoteAttestation API

Scheduler/Cluster Manager/Policy Manager…

Page 12: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

Trusted Docker Containers & VMs with OpenStack

Glance

Nova Scheduler

AttestationAuthority

(OAT)

Trusted

Host

Trusted

Host

TrustFilter

LocationFilter

OS, Initrd++

Docker Engine

Nova +

Agents

TPM v1.2

OS

QemuNova +

Agents

TPM v1.2

OS, Initrd++

DockerEngine

Nova

+Agents

TPM v1.2

Remote

Attestation

API

VM1 VM2

Trust Not Verified.

OS

QemuNova +

Agents

TPM v1.2

Trusted

Host

Trust VM launch

API Server

Horizon Trusted ContainerLaunch

1Horizon/API Server : Initiate Launch of Image (with Hypervisor_Type Property)

2 Nova Scheduler: ImageProp Filter excludes Hoststhat don’t met Image Hypervisor Type.

3 Nova Scheduler: Runs Trust/Location Filter to identifyTrusted Host (for VM or Docker Container)

4 Attestation Authority: Challenges Host to Attest.Provides Signed Attestation Report to Scheduler to use. – Identifies Trusted Host for VMs or Docker Containers.

ImageProp Filter

Nova Compute: Download Glance Image and Launch.For Docker Images: Nova uses DockerDriver to download, and loaded to Docker File system with Docker load Command.

5

1

2

3

4

5

5

TPM v1.2 TPM v1.2

Page 13: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

OpenStack changes1. Add hypervisor_type property to images

Value=qemu for VM images

Value=docker for docker images

2. Activate ImageProperties filter

filters out hosts that don’t match Value from Image Hypervisor Type

3. Activate Trust filter in openstack scheduler and trust properties in images

4. Configure Nova-compute to use docker driver.

DEFAULT] compute_driver =

novadocker.virt.docker.DockerDriver

Steps at: https://wiki.openstack.org/wiki/Docker)

Changes needed in OpenStack Infrastructure

[

Docker Specific changes

For Docker Image Integrity:

o Modified Docker daemon to intercept container launch request and call measurement agent before launch

o Manifest/trust-policy created and associated with each Docker layer

Infrastructure related changeso TXT/TPM hardware;

o TXT/TPM activation on the clusters

o Attestation Server is setup

Page 14: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

Demo

Page 15: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

o Intel’s focus: Enable Hardware-based Integrity assurance for Docker Containers –Trusted Docker Containerso Enabling the same model as we have done for VMs.o Intel TXT and Attestation Software becomes the foundation for asserting Docker Host

Integrity..o Intel iKGT (Kernel Guard Technology) can help in runtime integrity protection of the

Linux Kernel.

o OpenStack can launch VMs and Containers with the extensions that are already mainstream (Trusted Compute Pools)

o Get engaged, get started with Trusted VMs and OpenStack. Extensions to OpenStack for Trusted Docker containers, will be available in Q3 timeframe.

o iKGT is available now on 01.org. Download it and try it out.

Summary & Call to Action

Page 16: Trusted Docker Containers and Trusted VMs in OpenStack · Trusted Docker Containers and Trusted VMs in OpenStack ... o Reference Architecture with OpenStack ... Docker Engine –Runtime

Q & A