Visor: Private Video Analytics as a Cloud Serviceiot.stanford.edu/retreat19/sitp19-visor.pdf ·...

Visor: Private Video Analytics as a Cloud Service

Rishabh PoddarGanesh Ananthanarayanan, Srinath Setty,

Stavros Volos, and Raluca Ada Popa

UC Berkeley and Microsoft Research

• Image processing

• Speech recognition

• Video analytics

• …

Machine learning as a service

�2

Data

ClientCloud platform

Results

Key application: Video analytics

Decisions based on objects in videos


�4

Decisions based on objects in videos • E.g. Raise an alert if a human at the main door

!


�5

Decisions based on objects in videos • E.g. Raise an alert if a human at the main door • E.g. Change the traffic lights if less than 3 cars at the intersection

!

Cloud platform

Client source

CPU

Video decoding

Video analytics pipeline

�6

Pipeline of video processing modules

Background subtraction

Bounding box detection

Object cropping

CNN classification

Objects

GPU

�7


Video decoding



Object cropping

CNN classification

ObjectsClient source

Cloud platform

CPU GPU

Cheap filters to discard useless frames • Filter out moving objects • CPU is under-utilized, unlike GPU


�8


Video decoding



Object cropping

CNN classification


Cloud platform

CPU GPUResults

1.Red car 2.White van 3.Tree 4.…


Problem: Privacy

�9

Video decoding



Object cropping

CNN classification


Cloud platform

CPU GPUResults

1.Red car 2.White van 3.Tree 4.…

Hackers / co-tenants curious employees

Video stream is proprietary to clients

Related work: Privacy-preserving inference

�10

Cryptographic approaches: e.g. SMPC, Homomorphic encryption


�11

Cryptographic approaches: e.g. SMPC, Homomorphic encryption • High overhead

— Can’t sustain video frame rate


�12


— Can’t sustain video frame rate • Cannot be easily extended to video processing modules


�13



Trusted hardware-based approaches (enclaves): e.g. Slalom, Chiron


�14



Trusted hardware-based approaches (enclaves): e.g. Slalom, Chiron • Designed for CPU enclaves (using Intel SGX)

— Can’t sustain video frame rate


�15



Trusted hardware-based approaches (enclaves): e.g. Slalom, Chiron • Designed for CPU enclaves (using Intel SGX)

— Can’t sustain video frame rate • Side-channel leakage — e.g. memory access patterns

Illustration of memory access pattern leakage

�16

Example: Object detection

Location of accessed pixels

Original image (processed within

SGX enclave)Input image Leakage Recovered image: Leaks shapes

and positions of all objectsInput image Leakage

Visor

�17

�18

1 Hybrid enclave architecture (CPU + GPU) for secure computation

2 Remove memory side-channel leakage via data-obliviousness

Redesign all components such that memory access patterns

are independent of data

Key ideas

Client

Cloud platform

Operating System (untrusted)

CPU

CPU enclave (Intel SGX)

�19

Hybrid trusted execution environment (TEE)

Application (trusted)

• Intel SGX (CPU TEE)

GPU enclave (Graviton)

�20

Client

Hybrid trusted execution environment (TEE)

Operating System (untrusted)

Application (trusted) GPU runtime Trusted

process

Cloud platform

CPU enclave (Intel SGX)

CPU

Other processes (untrusted)

GPU

• Intel SGX (CPU TEE) + Graviton (GPU TEE)

�21

Visor: Hybrid TEE for video analytics

Cloud platformCPU enclave (Intel SGX)


Client source

• Ported to SGX using Graphene (extended to support required ioctl calls)

�22

Fram

e de

code

r



Client sourceEncrypted

video streamO

bjec

t det

ectio

n m

odul

es


CN

N c

lass

ifier

GPU

runt

ime

Object bufferObjects

Results

Visor: Hybrid TEE for video analytics

�23

Sources of leakage

Fram

e de

code

r




video streamO

bjec

t det

ectio

n m

odul

es


Object buffer

CN

N c

lass

ifier

GPU

runt

ime

Objects

Results

�24

Sources of leakage

Fram

e de

code

r

GPU

runt

ime

CN

N c

lass

ifier




video streamO

bjec

t det

ectio

n m

odul

esObjects

Object buffer

1

Encrypted traffic pattern • E.g. leaks times of activity in video stream

Results

�25

Sources of leakage

Fram

e de

code

r

GPU

runt

ime

CN

N c

lass

ifier




video streamO

bjec

t det

ectio

n m

odul

esObjects

Object buffer

2

SGX side-channel leakage • Vast majority of known attacks on SGX

exploit memory access pattern leakage

Results

�26

Sources of leakage

Fram

e de

code

r

GPU

runt

ime

CN

N c

lass

ifier




video streamO

bjec

t det

ectio

n m

odul

esObjects

Object buffer

CPU-GPU communication pattern • Number / size of objects in frames

3

Results

�27

Sources of leakage

Fram

e de

code

r

GPU

runt

ime

CN

N c

lass

ifier




video streamO

bjec

t det

ectio

n m

odul

esObjects

Object buffer

Graviton side-channel leakage • Similar in principle to SGX side-channel

leakage

4

Results

�28

Sources of leakage

Fram

e de

code

r

GPU

runt

ime

CN

N c

lass

ifier




video streamO

bjec

t det

ectio

n m

odul

esObjects

Object buffer

Results

1 2 3 4

Roadmap

�29

2 SGX memory side-channels

1 Input traffic pattern

3 CPU-GPU communication pattern

4 Graviton memory side-channels

Roadmap

�30





Pad frames to an upper bound while enabling a bandwidth /

performance trade-off

1

�31





Roadmap

Develop novel data-oblivious video processing algorithms

2

Roadmap

�32





Fixed communication schedule using dummy items, while minimizing GPU utilization

3

Roadmap

�33





Data-oblivious CNN implementation

4

Roadmap

�34





�35





Roadmap 2

Preventing SGX memory side-channels

�36

Redesign algorithms so that memory access patterns are independent of secret data


�37


• Identical set of operations per pixel


�38


• Identical set of operations per pixel • Design algorithms with structured memory accesses for efficiency


�39


• Identical set of operations per pixel • Design algorithms with structured memory accesses for efficiency

Implemented using a small set of oblivious primitives • Assumption: Register-to-register operations are data-oblivious • Wrappers around x86’s CMOV


�40

Data-oblivious primitives • oselect — conditional assignment

mov rcx, cond mov rdx, x mov rax, y test rcx, rcx cmovz rax, rdx retn

oselect(cond,x,y)if (cond) {

y = x; }

Can perform dummy operations by setting cond

to false


�41

Data-oblivious primitives1 • oselect — conditional assignment • osort — sorting using a bitonic network • oaccess — array accesses

1Ohrimenko et al.: “Oblivious multi-party machine learning on trusted processors” (Usenix Security ‘16)

Illustration: Bounding box detection

�42

Video decoding



Object cropping

CNN classification


Cloud platform

CPU GPU

�42


�43�43

Input: Binary image Output: Bounding boxes of all objects

Binary image


�44�44


Algorithm: Border following 1. Scan image row-wise until a white pixel

Binary image


�45�45



2. Examine neighbors and follow the borderclock-wise to obtain edge contour

Binary image


�46�46




3. Compute bounding box for each contour

Binary image


�47�47




3. Compute bounding box for each contour

Leakage: Shape and location of each object

Binary image

Oblivious Algorithm: 1. Scan image row-wise

2. For each white pixel:

a. Assign smallest label of its already-scanned neighbors, or a new label.

b. Update “parents” of all white neighbors to record that they are connected

c. Update bounding box of assigned label

3. For each black pixel: same set of operations, but with dummy values (zeros)

4. Merge all connected labels


�48�48

Binary image



a. Assign smallest label of its already-scanned neighbors, or a new label.






�49�49

Binary image



a. Assign a label — smallest already-scanned neighbor, or a new label.






�50�50

Binary image




b. Update “parent” of each neighboring label to record that they are connected





�51�51

Binary image

0 1 2 3 4 5 6 … … NParents


�52�52

Binary image

0 1 2 3 4 5 6 … … N

Parents

Oblivious Algorithm: 1. Scan image row-wise.




c. Update label’s bounding box


4. Merge all connected labelsBounding box coordinates

LeftRight

TopBottom


�53�53

Binary image

0 1 2 3 4 5 6 … … N

Parents







4. Merge all connected labelsBounding box coordinates

LeftRight

TopBottom


�54�54

Binary image

0

Parents








0000

Bounding box coordinates

LeftRight

TopBottom

0 1 2 3 4 5 6 … … N0


�55�55

Binary image

0 0 0 0 0 0 0 0 0 00 1

0 10 10 10 1


LeftRight

TopBottom

0 1 2 3 4 5 6 … … N0 1 Parents









�56�56

Binary image

0 0 0 0 0 0 0 0 0 00 1 1

0 10 20 10 1


0 1 2 3 4 5 6 … … N0 1 Parents

LeftRight

TopBottom









�57�57

Binary image

0 0 0 0 0 0 0 0 0 00 1 1 0 0 0 0 2

0 1 70 2 70 1 20 1 2


0 1 2 3 4 5 6 … … N0 1 2 Parents

LeftRight

TopBottom









�58�58

Binary image

0 0 0 0 0 0 0 0 0 00 1 1 0 0 0 0 2 2

0 1 70 2 80 1 20 1 2


0 1 2 3 4 5 6 … … N0 1 2 Parents

LeftRight

TopBottom









�59�59

Binary image

0 0 0 0 0 0 0 0 0 00 1 1 0 0 0 0 2 2 00 0 1 1 1 1 1

0 1 70 6 80 1 20 2 2


0 1 2 3 4 5 6 … … N0 1 1 Parents

LeftRight

TopBottom









�60�60

Binary image

0 0 0 0 0 0 0 0 0 00 1 1 0 0 0 0 2 2 00 0 1 1 1 1 1 1 0 00 0 1 1 1 1 1 1 0 00 0 0 0 1 1 0 0 0 00

0 1 70 7 80 1 20 4 2


0 1 2 3 4 5 6 … … N0 1 1 Parents

LeftRight

TopBottom









�61�61

Binary image

0 0 0 0 0 0 0 0 0 00 1 1 0 0 0 0 2 2 00 0 1 1 1 1 1 1 0 00 0 1 1 1 1 1 1 0 00 0 0 0 1 1 0 0 0 00 3 3

0 1 7 10 7 8 20 1 2 50 4 2 5


0 1 2 3 4 5 6 … … N0 1 1 3 Parents

LeftRight

TopBottom









�62�62

Binary image

0 0 0 0 0 0 0 0 0 00 1 1 0 0 0 0 2 2 00 0 1 1 1 1 1 1 0 00 0 1 1 1 1 1 1 0 00 0 0 0 1 1 0 0 0 00 3 3 1

0 1 7 10 7 8 20 1 2 50 5 2 5


0 1 2 3 4 5 6 … … N0 1 1 1 Parents

LeftRight

TopBottom









�63�63

Binary image

0 0 0 0 0 0 0 0 0 00 1 1 0 0 0 0 2 2 00 0 1 1 1 1 1 1 0 00 0 1 1 1 1 1 1 0 00 0 0 0 1 1 0 0 0 00 3 3 1 1 1 1 1 1 0

0 1 7 10 8 8 20 1 2 50 5 2 5


0 1 2 3 4 5 6 … … N0 1 1 1 Parents

LeftRight

TopBottom









�64�64

Binary image

0 0 0 0 0 0 0 0 0 00 1 1 0 0 0 0 2 2 00 0 1 1 1 1 1 1 0 00 0 1 1 1 1 1 1 0 00 0 0 0 1 1 0 0 0 00 3 3 1 1 1 1 1 1 00 0 0 0 0 0 0 0 0 00 0 0 4 4 4 4 0 0 00 0 0 4 4 4 4 0 0 00 0 0 0 0 0 0 0 0 0

0 1 7 1 30 8 8 2 60 1 2 5 70 5 2 5 8


0 1 2 3 4 5 6 … … N0 1 1 1 4 Parents

LeftRight

TopBottom









�65�65

Binary image

0 0 0 0 0 0 0 0 0 00 1 1 0 0 0 0 2 2 00 0 1 1 1 1 1 1 0 00 0 1 1 1 1 1 1 0 00 0 0 0 1 1 0 0 0 00 3 3 1 1 1 1 1 1 00 0 0 0 0 0 0 0 0 00 0 0 4 4 4 4 0 0 00 0 0 4 4 4 4 0 0 00 0 0 0 0 0 0 0 0 0

0 1 7 1 30 8 8 2 60 1 2 5 70 5 2 5 8


0 1 2 3 4 5 6 … … N0 1 1 1 4 Parents

LeftRight

TopBottom









�66�66

Binary image

0 0 0 0 0 0 0 0 0 00 1 1 0 0 0 0 1 1 00 0 1 1 1 1 1 1 0 00 0 1 1 1 1 1 1 0 00 0 0 0 1 1 0 0 0 00 1 1 1 1 1 1 1 1 00 0 0 0 0 0 0 0 0 00 0 0 4 4 4 4 0 0 00 0 0 4 4 4 4 0 0 00 0 0 0 0 0 0 0 0 0

0 1 7 1 30 8 8 2 60 1 2 5 70 5 2 5 8


0 1 2 3 4 5 6 … … N0 1 1 1 4 Parents

LeftRight

TopBottom









�67�67

Binary image

Enhancement via parallelization


�68�68

Binary image

Enhancement via parallelization 1. Divide the image into strips


�69�69

Binary image


2. Detect bounding boxes per strip in parallel


�70�70

Binary image


2. Detect bounding boxes per strip in parallel

3. Merge connected boxes across strip boundaries

Performance highlights

�71

Setup

�72

Testbed • Intel i7-8700K with 6 cores at 3.7 GHz • NVIDIA GTX 780 GPU with 2304 CUDA cores operating at 863 MHz

Setup

�73


Workload • 4 real video streams of resolution 1280 x 720 and 1024 x 768

— traffic cameras, security cameras

Setup

�74


Workload • 4 real video streams of resolution 1280 x 720 and 1024 x 768

— traffic cameras, security cameras

Baseline • Insecure video pipeline (without enclaves / obliviousness)

Overhead of side-channel protection

�75

Net t

hrou

ghpu

t (fra

mes

/s)

0

50

100

150

200

CNN model used in pipelineAlexNet ResNet-18 ResNet-50 VGG-16 VGG-19

BaselineOblivious

Net t

hrou

ghpu

t (fra

mes

/s)

0

50

100

150

200


BaselineOblivious


�76

Performance bottleneck shifts to the GPU with heavier models

Net t

hrou

ghpu

t (fra

mes

/s)

0

50

100

150

200


BaselineOblivious


�77

Performance bottleneck shifts to the GPU with heavier models, leaving more legroom for obliviousness on the CPU

5.9x 5.1x 2.5x 2.2x 2.2x

Overhead of enclaves

�78

Net l

aten

cy (m

s)

0

250

500

750

1000

Frame resolution1280 x 720 320 x 180

BaselineBaseline + enclavesOblivious + enclaves


�79

Net l

aten

cy (m

s)

0

250

500

750

1000




�80

Limited CPU enclave memory size increases baseline latency for large framesNe

t lat

ency

(ms)

0

250

500

750

1000



7.8x

2.5x


�81

Net l

aten

cy (m

s)

0

250

500

750

1000



2.4x

1.5x

Limited CPU enclave memory size increases baseline latency for large frames

• Hybrid enclaves for running MLaaS applications such

as video analytics pipelines

• Data-oblivious algorithms to remove memory side-

channels

• Overhead of oblivious techniques limited to 2x—6x

�82

Summary [email protected]

mailto:[email protected]

Thanks!

�83

Visor: Private Video Analytics as a Cloud Serviceiot.stanford.edu/retreat19/sitp19-visor.pdf ·...

Documents

Transcript of Visor: Private Video Analytics as a Cloud Serviceiot.stanford.edu/retreat19/sitp19-visor.pdf ·...