Machine Learning in the presence of adversaries · Machine Learning in the presence of adversaries...

Machine Learningin the presence of adversaries

(joint work with Mika Juuti, Jian Liu, Andrew Paverd and Samuel Marchal)

N. Asokan http://asokan.org/asokan/@nasokan

2

Machine Learning is ubiquitous

The ML market size is expected to grow by 44% annually over next five yearsIn 2016, companies invested up to $9 Billion in AI-based startups

Machine Learning and Deep Learningis getting more and attention...

2[1] http://www.marketsandmarkets.com/PressReleases/machine-learning.asp[2] McKinsey Global Institute, ”Artificial Intelligence: The Next Digital Frontier?”

http://www.marketsandmarkets.com/PressReleases/machine-learning.asp

Machine Learning for security/privacy

3

Access Control Deception DetectionMarchal et al., “Off-the-Hook: An Efficient and Usable Client-Side Phishing Prevention Application” 2017. IEEE Trans. Comput.https://ssg.aalto.fi/projects/phishing/

https://ssg.aalto.fi/projects/phishing/

4

How do we evaluate ML-based systems?

Effectiveness of inference• measures of accuracy

Performance• inference speed and memory consumption

...

Meeting these criteria even in the presence of adversaries?

Security & privacy ofmachine learning

6

Which class is this?School bus

Which class is this?Ostrich

Szegedy et al., “Intriguing Properties of Neural Networks” 2014. https://arxiv.org/abs/1312.6199v4

Skip to robust adversarial examples

+ 0.1⋅ =

Adversarial examples

https://arxiv.org/abs/1312.6199v4

7

Which class is this?Panda

Goodfellow et al., “Explaining and Harnessing Adversarial Examples” ICLR 2015

Which class is this?Gibbon

+ 0.007⋅ =


8


8Papernot et al, “Practical Black-Box Attacks against Machine Learning”, ASIACCS 2017 https://arxiv.org/abs/1602.02697

https://arxiv.org/abs/1602.02697

9

Which class is this?Cat

Which class is this?Desktop computer

Athalye et al. “Synthesizing Robust Adversarial Examples”. https://blog.openai.com/robust-adversarial-inputs/

https://blog.openai.com/robust-adversarial-inputs/

1010Zhang et al, “DolphinAttack: Inaudible Voice Commands”, ACM CCS 2017 https://arxiv.org/abs/1708.09537


11Fredrikson et al. “Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures”, ACM CCS 2015. https://www.cs.cmu.edu/~mfredrik/papers/fjr2015ccs.pdf

Model inversion

https://www.cs.cmu.edu/%7Emfredrik/papers/fjr2015ccs.pdf

12

Model inversion

12Hitaj et al. “Deep models under the GAN: information leakage from collaborative deep learning”, ACM CCS 2017. https://arxiv.org/pdf/1702.07464.pdf

https://arxiv.org/pdf/1702.07464.pdf

13

Machine Learning pipeline

Data owners

Analyst

𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

𝐿𝐿𝑇𝑇𝐿𝐿𝐿𝐿

ML model Client

Prediction Service Provider

API𝐷𝐷𝑇𝑇𝑎𝑎𝑇𝑇𝐿𝐿𝑇𝑇𝑎𝑎

Where is the adversary? What is its target?

14

Speed limit 80km/h



Compromised input

Data owners

Analyst

ML model


API

Dang et al., “Evading Classifiers by Morphing in the Dark”, ACM CCS ’17. https://arxiv.org/abs/1705.07535Evtimov et al., “Robust Physical-World Attacks on Deep Learning Models”. https://arxiv.org/abs/1707.08945Zhang et al., “DolphinAttack: Inaudible Voice Commands”, ACM CCS ’17. https://arxiv.org/abs/1708.09537

Evade model

𝐷𝐷𝑇𝑇𝑎𝑎𝑇𝑇𝐿𝐿𝑇𝑇𝑎𝑎ML

model Client

https://arxiv.org/abs/1705.07535https://arxiv.org/abs/1707.08945https://arxiv.org/abs/1708.09537

15



Malicious client

Data owners

Analyst

ML model


API

Shokri et al., “Membership Inference Attacks Against Machine Learning Models”. IEEE S&P ’16. https://arxiv.org/pdf/1610.05820.pdfFredrikson et al., “Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures”. ACM CCS’15. https://www.cs.cmu.edu/~mfredrik/papers/fjr2015ccs.pdf

Invert model, infer membership


model Client

Inference

𝐷𝐷𝑇𝑇𝑎𝑎𝑇𝑇𝐿𝐿𝑇𝑇𝑎𝑎

https://arxiv.org/pdf/1610.05820.pdfhttps://www.cs.cmu.edu/%7Emfredrik/papers/fjr2015ccs.pdf

16



Malicious client

Data owners

Analyst

ML model


API Client

Tramer et al., “Stealing ML models via prediction APIs”, Usenix SEC ’16. https://arxiv.org/abs/1609.02943

Extract/steal model


model

MLmodel


17


Malicious prediction service

Data owners

Analyst

𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 ML model


API Client X

Malmi and Weber. “You are what apps you use Demographic prediction based on user's apps”, ICWSM ‘16. https://arxiv.org/abs/1603.00059

Profile users

𝐷𝐷𝑇𝑇𝑎𝑎𝑇𝑇𝐿𝐿𝑇𝑇𝑎𝑎

𝐷𝐷𝑇𝑇𝑎𝑎𝑇𝑇𝐿𝐿𝑇𝑇𝐿𝐿𝑇𝑇Add: “X uses app”

Is this appmalicious?


18

Compromised toolchain: adversary inside training pipeline

Data owners

Analyst



ML model


API Client

Song et al., “Machine Learning models that remember too much”, ACM CCS ’17. https://arxiv.org/abs/1709.07886 Hitja et al., “Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning”, ACM CCS ’17. http://arxiv.org/abs/1702.07464

Sensitive query

Reveal trainingdata

Violate privacy



ML model𝐷𝐷𝑇𝑇𝑎𝑎𝑇𝑇𝐿𝐿𝑇𝑇𝑎𝑎𝐷𝐷𝑇𝑇𝑎𝑎𝑇𝑇𝐿𝐿𝑇𝑇𝑎𝑎

https://arxiv.org/abs/1709.07886http://arxiv.org/abs/1702.07464

19

Malicious data owner

Data owners

Analyst



ML model


API Client

https://www.theguardian.com/technology/2016/mar/26/microsoft-deeply-sorry-for-offensive-tweets-by-ai-chatbothttps://www.theguardian.com/technology/2017/nov/07/youtube-accused-violence-against-young-children-kids-content-google-pre-school-abuse

𝐷𝐷𝑇𝑇𝑎𝑎𝑇𝑇𝐿𝐿𝑇𝑇𝑎𝑎𝐷𝐷𝑇𝑇𝑎𝑎𝑇𝑇𝐿𝐿𝑇𝑇𝑎𝑎ML

model

Influence ML model (model poisoning)

https://www.theguardian.com/technology/2016/mar/26/microsoft-deeply-sorry-for-offensive-tweets-by-ai-chatbothttps://www.theguardian.com/technology/2017/nov/07/youtube-accused-violence-against-young-children-kids-content-google-pre-school-abuse

20

ML-based systems need to worry about adversarial behaviourAdversaries are multilateral, solutions are tooAdversarial machine learning is now a very active research area

Secure Systems Group Finnish Center for AIhttps://ssg.aalto.fi/ http://fcai.fi/

Summary

20

https://ssg.aalto.fi/http://fcai.fi/

Oblivious Neural NetworkPredictions via MiniONNTransformationsN. Asokan

http://asokan.org/asokan/@nasokan

(Joint work with Jian Liu, Mika Juuti, Yao Lu)By Source, Fair use, https://en.wikipedia.org/w/index.php?curid=54119040

https://en.wikipedia.org/w/index.php?curid=54119040

3

Machine learning as a service (MLaaS)

Predictions

Input

violation of clients’ privacy

4

Running predictions on client-side

Model

model theftevasionmodel inversion

Oblivious Neural Networks (ONN)

Given a neural network, is it possible to make it oblivious?

• server learns nothing about clients' input;

• clients learn nothing about the model.

5

6

Example: CryptoNets

FHE-encrypted input

FHE-encrypted predictions

[GDLLNW16] CryptoNets, ICML 2016

• High throughput for batch queries from same client • High overhead for single queries: 297.5s and 372MB (MNIST dataset)• Cannot support: high-degree polynomials, comparisons, …

FHE: Fully homomorphic encryption (https://en.wikipedia.org/wiki/Homomorphic_encryption)

http://proceedings.mlr.press/v48/gilad-bachrach16.pdfhttps://en.wikipedia.org/wiki/Homomorphic_encryption

7

MiniONN: Overview

Blinded input

Blinded predictions

oblivious protocols

• Low overhead: ~1s • Support all common neural networks

By Source, Fair use, https://en.wikipedia.org/w/index.php?curid=54119040

https://en.wikipedia.org/w/index.php?curid=54119040

8

Example

All operations are in a finite fieldx

y

'x

z

https://eprint.iacr.org/2017/452

Skip to performance


9

Core idea: use secret sharing for oblivious computation

cy

cx'

cy' sy'+z

client & server have shares and s.t.

client & server have shares and s.t.

Use efficient cryptographic primitives (2PC, additively homomorphic encryption)

Skip to performance

10

Secret sharing initial input

Note that xc is independent of x. Can be pre-chosen

xSkip to performance



11

Compute locally by the server

Dot-product

Oblivious linear transformationSkip to performance

W•xc

12

Oblivious linear transformation: dot-productHomomorphic

Encryption with SIMD

u + v = W•xc; Note: u, v, and W•xc are independent of x. generated/stored in a precomputation phase

Skip to performance

13


14


Oblivious activation/pooling functions

15

Piecewise linear functions e.g.,• ReLU:• Oblivious ReLU:

- easily computed obliviously by a garbled circuit

Skip to performance

)1/(1: )(cs yycs exx +−+=+

Oblivious activation/pooling functions

16

Smooth functions e.g.,• Sigmoid:• Oblivious sigmoid:

- approximate by a piecewise linear function- then compute obliviously by a garbled circuit- empirically: ~14 segments sufficient

18

Combining the final result

They can jointly calculate max(y1,y2)(for minimizing information leakage)

19

Core idea: use secret sharing for oblivious computation

cy

cx'

cy' sy'+z

19

22

PTB/Sigmoid 4.39 (+ 13.9) 474 (+ 86.7) Less than 0.5%(cross-entropy loss)

Performance (for single queries)

Pre-computation phase timings in parentheses

CIFAR-10/ReLU 472 (+ 72) 6226 (+ 3046) none

Model Latency (s) Msg sizes (MB) Loss of accuracy

MNIST/Square 0.4 (+ 0.88) 44 (+ 3.6) none

PTB = Penn Treebank

MiniONN pros and cons

300-700x faster than CryptoNets

Can transform any given neural network to its oblivious variant

Still ~1000x slower than without privacy

Server can no longer filter requests or do sophisticated metering

Assumes online connectivity to server

Reveals structure (but not params) of NN

23

Skip to End

Can trusted computing help?

Hardware support for- Isolated execution: Trusted Execution Environment- Protected storage: Sealing- Ability to report status to a remote verifier:

Attestation

24

Other Software

Trusted Software

Protected Storage

Root of Trust

https://www.ibm.com/security/cryptocards/ https://www.infineon.com/tpm https://software.intel.com/en-us/sgxhttps://www.arm.com/products/security-on-arm/trustzone

Cryptocards Trusted Platform Modules ARM TrustZone Intel Software Guard Extensions

https://www.ibm.com/security/cryptocards/https://www.infineon.com/tpmhttps://software.intel.com/en-us/sgxhttps://www.arm.com/products/security-on-arm/trustzone

25

Using a client-side TEE to vet input

1. Attest client’s TEE app3. Input

4. Input, “Input/Metering Certificate”

5. MiniONN protocol + “Input/Metering Certificate”

2. Provision filtering policy

MiniONN + policy filtering + advanced metering

26

3. Input

Using a client-side TEE to run the model

1. Attest client’s TEE app

4. Predictions + “Metering Certificate”

2. Provision model configuration, filtering policy

MiniONN + policy filtering + advanced metering+ disconnected operation + performance + better privacy- harder to reason about model secrecy

5. “Metering Certificate”

27

2. Input

Using a server-side TEE to run the model

1. Attest server’s TEE app

3. Provision model configuration, filtering policy

MiniONN + policy filtering + advanced metering- disconnected operation + performance + better privacy

1. Attest server’s TEE app

4. Prediction

29

MiniONN: Efficiently transform any givenneural network into oblivious form with no/negligible accuracy loss

Trusted Computing can help realize improved security and privacy for ML

ML is very fragile in adversarial settingshttps://eprint.iacr.org/2017/452ACM CCS 2017


ML-and-security.pdfMachine Learning�in the presence of adversariesMachine Learning is ubiquitousMachine Learning for security/privacyHow do we evaluate ML-based systems?Security & privacy of machine learningAdversarial examplesAdversarial examplesAdversarial examplesSlide Number 9Slide Number 10Model inversionModel inversionMachine Learning pipelineCompromised inputMalicious clientMalicious clientMalicious prediction serviceCompromised toolchain: �adversary inside training pipelineMalicious data ownerSummary

MiniONN.pdfOblivious Neural Network Predictions via MiniONN TransformationsMachine learning as a service (MLaaS)Running predictions on client-sideOblivious Neural Networks (ONN)Example: CryptoNetsMiniONN: OverviewExampleCore idea: use secret sharing for oblivious computationSecret sharing initial inputOblivious linear transformationOblivious linear transformation: dot-productOblivious linear transformationOblivious linear transformationOblivious activation/pooling functionsOblivious activation/pooling functionsCombining the final resultCore idea: use secret sharing for oblivious computationPerformance (for single queries)MiniONN pros and consCan trusted computing help?Using a client-side TEE to vet inputUsing a client-side TEE to run the modelUsing a server-side TEE to run the modelSlide Number 29

Machine Learning in the presence of adversaries · Machine Learning in the presence of adversaries...

Documents

Transcript of Machine Learning in the presence of adversaries · Machine Learning in the presence of adversaries...