Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22...
31
Model Inversion Attacks Against Collaborative Inference Zecheng He 1 , Tianwei Zhang 2 , and Ruby B. Lee 1 1 Princeton University 2 Nanyang Technological University 1
Transcript of Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22...
Model Inversion Attacks AgainstCollaborative Inference
Zecheng He1, Tianwei Zhang2, and Ruby B. Lee1
1Princeton University2Nanyang Technological University
1
Model Inversion Attacks AgainstCollaborative Inference
•Motivation and Background•White-box Attack• Black-box Attack• Access-free Attack• Defense and Conclusion
2
q Deep learning is everywhere• Computer Vision, Natural Language Processing, Robotics,
Medical, Autonomous Driving…
q Cloud-based deep learning systems become prevalent• Google Cloud AI, Amazon Sagemaker, Microsoft Azure AI…• Collaborative Training, Collaborative Inference
Motivation
3
Collaborative Training
Parameter server
Client
…Client
4
g2t
𝜽𝒕
gnt
𝜽𝒕
𝜽𝒕
gn-1t
Client
Client
Collaborative Inferenceq Split the DNN to multiple parties
• Hybrid IoT / Edge-Cloud Computation• Easy parts computed on the edge device, hard parts on the cloud• Save power and reduce latency
Cloud
Edge Device
Edge Device Edge Device
Edge Device
Attacks against Training Data Privacy
q Model inversion attacks• Recover representative input of each class• ‘Average’ of data instead of a specific input. • Not work well for deep NNs
q Membership attacks• Whether a given sample is in the training set?• Need to know candidate training samples
q Attribute/property attacks• Whether the training data has a property, e.g. color, gender• Obtain property information, not an individual sample
6
q Inference data privacy is less studied• More severe problem:
ü Unlike fixed number of training data, the number of inference data is increasing over time
ü Inference samples could be more sensitive
• More challenging problem:ü Trained model does not depend on inference dataü Inference samples vary significantly
No Attacks against Collaborative Inference
Is it possible to recover useful information of individual inference data input?
7
Threat Model
q Adversary: Untrusted or compromisedcloud provider
q Target:Recover sensitive inputs during inference
q Adversary’s capabilities1) White-box of the edge-side model2) Black-box of the edge-side model3) No query to the edge-side model
Edge-side model Cloud-side model
Attack works in all scenarios!8
Model Inversion Attacks AgainstCollaborative Inference
•Motivation and Background•White-box Attack• Black-box Attack• Access-free Attack• Defense and Conclusion
9
?0.850.020.410.19...
[
[
White-box Attack
Intermediate values𝑓$% 𝑥'
Insight: find an input x, such that1) 𝑓$% 𝑥 is close to 𝑓$% 𝑥'2) Domain knowledge: 𝑥 is a natural input
10
Known parameters 𝑓$%
White-box Attack
Regularized Maximum Likelihood Estimation (rMLE)
1) 𝐟𝛉𝟏 𝐱 is close to 𝐟𝛉𝟏 𝐱𝟎 => Euclidean Distance (ED)
2) 𝐱 is a natural sample => Small Total Variation (TV)
ED(𝑥, 𝑥') = ||𝑓$% 𝑥 − 𝑓$% 𝑥' ||88
TV(𝑥) = ;(|𝑥<=%,> − 𝑥<,>|8 + |𝑥<,>=% − 𝑥<,>|8)@/8<,>
𝒙∗ = 𝒂𝒓𝒈𝒎𝒊𝒏𝒙 𝑬𝑫 𝒙,𝒙𝟎 + 𝝀𝑻𝑽(𝒙)
S𝐨𝐥𝐯𝐞 𝒙 with Gradient Descent11
White-box Evaluation
Dataset MNIST CIFAR10Target Model 2 conv + 3 fc (LeNet5) 6 conv + 2 fcSplit Point • 1st conv layer (conv1)
• 2nd conv layer after activation (ReLU2)
• 1st conv layer (conv11)• 4th conv layer after
activation (ReLU22)• 6th conv layer after
activation (ReLU32)
q Target models and datasets
12conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2
White-box Evaluation
13
Model Inversion Attacks AgainstCollaborative Inference
•Motivation and Background•White-box Attack• Black-box Attack• Access-free Attack• Defense and Conclusion
14
?0.850.020.410.19...
[
[
𝑓$% 𝑥'
Black-box Attack
Insight: Train 𝑓$%S%, an ‘inverse network’ of 𝑓$%, 𝑚𝑎𝑝𝑝𝑖𝑛𝑔
𝑓$% 𝑥 back to x
The adversary can query 𝑓$%
15
Known parameters 𝑓$%
Black-box Attack
q Inverse Network 𝒇𝜽𝟏S𝟏1) Generate training samples2) Train the inverse network
𝑓$%S% = 𝑎𝑟𝑔𝑚𝑖𝑛\
1𝑚; ||𝑔 𝑓$% 𝑥< − 𝑥<||8]
<^%
𝑓$% 𝑥% , … , 𝑓$% 𝑥] ~(𝑥%,… , 𝑥])
𝑓$% 𝑓$%S%
16
Black-box Attack Evaluation
17
Black-box Attack Evaluation
q Is knowing training data / distribution important?
1) Train Inverse Network with same training data
2) Train Inverse Network with data from the same distribution
3) Train Inverse Network with random noise
𝑓$%S% = 𝑎𝑟𝑔𝑚𝑖𝑛\
1𝑚; ||𝑔 𝑓$% 𝑥< − 𝑥<||8]
<^%
18
Black-box Attack Evaluation
q Is knowing training data / distribution important?
Inverse Network trained with the same training data
Original inputs
Inverse Network trained with the same training distribution
Inverse Network trained with random noise 19
Black-box Attack Evaluation
q The data distribution is important• Random noise to train the inverse network can only
partially recover the inputs
20
q The exact training samples are not necessary
q Quantitative results in the paper• Peak Signal Noise Ratio (PSNR)• Structural Similarity (SSIM)
Model Inversion Attacks AgainstCollaborative Inference
•Motivation and Background•White-box Attack• Black-box Attack• Access-free Attack• Defense and Conclusion
21
?0.850.020.410.19...
[
[
𝑓$% 𝑥'
Access-free Attack
Known parameters 𝑓$%The adversary can query 𝑓$%
Insight: Reconstruct a shadow model 𝑓′$% , then perform a white-box attack on 𝑓′$%
22
Shadow Model Construction
The cloud-side model 𝑓$8, frozen
The shadow (edge-side) model 𝑓′$% , trainable
Cat
𝑓′$% = 𝑎𝑟𝑔𝑚𝑖𝑛\ ;CrossEntropy(𝑓$8 𝑔 𝑥< , 𝑦<) ]
<^%
𝑓′$% and 𝑓$8 (frozen) jointly perform well on the original classification task
23
White-box Attack on the Shadow Model
ED′(𝑥, 𝑥') = ||𝑓′$% 𝑥 − 𝑓$% 𝑥' ||88
𝒙∗ = 𝒂𝒓𝒈𝒎𝒊𝒏𝒙 𝑬𝑫′ 𝒙, 𝒙𝟎 + 𝝀𝑻𝑽(𝒙)
0.850.020.410.19...
[
[
𝑓′$% 𝑥'𝑓′$%
White-box rMLE attack against 𝑓′$%
24
Access-free Attack Evaluation
25
Access-free Attack Evaluationq Is knowing exact training data important?
26
Original input
Same training data
Same distribution
Original input
Same training data
Same distribution
Where should the model be split between edge and cloud?
PSRN and SSIM of query-free attackagainst v.s. split points
Query-free attack against v.s. split points
27
Model Inversion Attacks AgainstCollaborative Inference
•Motivation and Background•White-box Attack• Black-box Attack• Access-free Attack• Defense and Conclusion
28
Defenses
q Running FC layers before sending to the cloud, or make the edge-side network deeper• High overhead on edge-side
q Homomorphic Encryption• Too complex, not practical
q Differentially private model inference• Adding noise to the inference input: 𝑣 = 𝑓$%(𝑥 + 𝜀)• Adding noise to the intermediate value: 𝑣 = 𝑓$% 𝑥 + 𝜀• Tradeoff between accuracy (usability) and privacy
29
Conclusion
q Three new attacks in collaborative inference scenarios• White-box attack• Black-box attack• Query-free attack• Adversary can successfully recovers sensitive inputs• Exact training data is not important to the success of the attack, but
training data distribution is important
q Data privacy should be considered in collaborative inference system design
30
Code available
https://github.com/zechenghe/Inverse_Collaborative_Inference
Questions?
31
