Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity...
Transcript of Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity...
![Page 1: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/1.jpg)
High Accuracy and High Fidelity Extraction of Neural Networks
Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot
![Page 2: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/2.jpg)
MLaaS
5DigitCo
![Page 3: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/3.jpg)
Model Extraction
5DigitCo
![Page 4: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/4.jpg)
This Talk● Taxonomy
○ Motivation
● Learning Extraction○ Algorithms○ Limitations
● Direct Recovery Extraction○ Prior Work○ Improvements
![Page 5: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/5.jpg)
Why would someone do this?
Data and engineers are expensive (theft)...
![Page 6: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/6.jpg)
Why would someone do this?
Data and engineers are expensive (theft)... ...and models are vulnerable to
attack (reconnaissance)!
![Page 7: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/7.jpg)
Taxonomy● Theft
○ Accuracy
● Reconnaissance○ Fidelity○ Functional Equivalence
● Adversaries also have specific access restrictions○ Full model output vs class label○ Rate limiting
![Page 8: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/8.jpg)
Algorithms for Extraction● Consider a linear model: f(x) = w.x● We could try learning:
○ Do machine learning on (xi, f(xi)) pairs
● But notice also:○ f([1, 0, …, 0]) = w0○ f([0, 1, …, 0]) = w1
● We can directly recover linear models!● What about neural networks?
![Page 9: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/9.jpg)
Learning-based Extraction - Active Learning (also here!)
Active Learning: progressively growing a labeled dataset
Chandrasekharan et al: https://arxiv.org/abs/1811.02054
![Page 10: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/10.jpg)
Learning-based Extraction - Semi-Supervised Learning
Semi-Supervised Learning: label a small dataset, but also use the unlabeled dataset
![Page 11: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/11.jpg)
Learning-based Extraction● Semi-supervised learning
○ Scales to deep learning + complex datasets○ Requires large unlabeled dataset
● Label efficient!
Dataset Queries Baseline Accuracy SemiSup Accuracy
SVHN 250 79.25% 95.82%
CIFAR-10 250 53.35% 87.98%
ImageNet (top 5) ~140000 83.5% 86.17%
![Page 12: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/12.jpg)
Limitations of Learning-based Extraction - Nondeterminism
![Page 13: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/13.jpg)
Improving Fidelity - Direct Recovery (Milli et al.)● Linear model direct recovery isn’t
easily extended to neural networks
● We focus on 2-layer ReLU networks, following Milli et al. [1]
[1] Milli et al: https://arxiv.org/abs/1807.05185
ReLU(x) = max(0, x)
![Page 14: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/14.jpg)
Improving Fidelity - Direct Recovery (Milli et al.)
[1] Milli et al: https://arxiv.org/abs/1807.05185
![Page 15: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/15.jpg)
Improving Fidelity - Direct Recovery (Milli et al.)
[1] Milli et al: https://arxiv.org/abs/1807.05185
![Page 16: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/16.jpg)
Improving Fidelity - Direct Recovery (Milli et al.)
[1] Milli et al: https://arxiv.org/abs/1807.05185
![Page 17: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/17.jpg)
Improving Fidelity - Direct Recovery (Milli et al.)
[1] Milli et al: https://arxiv.org/abs/1807.05185
![Page 18: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/18.jpg)
Improving Fidelity - Direct Recovery (Milli et al.)
[1] Milli et al: https://arxiv.org/abs/1807.05185
![Page 19: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/19.jpg)
Improving Fidelity - Direct Recovery (Milli et al.)
[1] Milli et al: https://arxiv.org/abs/1807.05185
![Page 20: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/20.jpg)
Our Functionally Equivalent● Make Milli et al. work in practice - improved search and precision
Parameters 25,000 50,000 100,000
Fidelity 100% 100% 99.98%
Queries ~150,000 ~300,000 ~600,000
Effectiveness of our Direct Recovery Attack
![Page 21: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/21.jpg)
Wrapping Up● See the paper for more!
○ Hardness results○ Nondeterminism○ Adversarial example transferability○ Our functionally equivalent attack○ Hybrid attacks
● Future Work○ More efficient, realistic, effective attacks!○ Defenses for accuracy, fidelity,
functionally equivalent?
● Thank you! Ask me questions!
![Page 22: Alex Kurakin, and Nicolas Papernot High Accuracy and High ...High Accuracy and High Fidelity Extraction of Neural Networks Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex](https://reader035.fdocuments.in/reader035/viewer/2022070111/604f39fceb20b933064cc153/html5/thumbnails/22.jpg)
Credits● Papers
○ Chandrasekharan et al.: https://arxiv.org/abs/1811.02054○ Milli et al.: https://arxiv.org/abs/1807.05185
● Images○ Alice: Eysa Lee https://ccs.neu.edu/~eysa/○ Neural Network Diagram: http://alexlenail.me/NN-SVG/index.html○ Affiliations: https://ai.googleblog.com/, https://en.wikipedia.org/wiki/Northeastern_University,
https://en.wikipedia.org/wiki/University_of_Toronto○ Slide 6: https://hackernoon.com/hn-images/1*be2sR_HIKjY36cWuWRcu-Q.jpeg○ Slide 6: https://imgs.xkcd.com/comics/machine_learning_2x.png○ Slide 6: https://www.labsix.org/media/2017/10/31/cat_adversarial.png