With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning ·...
Transcript of With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning ·...
![Page 1: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/1.jpg)
With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning
Bolun Wang*, YuanshunYao, Bimal Viswanath§
Haitao Zheng, Ben Y. Zhao
University of Chicago, * UC Santa Barbara, § Virginia Tech
![Page 2: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/2.jpg)
Deep Learning is Data Hungry
• High-quality models are trained using large labeled datasets• Vision domain: ImageNet contains over 14 million labeled images
1
Where do small companies get such large datasets?
![Page 3: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/3.jpg)
A Prevailing Solution: Transfer Learning
2
Company X Limited Training Data Highly-trained Model
+
High-quality ModelStudent A Student B Student CRecommended by Google, Microsoft, and Facebook DL frameworks
Teacher
StudentTransfer and re-use pre-trained model
![Page 4: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/4.jpg)
Deep Learning 101
3Photo credit: Google
![Page 5: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/5.jpg)
Transfer Learning: Details
4
Output
Input
! Layers! − 1 Layers
In general, first $ layers can be directly transferred
($ = ! − 1)
Insight: high-quality features can be re-used
Teacher Teacher-specific classification layer
StudentOutputStudent-specific
classification layer
Input
![Page 6: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/6.jpg)
Transfer Learning: Example
• Face recognition: recognize faces of 65 people
5
Teacher (VGG-Face)
900 images/person2,622 people
StudentTransfer 15 out of 16 layers
10 images/person65 people
Classification AccuracyWithout Transfer Learning With Transfer Learning
1% 93.47%
Company X
![Page 7: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/7.jpg)
Is Transfer Learning Safe?
• Transfer Learning lacks diversity• Users have very limited choices of Teacher models
6
Same Teacher
Company A Attacker
Help attacker exploit all Student models
Company B
![Page 8: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/8.jpg)
In This Talk
• Adversarial attack in the context of Transfer Learning
• Impact on real DL services
• Defense solutions
7
![Page 9: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/9.jpg)
Background: Adversarial Attack
• Adversarial attack• Misclassify inputs by adding carefully engineered perturbation
8
+ " #Misclassified as
Imperceptible perturbation
![Page 10: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/10.jpg)
Attack Models of Prior Adversarial Attacks
• White-box attack: assumes full access to model internals• Find the optimal perturbation offline
• Black-box attack: assumes no access to model internals• Repeated query to reverse engineer the victim• Test intermediate result and improve
9
Not practical
Easily detected
![Page 11: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/11.jpg)
Our Attack Model
• We propose a new adversarial attack targeting Transfer Learning
• Attack model
10
StudentTeacher
Black-box
• Model internals are hidden and kept secure
White-box
• Model internals are known to the attacker
Default access model today• Teachers are made public by popular DL services• Students are trained offline and kept secret
![Page 12: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/12.jpg)
Attack Methodology: Neuron Mimicry
11
+SourceImage
TargetImage
Perturbation∆
"($)
"($)
&($)
&($)
'('(
If two inputs match at layer ), then they produce the same result regardless of changes above layer ).
"('()"('()
&(" '( )&(" '( )
Same as Teacher
![Page 13: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/13.jpg)
How to Compute Perturbation?
• Compute perturbation (∆) by solving an optimization problem• Goal: mimic hidden-layer representation• Constraint: perturbation should be indistinguishable by humans
12
"#: source image"$: target image
min ()*+,-./(12 "# + ∆ , 12("$))*. +. 7/8+98:_<,=-)+9>/ "# + ∆, "# < @ABCDE$
Minimize L2 distance between internal representations
Constrain perturbation
DSSIM: an objective measure for image distortion
12 " : internal representation at layer F of image "
![Page 14: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/14.jpg)
Attack Effectiveness
• Targeted attack: randomly select 1,000 source, target image pairs
• Attack success rate: percentage of images successfully misclassified into the target
13
Source Adversarial Target
Face recognition92.6% attack success rate
Source Adversarial Target
Iris recognition95.9% attack success rate
![Page 15: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/15.jpg)
Attack in the Wild
• Q1: given Student, how to determine Teacher?• Craft “fingerprint” input for each Teacher candidate• Query Student to identify Teacher among candidates
• Q2: would attack work on Students trained by real DL services?• Follow tutorials to build Student using following services
• Attack achieves >88.0% success rate for all three services
14
Student
Teacher A
Teacher B
Which Teacher is used?
Fingerprint input
![Page 16: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/16.jpg)
In This Talk
• Adversarial attack in the context of Transfer Learning
• Impact on real DL services
• Defense solutions
15
![Page 17: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/17.jpg)
Intuition: Make Student Unpredictable
• Modify Student to make internal representation deviate from Teacher• Modification should be unpredictable by the attacker → No countermeasure• Without impacting classification accuracy
16
min $%&''()*%&+,(,./01, ,3/14)'. *. 78 9 − ;8(9) < > >.? for 9 ∈ D./EFG
Maintain classification accuracyTeacher
RobustStudent
Transfer using an updated objective function
Updated objective function
Guarantee difference between Teacher and Student
![Page 18: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/18.jpg)
Effectiveness of Defense
Model Face Recognition Iris Recognition
Before Patching Attack Success Rate
92.6% 100%
After Patching
Attack Success Rate
30.87% 12.6%
Change of Classification
Accuracy↓ 2.86% ↑ 2.73%
17
![Page 19: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/19.jpg)
One More Thing
• Findings disclosed to Google, Microsoft, and Facebook
• What’s not included in the talk• Impact of Transfer Learning approaches• Impact of attack configurations• Fingerprinting Teacher• …
18
![Page 20: With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning · 2019-12-18 · With Great Training Comes Great Vulnerability: Practical Attacks against](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ecec0e86f1ab578be0e3841/html5/thumbnails/20.jpg)
Code, models, and datasets are available at https://github.com/bolunwang/translearn
19
Thank you!