Secure Kernel Machines against Evasion Attacks

PatternRecognitionandApplications Lab

UniversityofCagliari,Italy

DepartmentofElectricalandElectronic

Engineering

Secure Kernel Machines againstEvasion Attacks

PaoloRussu,AmbraDemontis,BattistaBiggio,GiorgioFumera,FabioRoli

[email protected]

Dept.OfElectrical andElectronicEngineeringUniversity ofCagliari,Italy

AISec2016– Vienna,Austria– Oct.,28th2016

http://pralab.diee.unica.it

Recent Applications of Machine Learning

• Consumer technologies for personal applications

2


iPhone 5s with Fingerprint Recognition…

3


… Cracked a Few Days After Its Release

4

EU FP7 Project: TABULA RASA


New Challenges for Machine Learning

• The use of machine learning opens up new big possibilitiesbut also new security risks

• Proliferation and sophisticationof attacks and cyberthreats– Skilled / economically-motivated

attackers (e.g., ransomware)

• Several security systems use machine learning to detect attacks– but … is machine learning secure enough?

5


Is Machine Learning Secure Enough?

• Problem: how to evade a linear (trained) classifier?

Start 2007 with a bang!Make WBFS YOUR PORTFOLIO’sfirst winnerof the year...

startbangportfoliowinneryear...universitycampus

11111...00

+6 > 0, SPAM(correctlyclassified)

f (x) = sign(wT x)

x


+2+1+1+1+1...-3-4

w

x’

St4rt 2007 with a b4ng!Make WBFS YOUR PORTFOLIO’sfirst winnerof the year... campus


00111...01

+3 -4 < 0, HAM(misclassifiedemail)

f (x) = sign(wT x)

6But…what if theclassifier is non-linear?


Gradient-based Evasion

• Goal: maximum-confidence evasion

• Attack strategy:

• Non-linear, constrained optimization– Gradient descent: approximate

solution for smooth functions

• Gradients of g(x) can be analyticallycomputed in many cases– SVMs, Neural networks

−2−1.5

−1−0.5

00.5

11.5

x

f (x) = sign g(x)( ) =+1, malicious−1, legitimate

"#$

%$

minx 'g(x ')

s.t. d(x, x ') ≤ dmax

x '

7

d(x, !x ) ≤ dmaxFeasible domain

[Biggio et al., ECML 2013]


Computing Descent Directions

Support vector machines

Neural networks

x1

xd

d1

dk

dm

xf g(x)

w1

wk

wm

v11

vmd

vk1

……

……

g(x) = αi yik(x,i∑ xi )+ b, ∇g(x) = αi yi∇k(x, xi )

i∑

g(x) = 1+ exp − wkδk (x)k=1

m

∑#

$%

&

'(

)

*+

,

-.

−1

∂g(x)∂x f

= g(x) 1− g(x)( ) wkδk (x) 1−δk (x)( )vkfk=1

m

∑

RBF kernel gradient: ∇k(x,xi ) = −2γ exp −γ || x − xi ||2{ }(x − xi )

8But…what if theclassifier is non-differentiable?

[Biggio et al., ECML 2013]


Evasion of Non-differentiable Classifiers

9

PD(X,Y)data

Surrogate training data

f(x)

Send queriesGet labels

Learnsurrogate classifier

f’(x)


Dense and Sparse Evasion Attacks

• L2-norm noise corresponds to dense evasion attacks– All features are modified by

a small amount

• L1-norm noise corresponds to sparse evasion attacks– Few features are significantly

modified

10

min$% 𝑔 𝑥%𝑠. 𝑡. |𝑥 − 𝑥%|-- ≤ 𝑑01$

min$% 𝑔 𝑥%𝑠. 𝑡. |𝑥 − 𝑥%|2 ≤ 𝑑01$


Goal of This Work

• Secure learning against evasion attacks exploits game-theoretical models, robust optimization, multiple classifiers, adversarial training, etc.

• Practical adoption of current secure learning algorithms is hindered by several factors:– strong theoretical requirements– complexity of implementation– scalability issues (computational time and space for training)

11

Our goal: to develop secure kernel machinesthat are not computationally more demanding

than their non-secure counterparts


Security of Linear Classifiers

12


Secure Linear Classifiers

• Intuition in previous work on spam filtering[Kolcz and Teo, CEAS 2007; Biggio et al., IJMLC 2010]– the attacker aims to modify few features– features assigned to highest absolute weights are modified first– heuristic methods to design secure linear classifiers with more evenly-

distributed weights

• We know now that the aforementioned attack is sparse– l1-norm constrained

13

Then, what does more evenly-distributed weights mean from a more theoretical perspective?


Robustness and Regularization[Xu et al., JMLR 2009]

• SVM learning is equivalent to a robust optimization problem– regularization depends on the noise on training data!

14

minw,b

12wTw+C max 0,1− yi f (xi )( )

i∑

minw,b

maxui∈U

max 0,1− yi f (xi +ui )( )i∑

l2-norm regularization is optimal against l2-norm noise!

infinity-norm regularization is optimal against l1-norm noise!


Infinity-norm SVM (I-SVM)

• Infinity-norm regularizer optimal against sparse evasion attacks

15

minw,b

w∞+C max 0,1− yi f (xi )( )

i∑ , w

∞=max

i=1,...,dwi

wei

ghts

wei

ghts


Cost-sensitive Learning

• Unbalancing cost of classification errors to account for different levels of noise over the training classes[Katsumada and Takeda, AISTATS ‘15]

• Evasion attacks: higher amount of noise on malicious data

16


Experiments on MNIST Handwritten Digits

17

• 8 vs 9, 28x28 images (784 features – grey-level pixel values)

• 500 training samples, 500 test samples, 5 repetitions

• Parameter tuning (max. detection rate at 1% FP)

0 200 400 6000

0.2

0.4

0.6

0.8

1Handwritten digits (dense attack)

TP

at F

P=

1%

d max

SVMcSVMI−SVMcI−SVM

0 2000 4000 60000

0.2

0.4

0.6

0.8

1Handwritten digits (sparse attack)

TP

at F

P=

1%

d max


vs


Examples of Manipulated MNIST Digits

18

original sample

5 10 15 20 25

5

10

15

20

25

SVM g(x)= −0.216

5 10 15 20 25

5

10

15

20

25

I−SVM g(x)= 0.112

5 10 15 20 25

5

10

15

20

25

cI−SVM g(x)= 0.148

5 10 15 20 25

5

10

15

20

25

cSVM g(x)= −0.158

5 10 15 20 25

5

10

15

20

25

Sparseevasionattacks(l1-normconstrained)

original sample

5 10 15 20 25

5

10

15

20

25

cI−SVM g(x)= −0.018

5 10 15 20 25

5

10

15

20

25

I−SVM g(x)= −0.163

5 10 15 20 25

5

10

15

20

25

cSVM g(x)= 0.242

5 10 15 20 25

5

10

15

20

25

SVM g(x)= 0.213

5 10 15 20 25

5

10

15

20

25

Denseevasionattacks(l2-normconstrained)


Experiments on Spam Filtering

• 5000 samples from TREC 07 (spam/ham emails)• 200 features (words) selected to maximize information gain• Parameter tuning (max. detection rate at 1% FP)• Results averaged on 5 repetitions

19

0 5 10 150

0.2

0.4

0.6

0.8

1Spam filtering (sparse attack)

TP

at

FP

=1

%

d max



Security of Non-linear Classifiers

20


Secure Nonlinear Classifiers (Intuition)

21


Secure Nonlinear Classifiers (Intuition)

22


Secure Kernel Machines

• Key Idea: to better enclose benign data (eliminate blind spots)– Adversarial Training / Game-theoretical models

• We can achieve a similar effect by properly modifying the SVM parameters (classification costs and kernel parameters)

23

StandardSVM Cost-sensitiveLearning KernelModification


• Lux0R [Corona et al., AISec ‘14]

• Adversary’s capability– adding up to dmax API calls– removing API calls may

compromise the embeddedmalware code

classifier

benign

malicious

API reference extraction

API reference selection

learning-based model

runtime analysis

known label

JavaScript

API references

Suspiciousreferences

Experiments on PDF Malware Detection

minx 'g(x ')

s.t. d(x, x ') ≤ dmax

x ≤ x '

24

evalisNaNthis.getURL...


Experiments on PDF Malware Detection

• Lux0R Data: Benign / Malicious PDF files with Javascript– 5,000 training samples, 5,000 test samples, 5 repetitions– 100 API calls selected from training data

• Detection rate (TP) at FP=1% vs max. number of added API calls

25numberofaddedAPIcalls


Conclusions and Future Work

• Classifier security can be significantly improved by properly tuning classifier parameters– regularization terms– cost-sensitive learning, kernel parameters

Future Work• Security / complexity comparison against current adversarial

approaches– Adversarial Training / Game-theoretical models

• More theoretical insights on classifier / feature vulnerability

26


?Any questionsThanks foryour attention!

27

Secure Kernel Machines against Evasion Attacks

Education

Transcript of Secure Kernel Machines against Evasion Attacks