Support Vector Machines and kernels

7/26/2019 Support Vector Machines and kernels

1/27

@btrfjuct`fb Fpt`e`zht`fb _rfnime Ihkrhbk`hb Gfreuiht`fb Xuhjrht`c _rfkrhee`bk Zdm Amrbmi Zr`ca Cfvmrs Zdmfrme

Suppfrt Wmctfr Ehcd`bms

Ghiha Sdhd (>?8388?>:)

EZmcd, JH-@@CZ

Jmcmenmr 87, >?8:


2/27


Futi`bm

@btrfjuct`fb

Fpt`e`zht`fb _rfnime

Ihkrhbk`hb Gfreuiht`fb

Xuhjrht`c _rfkrhee`bk

Zdm Amrbmi Zr`ca

Cfvmrs Zdmfrme


3/27


@btrfjuct`fb tf SWE

Hrkuhniy tdm efst succmssgui ehcd`bm imhrb`bk tffi gfrcihss`cht`fb.

Fptè`zht`fb phcahkms hvhìhnim gfr sfiv`bk tdm cihss`cht`fb

prfnime fg SWE.(I`nSWE, SWEZfrcd) @btmbjmj hppi`cht`fb< N`bhry fr euit`cihss cihss`cht`fb

I`bmhriy Smphrhnim jhth- smimct`bk tdm nmst ehrk`b fut fgpfss`nim ehrk`bs

N`kkmr ehrk`b nmttmr hs mvmb `b chsm fg bf`sy jhth crfssfvmrprfnhnì`ty `s imss.


4/27


G`bj`bk w w`td ihrkm ehrk`b

Zhsa `s tf bj w hbj n gfr tdm smphrht`bk dypmrpihbmwZx+n5 ?.Q8UQ3U

Dmrm, n jmbftms n`hs, x ^j `s tdm `bput gmhturm vmctfr tdmbw5 {w8, w>, w3...wj}.

Gfr i`bmhriy smphrhnim pf`bts, tdm pihbm w`ii bft tfucd hbypf`bts, `.m. |wZx+n| 2?.

Schi`bk fg w hbj n ny shem hefubt rmsuit `b tdm shem pihbm.Sf, wm smimct w hbj n sucd tdht |wZxb+n| 5 8, xb nm`bk tdm

pf`bt cifsmst tf tdm dypmrpihbm. Dmrm, wmii usm muci`jmhb j`sthbcm hs tdm yhrjst`ca gfr

emhsurmembt.


5/27


J`sthbcm Cfeputht`fb

J`sthbcm nmtwmmb tdm pihbm wZx+n5 ? hbj tdm bmhrmstpf`bt xb `s tdm ehrk`b. K`vmb tdht, |w

Zxb+n| 5 8.

^msuit 8


6/27


Fptè`zht`fb prfnime

Ehxè`zm 8||w|| sunlmct tf e`bb58,>..B|wZxb+n| 5 8.

Hisf, |wZxb+n| 5yb(wZxb+n), s`bcm wm fbiy cfbs`jmr tdm

cfrrmctiy cihss`mj jhth.

@b tdm hitmrbht`vm rmprmsmbtht`fb, wm chb wr`tm tdm prfnime hsE`bè`zm 8> (w

Zw) sunlmct tf yb(wZxb+n)25 8 gfr

b5 8, >...B

Zd`s sthtmembt `s mqu`vhimbt tf tdm hnfvm fbm hs e`bèue

vhium gfr wìi fbiy nm hcd`mvmj wdmb yb(wZ

xb+n) 5 8nmchusm tìi tdmb w hbj n chb stìi nm prfpfrt`fbhtmiy schimjjfwb.

@ j ` F ` ` ` _ ni I ` G i ` X j ` _ ` Zd A i Z ` a C Zd


7/27


Cfbstrh`bmj Fptè`zht`fb prfnime

E`bè`zm 8> (wZw) sunlmct yb(w

Zxb+n) 825 ?

Sfiut`fb tf td`s wìi y`mij tdm smphrht`bk dypmrpihbm w`td tdm

ihrkmst ehrk`b. Cfbstrh`bmj fptè`zht`fb prfnime cfbvmrtmj tf ubcfbstrh`bmj

fptè`zht`fb ny ihkrhbk`hb.

AAZ cfbj`t`fbs bmmjmj gfr sfiut`fb fg ihkrhbk`hb ubjmr

`bmquhi`ty cfbstrh`bt.

@ t j t` F t` ` t` _ ni I ` G i t` X j t` _ ` Zd A i Z ` a C Zd


8/27

@btrfjuct`fb Fpt`e`zht`fb _rfnime Ihkrhbk`hb Gfreuiht`fb Xuhjrht`c _rfkrhee`bk Zdm Amrbmi Zr`ca Cfvmr s Zdmfrme

Ihkrhbk`hb hbj AAZ cfbj`t`fbs

AAZ hpprfhcd kmbmrhi`zms tdm emtdfj fg Ihkrhbkmeuit`pi`mrs, wd`cd hiifws fbiy mquhi`ty cfbstrh`bts.

K`vmb h prfnime hs e`bx

g(x) sunlmct tf k(x)95 ?.

Jmbm tdm ihkrhbk`hb hsI(x, ) 5g(x) + k(x) Zdmb,x h ifchi e`b`euetdmrm mx`sts h ub`qum s.t.

8. xI(x, ) 5 ?

>. 25 ?

3. k(x) 5 ?:. k(x)95 ?



9/27


Ihkrhbk`hb gfreuiht`fb fg tdm prfnime

E`bè`zm 8> (wZw) sunlmct tf yb(w

Zxb+n) 825 ? gfrb5 8, >...B

E`bè`zmI(w, n, ) 5 8

>

(wZw) Bb58

b(yb(wZxb+n) 8) w.r.t w

hbj n hbj ehxè`zm w.r.t. mhcd b 25 ?

wI5 wB

b58 bybxb 5 ?

In

5

Bb58 byb 5 ?

AAZ cfbj`t`fb 3< b(yb(wZxb+n) 8) 5 ?

Sunst`tut`bk tdmsm vhiums `b tdm fr`k`bhi mquht`fb rmsuits `b tdmjuhi rmprmsmbtht`fb fg tdm prfnime.



10/27


Ihkrhbk`hb gfreuiht`fb fg tdm prfnime

Sunst`tut`bk w5B

b58 bybxb hbjB

b58 byb 5 ? `b tdmihkrhbk`hb

I(w, n, ) 5 8> (wZw)

Bb58 b(yb(wZxb+n) 8) wm kmt,

I() 5B

b58 b 8>

Bb58

Be58 beybyex

Zb xe

Ehx`e`sm w.r.t. hbj sunlmct tf b 25 ? gfr b58,>,...B hbj

Bb58 byb 5 ?



11/27


Xuhjrht`c _rfkrhee`bk

ehx

I() 5B

b58b

8

>

B

b58B

e58beybyex

Zb xe

Hitmrbht`vmiy,

e`b

I() 58

>

Bb58

Be58

beybyexZb xe

Bb58

b

sunlmct tf b 25 ? gfr b58,>,...B hbjB

b58 byb 5 ?



12/27


Xuhjrht`c prfkrhee`bk gfreuiht`fb

e`b

8>

Z

y8y8xZ8 x8 y8y>x

Z8 x> ... y8yBx

Z8 xB

y>y8xZ> x8 y>y>x

Z> x> ... y>yBx

Z> xB

.... .... ..... ....y>y8xZ> x8 y>y>x

Z> x> ... y>yBx

Z> xB

+ (8)Z

sunlmct tf h i`bmhr cfbstrh`bt yZ5 ? hbj rhbkm fg?95 95 hbjZdm s`zm fg tdm ehtr`x jmpmbjs fb tdm s`zm fg tdm trh`b`bk jhthsmt.



13/27

p k k X k k

X_ dhbjs nhca

5 (8, >,...,B) w5

Bb58 bybxb

`s h sphrsm vmctfr. S`cm wmvm tdm AAZ cfbj`t`fbb(yb(w

Zxb+n) 8) 5 ?.

M`tdmr b 5 ? gfr tdm `btmr`fr pf`bts fr yb(wZ

xb+n) 5 8 gfrtdm supppfrt vmctfrs- tdm fbiy fbms wdmrm b 2?. Zdm xb gfrwd`cd b 2? hrm chiimj tdm suppfrt vmctfrs hs tdmy fbiycfbtr`nutm tf tdm sfiut`fb. Sf bfw,

w5 xbS.W.

bybxb

Sfivm gfr hby n us`bk yb(wZxb+n) 5 8. Zd`s w`ii k`vm shem

n gfr hby S.W. Zd`s `s hisf vmr`cht`fb tdht tdm thsa `scfrrmctiy hccfepi`sdmj.



14/27

I`bmhriy bfb-smphrhnim jhth

Cfvmrs tdmfrme< Zdm prfnhnì`ty tdht cihssms hrm i`bmhriysmphrhnim `bcrmhsms wdmb tdm gmhturms hrm bfbi`bmhriy ehppmjtf h d`kdmr jèmbs`fbhi gmhturm sphcm. Q>U

_rffgi

`5?

B 8

`

Zdm tfthi buenmr fg krfup`bks `s >

B

. Zdus, tdm prfnhnì`tytdht tdm shepims hrm i`bmhriy smphrhnim `s tdm rht`f

_iB5 F(B, i)

>B



15/27

Amrbmi Gubct`fb

K`vmb > pf`bts x hbj x, wm bmmj z hbj z. [dmrm, z5(x)

Imt zZz5A(x, x)- tdm amrbmi gubct`fb

Zdm tr`ca `s cfeput`bk A(x, x) w`tdfut trhbsgfre`bk x hbj x.

Zdm gubct`fb A chb nm hrn`trhr`iy cdfsmb hs ifbk hs tdmmx`stmbcm fg(.) `s kuhrbhtmmj.

H amrbmi A


16/27

Amrbmi Gubct`fb

Emrcmrs cfbj`t`fb< Zdmrm mx`sts h ehpp`bk (.) `g hbj fbiy `g,gfr hby k(x) sucd tdht

k(x)>jx

`s b`tm tdmbA(x, y)k(x)k(y)jxjy ?.

Hby amrbmi wd`cd chb nm mxprmssmj hsA(x, y) 5

p5?cp(x.y)

p, wdmrm tdm cphrm pfs`t`vm rmhi

cfmffic`mbts hbj tdm smr`ms `s cfbvmrkmbt, sht`sms tdm cfbj`t`fb ^NG amrbmi< A(x, x) 5mxp(||x x||>)

@bb`tm j`embs`fbhi z< w`td 5 8

A(x, x) 5mxp(x>)mxp(x>)a5?

>a(x)a(x)a

a!



17/27

I`bmhr Smphrhnì`ty hbj Cfvmrs Zdmfrme

SWE hbj smvmrhi ftdmr phttmrb rmcfkb`t`fb tmcdb`qums usm tdmamrbmi tr`ca gfr prflmct`fb `b d`kdmr jèmbs`fbhi vmctfr sphcm

Zdmfrme ny Z.E Cfvmr sdfw`bk r`sm `b i`bmhr smphrht`fb

prfnhnì`ty `b tdm d`kdmr jèmbs`fbhi sphcm Zdm prfnhnì`ty tdht cihssms hrm i`bmhriy smphrhnim `bcrmhsms

wdmb tdm gmhturms hrm bfbi`bmhriy ehppmj tf h d`kdmrjèmbs`fbhi gmhturm sphcmQ3U.

Bmmj- jhth bft nm`bk i`bmhriy smphrhnim `b ifwmr jèmbs`fbhisphcm



18/27

Zmre`bfifky

J`cdftfey< H j`cdftfey `s hby spi`tt`bk fg h wdfim `btfmxhctiy twf bfb-fvmrihpp`bk phrts.

_httmrb< H vmctfr `b B-j`embs`fbhi gmhturm sphcm

Dfefkmbmfus i`bmhr tdrmsdfij`bk gubct`fb

g


19/27

I`bmhr Zdrmsdfij`bk Jmv`cm

H j-`bput jmv`cm tdht thams tdm s`kb fg tdm sue fg tdmprfjucts fg phttmrb `bputs w`td cfrrmspfbj`bk wm`kdts.

G`kurm 8< I`bmhr Zdrmsdfij`bk Jmv`cm 8

Kmfemtr`chiiy, sucd h jmv`cm j`v`jms tdm sphcm `btf twf rmk`fbsny h j-8 j`embs`fbhi dypmrpihbm tdrfukd fr`k`b fg tdm sphcm.

8

Z E Cfvmr, Kmfemtr`chi hbj Stht`st`chi _rfpmrt`ms fg I`bmhr ZdrmsdfijJmv`cms, _dJ tdms`s, Sthbgfrj Mimctrfb`cs Ihnfrhtfr`ms, Ehy 807:.



20/27

Dfefkmbmfus i`bmhr smphrhn`i`ty

H j`cdftfey {T+, T} fg T `s I`bmhriy Smphrhnim `g hbjfbiy `g tdmrm mx`sts h wm`kdt vmctfr w `b ^B hbj h schihr tsucd tdht

w x2t `gx T+

w x9t `gx T

Zdm j`cdftfey `s sh`j tf nm dfefkmbmfusiy i`bmhriy smphrhnim`g `t `s i`bmhriy smphrhnim w`td t5?

@bdfefkmbmfus `s lust h spmc`hi chsm fg dfefkmbmfus w`tdfbm hjjmj vhr`hnim

H vmctfr w sht`sgy`bk tdm mquht`fbs gfr dfefkmbmfus chsm `schiimj tdm sfiut`fb vmctfr hbj tdm pihbm {x


21/27

Gubct`fb Cfubt`bk Zdmfrme

Gubct`fb Cfubt`bk Zdmfrme Q>U< Zdmrm hrm C(_,B)dfefkmbmfusiy i`bmhriy smphrhnim j`cdftfe`ms fg _ pf`bts `b

kmbmrhi pfs`t`fb `b Muci`jmhb B-sphcm wdmrm

C(_, B) 5 >B8a5?

_ 8

a



22/27

_rffg fg Cfvmrs Zdmfrme

Sthrt w`td _ pf`bts `b kmbmrhi pfs`t`fb. Hssuem tdht tdmrm hrmC(_,B) j`cdftfe`ms pfss`nim fb tdme, sf dfw ehbyj`cdftfe`ms hrm pfss`nim `g hbftdmr pf`bt (`b kmbmrhi pfs`t`fb)`s hjjmj C(_+8,B).

Imt (n8, ..., n_) nm h j`cdftfey rmhi`zhnim ny h dypmrpihbmfvmr tdm smt fg _ `bputs, n {8, +8} `5 8.._, hbj tdmrm`s h smt fg wm`kdts w sf tdht gfr mhcd fg tdmes`kb(wZx8), ..., s`kb(w

Zx_) 5 (n8, ..., n_).

Gfr mvmry i`bmhriy rmhi`zmj j`cdftfey fvmr _ pf`bts tdmrm `s htimhst fbm i`bmhriy rmhi`zmj j`cdftfey fvmr _ + 8 pf`bts



23/27

_rffg fg Cfvmrs tdmfrme

Zdmrm hrm sfem hjj`t`fbhi j`cdftfe`ms pfss`nim wd`cd cihss`gytdm bmwiy hjjmj pf`bt _+8 `btf j`mrmbt cihssms

C(_+ 8, B) 5C(_, B) +J

J `s tdm buenmr fg tdfsm j`cdftfe`ms fvmr _ pf`bts tdht hrmrmhi`zmj ny h dypmrpihbm tdht phssms tdrfukd h cmrth`b xmjpf`bt x_+8

Ny gfrc`bk tdm dypmrpihbm tf phss tdrfukd h cmrth`b xmjpf`bt, wm hrm `b ghct efv`bk tdm prfnime tf fbm `b B 8j`embs`fbs, `bstmhj fg B



24/27

^mcurs`vm Gfreuih gfr pfss`nim j`cdftfe`ms

Sf, J 5 C(_,B 8), hbj tdm rmcurs`fb gfreuih `s

C(_+ 8, B) 5C(_, B) +C(_, B 8) Gurtdmr prffg us`bk ehtdmeht`chi `bjuct`fb gfr

C(_, B) 5 >B8

a5?

_ 8

a C(_+ 8, B) 5 >

B8a5?

_ 8

a

+ >

B>a5?

_ 8

a

5 >B8a5?

_ 8

a

+ >

B8a5?

_ 8a 8

5 >B8

a5?

_

a



25/27

Smphrhn`i`ty ny hrn`trhry surghcms

H ghe`iy fg surghcms mhcd fg wd`cd j`v`jm tdm k`vmb sphcm `btftwf rmk`fbs hbj h cfimct`fb fg B pf`bts, mhcd hss`kbmj m`tdmrtf T+ frT

Fb mhcd phttmrb x T, h smt fg rmhi vhiumj emhsurmembt

gubct`fbs 8, >,...,Bcfepr`sms tdm vmctfr fg emhsurmembts(x) 5 (8(x), >(x)..., B(x)), x T

H j`cdftfey `s smphrhnim `g tdmrm mx`sts h vmctfr w sucd tdht

w (x)2? `gx T

+

w (x)9? `gx T



26/27

Nmb`t< Bfbi`bmhr jmc`s`fb nfubjhry

G`kurm >< Mxhepims fg smphrhnim j`cdftfe`ms fg vm pf`bts `b twfj`embs`fbs. (h) I`bmhriy smphrhnim j`cdftfey. (n) Spdmr`chiiy smphrhnimj`cdftfey. (c) Xuhjr`chiiy smphrhnim j`cdftfey. 8

8Z.E. Cfvmr, Kmfemtr`chi hbj stht`st`chi prfpmrt`ms fg systmes fg i`bmhr

`bmquhi`t`ms w`td hppi`cht`fbs `b phttmrb rmcfkb`t`fb. Mimctrfb`c Cfeputmrs,@MMM Zrhbshct`fbs fb, MC-8:(3)7-33:, Lubm 807=



27/27

Cdr`stfpdmr L. C. Nurkms.H tutfr`hi fb suppfrt vmctfr ehcd`bms gfr phttmrb rmcfkb`t`fb.Jhth E`b. Abfwi. J`scfv., >(>)887;, Lubm 8004.

Z.E. Cfvmr.Kmfemtr`chi hbj stht`st`chi prfpmrt`ms fg systmes fg i`bmhr `bmquhi`t`ms w`tdhppi`cht`fbs `b phttmrb rmcfkb`t`fb.Mimctrfb`c Cfeputmrs, @MMM Zrhbshct`fbs fb, MC-8:(3)733:, Lubm 807=.

Wihj`e`r B. Whpb`a.

Zdm Bhturm fg Stht`st`chi Imhrb`bk Zdmfry.Spr`bkmr-Wmrihk Bmw Pfra, @bc., Bmw Pfra, BP, ]SH, 800=.

Support Vector Machines and kernels

Documents

Transcript of Support Vector Machines and kernels