LaFalda Workshop-07.2012
-
Upload
elinor-velasquez -
Category
Documents
-
view
28 -
download
1
Transcript of LaFalda Workshop-07.2012
![Page 1: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/1.jpg)
Generaliza)ons,of,support,vector,machines,
Elinor,Velasquez,,Bioinforma)cs,Department,
University,of,California,,Santa,Cruz,
![Page 2: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/2.jpg)
Outline,
• The,problem,that,we,are,trying,to,solve,• An,example,
• Support,vector,machines,defined,
• Rela)on,to,convolu)onal,neural,nets,• SVMs,generalized,to,Gelfand,pairs,
• Some,new,geometrical,generaliza)ons,for,learning,new,features,and,for,classifica)on,
![Page 3: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/3.jpg)
The,problem,we,are,trying,to,solve,
• Given,data,from,a,cancer,pa)ent,,classify,the,subtype,of,the,cancer.,
![Page 4: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/4.jpg)
The,problem,we,are,trying,to,solve,
• What,is,the,data?,Suppose,we,have,gene,expression,intensity,data.,
• The,gene,expression,intensity,increases,when,the,gene,is,“on”,(upPregulated),and,decreases,when,the,gene,is,“off”,(downPregulated).,
Microarray,datachip,(Ohio,State,CompBio,Lab),
![Page 5: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/5.jpg)
Example,
• Suppose,there,are,only,two,types,of,lung,cancer,,named,AC,and,SCC.,
• Data,=,{(xi,,yi)}i=1,,…,,N,with,xi,a,MPdimensional,vector,of,gene,expression,intensity,values,(M,genes),
• yi,=,Dirac,delta,func)on,(1,if,AC,,0,if,SCC),for,,,,,,,i,=,1,,…,,N,pa)ent,samples.,
![Page 6: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/6.jpg)
We,want,to,classify,the,data,samples,
• How,to,do,this?,• Use,supervised,learning,method:,Support,vector,machines,
• Supervised,learning,means,to,predict/classify,
• Unsupervised,learning,is,used,to,uncover,structure,in,the,data,,
![Page 7: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/7.jpg)
A,support,vector,machine,(SVM),is,a,type,of,graphical,model,(Vapnik,,1995),
x1,
x2,
xM,
+1,
Output,=,f(wTx,+,b),,,,,,,,,,,,,,,=,ypredicted(
Loss,func)on,=,max,(0,,1,–,yf(x)),
Input,
Supervised,learning,method,
![Page 8: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/8.jpg)
Generalize,SVMs,to,convolu)onal,neural,nets,
• Why?,,• Convolu)onal,neural,nets,are,performing,beeer,than,SVMs,for,a,variety,of,data,types.,
![Page 9: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/9.jpg)
Convolu)onal,neural,net,is,a,graphical,learning,model,(LeCun,,2006),
x1,
x2,
xN,
+1,
h1,
h1,
h1,
h2,
h2,
h2,
Input,Output,
Supervised,learning,method,
![Page 10: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/10.jpg)
We,want,our,data,to,be,invariant,under,rigid,mo)ons,and,small,deforma)ons,
• Mallat,(2010),solved,this,problem,for,data,learned,in,convolu)onal,neural,nets:,He,used,a,scaeering,operator,,S,,on,f,ε,L2(RM),which,was,composed,of,a,wavelet,transform,,convolu)on,operator,and,modulus,operator.,,
• But,data,lies,on,a,more,complicated,manifold.,
![Page 11: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/11.jpg)
Construc)ng,data,invariant,to,rigid,mo)ons,and,small,deforma)ons,
• Let,G,be,a,locally,compact,group,and,K,a,compact,subgroup,(Gelfand,pair),
• G,approximates,a,complicated,manifold.,• Example:,Let,G,=,group,of,inver)ble,matrices,,K,=,orthogonal,group,
• Suppose,we,can,map,the,data,to,G,(True,for,Lie,groups),• Then,,build,a,convolu)onal,neural,net,for,the,data,on,G/K,,an,“invariant”,space.,
• Will,extend,Mallat’s,work,(work,in,progress),
![Page 12: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/12.jpg)
An,ideal,data,space,
• The,data,actually,lives,on,a,manifold,X.,• Map,the,data,to,X/G,,with,X,a,manifold,and,G,,its,group,of,invariances,,using,geometric,invariant,theory,tools.,
• Build,a,convolu)onal,neural,network,on,this,data,space,,X/G.,,
• (Work,in,progress),
![Page 13: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/13.jpg)
Can,we,improve,upon,convolu)onal,neural,nets?,
x1,
x2,
xN,
h1,
h1,
h1,
h2,
h2,
h2,
h3,
A,cylindrical,neural,net,is,called,a,Recurrent,Neural,Net,(Graves,1980s),
![Page 14: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/14.jpg)
We,can,make,a,convolu)onal,recurrent,neural,net,for,classifica)on,
x1,
x2,
xN,
h1,
h1,
h1,
h2,
h2,
h2,
h3, Aeach,a,recursive,neural,net,to,the,recurrent,neural,net:,
h1,
h2,
Cylinder,(Recurrent,neural,net),
Triangle,(Recursive,neural,net),
Output,
![Page 15: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/15.jpg)
We,can,pinch,the,cylinder,on,each,end,to,make,a,convolu)onal,neural,net,
sphere,minus,the,south,pole,
h3,
h2,
h1,
x1,
h3,
h2,
h1,
x2,
h1, h1,
h2, h2,
h3,
Output,
![Page 16: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/16.jpg)
We,can,construct,a,convolu)onal,neural,net,on,a,smooth,manifold,of,
genus,G,
h9,
h2,
h3,
h4,
h5,
h6,
h7,h8,
h1,
Output,
h1,
h2,
h3,
h4,
h5,h9,
h6,
h7,
h8,
![Page 17: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/17.jpg)
Summary,
• Convolu)onal,neural,nets,are,learning,models,which,classify,data.,
• Data,actually,lives,on,a,complicated,manifold.,
• We,can,remove,data,invariances,before,learning,to,remove,redundancy.,
• The,future,holds,many,possibili)es,for,learning,models.,
![Page 18: LaFalda Workshop-07.2012](https://reader034.fdocuments.in/reader034/viewer/2022042615/55c4713fbb61eb5b288b478d/html5/thumbnails/18.jpg)
Thank,you!,