Kernel Properties
description
Transcript of Kernel Properties
![Page 1: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/1.jpg)
Kernel Properties2012 Computer Science PhD Showcase
17 February 2012
Roberto Valerio
Dr. Ricardo Vilalta
Pattern Analysis Lab
![Page 2: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/2.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Kernel Properties
Agenda• Introduction• Objective• Current work• Experiments• Conclusions• Publications
![Page 3: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/3.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Introduction
• Machine Learning– What is it?
• Kernel methods– What are kernel methods?
![Page 4: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/4.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Support Vector Machine
• Constructs a hyper plane in a high dimensional space with the largest margin.
![Page 5: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/5.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Support Vector Machine
?
Feature 1
Feature 2
.
.
Feature n
Infinite Dimensional
Space
![Page 6: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/6.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Kernel Trick
• Avoid explicit mapping of the infinite dimensional space
• By using this mapping we avoid dealing with a high dimensional space and we can find a separating hyper plane with the kernel matrix
![Page 7: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/7.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Which kernel?
Linear Polynomial Gaussian Hyperbolic ?
![Page 8: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/8.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Objective
• Analyze the behaviors of different kernels to generate properties that allow us to determine the
optimal kernel.
![Page 9: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/9.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Current Work
• Kernel Matrices evaluations
• Behavioral evaluation of the Kernel transformation in varied data density situations
• Identifying key points in the hyper plane construction and kernel mappings
![Page 10: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/10.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Experiments
Toy Data sets
Bayes Error Non Linear Non linear and Bayes error
![Page 11: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/11.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Experiments
Linear Kernel Matrix
Bayes Error Non Linear Non linear and Bayes error
![Page 12: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/12.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Experiments
Polynomial Kernel Degree 4 Kernel
Bayes Error Non Linear Non linear and Bayes error
![Page 13: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/13.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Experiments
Linear Kernel Density Evaluation
Bayes Error Non Linear Non linear and Bayes error
![Page 14: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/14.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Experiments
Polynomial Kernel Degree 4 Density evaluation
Bayes Error Non Linear Non linear and Bayes error
![Page 15: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/15.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Experiments
Linear
Poly 2
Poly 3
Poly 4
RBF 0.5
RBF 0.25
95
95.3
94.3
90.5
95.1
95
66.67
66.67
66.67
66.67
66.67
66.67
66.67
66.67
66.67
66.67
66.67
66.67
Accuracy Results
NonLinear Overlap Non Linear Bayes Error
![Page 16: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/16.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Conclusions
• Each kernel has its own pattern
• We can take advantage of these patterns to generate more accurate classifications.
![Page 17: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/17.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Future work
• Identify the relationship between the kernel pattern and the misclassification error
• Use this relationship to select the optimal kernel or as a guideline to construct new kernels.
![Page 18: Kernel Properties](https://reader034.fdocuments.in/reader034/viewer/2022051517/5681505d550346895dbe5f2d/html5/thumbnails/18.jpg)
Kernel Properties – Roberto Valerio 2012 Computer Science PhD Showcase -17 February 2012
Publications
Classification of Sources of Ionizing Radiation in Space Missions: A Machine Learning Approach.
Vilalta, R., Kuchibhotla, S., Hoang, S., Valerio, R., Ocegueda, F., and Pinsky, L., (2012) Acta Futura, 5, pp.111-119, 2012.
Development of Pattern Recognition Software for Tracks of Ionizing Radiation in Medipix2-Based (TimePix) Pixel Detector Devices.
Vilalta R., Valerio R., Kuchibhotla S., Pinsky L. (2010) 18th International Conference on Computing in High Energy and Nuclear Physics (CHEP-10), Taipei, Taiwan. Journal of Physics: Conference Series.
The Effect of the Fragmentation Problem in Decision Tree Learning Applied to the Search for Single Top Quark Production.
Vilalta R., Valerio R., Ocegueda-Hernandez F., Watts G. (2009) 17th International Conference on Computing in High Energy and Nuclear Physics (CHEP-09), Prague, Czech Republic. Journal of Physics: Conference Series.