Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The...
Transcript of Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The...
![Page 1: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/1.jpg)
Support Vector Machine(Part 2)
![Page 2: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/2.jpg)
OUTLINE
• Multi-class classification
• Nonlinear mapping
• Kernel
![Page 3: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/3.jpg)
The Solution of Quiz-2• The maximum margin weight vector will be parallel to the shortest line connecting points of the
two classes, that is, the line between (1,1) and (2,3), giving a weight vector of (1,2)
• Working algebraically, with the standard constraint that:
Minimize ||w|| subject to
• For some a;
a+2a + b = -1
2a + 6a + b = 1
a= 2/5, b=- 11/5, so the optimal hyperplane is given by w = (2/5 , 4/5) and b = -11/5.
The large margin M is 2/ ||w|| 2/4
25+
16
25= 5
![Page 4: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/4.jpg)
Multi-class Classification
![Page 5: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/5.jpg)
Multi-class Classification
![Page 6: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/6.jpg)
Multi-class Classification
Source code example: http://scikit-learn.org/stable/modules/multiclass.html
![Page 7: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/7.jpg)
Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]
![Page 8: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/8.jpg)
![Page 9: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/9.jpg)
Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]
![Page 10: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/10.jpg)
Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]
![Page 11: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/11.jpg)
Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]
![Page 12: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/12.jpg)
Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]
![Page 13: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/13.jpg)
Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]
![Page 14: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/14.jpg)
Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]
![Page 15: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/15.jpg)
Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]
![Page 16: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/16.jpg)
![Page 17: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/17.jpg)
Source: [http://sli.ics.uci.edu/Classes/2016W-178?action=download&upname=07_svm.pdf]
![Page 18: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/18.jpg)
![Page 19: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/19.jpg)
![Page 20: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/20.jpg)
![Page 26: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/26.jpg)
Gaussian Kernel Example (1)
Source: [https://class.coursera.org/ml-005/lecture/74]
![Page 27: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/27.jpg)
Gaussian Kernel Example (2)
Source: [https://class.coursera.org/ml-005/lecture/74]
![Page 28: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/28.jpg)
Gaussian Kernel Example (3)
Source: [https://class.coursera.org/ml-005/lecture/74]
![Page 33: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/33.jpg)
Demo
• http://scikit-learn.org/stable/modules/svm.html
• http://scikit-learn.org/stable/auto_examples/plot_multilabel.html#example-plot-multilabel-py
• http://scikit-learn.org/stable/auto_examples/svm/plot_svm_nonlinear.html
• http://scikit-learn.org/stable/auto_examples/svm/plot_svm_kernels.html
• http://scikit-learn.org/stable/auto_examples/svm/plot_custom_kernel.html
![Page 34: Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The maximum margin weight vector will be parallel to the shortest line connecting points](https://reader034.fdocuments.in/reader034/viewer/2022051811/601f83c87a99bc7c89086b75/html5/thumbnails/34.jpg)
References
• http://cs229.stanford.edu/notes/cs229-notes3.pdf
• https://class.coursera.org/ml-005/lecture/74
• http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf
• http://nlp.stanford.edu/IR-book/html/htmledition/support-vector-machines-the-linearly-separable-case-1.html
• http://www.holehouse.org/mlclass/12_Support_Vector_Machines.html