Henrik Skov Midtiby [email protected]...
Transcript of Henrik Skov Midtiby [email protected]...
![Page 2: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/2.jpg)
The classification problem
![Page 3: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/3.jpg)
Supervised classification
Instance: Feature vector ~x ∈ Rn and class relation shipy ∈ −1, 1.
Training: Given a set of feature vectors and corresponding classlabel.
Classification: Predict which class a new feature vector belongs to.
![Page 4: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/4.jpg)
Nearest neighbor
![Page 5: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/5.jpg)
Nearest neighbor
Approach
I Find nearest known sample
I Return class of that sample
Properties
I instance based learning
I distance metric
![Page 6: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/6.jpg)
Nearest neighbor
Advantages
I easy training
Disadvantages
I slow classification
I large memory requirements
![Page 7: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/7.jpg)
k Nearest neighbor
![Page 8: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/8.jpg)
k Nearest Neighbor
I Generalization of Nearest Neighbor
I Find the k nearest neighbours
I Determine class by majority vote
http://en.wikipedia.org/wiki/K-nearest_neighbor_algorithm
![Page 9: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/9.jpg)
Bayes rule
![Page 10: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/10.jpg)
Bayes rule
The probability of A and B can be expressed in two ways
P(A,B) = P(A) · P(B|A) = P(B) · P(A|B)
which is rearranged to Bayes rule
P(B|A) =P(B) · P(A|B)
P(A)
often it is used as
P(B|A) =P(B,A)
P(B,A) + P(¬B,A)
![Page 11: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/11.jpg)
Naive Bayes
Approximates conditional probability densitiesUses Bayes rule to infer class membership probabilities based onobservations and the conditional probability densities.
p(C = 1|F1,F2,F3) =p(C = 1,F1,F2,F3)
p(C = 1,F1,F2,F3) + p(C = 2,F1,F2,F3)
![Page 12: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/12.jpg)
Naive Bayes
Advantages
I fast classification
I low memory requirements
I probability of belonging to certain classes
I requires relative few training samples
Disadvantages
I assuming independent features
http://en.wikipedia.org/wiki/Naive_Bayes_classifier
![Page 13: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/13.jpg)
Bayes theorem
Joint probability
p(C ,F1,F2,F3) = p(F1,F2,F3|C ) p(C )
= p(F1,F2|F3,C ) P(F3|C ) p(C )
= p(F1|F2,F3,C ) P(F2, |F3,C ) P(F3|C ) p(C )
Naive assumption
p(C ,F1,F2,F3) ' p(F1|C ) P(F2, |C ) P(F3|C ) p(C )
'[
3∏
k=1
p(Fk |C )
]· p(C )
http://en.wikipedia.org/wiki/Bayes%27_theorem
![Page 14: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/14.jpg)
Conditional probabilities
![Page 15: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/15.jpg)
Support vector machines
Advantages
I state of art classification results
Disadvantages
I binary classification
![Page 16: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/16.jpg)
Perceptron
f (~x) = ~x · ~w
![Page 17: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/17.jpg)
Perceptron
Invented in 1957 by Frank RosenblattWorks on linearly separable problemsDecision rule
f (~x) =
{1 if ~w · ~x + b > 0
−1 otherwise
Demonstration athttps://www.khanacademy.org/cs/perceptron-classifier/
993241235
http://en.wikipedia.org/wiki/Perceptron
![Page 18: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/18.jpg)
Perceptron classifier
![Page 19: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/19.jpg)
Perceptron classifier
![Page 20: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/20.jpg)
Perceptron Algorithm
Alexander J. Smola: An Introduction to Machine Learning with Kernels, Page 10
argument: X := {x1, . . . , xm} ⊂ X (data)Y := {y1, . . . , ym} ⊂ {±1} (labels)
function (w, b) = Perceptron(X, Y, η)initialize w, b = 0repeat
Pick (xi, yi) from dataif yi(w · xi + b) ≤ 0 then
w′ = w + yixib′ = b + yi
until yi(w · xi + b) > 0 for all iend
![Page 21: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/21.jpg)
Perceptron example
Demonstration athttps://www.khanacademy.org/cs/perceptron-classifier/
993241235
![Page 22: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/22.jpg)
Perceptron training as a optimization problem
Minimize:f (~w , b) = 0
Subject to:1− yi (~w
T~xi + b) ≤ 0
Find a feasible solution.
![Page 23: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/23.jpg)
Linearly non separable problems
![Page 24: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/24.jpg)
Linearly non separable problems – Decision surface
http://www.youtube.com/watch?v=3liCbRZPrZA
http://www.youtube.com/watch?v=9NrALgHFwTo
![Page 25: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/25.jpg)
Derived features
If the classification problem is not linearly separable. . .
Map the input feature space to an extended feature space.
An example
φ([x1, x2, x3]) = [x21 , x22 , x
23 , x1x2, x1x3, x2x3]
φ([z1, z2, z3]) = [z21 , z22 , z
23 , z1z2, z1z3, z2z3]
![Page 26: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/26.jpg)
Derived features
![Page 27: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/27.jpg)
Derived features
![Page 28: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/28.jpg)
Perceptron Algorithm
Alexander J. Smola: An Introduction to Machine Learning with Kernels, Page 10
argument: X := {x1, . . . , xm} ⊂ X (data)Y := {y1, . . . , ym} ⊂ {±1} (labels)
function (w, b) = Perceptron(X, Y, η)initialize w, b = 0repeat
Pick (xi, yi) from dataif yi(w · xi + b) ≤ 0 then
w′ = w + yixib′ = b + yi
until yi(w · xi + b) > 0 for all iend
![Page 29: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/29.jpg)
Perceptron on Features
Alexander J. Smola: An Introduction to Machine Learning with Kernels, Page 36
argument: X := {x1, . . . , xm} ⊂ X (data)Y := {y1, . . . , ym} ⊂ {±1} (labels)
function (w, b) = Perceptron(X, Y, η)initialize w, b = 0repeat
Pick (xi, yi) from dataif yi(w · Φ(xi) + b) ≤ 0 then
w′ = w + yiΦ(xi)b′ = b + yi
until yi(w · Φ(xi) + b) > 0 for all iend
Important detailw =
∑
j
yjΦ(xj) and hence f (x) =∑
j yj(Φ(xj) · Φ(x)) + b
![Page 30: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/30.jpg)
Kernel Perceptron
Alexander J. Smola: An Introduction to Machine Learning with Kernels, Page 41
argument: X := {x1, . . . , xm} ⊂ X (data)Y := {y1, . . . , ym} ⊂ {±1} (labels)
function f = Perceptron(X,Y, η)initialize f = 0repeat
Pick (xi, yi) from dataif yif (xi) ≤ 0 then
f (·)← f (·) + yik(xi, ·) + yiuntil yif (xi) > 0 for all i
end
Important detailw =
∑
j
yjΦ(xj) and hence f (x) =∑
j yjk(xj, x) + b.
![Page 31: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/31.jpg)
The kernel trick
Use the derived features from earlier
φ([x1, x2, x3]) = [x21 , x22 , x
23 ,√
2x1x2,√
2x1x3,√
2x2x3]
φ([z1, z2, z3]) = [z21 , z22 , z
23 ,√
2z1z2,√
2z1z3,√
2z2z3]
Now the dot product of φ(x) and φ(z) can be computed as follows:
φ([x1, x2, x3]) · φ([z1, z2, z3])
= x21 z21 + x22 z
22 + x23 z
23 + 2x1x2z1z2 + 2x1x3z1z3 + 2x2x3z2z3
([x1, x2, x3] · [z1, z2, z3])2
= x21 z21 + x22 z
22 + x23 z
23 + 2x1x2z1z2 + 2x1x3z1z3 + 2x2x3z2z3
Two different methods of computing the same value, lets use thefastest one.
![Page 32: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/32.jpg)
Kernel perceptron example
Demonstration athttps://www.khanacademy.org/cs/
perceptron-classifier-kernel-version/1116691580
![Page 33: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/33.jpg)
SVM
![Page 34: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/34.jpg)
SVM
The ”best possible” perceptron (largest margin).Soft classifier (for handling errors).
http://en.wikipedia.org/wiki/Support_vector_machine
![Page 35: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/35.jpg)
Perceptron training as a optimization problem
Minimize:
f (~w , b) = 0 (1)
Subject to:
yi (~wT~xi + b) ≥ 1 (2)
Find a feasible solution.
![Page 36: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/36.jpg)
SVM optimization problem
Minimize:
f (~w , b) =1
2||~w ||2 (3)
Subject to:
yi (~wT~xi + b) ≥ 1 (4)
Find maximum margin separating hyperplane.
![Page 37: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/37.jpg)
SVM optimization problem - non separable case
Minimize:
f (~w , b) =1
2||~w ||2 + C
∑
i
εi (5)
Subject to:
yi (~wT~xi + b)− εi ≥ 1 (6)
Find maximum margin separating hyperplane allowing a fewmisclassifications.
![Page 38: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/38.jpg)
SVM optimization problem - kernelized
Minimize:
f (~w , b) =1
2||~w ||2 + C
∑
i
εi (7)
Subject to:
yi
∑
j
wjk(~xj , ~xi ) + b
− εi ≥ 1 (8)
Find maximum margin separating hyperplane allowing a fewmisclassifications.
![Page 39: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/39.jpg)
SV Classification Machine
Alexander J. Smola: An Introduction to Machine Learning with Kernels, Page 48
![Page 40: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/40.jpg)
Boosting
Combination of several simple classifiers, which are allowed tomake certain error types
Advantages
I state of art classification results
Disadvantages
I binary classification
Used for face recognition inViola, P., & Jones, M. J. (2004). Robust Real-Time FaceDetection. International Journal of Computer Vision, 57(2),137154. doi:10.1023/B:VISI.0000013087.49260.fb
![Page 41: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/41.jpg)
Boosting
![Page 42: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/42.jpg)
Boosting
![Page 43: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/43.jpg)
Boosting
![Page 44: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/44.jpg)
Boosting
![Page 45: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/45.jpg)
Boosting
![Page 46: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/46.jpg)
Boosting
![Page 47: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/47.jpg)
Decision trees
Go golfing based on the parameters
I weather forecast
I wind
I humidity
![Page 48: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/48.jpg)
Decision trees
http://www.saedsayad.com/decision_tree.htm
![Page 49: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/49.jpg)
Random forests
Combination of a large quantity of decision treesAdvantages
I state of art classification results
Disadvantages
I difficult to explain . . .
I ?
![Page 50: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/50.jpg)
Cross validation
Method for using the entire dataset for training and testing.
![Page 51: Henrik Skov Midtiby hemi@mmmi.sdu.dk 2014-11-05henrikmidtiby.github.io/downloads/2014-11-05classifierspresentation.pdf · hemi@mmmi.sdu.dk 2014-11-05. The classi cation problem. Supervised](https://reader034.fdocuments.in/reader034/viewer/2022052103/603e4402a56b564b250d7644/html5/thumbnails/51.jpg)
Ressources
An Introduction to Machine Learning with Kernels, Alexander J.Smola, 2004