Post on 14-Jan-2016
description
22.10.2005 1Prof. Pushpak Bhattacharyya, IIT Bombay
CS 621 Artificial Intelligence
Lecture 29 – 22/10/05
Prof. Pushpak Bhattacharyya
SOM, Theoretical Aspects Of Machine Learning, Probably
Approximately Correct Learning.
22.10.2005 2Prof. Pushpak Bhattacharyya, IIT Bombay
…
… … .
P neurons o/p layer
n neurons
Clusters:
AA :
A
Wp
B :
C :
: :
22.10.2005 3Prof. Pushpak Bhattacharyya, IIT Bombay
Wp(n+1) = Wp
(n) + η (I(n) – Wp(n))
Competitive learning: Weights to the winner neuron only are changed.
22.10.2005 4Prof. Pushpak Bhattacharyya, IIT Bombay
Generating function application to recurrence relation solving.
Example: Max. No of regions on 2-Dim plane produced by n lines.
22.10.2005 5Prof. Pushpak Bhattacharyya, IIT Bombay
L2L1L3
L4
0 x
y R1 = 2
R2 = R1 + 2
= 2+ 2
= 4
R3 = R2 + 3
= 4 + 3 = 7
R4 = 7 + 4 = 11 Rn = Rn-1 + n R1 = 2
22.10.2005 6Prof. Pushpak Bhattacharyya, IIT Bombay
Rn = Rn-1 + n
Rn-1 = Rn-2 + (n-1)
Rn-2 = Rn-3 + (n-2)
.
.
.
R2 = R1 + 2
22.10.2005 7Prof. Pushpak Bhattacharyya, IIT Bombay
Rn = n + (n-1) + (n-2)+ …..+R1 + 2
= n+(n-1)+(n-2)+…..+2+2
=n(n+1)/2 + 1
22.10.2005 8Prof. Pushpak Bhattacharyya, IIT Bombay
Generating Function Method
Key idea: Rns are generated by infinite
series function ,i.e,
F(x) = R1 + R2x + R3x2 + ………………+ Rnxn-1+ ….. ∞
Rn = Coeff of xn-1
22.10.2005 9Prof. Pushpak Bhattacharyya, IIT Bombay
Rn = Rn-1 + n
Rn - Rn-1 = n
R2 - R1 = 2
R3 – R2 = 3
.
.
(Continued)
22.10.2005 10Prof. Pushpak Bhattacharyya, IIT Bombay
F(x) = R1 + R2x + R3x2 + ………………+ Rnxn-1+
….. ∞ (1)
xF(x) = R1x + R2x2 + R3x3 + ……………+ Rnxn+
….. ∞ (2)
1 - 2 gives
(1-x) F(x) = R1 + (R2 – R1)x + (R3 – R2)x2 +
……………+ (Rn – Rn-1)xn-1+ ….. ∞
(Continued)
22.10.2005 11Prof. Pushpak Bhattacharyya, IIT Bombay
(1-x) F(x) = 2 + 2x + 3x2 + ……………+ nxn-1+ ….. ∞
F(x) = (1-x)-1(2 + 2x + 3x2 + ……………+ nxn-1+ ….. ∞)
= ( 1 + x + x2 + x3 +…+ xn-1+…..)
(2 + 2x + 3x2 + 4x3 +…+ nxn-1+…..)
(Continued)
22.10.2005 12Prof. Pushpak Bhattacharyya, IIT Bombay
Co eff of Xn-1 from the expression for f(x)
= ( n + n-1 + n-2 + … + 2+ 2 )
n(n+1)/ 2 + 1
(Continued)
22.10.2005 13Prof. Pushpak Bhattacharyya, IIT Bombay
Clustering Algos
1. Competitive learning
2. K – means clustering
3. Self organization / Kohonen net
4. Counter Propagation
22.10.2005 14Prof. Pushpak Bhattacharyya, IIT Bombay
Two packages
1. SNNS – Stuttgart Neural Net Simulator
2. WEKA Package
3. MATLAB
22.10.2005 15Prof. Pushpak Bhattacharyya, IIT Bombay
Data to train
1) IRIS data: 3 - class data
2) X- OR, parity, Majority
22.10.2005 16Prof. Pushpak Bhattacharyya, IIT Bombay
IRIS Data
Classification of flowers based on four attributes in 3 classes c1, c2 and C2
1 Petal length A1
2 Petal width A2
3 Sepal length A3
4 Sepal width A4
22.10.2005 17Prof. Pushpak Bhattacharyya, IIT Bombay
Google on “Machine learning Data”
IRIS
120 – data points
< A1, A2, A3, A4 >1 --- C1
< A1, A2, A3, A4 >2 --- C3
……….
……….
22.10.2005 18Prof. Pushpak Bhattacharyya, IIT Bombay
C1, C2, C3 each has 50 data points in it
80 – 20 rule for ML
Training – 40 and Testing -- 10
22.10.2005 19Prof. Pushpak Bhattacharyya, IIT Bombay
What you have to do for IRIS choose 40 points randomly from C1, C2 and C3
1. Run BP on FF n/w
2. Run SOM assuming unlabelled data. See if you can discover 3 cluster
22.10.2005 20Prof. Pushpak Bhattacharyya, IIT Bombay
Two part assignment
Supervised
Hidden layer to be found
4 neurons for 4 attributes
22.10.2005 21Prof. Pushpak Bhattacharyya, IIT Bombay
BP for IRIS error
Training iterations
EPOCHS
1%
Very good performance on test data should be achieved.
22.10.2005 22Prof. Pushpak Bhattacharyya, IIT Bombay
A1 A2 A3
4 I/p neuronsA4
Cluster Discovery By SOM/Kohenen Net
22.10.2005 23Prof. Pushpak Bhattacharyya, IIT Bombay
K – means clustering
K o/p neurons are required from the knowledge of k clusters being present.
26 neurons
Full connection
n neurons
……
……
22.10.2005 24Prof. Pushpak Bhattacharyya, IIT Bombay
Steps
1. Initialize the weights randomly.
2. Ik is the vector presented at kth iteration.
3. Find W* such that
|w* - Ik| < |wj - Ik| for all j
22.10.2005 25Prof. Pushpak Bhattacharyya, IIT Bombay
4. make W*(new) = W* (old) + η(Ik - w* ).
5. k K +1 ; go to 3.
6. Go to 2 until the error is below a threshold.
22.10.2005 26Prof. Pushpak Bhattacharyya, IIT Bombay
K means
1. Initialize weights randomly.
2. Ik in the I/p vector presented at kth.
step of one iteration over all the training data (EPOCH).
3. Find the W* such that |w* - Ik| is minimum.
4. Find W* for Ik+1..
22.10.2005 27Prof. Pushpak Bhattacharyya, IIT Bombay
5. See the partitions at the end of the epoch.
6. Find centroid of each partition.
7. Make the weight equal to the centroid of the partition.
8. Take next epoch.
22.10.2005 28Prof. Pushpak Bhattacharyya, IIT Bombay
9. Keep doing until the error is below a threshold.
Key idea form competitive learning: Weight change incorporate after every epoch.