FOUNDATIONS OF DATA MINING NEURAL NETWORKS, MLP
15
FOUNDATIONS OF DATA MINING NEURAL NETWORKS, MLP Mohammad Javad Fadaeieslam
Transcript of FOUNDATIONS OF DATA MINING NEURAL NETWORKS, MLP
FOUNDATIONS OF DATA MINING
NEURAL NETWORKS, MLP
Mohammad Javad Fadaeieslam
Foundations of Data Mining
THE XOR PROBLEM
2
No single straight line exists that separates the two classes.
Foundations of Data Mining
AND, OR
3
Foundations of Data Mining
THE PERCEPTRON TO REALIZE OR
4
Foundations of Data Mining
SOLVING XOR USING TWO LINES
To separate the two classes Aand B in the XOR problem, afirst thought that comes tomind is to draw two, insteadof one, straight lines.
5
Foundations of Data Mining
THE TWO-LAYER PERCEPTRON
6
Foundations of Data Mining
THE TWO-LAYER PERCEPTRON
7
Input layer Hidden layer Output layer
A careful look at the two-layer perceptron reveals that the action of
the neurons of the hidden layer is actually a mapping of the input
space x onto a linearly separable one.
Foundations of Data Mining
CLASSIFICATION CAPABILITIES OF THE TWO-LAYER PERCEPTRON
8
Foundations of Data Mining
POLYHEDRAL FORMED BY THE
NEURONS OF THE FIRST HIDDEN LAYER
OF A MULTILAYER PERCEPTRON.
9
Foundations of Data Mining
CLASSIFICATION CAPABILITIES OF THE TWO-LAYER PERCEPTRON
The first layer of neurons divides the input l-dimensionalspace into polyhedral, which are formed by hyperplaneintersections. All vectors located within one of thesepolyhedral regions are mapped onto a specific vertex of theunit Hp dimensional Hypercube. The output neuronsubsequently realizes another hyperplane, which separatesthe hypercube into two parts, having some of its vertices onone side and some on the other side.
10
Foundations of Data Mining
CLASSIFICATION CAPABILITIES OF THE TWO-LAYER PERCEPTRON
11
Foundations of Data Mining
THREE-LAYER PERCEPTRON
12
Foundations of Data Mining
THREE-LAYER PERCEPTRON
13
A three-layer perceptron can separate any union of
polyhedral regions.
Foundations of Data Mining
THE DRAWBACKS OF MULTILAYER BASIC
PERCEPTRON
To assume that in practice we know the regions where the data
are located and we can compute the respective hyperplane
equations analytically is no doubt wishful thinking.
All we know in practice is a set of training points with therespective class labels.
14
Foundations of Data Mining
THE BACKPROPAGATION ALGORITHM
The backpropagation algorithm was introduced to find
weights of neural network from data.
It works by computing the gradient of the loss function with respect to each weight by the chain rule.
15
