Digital Image Processing Introduction to M-function Programming.
DIGITAL IMAGE PROCESSING Dr J. Shanbehzadeh M. Hosseinajad ( J.Shanbehzadeh M. Hosseinajad)
-
Upload
basil-fields -
Category
Documents
-
view
214 -
download
0
Transcript of DIGITAL IMAGE PROCESSING Dr J. Shanbehzadeh M. Hosseinajad ( J.Shanbehzadeh M. Hosseinajad)
DIGITAL IMAGE PROCESSING
Dr J. [email protected]
M. Hosseinajad
Chapter 12 – Object RecognitionPart 2 – Neural Networks And Structural
Methods
( J.Shanbehzadeh M. Hosseinajad)
Road map of chapter 8
12.1 12.312.1
12.1 Patterns and Pattern Classes12.2 Recognition Based on Decision-Theoretic Methods12.3 Structural Methods
Patterns and Pattern Classes
12.212.2 12.3
Some Basic Compression MethodsStructural Methods
( J.Shanbehzadeh M. Hosseinajad)
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
Matching
Optimum Statistical Classifiers
Neural NetworksNeural Networks
( J.Shanbehzadeh M. Hosseinajad)
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
Try to mimic the structure and function of our nervous system.Neuron:
- A function unit of the nervous system.- Functioning through complex interconnection.- Functions in parallel.
( J.Shanbehzadeh M. Hosseinajad)
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Biological Neurons: Each neuron receives thousands of signal from other neurons.If the integrated signal exceeds a threshold, the cell fires and generates an action potential or spike.
Artificial Neurons:A value of weight ‘w’ indicates the strength of the signal.Neuron B is stimulated if there is sufficient signal sent from neuron A.
Neural Networks
Mathematic Models12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Neural Networks
Mathematic Models12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Perceptron is the simplest NN model consisting of a single neuron.Two main parameters:
WeightThreshold
Example: AND operation
Perceptron
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
The rule which govern how exactly weights or thresholds are changed, is called as the learning algorithm.
Different types of neural networks may have different learning algorithms:
Fixed incremental correction ruleDelta ruleGradient decent rule…
Training
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Perceptron Learning Rule
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Perceptron Learning Rule
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Perceptron Learning Rule
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Perceptron Learning Rule
Combine both rulesw’ = w + α(r-o)x, w = α (r-o)x
where 0< α <1r = desired outputo = output from the network
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Fixed Increment CorrectionFor linearly separable classes:
The weight is updated when an error occurs.
case I:if y(k) w1 and w∈ T(k)y(k) ≤0 w(k+1) = w(k) + cy(k), c > 0
case II:if y(k) w2 and w∈ T(k)y(k) ≥0 w(k+1) = w(k) - cy(k), c > 0
case III:else w(k+1) = w(k)
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Nonseparable Classes
The Delta Rule
Can be used both with separable and non-separable classes.
Main Idea:– Minimizes the error between the actual and desired response at any training step.
– This can be achieved by using a technique called
Gradient descent.
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Gradient Descent
Gradient descent searches the space of weight vectors to find weights that best fit the training examples.
The objective is to minimize the following error:
E( w ) = ( ½ ) ∑( r – o )2o = wyr : desired outputo: output from the network
The training is a process of minimizing the error E( w ) in the direction of the steepest most rapid decrease, that is in direction opposite to the gradient.
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Gradient Descent
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Learning Rate
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
How about XOR Problem?!
Two planes are required for correct classification.
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Multi-layer NN
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Learning Rule
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
An Example
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Overfitting (overtraining): when the NN learns too many I/O examples it may end up memorizing the training data.
The size of the training set.
The architecture of the NN (too many hidden layers often causes overfitting problem).
The complexity of the problem at hand.
Since the algorithm attempts to minimize error, the algorithm can fall into any local minima.
Problems
Neural Networks
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Number of Layers
The complexity of the decision surfaces can increase number of layers
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
Matching Shape Numbers
String Matching
Matching Shape Numbers
( J.Shanbehzadeh M. Hosseinajad)
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Matching Shape Numbers
Structural Pattern Recognition
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Matching Shape Numbers
The Order of Shape Number
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
Matching Shape Numbers
The Order of Shape Number
Similarity Degree (k) between two region boundaries is defined as the largest order for which their shape numbers still coincide.
Distance between shape a and b = D(a,b)= 1/k
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
Matching Shape Numbers
String MatchingString Matching
( J.Shanbehzadeh M. Hosseinajad)
12.1 Patterns and Pattern Classes
12.2 Recognition Based on Decision-Theoretic Methods
12.3 Structural Methods
( J.Shanbehzadeh M. Hosseinajad)
String Matching
Region a is coded into string denoted by a1 a2 ... an
Region b is coded into string denoted by b1 b2 … bn
The number of symbols that do not match is
A similarity b/w region a and b is the ratio R