Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern...
Transcript of Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern...
![Page 1: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/1.jpg)
Image Analysis
Lecture 9.3 - Introduction to Machine Learning
Idar Dyrdal
![Page 2: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/2.jpg)
Machine learning (Pattern recognition)
• Recognition of individuals (instance recognition)
• Discrimination between classes (pattern recognition, classification)
Training Labels
Training
ImagesClassifier Training
Supervised learning
Image Features
Trained Classifier
2
![Page 3: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/3.jpg)
Pattern recognition in practice
Working applications of Image Pattern recognition:
• Reading license plates, postal codes, bar codes
• Character recognition
• Automatic diagnosis of medical samples
• Fingerprint recognition
• Face detection and recognition
• …
3
![Page 4: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/4.jpg)
Classification system
4
Feature extractor Classifier
Pine
Birch
Camera
![Page 5: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/5.jpg)
Image features for object recognition
5
![Page 6: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/6.jpg)
Feature vector and feature space
6
![Page 7: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/7.jpg)
Training of classifiers
7
Learn a function to predict the class from
the given featuresRegion 1
Region 2
Decision
boundary
![Page 8: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/8.jpg)
Classifiers and training methods
• Bayes classifier
• Nearest-neighbors and K-nearest-neighbors
• Parzen windows
• Linear and higher order discriminant functions
• Neural nets
• Support Vector Machines (SVM)
• …
8
![Page 9: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/9.jpg)
Class conditional probability density functions
9
![Page 10: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/10.jpg)
Bayesian decision theory
Overview
10
x
Class conditional densities:
Prior probabilities:
Posterior probabilities given by Bayes rule:
Minimum error rate classification:
Assign the unknown object to the class with maximum posterior probability!
![Page 11: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/11.jpg)
Density estimation
Parametric methods:
• Assume a given shape of the density function
• Use the training set to estimate the unknown parameters.
Non-parametric (distribution free) methods:
• Point estimation of the density using the training set directly
• Parzen windows
• Nearest neighbor estimation (leads directly to the nearest-neighbor and k-nearest-neighbor classifiers).
11
Example – Gaussian distribution:
Parameters:
![Page 12: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/12.jpg)
Parameter estimation
12
Parameter estimates:
![Page 13: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/13.jpg)
Discriminant functions
13
Estimate of the density in a given point:
From Bayes rule:
Examples of discriminant functions:
Decision rule:
Choose the class with maximum discriminant function value.
![Page 14: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/14.jpg)
Example - linear classifier
14
Uncorrelated features and
common covariance matrices
Linear decision boundaries
![Page 15: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/15.jpg)
Dimensionality reduction - linear transformations
• PCA, ICA, LLE, Isomap
• PCA (Principal Components Analysis)
is one of the most important techniques
for dimensionality reduction
• It takes advantage of correlations
between the features to produce the
best possible lower dimensional
representation of the data with respect
to reconstruction error
• The eigenvectors of the lumped
covariance matrix defines the new
features in the transformed feature
space.
15
![Page 16: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination](https://reader035.fdocuments.in/reader035/viewer/2022071018/5fd206be67fde61b2173dd84/html5/thumbnails/16.jpg)
Summary
Machine learning:
• Pattern classification
• Training of classifiers (supervised learning)
• Parametric and non-parametric methods
• Discriminant functions
• Dimensionality reduction
More information: Szeliski 14.1
16