13 cv mil_preprocessing
-
Upload
zukun -
Category
Technology
-
view
196 -
download
0
Transcript of 13 cv mil_preprocessing
![Page 1: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/1.jpg)
Computer vision: models, learning and inference
Chapter 13
Image preprocessing and feature extraction
Please send errata to [email protected]
![Page 2: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/2.jpg)
Preprocessing
• The goal of pre-processing is – to try to reduce unwanted variation in image due
to lighting, scale, deformation etc.
– to reduce data to a manageable size
• Give the subsequent model a chance
• Preprocessing definition: deterministic transformation to pixels p to create data vector x
• Usually heuristics based on experience
2Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 3: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/3.jpg)
Structure
33Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Per-pixel transformations
• Edges, corners, and interest points
• Descriptors
• Dimensionality reduction
![Page 4: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/4.jpg)
Normalization
• Fix first and second moments to standard values
• Remove contrast and constant additive luminance variations
Before
After 4Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 5: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/5.jpg)
Histogram Equalization
Make all of the moments the same by forcing the histogram of intensities to be the same
Before/ normalized/ Histogram Equalized
5Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 6: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/6.jpg)
Histogram Equalization
6Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 7: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/7.jpg)
Convolution
Takes pixel image P and applies a filter F
Computes weighted sum of pixel values, where weights given by filter.
Easiest to see with a concrete example
7Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 8: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/8.jpg)
Blurring (convolve with Gaussian)
8Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 9: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/9.jpg)
Gradient Filters
• Rule of thumb: big response when image matches filter
9Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 10: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/10.jpg)
Gabor Filters
10Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 11: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/11.jpg)
Haar Filters
11Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 12: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/12.jpg)
Local binary patterns
12Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 13: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/13.jpg)
Textons
• An attempt to characterize texture
• Replace each pixel with integer representing the texture ‘type’
13Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 14: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/14.jpg)
Computing Textons
Take a bank of filters and apply to lots of images
Cluster in filter space
For new pixel, filter surrounding region with same bank, and assign to nearest cluster
14Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 15: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/15.jpg)
Structure
1515Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Per-pixel transformations
• Edges, corners, and interest points
• Descriptors
• Dimensionality reduction
![Page 16: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/16.jpg)
Edges
(from Elder andGoldberg 2000)
16Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 17: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/17.jpg)
Canny Edge Detector
17Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 18: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/18.jpg)
Canny Edge Detector
Compute horizontal and vertical gradient images h and v
18Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 19: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/19.jpg)
Canny Edge Detector
Quantize to 4 directions19Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 20: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/20.jpg)
Canny Edge Detector
Non-maximal suppression
20Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 21: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/21.jpg)
Canny Edge Detector
Hysteresis Thresholding21Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 22: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/22.jpg)
Harris Corner Detector
Make decision based on image structure tensor
22Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 23: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/23.jpg)
SIFT Detector
Filter with difference of Gaussian filters at increasing scalesBuild image stack (scale space)Find extrema in this 3D volume
23Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 24: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/24.jpg)
SIFT Detector
Identified Corners Remove those on edges
Remove those where contrast is
low
24Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 25: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/25.jpg)
Assign Orientation
Orientation assigned by looking at intensity gradients in region around point
Form a histogram of these gradients by binning.
Set orientation to peak of histogram.
25Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 26: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/26.jpg)
Structure
2626Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Per-pixel transformations
• Edges, corners, and interest points
• Descriptors
• Dimensionality reduction
![Page 27: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/27.jpg)
Sift Descriptor
Goal: produce a vector that describes the region around the interest point.
All calculations are relative to the orientation and scale of the keypoint
Makes descriptor invariant to rotation and scale
27Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 28: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/28.jpg)
Sift Descriptor
1. Compute image gradients 2. Pool into local histograms
3. Concatenate histograms4. Normalize histograms
28Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 29: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/29.jpg)
HoG Descriptor
29Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 30: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/30.jpg)
Bag of words descriptor
• Compute visual features in image• Compute descriptor around each• Find closest match in library and assign index• Compute histogram of these indices over the region• Dictionary computed using K-means
30Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 31: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/31.jpg)
Shape context descriptor
31Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 32: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/32.jpg)
Structure
3232Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Per-pixel transformations
• Edges, corners, and interest points
• Descriptors
• Dimensionality reduction
![Page 33: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/33.jpg)
Dimensionality Reduction
where is a function that takes the hidden variable and a set of parameters q.
Dimensionality reduction attempt to find a low dimensional (or hidden) representation h which can approximately explain the data x so that
Typically, we choose the function family and then learn hand q from training data
33Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 34: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/34.jpg)
Least Squares Criterion
Choose the parameters q and the hidden variables h so that they minimize the least squares approximation error (a measure of how well they can reconstruct the data x).
34Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 35: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/35.jpg)
Simple Example
Approximate each data example x with a scalar value h.
Data is reconstructed by multiplying h by a parameter f and adding the mean vector m.
... or even better, lets subtract m from each data example to get mean-zero data
35Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 36: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/36.jpg)
Simple Example
Approximate each data example x with a scalar value h.
Data is reconstructed by multiplying h by a factor f.
Criterion:
36Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 37: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/37.jpg)
Criterion
Problem: the problem is non-unique. If we multiply f by any constant a and divide each of the hidden variables h1...I by the same constant we get the same cost. (i.e. (fa) (hi/a) = fhi)
Solution: We make the solution unique by constraining the length of f to be 1 using a Lagrange multiplier.
37Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 38: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/38.jpg)
Criterion
Now we have the new cost function:
To optimize this we take derivatives with respect to f and hi, equate the resulting expressions to zero and re-arrange.
38Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 39: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/39.jpg)
Solution
To compute the hidden value, take dot product with the vector f
39Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 40: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/40.jpg)
Solution
To compute the hidden value, take dot product with the vector f
orwhere
To compute the vector f, compute the first eigenvector of the scatter matrix .
40Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 41: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/41.jpg)
Computing h
To compute the hidden value, take dot product with the vector f
41Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 42: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/42.jpg)
Reconstruction
To reconstruct, multiply the hidden variable h by vector f.42Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 43: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/43.jpg)
Principal Components Analysis
Same idea, but not the hidden variable h is multi-dimensional. Each components weights one column of matrix F so that data is approximated as
This leads to cost function:
This has a non-unique optimum so we enforce the constraint that F should be a (truncated) rotation matrix and FTF=I
43Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 44: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/44.jpg)
PCA Solution
To compute the hidden vector, take dot product with each column of F.
To compute the matrix F, compute the first eigenvectors of the scatter matrix .
The basis functions in the columns of F are called principal components and the entries of h are called loadings
44Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 45: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/45.jpg)
Dual PCA
Problem: PCA as described has a major drawback. We need to compute the eigenvectors of the scatter matrix
But this has size Dx x Dx. Visual data tends to be very high dimensional, so this may be extremely large.
Solution: Reparameterize the principal components as weighted sums of the data
...and solve for the new variables Y.45Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 46: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/46.jpg)
Geometric Interpretation
Each column of F can be described as a weighted sum of the original datapoints.
Weights given in the corresponding columns of the new variable Y.
46Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 47: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/47.jpg)
Motivation
Solution: Reparameterize the principal components as weighted sums of the data
...and solve for the new variables Y.
Why? If the number of datapoints I is less than the number of observed dimension Dx then the Y will be smaller than F and the resulting optimization becomes easier.
Intuition: we are not interested in principal components that are not in the subspace spanned by the data anyway.
47Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 48: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/48.jpg)
Cost functions
...subject to FTF=I or YTXTXY=I.
...subject to FTF=I.
Principal components analysis
Dual principal components analysis
48Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 49: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/49.jpg)
Solution
To compute the hidden vector, take dot product with each column of F=YX.
49Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 50: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/50.jpg)
Solution
To compute the matrix Y, compute the first eigenvectors of the inner product matrix XTX.
The inner product matrix has size I x I.
If the number of examples I is less than the dimensionality of the data Dx then this is a smaller eigenproblem.
50Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
To compute the hidden vector, take dot product with each column of F=YX.
![Page 51: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/51.jpg)
K-Means algorithm
Approximate data with a set of means
51Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Least squares criterion
Alternate minimization
![Page 52: 13 cv mil_preprocessing](https://reader034.fdocuments.in/reader034/viewer/2022052601/5597e24d1a28abf30e8b456a/html5/thumbnails/52.jpg)
52Computer vision: models, learning and inference. ©2011 Simon J.D. Prince