COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION...
Transcript of COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION...
![Page 1: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/1.jpg)
COMPUTER VISION: DEEP LEARNING LAB COURSE
DAY 2 – FEATURE-BASED IMAGE CLASSIFICATION
SEBASTIAN HOUBEN
![Page 2: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/2.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 2
Schedule
Today
Histogram of Oriented Gradients (HOG)
Dimensionality Reduction with Principal
Component Analysis (PCA)
Going Deeper into Classification
Underfitting / Overfitting
Training-Test-Validation
Support Vector Machine (SVM)
Multi-Class SVM
![Page 3: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/3.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 3
Classifier
{cat,dog}
Feature Extraction
Classification pipeline
![Page 4: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/4.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 4
Multi-class
SVM
{speed limit 20, speed limit 30, …, derestriction, yield way, …}
Histogram-of-Oriented-Gradients
Classification pipeline (Multi-class)
Feature Extraction
![Page 5: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/5.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 5
Histogram-of-Oriented-Gradients
Dalal & Triggs 2005
Initially used for pedestrian detection
Describes local gradient orientation distribution
![Page 6: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/6.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 6
Histogram-of-Oriented-Gradients
Dalal & Triggs 2005
Initially used for pedestrian detection
Describes local gradient orientation distribution
Compute gradients
Convolute image with
and
Yields pixel-wise orientation
![Page 7: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/7.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 7
Histogram-of-Oriented-Gradients
Dalal & Triggs 2005
Initially used for pedestrian detection
Describes local gradient orientation distribution
Compute gradients
Convolute image with
and
Yields pixel-wise orientation
Divide image into cells (e.g., 8x8 pixels)
![Page 8: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/8.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 8
Histogram-of-Oriented-Gradients
Dalal & Triggs 2005
Initially used for pedestrian detection
Describes local gradient orientation distribution
Compute gradients
Convolute image with
and
Yields pixel-wise orientation
Divide image into cells (e.g., 8x8 pixels)
Compute a histogram of all orientations present in each cell
Weigh the contribution of each pixel with its absolute
gradient magnitude
![Page 9: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/9.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 9
Histogram-of-Oriented-Gradients
Dalal & Triggs 2005
Initially used for pedestrian detection
Describes local gradient orientation distribution
Compute gradients
Convolute image with
and
Yields pixel-wise orientation
Divide image into cells (e.g., 8x8 pixels)
Compute a histogram of all orientations present in each cell
Weigh the contribution of each pixel with its absolute
gradient magnitude
Combine neighbouring cells to blocks (e.g. 2x2 cells) and normalize
histograms with respect to sum of all pixel gradients magnitudes
![Page 10: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/10.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 10
Histogram-of-Oriented-Gradients
Dalal & Triggs 2005
Initially used for pedestrian detection
Describes local gradient orientation distribution
Compute gradients
Convolute image with
and
Yields pixel-wise orientation
Divide image into cells (e.g., 8x8 pixels)
Compute a histogram of all orientations present in each cell
Weigh the contribution of each pixel with its absolute
gradient magnitude
Combine neighbouring cells to blocks (e.g. 2x2 cells) and normalize
histograms with respect to sum of all pixel gradients magnitudes
For all blocks for all cells concatenate the histograms
![Page 11: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/11.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 11
Histogram-of-Oriented-Gradients
![Page 12: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/12.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 12
Visualizing High-dimensional Feature Spaces
High-dimensional vectors are hard to interpret
Visualizing in 2d or 3d is preferable
Dimensionality reduction / embedding
Several methods:
PCA (Principal Component Analysis)
t-SNE (t-distributed Stochastic Nearest-
Neighbour Embedding)
LLE (Locally-Linear Embedding)
MDS (Multi-Dimensional Scaling)
![Page 13: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/13.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 13
Visualizing High-dimensional Feature Spaces
![Page 14: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/14.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 14
Visualizing High-dimensional Feature Spaces
Find function that maps data points to 2
dimensions:
Make it easy: Linear
Thus, can be represented by a 2 x n matrix
But linear means 0 is mapped to 0
Subtract mean value from dataset beforehand
Consists of two rows
Rows represent the axes of main variance
(principal axes)
![Page 15: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/15.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 15
Visualizing High-dimensional Feature Spaces
Rows represent the axes of main variance
(principal axes)
Row vector maximizing this, is given by
eigenvector of
w.r.t. largest eigenvalue (covariance matrix C)
Generally: Take the eigenvectors corresponding to
the largest eigenvalues of the covariance matrix
and project the zero-mean dataset to these
vectors
![Page 16: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/16.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 16
![Page 17: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/17.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 17
Image Classification
Linear classifier finds hyperplane to seperate
sets of points
A more complex classifier might find a better
way to seperate the two datasets
![Page 18: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/18.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 18
Image Classification
Linear classifier finds hyperplane to seperate
sets of points
A more complex classifier might find a better
way to seperate the two datasets
Many ML methods have hyper-parameters that
control the complexity of the function to fit
But: In general, very complex functions tend to
perform worse on unseen data
![Page 19: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/19.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 19
Image Classification
Linear classifier finds hyperplane to seperate
sets of points
A more complex classifier might find a better
way to seperate the two datasets
Many ML methods have hyper-parameters that
control the complexity of the function to fit
But: In general, very complex functions tend to
perform worse on unseen data
Complexity
Erro
r
Test error
Training error
![Page 20: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/20.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 20
Complexity
Erro
r
Test error
Training error
Image Classification
Underfitting Good fit
Overfitting
![Page 21: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/21.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 21
Complexity
Erro
r
Test error
Training error
Image Classification
Underfitting Good fit
Overfitting
![Page 22: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/22.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 22
Complexity
Erro
r
Test error
Training error
Image Classification
Underfitting Good fit
Overfitting
![Page 23: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/23.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 23
Image Classification
Linear classifier finds hyperplane to seperate
sets of points
A more complex classifier might find a better
way to seperate the two datasets
Many ML methods have hyper-parameters that
control the complexity of the function to fit
But: In general, very complex functions tend to
perform worse on unseen data
Need to estimate the training error: split dataset
into training-validation-test
Complexity
Erro
r
Test error
Training error
![Page 24: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/24.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 24
Support Vector Machines
![Page 25: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/25.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 25
Support Vector Machines
Labelled Data:
Solve:
![Page 26: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/26.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 26
Support Vector Machines
Labelled Data:
Solve:
![Page 27: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/27.jpg)
INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION 27
Support Vector Machines
Labelled Data:
Solve:
are a hyper-parameter that control complexity
Multiclass: One-vs-All
most confident classifier wins
Confidence ist given by distance to border
Multiclass: One-vs-One
![Page 28: COMPUTER VISION: DEEP LEARNING LAB COURSE · 10 INTRODUCTION TO DEEP LEARNING FOR COMPUTER VISION Histogram-of-Oriented-Gradients Dalal & Triggs 2005 Initially used for pedestrian](https://reader030.fdocuments.in/reader030/viewer/2022041013/5ec461c217d06d7cdf35bb8b/html5/thumbnails/28.jpg)
QUESTIONS?
EXERCISES.