CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4....
Transcript of CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4....
![Page 1: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/1.jpg)
CS4501: Introduction to Computer VisionNeural Networks +Backpropagation
![Page 2: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/2.jpg)
• Softmax Classifier• Generalization / Overfitting• Pytorch
Last Class
![Page 3: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/3.jpg)
• Global Features• The perceptron model• Neural Networks – multilayer perceptron model (MLP)• Backpropagation
Today’s Class
![Page 4: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/4.jpg)
Prediction
Supervised Machine Learning StepsTraining Labels
Training Images
Training
Training
Image Features
Image Features
Testing
Test Image
Learned model
Learned model
Slide credit: D. Hoiem
![Page 5: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/5.jpg)
5
Supervised Learning –Softmax Classifier
!" = [!"%!"'!"(!")]Extract features
+, = -,%!"% + -,'!"' + -,(!"( + -,)!") + /,+0 = -0%!"% + -0'!"' + -0(!"( + -0)!") + /0+1 = -1%!"% + -1'!"' + -1(!"( + -1)!") + /1
2, = 345/(345+348 + 349)20 = 348/(345+348 + 349)21 = 349/(345+348 + 349)
Run features through classifier
;<" = [2,2021]
Get predictions
![Page 6: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/6.jpg)
6
Last Class:(mini-batch) Stochastic Gradient Descent (SGD)
!(#, %) =(−log./,01230(#, %)/∈5
6 = 0.01
for e = 0, num_epochs do
end
Initialize w and b randomly
:!(#, %)/:# :!(#, %)/:%Compute: and
Update w:
Update b:
# = # − 6:!(#, %)/:#% = % − 6:!(#, %)/:%
Print: !(#, %) // Useful to see if this is becoming smaller or not.
end
for b = 0, num_batches do B is a smallset of trainingexamples.
![Page 7: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/7.jpg)
Generalization• Generalization refers to the ability to correctly classify never before
seen examples• Can be controlled by turning “knobs” that affect the complexity of
the model
Training set (labels known) Test set (labels unknown)
![Page 8: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/8.jpg)
Overfitting
!"## $ is high !"## $ is low !"## $ is zero!
OverfittingUnderfittingHigh Bias High Variance
% is linear % is cubic% is a polynomial of
degree 9
![Page 9: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/9.jpg)
Pytorch: Project Assignment 4
• http://vicenteordonez.com/vision/
![Page 10: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/10.jpg)
Image Features• In your Project 4: Nearest Neighbors + Softmax Classifier features are:
Image: 3x32x32Feature: 3072-dim vector
![Page 11: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/11.jpg)
Image Features: Color
Photo by: marielito
slide by Tamara L. Berg
![Page 12: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/12.jpg)
Image Features: Color
![Page 13: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/13.jpg)
Image Features: Color
However, these are all images of people but thecolors in each image are very different.
![Page 14: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/14.jpg)
Image Features: HoG
Scikit-image implementation
Paper by Navneet Dalal & Bill Triggs presented at CVPR 2005 for detecting people.
![Page 15: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/15.jpg)
Image Features: HoG
Paper by Navneet Dalal & Bill Triggs presented at CVPR 2005 for detecting people.Figure from Zhuolin Jiang, Zhe Lin, Larry S. Davis, ICCV 2009 for human action recognition.
+ Block Normalization
![Page 16: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/16.jpg)
Image Features: GIST
The “gist” of a scene: Oliva & Torralba, 2001
![Page 17: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/17.jpg)
Image Features: GIST
Oriented edge response at multiple scales (5 spatial scales, 6 edge orientations) Hays and Efros,
SIGGRAPH 2007
![Page 18: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/18.jpg)
Image Features: GIST
Aggregated edge responses over 4x4 windowsHays and Efros, SIGGRAPH 2007
![Page 19: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/19.jpg)
Image Features: Bag of (Visual) Words Representation
slide by Fei-fei Li
![Page 20: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/20.jpg)
slide by Fei-fei Li
![Page 21: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/21.jpg)
Summary: Image Features
• Many other features proposed• LBP: Local Binary Patterns: Useful for recognizing faces.• Dense SIFT: SIFT features computed on a grid similar to the HOG features.• etc.
• Largely replaced by Neural networks• Still useful to study for inspiration in designing neural networks that
compute features.
![Page 22: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/22.jpg)
Perceptron ModelFrank Rosenblatt (1957) - Cornell University
More: https://en.wikipedia.org/wiki/Perceptron
! " = $1, if* +,",-
,./+ 1 > 0
0, otherwise
";
"<
"=
">
*
+;+<+=+>
Activationfunction
![Page 23: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/23.jpg)
Perceptron ModelFrank Rosenblatt (1957) - Cornell University
More: https://en.wikipedia.org/wiki/Perceptron
! " = $1, if* +,",-
,./+ 1 > 0
0, otherwise
";
"<
"=
">
*
+;+<+=+>
!?
![Page 24: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/24.jpg)
Perceptron ModelFrank Rosenblatt (1957) - Cornell University
More: https://en.wikipedia.org/wiki/Perceptron
! " = $1, if* +,",-
,./+ 1 > 0
0, otherwise
";
"<
"=
">
*
+;+<+=+>
Activationfunction
![Page 25: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/25.jpg)
Activation Functions
ReLU(x) = max(0, x)Tanh(x)
Sigmoid(x)Step(x)
![Page 26: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/26.jpg)
Pytorch - Perceptron
![Page 27: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/27.jpg)
Two-layer Multi-layer Perceptron (MLP)
!"!#!$!%
&
'"'#'$'%
&
&
&
&
()"
”hidden" layer
("
Loss / Criterion
![Page 28: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/28.jpg)
Forward pass
!"!#!$!%
&
'"'#'$'%
&
&
&
&
()" ("
*+ =& -"+.'+/
+01+ 3" !+ = 567896:(*+)
=" =& -#+!+/
+01+ 3#
(" = 567896:(=+)
>9?? = >((", ()")
![Page 29: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/29.jpg)
Backward pass
!"!#!$!%
&
'"'#'$'%
&
&
&
&
()" ("
*+*,-
= **,-
/012304(,-)*+*!7
898:;
= 88:;
/012304(<-) 898=);
898=);
= 88=);
+((", ()")
*+*'7
= ( **'7& ?"-@'-
A
-BC+ E")
*+*,-
*+*?"-@
= *'7*?"-@
*+*'7
*+*!7
= ( **!7& ?#-!-
A
-BC+ E#)
*+*<"
*+*?#-
= *!7*?#-
*+*!7
GradInputs
GradParams
![Page 30: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/30.jpg)
Pytorch – Two-layer MLP + Regression
![Page 31: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/31.jpg)
Pytorch – Two-layer MLP + LogSoftmax
# of Hidden Units
![Page 32: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/32.jpg)
Pytorch – Two-layer MLP + LogSoftmax
LogSoftmax + Negative Likelihood Loss
![Page 33: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/33.jpg)
Bookmark Opportunity!
![Page 34: CS4501: Introduction to Computer Vision Neural Networksvicente/vision/2018/slides/... · 2018. 4. 4. · Summary: Image Features •Many other features proposed • LBP: Local Binary](https://reader036.fdocuments.in/reader036/viewer/2022071516/6139282da4cdb41a985b8692/html5/thumbnails/34.jpg)
Questions?
34