CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer...
Transcript of CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer...
![Page 1: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/1.jpg)
CAP5415-ComputerVisionLecture20-FaceRecognition,Haar Features,
LocalBinaryPatterns,andBoosting
Lecture20
1
![Page 2: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/2.jpg)
Reminders• ProjectDeadlines/Return
– 6December,from1pm– 4.30pm– 8December,from1pm– 4.30pm– Location:HEC221– 5minutesPowerpoint presentation+Demo– Return
• Your.ppt/pdfpresentationandcode.
2
Lecture20
![Page 3: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/3.jpg)
FaceDetectionandRecognition
3
Lecture20
Detection Recognition “Sally”
![Page 4: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/4.jpg)
• Whywasn’tMassachusetss BomberidentifiedbytheMassachusettsDepartmentofMotorVehiclessystemfromthevideosurveillanceimages?
• HewasenrolledinMADMVDatabase!
4
Lecture20
DMV Face Recognition
System ?SlideCreditstoAnimetrics,Dr.MarcValliant,VP&CTO
![Page 5: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/5.jpg)
• Today’sFRtechnologywillreliablyfindcontrolledfacialphoto inamugshot databaseofcontrolleddatabase.
5
Lecture20
SlideCreditstoAnimetrics,Dr.MarcValliant,VP&CTO
![Page 6: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/6.jpg)
ControlledFacialPhoto• Today’sFRtechnologywillreliablyfindcontrolledfacialphotoinamugshot databaseofcontrolleddatabase.
• However,thereareconfoundingvariablesinuncontrolledfacialphotos
6
Lecture20
SlideCreditstoAnimetrics,Dr.MarcValliant,VP&CTO
![Page 7: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/7.jpg)
ControlledFacialPhoto• Today’sFRtechnologywillreliablyfindcontrolledfacialphotoinamugshot databaseofcontrolleddatabase.
• However,thereareconfoundingvariablesinuncontrolledfacialphotos– Resolution (not enough pixels)– Facial Pose – angulated– Illumination– Occluded facial areas
7
Lecture20
SlideCreditstoAnimetrics,Dr.MarcValliant,VP&CTO
![Page 8: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/8.jpg)
FurtherDifficulties
8
Lecture20
![Page 9: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/9.jpg)
Threegoals
9
Lecture20
FeatureComputation
• features must be computed as quickly as possible
FeatureSelection
• select the most discriminating features
Realtimeliness
• must focus on potentially positive image areas (that contain faces)
![Page 10: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/10.jpg)
FaceDetection• Beforefacerecognitioncanbeappliedtoageneralimage,the
locationsandsizesofanyfacesmustbefirstfound.
• Rowley,Baluja,Kanade (1998) 10
Lecture20
![Page 11: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/11.jpg)
FaceDetection/RecognitionusingMobileDevices
11
Lecture20
Facedetection(cameraautomaticallyAdjustthefocusbasedondetectedFaces)
Auto-loginwithrecognizedfaces
![Page 12: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/12.jpg)
FaceDetection
Feature-basedEye,mouth,..
Template-basedAAM,…
Appearance-BasedPatches,…
12
Lecture20
![Page 13: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/13.jpg)
Someoftherepresentativeworks
13
Lecture20
![Page 14: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/14.jpg)
Rectangle(Haar-like)Features
14
Lecture20
“Rectangle filters”
Value =
∑ (pixels in white area) –∑ (pixels in black area)
![Page 15: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/15.jpg)
FastComputationwithIntegralImages
15
Lecture20
• This can quickly be computed in one pass through the image
(x,y)( ) ( )
( ) ( ) ( )( ) ( ) ( )
' , '
formal definition:, ', '
Recursive definition:, , 1 ,
, 1, ,
x x y yii x y i x y
s x y s x y i x y
ii x y ii x y s x y
≤ ≤
=
= − +
= − +
∑
0 1 1 11 2 2 31 2 1 11 3 1 0
IMAGE
0 1 2 31 4 7 112 7 11 163 11 16 21
INTEGRAL IMAGE
![Page 16: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/16.jpg)
FeatureSelection
16
Lecture20
• For a 24x24 detection region, the number of possible rectangle features is ~160,000!
![Page 17: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/17.jpg)
FeatureSelection
17
Lecture20
• For a 24x24 detection region, the number of possible rectangle features is ~160,000! PCA
![Page 18: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/18.jpg)
LocalBinaryPatterns(LBP):AlternativeFeatures
• Gray-scaleinvarianttexturemeasure• Derivedfromlocalneighborhood• Powerfultexturedescriptor• Computationallysimple• Robustagainstmonotonicgray-scalechanges
18
Lecture20
![Page 19: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/19.jpg)
LocalBinaryPatterns(LBP):AlternativeFeatures
19
Lecture20
(LBPfromdynamic/videotexture)
![Page 20: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/20.jpg)
SimpleFRforMobileDevices
20
Lecture20
(LBP:localbinarypatterns)
![Page 21: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/21.jpg)
PrincipalComponentAnalysis(PCA)
21
Lecture20
• Mappingfromtheinputsintheoriginald-dimensionalspacetoanew(k<d)-dimensionalspace,withminimumlossofinformation.
![Page 22: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/22.jpg)
PrincipalComponentAnalysis(PCA)
22
Lecture20
• Mappingfromtheinputsintheoriginald-dimensionalspacetoanew(k<d)-dimensionalspace,withminimumlossofinformation.
• PCAisanunsupervisedmethod, itdoesnotuseoutput information.
![Page 23: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/23.jpg)
PrincipalComponentAnalysis(PCA)
23
Lecture20
• Mappingfromtheinputsintheoriginald-dimensionalspacetoanew(k<d)-dimensionalspace,withminimumlossofinformation.
• PCAisanunsupervisedmethod, itdoesnotuseoutput information.
PCAcentersthesampleandthenrotatestheaxestolineupwiththedirectionsofhighestvariance.
![Page 24: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/24.jpg)
PrincipalComponentAnalysis(PCA)
• Theprojectionofx onthedirectionof wis
24
Lecture20
z = w
Tx
Originalaxes
**
***
*
* **
****
*
*
*
** ***
*
**
Datapoints
FirstprincipalcomponentSecondprincipalcomponent
![Page 25: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/25.jpg)
PrincipalComponentAnalysis(PCA)
• Theprojectionofx onthedirectionof wis
• Theprincipalcomponentusw1 suchthatthesample,afterprojectiononw1,ismostspreadoutsothatthedifferencebetweenthesamplepointsbecomesmostapparent.
25
Lecture20
z = w
Tx
![Page 26: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/26.jpg)
PrincipalComponentAnalysis(PCA)
• Theprojectionofx onthedirectionof wis
• Theprincipalcomponentusw1 suchthatthesample,afterprojectiononw1,ismostspreadoutsothatthedifferencebetweenthesamplepointsbecomesmostapparent.
• Tohaveuniquesolution,
26
Lecture20
z = w
Tx
||w1|| = 1
![Page 27: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/27.jpg)
PrincipalComponentAnalysis(PCA)
• Theprojectionofx onthedirectionof wis
• Theprincipalcomponentusw1 suchthatthesample,afterprojectiononw1,ismostspreadoutsothatthedifferencebetweenthesamplepointsbecomesmostapparent.
• Tohaveuniquesolution,• with
27
Lecture20
z = w
Tx
||w1|| = 1
z1 = w1Tx
Cov(x) = ⌃
![Page 28: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/28.jpg)
PrincipalComponentAnalysis(PCA)
• Theprojectionofx onthedirectionof wis
• Theprincipalcomponentusw1 suchthatthesample,afterprojectiononw1,ismostspreadoutsothatthedifferencebetweenthesamplepointsbecomesmostapparent.
• Tohaveuniquesolution,• with• Then,
28
Lecture20
z = w
Tx
||w1|| = 1
z1 = w1Tx
Cov(x) = ⌃V ar(z1) = w1
T⌃w1
![Page 29: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/29.jpg)
PrincipalComponentAnalysis(PCA)
• Theprojectionofx onthedirectionof wis
• Theprincipalcomponentusw1 suchthatthesample,afterprojectiononw1,ismostspreadoutsothatthedifferencebetweenthesamplepointsbecomesmostapparent.
• Tohaveuniquesolution,• with• Then,• SEEKw1 suchthatVar(z1)ismaximized!
29
Lecture20
z = w
Tx
||w1|| = 1
z1 = w1Tx
Cov(x) = ⌃V ar(z1) = w1
T⌃w1
![Page 30: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/30.jpg)
SolutionofPCA• WriteitasaLagrangeproblem,takederivativesw.r.t tow,then
wheremisthesamplemean
(=D diagonal)(S:spectraldecomp.)
30
Lecture20
z = W
T (x�m)
Cov(z) = WTSW
XTX = WDWT
![Page 31: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/31.jpg)
SolutionofPCA
• Letussaywewanttoreducedimensionalitytok<d,wetakethefirstkcolumnsofW(withthehighesteigenvalues).
31
Lecture20
XTX = WDWT
![Page 32: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/32.jpg)
SolutionofPCA
• Letussaywewanttoreducedimensionalitytok<d,wetakethefirstkcolumnsofW(withthehighesteigenvalues).
i=1,…k,t=1,...,N
32
Lecture20
XTX = WDWT
zti = w
Ti x
t
(XTX)wi = �iwi
X = USVT
U = evec(XXT )
V = evec(XTXT )
S2 = eval(XXT )
![Page 33: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/33.jpg)
Screeplot:AbilityofPCstoexplainvariationindata
• Enough PCs (principal components) to have a cumulative variance explained by the PCs thatis >50-70%
• Kaiser criterion: keep PCs with eigenvalues >1
33
Lecture20
λ
λN
![Page 34: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/34.jpg)
Recap:PCAcalculationsincartoon
34
Lecture20
StepsinPCA:#1CalculateAdjustedDataSet
…ndims
datasamples
DataSet:D Meanvalues:M
-…
AdjustedDataSet:A
=Mi iscalculatedbytakingthemeanofthevaluesindimensioni
![Page 35: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/35.jpg)
Recap:PCAcalculationsincartoon
35
Lecture20
StepsinPCA:#2CalculateCo-variancematrix,C,fromAdjustedDataSet,A
Co-varianceMatrix:C
n
n
Cij =cov(i,j)
Note:Sincethemeansofthedimensions intheadjusteddataset,A,are0,thecovariancematrixcansimplybewrittenas:
C=AAT/(n-1)
![Page 36: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/36.jpg)
Recap:PCAcalculationsincartoon
36
Lecture20
StepsinPCA:#3CalculateeigenvectorsandeigenvaluesofC
Eigenvectors
Eigenvalues
Ifsomeeigenvaluesare0orverysmall,wecanessentially discardthoseeigenvaluesandthecorrespondingeigenvectors,hencereducingthedimensionality ofthenewbasis.
Eigenvectors
Eigenvalues
xx
MatrixEMatrixE
![Page 37: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/37.jpg)
Recap:PCAcalculationsincartoon
37
Lecture20
StepsinPCA:#4Transformingdatasettothenewbasis
F=ETA
where:• Fisthetransformeddataset• ET isthetransposeoftheEmatrixcontainingtheeigenvectors• Aistheadjusteddataset
Notethatthedimensions ofthenewdataset,F,arelessthanthedatasetA
TorecoverAfromF:
(ET)-1F=(ET)-1ETA(ET)TF=AEF=A
*Eisorthogonal,thereforeE-1 =ET
![Page 38: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/38.jpg)
HolisticFR:Eigenfaces
38
Lecture20
Eigenfaces,fisherfaces,tensorfaces…..
![Page 39: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/39.jpg)
GaborFeature-basedFR• EarlierFRmethodsaremostlyfeature-based.• Themostsuccessfulfeature-basedFRistheelasticbunchgraph
matchingsystemwithGaborfiltercoefficientsasfeatures:
39
Lecture20
(scale)
![Page 40: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/40.jpg)
GaborFeatures
40
Lecture20
Scale(5) Orientation(8)
![Page 41: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/41.jpg)
PCAonFaces:“Eigenfaces”
41
Lecture20
Averageface
Firstprincipalcomponent
Othercomponents
Forallexceptaverage,“gray” =0,“white” >0,“black” <0
![Page 42: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/42.jpg)
Eigenfaces example
42
Lecture20
Trainingfaces
![Page 43: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/43.jpg)
Eigenfaces example
43
Lecture20
Top eigenvectors: u1,…uk
Mean: μ
![Page 44: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/44.jpg)
Applicationtofaces
44
Lecture20
• Representing faces onto this basis
Facereconstruction:
![Page 45: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/45.jpg)
SimplestApproachtoFR
45
Lecture20
− The simplest approach is to think of it as a template matching problem
− Problems arise when performing recognition in a high-dimensional space.
− Significant improvements can be achieved by first mapping the data into a lower dimensionality space.
![Page 46: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/46.jpg)
FRusingeigenfaces
46
Lecture20
![Page 47: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/47.jpg)
FRusingeigenfaces
47
Lecture20
− The distance er is called distance within face space (difs)
− The Euclidean distance can be used to compute er, however, the Mahalanobis distance has shown to work better:
2
1|| || ( )
Kk k
i iiw w
=
Ω−Ω = −∑
Mahalanobis distance
Euclidean distance
![Page 48: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/48.jpg)
FaceDetection
48
Lecture20
(iPhoto)
![Page 49: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/49.jpg)
FaceDetection
49
Lecture20
NikonS60
![Page 50: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/50.jpg)
FaceDetection
50
Lecture20
NikonS60finds12faces…
![Page 51: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/51.jpg)
TheViola/JonesFaceDetector• Aseminalapproachtoreal-timeobjectdetection• Trainingisslow,butdetectionisveryfast• Keyideas
– Integralimages forfastfeatureevaluation– Boosting forfeatureselection– Attentional cascade forfastrejectionofnon-facewindows
51
Lecture20
P.ViolaandM.Jones.Rapidobjectdetectionusingaboostedcascadeofsimplefeatures. CVPR2001.
P.ViolaandM.Jones.Robustreal-timefacedetection. IJCV57(2),2004.
![Page 52: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/52.jpg)
TheViola/JonesFaceDetector-Training
• Initially,weighteachtrainingexampleequally• Ineachboostinground:
• Findtheweaklearner thatachievesthelowestweighted trainingerror• Raisetheweightsoftrainingexamplesmisclassifiedbycurrentweak
learner
• Computefinalclassifieraslinearcombinationofallweaklearners(weightofeachlearnerisdirectlyproportionaltoitsaccuracy)• Exactformulasforre-weightingandcombiningweaklearnersdependon
theparticularboostingscheme (e.g.,AdaBoost)
52
Lecture20
P.ViolaandM.Jones.Rapidobjectdetectionusingaboostedcascadeofsimplefeatures. CVPR2001.
P.ViolaandM.Jones.Robustreal-timefacedetection. IJCV57(2),2004.
![Page 53: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/53.jpg)
TheViola/JonesFaceDetector-Testing
53
Lecture20
• Firsttwofeaturesselectedbyboosting:
Thisfeaturecombinationcanyield100%detectionrateand50%falsepositiverate
![Page 54: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/54.jpg)
TheViola/JonesFaceDetector-Testing
54
Lecture20
• A200-featureclassifiercanyield95%detectionrateandafalsepositiverateof1in14084
Notgoodenough!
![Page 55: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/55.jpg)
Attentional Cascade
55
Lecture20
FACEIMAGESUB-WINDOW
Classifier 1T
Classifier 3T
F
NON-FACE
TClassifier 2
T
F
NON-FACE
F
NON-FACE
•Westartwithsimpleclassifierswhichrejectmanyofthenegativesub-windowswhiledetectingalmostallpositivesub-windows•Positiveresponsefromthefirstclassifiertriggerstheevaluationofasecond(morecomplex)classifier,andsoon•Anegativeoutcomeatanypointleadstotheimmediaterejectionofthesub-window
vsfalse neg determined by
% False Pos
% D
etec
tion
0 50
0
100
Receiveroperatingcharacteristic
![Page 56: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/56.jpg)
CascadedClassifiers(Boosting)
56
Lecture20
input
Base-learners
Output
![Page 57: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/57.jpg)
BoostingforFR
57
Lecture20
Weak Classifier 1
![Page 58: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/58.jpg)
BoostingforFR
58
Lecture20
WeightsIncreased
![Page 59: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/59.jpg)
BoostingforFR
59
Lecture20
Weak Classifier 2
![Page 60: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/60.jpg)
BoostingforFR
60
Lecture20
WeightsIncreased
![Page 61: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/61.jpg)
BoostingforFR
61
Lecture20
Weak Classifier 3
![Page 62: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/62.jpg)
BoostingforFR
62
Lecture20
Final classifier is a combination of weak classifiers
![Page 63: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/63.jpg)
AdaBoost Algorithm
63
Lecture20
• Given ,• Initialize• For
– For each classifier that minimizes the error with respect to the distribution
• is the weighted error rate of classifier– If , then stop – Choose , typically – Update
• where is a normalized factor (choose so that Dt+1 will sum_x=1)
1 1( , ),..., ( , )m mx y x y , { 1, 1}i ix X y Y∈ ∈ = − +
11( ) , 1,..., ,D i i mm
= =
1,...,t T=: { 1, 1}th X → − +
tD
argmint
t th H
h ε∈
= ( )[ ( )]t t i t iD i y h xε = ≠∑
0.5tε ≥t Rα ∈ 11 ln
2t
tt
εαε−=
tε th
1( ) exp( ( ))( ) t t i t i
tt
D i y h xD iZα
+−=
tZ
![Page 64: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/64.jpg)
BoostingforFR
64
Lecture20
• Define weak learners based on rectangle features
1( ) ( )
T
t tt
H x sign a h x=
⎛ ⎞= ⎜ ⎟⎝ ⎠∑
![Page 65: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/65.jpg)
Boosting&SVM• Advantages of boosting
– Integrates classification with feature selection– Complexity of training is linear instead of
quadratic in the number of training examples– Flexibility in the choice of weak learners, boosting
scheme– Testing is fast– Easy to implement
• Disadvantages– Needs many training examples– Often does not work as good as SVM
65
Lecture20
![Page 66: CAP5415-Computer Vision Lecture 20-Face …bagci/teaching/computervision16/Lec20.pdfCAP5415-Computer Vision Lecture 20-Face Recognition, HaarFeatures, Local Binary Patterns, and Boosting](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fd3e5cf3b34875cb05876c8/html5/thumbnails/66.jpg)
References&SliceCredits
66
Lecture20
• Animetrics,Dr.MarcValliant,VP&CTO• M. Turk, A. Pentland, "Eigenfaces for Recognition", Journal of Cognitive
Neuroscience, vol. 3, no. 1., 1991.• Y.FreundandR.Schapire,Ashortintroductiontoboosting,JournalofJapanese
SocietyforArtificialIntelligence,14(5):771-780,September,1999.• S.Li, et al. Handbook of Face Recognition, Springer.• Paul A. Viola and Michael J. Jones, Intl. J. Computer Vision
57(2), 137–154, 2004, (originally in CVPR’2001)• Some slides adapted from Bill Freeman, MIT 6.869, April 2005)• Friedman, J., Hastie, T. and Tibshirani, R. Additive Logistic Regression: a
Statistical View of Boostinghttp://www-stat.stanford.edu/~hastie/Papers/boost.ps