Learning Visual Object Categories with Global Descriptors and ...
Evaluation Of Color Descriptors For Object And Scene
-
Upload
shao-chuan-wang -
Category
Technology
-
view
2.452 -
download
0
Transcript of Evaluation Of Color Descriptors For Object And Scene
![Page 1: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/1.jpg)
Evaluation of Color Descriptors for Object and Scene Recognition
Authors: Koen E. A. van de Sande, Theo Gevers, and Cees G. M. Snoek @
University of AmsterdamPresenter: Shao-Chuan Wang
![Page 2: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/2.jpg)
Evaluation of Color Descriptors for Object and Scene Recognition
• Focus: Color features/descriptors on obj. and scene recognition
• Summary: The invariance of photometric transform aces and its effect on discriminative power.
• Conclusion: The usefulness of invariance is category-specific!
![Page 3: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/3.jpg)
Photometric transforms (1/2)
• Light intensity scale invariant
• Light intensity shift invariant
B
G
R
a
a
a
00
00
00
3
2
1
o
o
o
B
G
R
![Page 4: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/4.jpg)
Photometric transforms (1/2)
• Light intensity scale and shift invariant
• Light color change
• Light color change and shift
3
2
1
00
00
00
o
o
o
B
G
R
a
a
a
B
G
R
c
b
a
00
00
00
3
2
1
00
00
00
o
o
o
B
G
R
c
b
a
![Page 5: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/5.jpg)
Color Descriptors (1/1)
• Histograms – RGB, Hue, Saturation, rgHistogram, …
• Color Moments: contain spatial info.
• Color SIFT: combined color and SIFT– HSV-SIFT, Hue-SIFT, …
dxdyyxIyxIyxIyxM cB
bG
aR
qpabcpq )],([)],([)],([
![Page 6: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/6.jpg)
Color Histograms (1/2)
• RGB-histogram• Hue-histogram– H and S are scale-invariant and
shift-invariant w.r.t light intensity
• rg-histogram– Scale-invariant– Not shift-invariant– Not that b is redudant
BGRB
BGRG
BGRR
b
g
r
Image from wikipedia
![Page 7: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/7.jpg)
Color Histograms (2/2)
• Transformed color– Scale and shift-invariant w.r.t
light intensity.
• Opponent color histogram– O1,O2 shift invariant– O3: intensity, no invariant
B
B
G
G
R
R
B
G
R
B
G
R
3
622
3
2
1
BGR
BGR
GR
O
O
O
![Page 8: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/8.jpg)
Color SIFT Descriptors (1/3)
• HSV-SIFT– SIFT over HSV channels– Hue is unstable in gray axis
• Hue-SIFT (Van de Weijer 2006)
– Used Hue histogram weighing by its saturation
– Concatenate Hue histogram with SIFT– Only SIFT is invariant; Hue histogram is not!
(partial invariance)
Hue Instability
![Page 9: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/9.jpg)
Color SIFT Descriptors (2/3)
• OpponentSIFT– SIFT over all channels in the opponent color space.– Shift-invariant to light intensity.
• W-SIFT– Eliminate O1 and O2’s intensity information– Scale-invariant to light intensity
• rg-SIFT– SIFT over r,g spaces– Scale and shift invariant, but not invariant to light color
changes/shifts
32
31
2
1
OO
OO
O
O
![Page 10: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/10.jpg)
Color SIFT Descriptors (3/3)
• Transformed color SIFT– SIFT over normalized transformed channels.– Scale- and shift-invariant to light color changes
and shift.
![Page 11: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/11.jpg)
Experiments
• Implementation:– Scale-invariants points are detected by Harris-
Laplace point detectors– Color descriptors are computed over the area
around the points; all regions are proportionally re-sampled to a uniform 60x60 patch.
– Cluster descriptors with k = 40 (images) k = 4000 (video)
– SVM classifier with EMD/chi-square kernel
![Page 12: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/12.jpg)
Benchmark (1/3)
• Image: PASCAL VOC 2007, over 20 object categories
![Page 13: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/13.jpg)
Benchmark (2/3)
• Most objs were categorized better under scale- and shift- invariant to light intensity
• Some, such as car and dining table, do not benefit from such invariance.
![Page 14: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/14.jpg)
Benchmark (3/3)
• Video: Mediamill Challenge, 39 object and scene categories
![Page 15: Evaluation Of Color Descriptors For Object And Scene](https://reader036.fdocuments.in/reader036/viewer/2022062405/556104add8b42a424d8b58aa/html5/thumbnails/15.jpg)
Evaluation of Color Descriptors for Object and Scene Recognition
• Conclusion:– W-SIFT and rgSIFT, in general, outperform other
color descriptors.– Light intensity info. Is important for some
categories– Usefulness of invariance is category-specific.