ECCV2010: distance function and metric learning part 1
Transcript of ECCV2010: distance function and metric learning part 1
![Page 1: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/1.jpg)
Distance Functions and Metric Learning:
Part 1
Michael Werman Ofir Pele
The Hebrew Universityof Jerusalem
12/1/2011 version. Updated Version:http://www.cs.huji.ac.il/~ofirpele/DFML_ECCV2010_tutorial/
![Page 2: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/2.jpg)
Rough Outline
• Bin-to-Bin Distances.
• Cross-Bin Distances:
– Quadratic-Form (aka Mahalanobis) / Quadratic-Chi.
– The Earth Mover’s Distance.
• Perceptual Color Differences.
• Hands-On Code Example.
![Page 3: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/3.jpg)
Distance ?
![Page 4: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/4.jpg)
Metric
² Non-negativity:D(P;Q) ¸ 0
² Identity of indiscernibles:D(P;Q) = 0 i®P = Q
² Symmetry:D(P;Q) = D(Q;P )
² Subadditivity (triangle inequality):D(P;Q) · D(P;K ) + D(K ;Q)
![Page 5: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/5.jpg)
² Non-negativity:D(P;Q) ¸ 0
² Property changed to:D(P;Q) = 0 if P = Q
² Symmetry:D(P;Q) = D(Q;P )
² Subadditivity (triangle inequality):D(P;Q) · D(P;K ) + D(K ;Q)
Pseudo-Metric (aka Semi-Metric)
![Page 6: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/6.jpg)
Metric
An object is most similar to itself
² Non-negativity:D(P;Q) ¸ 0
² Identity of indiscernibles:D(P;Q) = 0 i®P = Q
![Page 7: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/7.jpg)
Metric
² Symmetry:D(P;Q) = D(Q;P )
² Subadditivity (triangle inequality):D(P;Q) · D(P;K ) + D(K ;Q)
Useful for many algorithms
![Page 8: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/8.jpg)
Minkowski-Form Distances
Lp(P;Q) =
ÃX
i
jPi ¡ Qi jp! 1
p
![Page 9: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/9.jpg)
Minkowski-Form Distances
L2(P;Q) =
s X
i
(Pi ¡ Qi )2
L1(P;Q) =X
i
jPi ¡ Qi j
L1 (P;Q) = maxi
jPi ¡ Qi j
![Page 10: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/10.jpg)
Kullback-Leibler Divergence
K L(P;Q) =X
i
Pi logPi
Qi
• Information theoretic origin.
• Non symmetric.
• Qi = 0 ?
![Page 11: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/11.jpg)
Jensen-Shannon Divergence
J S(P;Q) =12K L(P;M ) +
12K L(Q;M )
M =12(P + Q)
J S(P;Q) =12
X
i
Pi log2Pi
Pi + Qi+
12
X
i
Qi log2Qi
Pi + Qi
![Page 12: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/12.jpg)
Jensen-Shannon Divergence
• Information Theoretic origin.
• Symmetric.
• is a metric.
J S(P;Q) =12K L(P;M ) +
12K L(Q;M )
M =12(P + Q)
pJ S
![Page 13: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/13.jpg)
Jensen-Shannon Divergence
• Using Taylor extension and some algebra:
J S(P;Q) =1X
n=1
12n(2n ¡ 1)
X
i
(Pi ¡ Qi )2n
(Pi + Qi )2n¡ 1
=12
X
i
(Pi ¡ Qi )2
(Pi + Qi )+
112
X
i
(Pi ¡ Qi )4
(Pi + Qi )3+ :::
![Page 14: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/14.jpg)
Histogram Distance Â2
• Statistical origin.
• Experimentally results are very similar to JS.
• Reduces the effect of large bins.
• is a metric
Â2(P;Q) =12
X
i
(Pi ¡ Qi )2
(Pi + Qi )
pÂ2
![Page 15: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/15.jpg)
Histogram Distance Â2
<Â2( ; )Â2( ; )
![Page 16: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/16.jpg)
Histogram Distance Â2
>L1( ; ) L1( ; )
![Page 17: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/17.jpg)
Histogram Distance Â2
• Experimentally better than .L2
Â2(P;Q) =12
X
i
(Pi ¡ Qi )2
(Pi + Qi )
![Page 18: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/18.jpg)
L1( ; )
Bin-to-Bin Distances
• Bin-to-Bin distances such as L2 are sensitive to quantization:
L1;L2;Â2
> L1( ; )
![Page 19: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/19.jpg)
robustness distinctiveness
robustness distinctiveness
• #bins
• #bins
Bin-to-Bin Distances
>L1( ; ) L1( ; )
![Page 20: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/20.jpg)
robustness distinctiveness
robustness distinctiveness
• #bins
• #bins
Bin-to-Bin Distances
Can we achieve robustnessrobustness
and
distinctivenessdistinctiveness ?
![Page 21: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/21.jpg)
The Quadratic-Form Histogram Distance
• is the similarity between bin i and j.
• If is the inverse of the covariance matrix, QF is called Mahalanobis distance.
A i j
=
s X
i j
(Pi ¡ Qi )(Pj ¡ Qj )A i j
QF A (P;Q) =q
(P ¡ Q)T A(P ¡ Q)
A
![Page 22: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/22.jpg)
The Quadratic-Form Histogram Distance
QF A (P;Q) =q
(P ¡ Q)T A(P ¡ Q)
=
s X
i j
(Pi ¡ Qi )(Pj ¡ Qj )A i j
A = I
=
s X
i j
(Pi ¡ Qi )2 = L2(P;Q)
![Page 23: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/23.jpg)
The Quadratic-Form Histogram Distance
• Does not reduce the effect of large bins.
• Alleviates the quantization problem.
• Linear time computation in # non zero . A i j
QF A (P;Q) =q
(P ¡ Q)T A(P ¡ Q)
![Page 24: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/24.jpg)
The Quadratic-Form Histogram Distance
• If A is positive-semidefinite then QF is a pseudo-metric.
QF A (P;Q) =q
(P ¡ Q)T A(P ¡ Q)
![Page 25: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/25.jpg)
² Non-negativity:D(P;Q) ¸ 0
² Property changed to:D(P;Q) = 0 if P = Q
² Symmetry:D(P;Q) = D(Q;P )
² Subadditivity (triangle inequality):D(P;Q) · D(P;K ) + D(K ;Q)
Pseudo-Metric (aka Semi-Metric)
![Page 26: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/26.jpg)
The Quadratic-Form Histogram Distance
• If A is positive-definite then QF is a metric.
QF A (P;Q) =q
(P ¡ Q)T A(P ¡ Q)
![Page 27: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/27.jpg)
The Quadratic-Form Histogram Distance
QF A (P;Q) =q
(P ¡ Q)T A(P ¡ Q)
=q
(P ¡ Q)T WT W(P ¡ Q)
= L2(WP;WQ)
![Page 28: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/28.jpg)
The Quadratic-Form Histogram Distance
QF A (P;Q) =q
(P ¡ Q)T A(P ¡ Q)
= L2(WP;WQ)
We assume there is a linear transformationlinear transformation
that makes bins independent independent
There are cases where this is not truenot true
e.g. COLORCOLOR
![Page 29: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/29.jpg)
The Quadratic-Form Histogram Distance
QF A (P;Q) =q
(P ¡ Q)T A(P ¡ Q)
• Converting distance to similarity (Hafner et. al 95):
A i j = 1¡D i j
maxi j (D i j )
![Page 30: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/30.jpg)
The Quadratic-Form Histogram Distance
QF A (P;Q) =q
(P ¡ Q)T A(P ¡ Q)
• Converting distance to similarity (Hafner et. al 95):
A i j = e¡ ® D ( i ; j )maxi j (D ( i ; j ) )
If ®is large enough,A will be positive-de nitive
![Page 31: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/31.jpg)
The Quadratic-Form Histogram Distance
QF A (P;Q) =q
(P ¡ Q)T A(P ¡ Q)
• Learning the similarity matrix: part 2 of this tutorial.
![Page 32: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/32.jpg)
• is the similarity between bin i and j.
• Generalizes and .
• Reduces the effect of large bins.
• Alleviates the quantization problem.
• Linear time computation in # non zero .
The Quadratic-Chi Histogram Distance
Â2
QCAm(P;Q) =
vuut
X
i j
(Pi ¡ Qi )(Pj ¡ Qj )Ai j
(P
c(Pc +Qc)Aci )m(P
c(Pc +Qc)Acj )m
A i j
A i j
QF
![Page 33: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/33.jpg)
QCAm(P;Q) =
vuut
X
i j
(Pi ¡ Qi )(Pj ¡ Qj )Ai j
(P
c(Pc +Qc)Aci )m(P
c(Pc +Qc)Acj )m
The Quadratic-Chi Histogram Distance
• is non-negative if A is positive-semidefinite.
• Symmetric.
• Triangle inequality unknown.
• If we define and , QC is continuous. 0· m < 100 = 0
![Page 34: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/34.jpg)
Similarity-Matrix-Quantization-Invariant Property
( , )D )( , = D
![Page 35: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/35.jpg)
=
Sparseness-Invariant
D( , )D )( ,
Empty bins
![Page 36: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/36.jpg)
f unct i on di st= Quadrat i cChi (P, Q, A, m)
Z= (P+Q)*A;%1 can be any number as Z_i ==0 i f f D_i =0Z(Z==0)= 1;Z= Z. m;D= (P- Q) . / Z;%max i s redundant i f A i s%posi t i ve- semi def i ni tedi st= sqrt ( max(D*A*D' , 0) ) ;
The Quadratic-Chi Histogram Distance Code
![Page 37: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/37.jpg)
f unct i on di st= Quadrat i cChi (P, Q, A, m)
Z= (P+Q)*A;%1 can be any number as Z_i ==0 i f f D_i =0Z(Z==0)= 1;Z= Z. m;D= (P- Q) . / Z;%max i s redundant i f A i s%posi t i ve- semi def i ni tedi st= sqrt ( max(D*A*D' , 0) ) ;
The Quadratic-Chi Histogram Distance Code
TimeComplexity: O(#A i j 6= 0)
![Page 38: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/38.jpg)
f unct i on di st= Quadrat i cChi (P, Q, A, m)
Z= (P+Q)*A;%1 can be any number as Z_i ==0 i f f D_i =0Z(Z==0)= 1;Z= Z. m;D= (P- Q) . / Z;%max i s redundant i f A i s%posi t i ve- semi def i ni tedi st= sqrt ( max(D*A*D' , 0) ) ;
The Quadratic-Chi Histogram Distance Code
What about sparse (e.g. BoW) histograms ?
![Page 39: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/39.jpg)
The Quadratic-Chi Histogram Distance Code
0 0 0 0 0 00 00 4 50 -3
P ¡ Q =
What about sparse (e.g. BoW) histograms ?
TimeComplexity: O(SK )
![Page 40: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/40.jpg)
The Quadratic-Chi Histogram Distance Code
0 0 0 0 0 00 00 4 50 -3
P ¡ Q =
#(P 6= 0) + #(Q 6= 0)
What about sparse (e.g. BoW) histograms ?
TimeComplexity: O(SK )
![Page 41: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/41.jpg)
Averageof non-zero entries
The Quadratic-Chi Histogram Distance Code
0 0 0 0 0 00 00 4 50 -3
P ¡ Q =
in each row of A
What about sparse (e.g. BoW) histograms ?
TimeComplexity: O(SK )
![Page 42: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/42.jpg)
The Earth Mover’s Distance
![Page 43: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/43.jpg)
The Earth Mover’s Distance
• The Earth Mover’s Distance is defined as the minimal cost that must be paid to transform one histogram into the other, where there is a “ground distance” between the basic features that are aggregated into the histogram.
![Page 44: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/44.jpg)
The Earth Mover’s Distance
≠
![Page 45: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/45.jpg)
The Earth Mover’s Distance
=
![Page 46: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/46.jpg)
EM DD (P;Q) = minF =f F i j g
X
i ;j
F i j Di j
s:t :X
j
F i j = Pi
X
i
F i j = Qj
X
i ;j
F i j =X
i
Pi =X
j
Qj = 1
F i j ¸ 0
The Earth Mover’s Distance
![Page 47: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/47.jpg)
The Earth Mover’s Distance
E M DD (P;Q) = minF =f F i j g
Pi ;j F i j Di jP
i F i j
s:t :X
j
F i j · Pi
X
i
F i j · Qj
X
i ;j
F i j = min(X
i
Pi ;X
j
Qj )
F i j ¸ 0
![Page 48: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/48.jpg)
• Pele and Werman 08 – , a new EMD definition.
The Earth Mover’s Distance
\E M D
![Page 49: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/49.jpg)
Definition:
\EM DD
C (P;Q) = minF =f F i j g
X
i ;j
F i j Di j +
¯¯¯¯¯¯
X
i
Pi ¡X
j
Qj
¯¯¯¯¯¯£ C
s:t :X
j
F i j · Pi
X
i
F i j · Qj
X
i ;j
F i j = min(X
i
Pi ;X
j
Qj ) F i j ¸ 0
![Page 50: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/50.jpg)
Supplier
E M D
Demander
Di j = 0
PDemander ·
PSupplier
![Page 51: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/51.jpg)
Demander Supplier
PDemander ·
PSupplier
Di j = C
\E M D
![Page 52: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/52.jpg)
When to Use
• When the total mass of two histograms is important.
E M D¡
;¢
= E M D¡
;¢
\E M D
![Page 53: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/53.jpg)
When to Use
• When the total mass of two histograms is important.
¡;
¢<
¡;
¢
E M D¡
;¢
= E M D¡
;¢
\E M D
\E M D \E M D
![Page 54: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/54.jpg)
When to Use
• When the difference in total mass between histograms is a distinctive cue.
E M D¡
;¢
= E M D¡
;¢= 0
\E M D
![Page 55: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/55.jpg)
E M D¡
;¢
= E M D¡
;¢
When to Use
• When the difference in total mass between histograms is a distinctive cue.
= 0
\E M D
¡;
¢<
¡;
¢\E M D \E M D
![Page 56: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/56.jpg)
When to Use
• If ground distance is a metric:
² E M D is a metric only for normalized histograms.
² \E M D is a metric for all histograms (C ¸ 12¢ ).
\E M D
![Page 57: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/57.jpg)
C = 1Di j =
(0 if i = j2 otherwise
\EM DD
C (P;Q) = minF =f F i j g
X
i ;j
F i j Di j +
¯¯¯¯¯¯
X
i
Pi ¡X
j
Qj
¯¯¯¯¯¯£ C
² \E M D = L1 if:
\E M D - a Natural Extension to L1
![Page 58: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/58.jpg)
The Earth Mover’s Distance Complexity Zoo
• General ground distance:
– Orlin 88
• normalized 1D histograms
– Werman, Peleg and Rosenfeld 85
L1
O(N 3 logN )
O(N )
![Page 59: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/59.jpg)
The Earth Mover’s Distance Complexity Zoo
• normalized 1D cyclic histograms
– Pele and Werman 08 (Werman, Peleg, Melter, and Kong 86)
L1 O(N )
![Page 60: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/60.jpg)
O(N 2 logN (D + logN))
The Earth Mover’s Distance Complexity Zoo
• Manhattan grids
– Ling and Okada 07
L1
![Page 61: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/61.jpg)
• What about N-dimensional histograms with a cyclic dimensions?
The Earth Mover’s Distance Complexity Zoo
![Page 62: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/62.jpg)
• What about N-dimensional histograms with a cyclic dimensions?
The Earth Mover’s Distance Complexity Zoo
![Page 63: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/63.jpg)
• What about N-dimensional histograms with a cyclic dimensions?
The Earth Mover’s Distance Complexity Zoo
![Page 64: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/64.jpg)
The Earth Mover’s Distance Complexity Zoo
• general histograms
– Gudmundsson, Klein, Knauer and Smid 07
L1
p1p2
p3p4
p5p6
p7
p8
l2 l1 l`2
O(N 2 log2D ¡ 1 N)
![Page 65: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/65.jpg)
The Earth Mover’s Distance Complexity Zoo
• 1D linear/cyclic histograms
– Pele and Werman 08
min(L1;2) O(N )f1;2;::: ;¢ g
![Page 66: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/66.jpg)
The Earth Mover’s Distance Complexity Zoo
• 1D linear/cyclic histograms
– Pele and Werman 08
min(L1;2) O(N )f1;2;::: ;¢ g
![Page 67: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/67.jpg)
The Earth Mover’s Distance Complexity Zoo
• general histograms
is the number of edges with cost 1
– Pele and Werman 08
min(L1;2) f1;2;::: ;¢ gD
K O(N 2K log(NK ))
![Page 68: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/68.jpg)
The Earth Mover’s Distance Complexity Zoo
• Any thresholded distance
– Pele and Werman 09
O(N 2 logN (K + logN))
number of edges with cost
different from the threshold
K = O(logN)O(N 2 log2 N)
![Page 69: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/69.jpg)
• EMD with a thresholded ground distance is
not an approximation of EMD.
• It has better performance.
Thresholded Distances
![Page 70: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/70.jpg)
The Flow Network Transformation
OriginalNetwork
SimplifiedNetwork
![Page 71: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/71.jpg)
The Flow Network Transformation
OriginalNetwork
SimplifiedNetwork
![Page 72: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/72.jpg)
The Flow Network Transformation
Flowing the Monge sequence (if ground distance is a metric,
zero-cost edges are a Monge sequence)
![Page 73: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/73.jpg)
The Flow Network Transformation
Removing Empty Binsand their edges
![Page 74: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/74.jpg)
The Flow Network Transformation
We actually finished here….
![Page 75: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/75.jpg)
Combining Algorithms
• EMD algorithms can be combined.
• For example :L1
![Page 76: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/76.jpg)
Combining Algorithms
• EMD algorithms can be combined.
• For example, thresholded :L1
![Page 77: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/77.jpg)
The Earth Mover’s Distance Approximations
![Page 78: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/78.jpg)
• Charikar 02, Indyk and Thaper 03 – approximated EMD on by embedding it into the norm.
Time complexity:
Distortion (in expectation):
The Earth Mover’s Distance Approximations
L1
f1;: :: ;¢ gd
O(TNdlog¢ )
O(dlog¢ )
![Page 79: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/79.jpg)
The Earth Mover’s Distance Approximations
• Grauman and Darrell 05 – Pyramid Match Kernel (PMK) same as Indyk and Thaper, replacing with histogram intersection.
• PMK approximates EMD with partial matching.
• PMK is a mercer kernel.
• Time complexity & distortion – same as Indyk and Thaper (proved in Grauman and Darrell 07).
L1
![Page 80: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/80.jpg)
The Earth Mover’s Distance Approximations
• Lazebnik, Schmid and Ponce 06 – used PMK in the spatial domain (SPM).
![Page 81: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/81.jpg)
The Earth Mover’s Distance Approximations
+
+++
+
+
+++
++
++
+++
+
+
+++
++
+
x1/8 x1/4 x1/2
+
+
+++
+
+
+++
++
+
+ +
level 0 level 1 level 2+
+++
+
+
+++
++
+ +
+++
+
+
+++
++
+
![Page 82: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/82.jpg)
The Earth Mover’s Distance Approximations
![Page 83: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/83.jpg)
The Earth Mover’s Distance Approximations
Difference
P
Q
Wavelet
Transform j=0
j=1
Absolutevalues
+
• Shirdhonkar and Jacobs 08 - approximated EMD using the sum of absolute values of the weighted wavelet coefficients of the difference histogram.
£2¡ 2£ 0
£2¡ 2£ 1
![Page 84: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/84.jpg)
The Earth Mover’s Distance Approximations
• Khot and Naor 06 – any embedding of the EMD over the d-dimensional Hamming cube into must incur a distortion of .
• Andoni, Indyk and Krauthgamer 08 - for sets with cardinalities upper bounded by a parameter the distortion reduces to .
• Naor and Schechtman 07 - any embedding of the EMD over must incur a distortion of .
L1
(d)
sO(logslogd)
f0;1;::: ;¢ g2
(p
log¢ )
![Page 85: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/85.jpg)
Robust Distances
• Very high distances outliers same difference.
![Page 86: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/86.jpg)
Robust Distances
• With colors, the natural choice.
00
20
40
60
80
100
120C
IED
E20
0 D
ista
nce
from
Blu
e
![Page 87: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/87.jpg)
Robust Distances
(blue;red) = 56
¢E 00(blue;yellow) = 102
¢E 00
![Page 88: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/88.jpg)
Robust Distances - Exponent
• Usually a negative exponent is used:
² Let d(a;b) be a distance measure between twofeatures - a;b.
² The negative exponent distance is:
de(a;b) = 1¡ e¡ d(a ;b)
¾
![Page 89: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/89.jpg)
Robust Distances - Exponent
• Exponent is used because (Ruzon and Tomasi 01):
robust, smooth, monotonic, and a metric
Input is always discrete anyway …
![Page 90: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/90.jpg)
Robust Distances - Thresholded
² Let d(a;b) be a distance measure between twofeatures - a;b.
² The thresholded distance with a thresholdof t > 0 is:
dt(a;b) = min(d(a;b);t).
![Page 91: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/91.jpg)
Thresholded Distances
• Thresholded metrics are also metrics (Pele and Werman ICCV 2009).
• Better results.
• Pele and Werman ICCV 2009 algorithm computes EMD with thresholded ground distances much faster.
• Thresholded distance corresponds to sparse similarities matrix -> faster QC / QF computation.
A i j = 1¡D i j
maxi j (D i j )
![Page 92: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/92.jpg)
• Thresholded vs. exponent:
– Fast computation of cross-bin distances with a thresholded ground distance.
– Exponent changes small distances – can be a problem (e.g. color differences).
Thresholded Distances
![Page 93: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/93.jpg)
Thresholded Distances
• Color distance should be thresholded (robust).
0
50
100
150
dist
ance
fro
m b
lue
¢ E00min(¢ E00;10)10(1¡ e
¡ ¢ E 004 )
10(1¡ e¡ ¢ E 00
5 )
![Page 94: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/94.jpg)
Thresholded Distances
¢ E00min(¢ E00;10)10(1¡ e
¡ ¢ E 004 )
10(1¡ e¡ ¢ E 00
5 )
0
5
10
dist
ance
fro
m b
lue
Exponent changes small distances
![Page 95: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/95.jpg)
A Ground Distance for SIFT
dR =jj(xi ;yi ) ¡ (xj ;yj )jj2+
min(joi ¡ oj j;M ¡ joi ¡ oj j)
dT =min(dR ;T)
The ground distance between two SIFT bins(xi ;yi ;oi ) and (xj ;yj ;oj ):
![Page 96: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/96.jpg)
A Ground Distance for Color Image
The ground distances between two LAB image bins(xi ;yi ;L i ;ai ;bi ) and (xj ;yj ;L j ;aj ;bj ) we use are:
dcT = min((jj(xi ;yi ) ¡ (xj ;yj )jj2)+
¢ 00((L i ;ai ;bi ); (L j ;aj ;bj ));T)
![Page 97: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/97.jpg)
Perceptual Color Differences
![Page 98: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/98.jpg)
Perceptual Color Differences
• Euclidean distance on L*a*b* space is widely considered as perceptual uniform.
0
100
200
300
dist
ance
fro
m b
lue jjLabjj2
![Page 99: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/99.jpg)
Perceptual Color Differences
0
20
40
60adist2jjLabjj2
dist
ance
fro
m b
lue
Purples before blues
![Page 100: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/100.jpg)
Perceptual Color Differences
• on L*a*b* space is better.
Luo, Cui and Rigg 01. Sharma, Wu and Dalal 05.
dist
ance
fro
m b
lue
0
5
10
15
20¢ E00
¢ E00
![Page 101: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/101.jpg)
• on L*a*b* space is better.¢ E00
Perceptual Color Differences
jjLabjj2
¢ E00
![Page 102: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/102.jpg)
Perceptual Color Differences
0
50
100
150
dist
ance
fro
m b
lue ¢ E00
• on L*a*b* space is better.¢ E00
![Page 103: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/103.jpg)
Perceptual Color Differences• on L*a*b* space is better.
• But still has major problems.
• Color distance should be thresholded (robust).
(blue;red) = 56
¢E 00(blue;yellow) = 102
¢E 00
¢ E00
![Page 104: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/104.jpg)
Perceptual Color Differences
• Color distance should be saturated (robust).
0
50
100
150
dist
ance
fro
m b
lue
¢ E00min(¢ E00;10)10(1¡ e
¡ ¢ E 004 )
10(1¡ e¡ ¢ E 00
5 )
![Page 105: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/105.jpg)
Thresholded Distances
¢ E00min(¢ E00;10)10(1¡ e
¡ ¢ E 004 )
10(1¡ e¡ ¢ E 00
5 )
0
5
10
dist
ance
fro
m b
lue
Exponent changes small distances
![Page 106: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/106.jpg)
Perceptual Color Descriptors
![Page 107: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/107.jpg)
•11 basic color terms. Berlin and Kay 69.
Perceptual Color Descriptors
white red green yellow blue brown
purple pink orange grey
black
![Page 108: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/108.jpg)
•11 basic color terms. Berlin and Kay 69.
Perceptual Color Descriptors
![Page 109: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/109.jpg)
•11 basic color terms. Berlin and Kay 69.
Perceptual Color Descriptors
• Image copyright by Eric Rolph. Taken from: http://upload.wikimedia.org/wikipedia/commons/5/5c/Double-alaskan-rainbow.jpg
![Page 110: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/110.jpg)
•11 basic color terms. Berlin and Kay 69.
Perceptual Color Descriptors
![Page 111: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/111.jpg)
•How to give each pixel an “11-colors” description ?
Perceptual Color Descriptors
![Page 112: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/112.jpg)
• Learning Color Names from Real-World Images, J. van de Weijer, C. Schmid, J. Verbeek CVPR 2007.
Perceptual Color Descriptors
![Page 113: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/113.jpg)
• Learning Color Names from Real-World Images, J. van de Weijer, C. Schmid, J. Verbeek CVPR 2007.
Perceptual Color Descriptors
![Page 114: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/114.jpg)
• Learning Color Names from Real-World Images, J. van de Weijer, C. Schmid, J. Verbeek CVPR 2007.
Perceptual Color Descriptors
chip-based real-world
![Page 115: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/115.jpg)
• Learning Color Names from Real-World Images, J. van de Weijer, C. Schmid, J. Verbeek CVPR 2007.
• For each color returns a probability distribution over the 11 basic colors.
Perceptual Color Descriptors
white red green yellow blue brown
purple pink orange grey
black
![Page 116: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/116.jpg)
• Applying Color Names to Image Description, J. van de Weijer, C. Schmid ICIP 2007.
• Outperformed state of the art color descriptors.
Perceptual Color Descriptors
![Page 117: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/117.jpg)
• Using illumination invariants – black, gray and white are the same.
• “Too much invariance” happens in other cases (Local features and kernels for classification of texture and object categories: An in-depth study - Zhang, Marszalek, Lazebnik and Schmid. IJCV 2007, Learning the discriminative power-invariance trade-off - Varma and
Ray. ICCV 2007).
• To conclude: Don’t solve imaginary problems.
Perceptual Color Descriptors
![Page 118: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/118.jpg)
• This method is still not perfect.
• 11 color vector for purple(255,0,255) is:
• In real world images there are no such
over-saturated colors.
Perceptual Color Descriptors
0 0 00 0 0 00010
![Page 119: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/119.jpg)
• EMD variant that reduces the effect of large bins.
• Learning the ground distance for EMD.
• Learning the similarity matrix and normalization factor for QC.
Open Questions
![Page 120: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/120.jpg)
Hands-On Code Example
http://www.cs.huji.ac.il/~ofirpele/FastEMD/code/
http://www.cs.huji.ac.il/~ofirpele/QC/code/
![Page 121: ECCV2010: distance function and metric learning part 1](https://reader035.fdocuments.in/reader035/viewer/2022062405/555093ccb4c905235b8b537f/html5/thumbnails/121.jpg)
Tutorial:
Or “Ofir Pele”
http://www.cs.huji.ac.il/~ofirpele/DFML_ECCV2010_tutorial/