Ipta2010

Binary Pattern Matching from a Local Dissimilarity Measure

F. Morain-Nicolier - Jérôme Landré - Su RuanCReSTIC - URCA - IUT Troyes - FRANCE

http://pixel-shaker.frIPTA 2010

1

1mercredi 7 juillet 2010

2

?


Outline

• Local Dissimilarity Measure

• LDM as Shape Matcher

• Results

3


Local Dissimilarity Map

4

E. Baudrier, F. Nicolier, G. Millon, S. Ruan, ”Binary-image comparison with local-dissimilarity quantification”, Pattern Recognition, vol. 41, n. 5, pp. 1461–1478, jan. 2008


HD(A,B) = max(h(A,B), h(B,A))

h(A,B) = maxa∈A

(minb∈B

d(a, b))with

5

Hausdorff Distance


6

1468 É. Baudrier et al. / Pattern Recognition 41 (2008) 1461–1478

Fig. 3. Behavior of the LDMap on simple patterns. A vertical line, a horizontal one, a square and their LDMaps. The darker the pixel, the higher the distancemeasure.

Fig. 4. Letters “co” et “et” and their LDMap. The obtained LDMap (c) shows clearly both locations and quantification of the dissimilarities.

Fig. 5. The ten-error game. The two images to compare (A and B), the absolute difference C = |B ! A| and their LDMap (D) where we have circled theerrors in black.




7








8








LDMA,B(p) = |A(p)−B(p)|max(dtA(p), dtB(p))

Iterative algorithm = slow

9


: distance transform(distance to nearest foreground pixel)

dt



LDMA,B(p) = |A(p)−B(p)|max(dtA(p), dtB(p))

For binary images :

LDMA,B = BdtA +AdtB

10



11

1468 É. Baudrier et al. / Pattern Recognition 41 (2008) 1461– 1478




Localized and quantified


corresponds to optimal straight-line fitted to the local powerspectrum. Its orientation gives the dominant orientation ofthe local neighborhood. The obtained vectors are next usedto build orientation images in predefined directions. Radio-grams are obtained by projecting the corresponding orien-tation images along their pass orientations. Figures 3. (b)(c) gives examples of radiograms computed from the image(a). Fourier coefficients are next computed for each radio-gram. The coefficient vectors are compared with an Eu-clidean distance to obtain similarity measures between thecorresponding radiograms. This process is fast enough to berun online on large databases5. Its drawback is the scalinginvariance. Symmetry vectors are obtained by local com-putation involving to use images at a same resolution level(around 200 dpi in Passe-Partout4). Another drawback isthe specificity of the radiograms to fleuron ornaments. Asan example, straight-lines are difficult to detect within im-ages of initials, mainly composed of textures.

The system of [6] is employed for the retrieval of em-blem images. In order to reduce the time processing of sucha comparison, this system works with points of interest ex-tracted from images. The global score of similarity will cor-respond to the number of similar points between two im-ages, using feature vectors computed locally. The pointsare first extracted using a modified Harris detector. Then,Zernike moments are computed locally from each of them.These moments are compared using a maximum likelihoodestimation and a T threshold i.e. when the estimation doesnot overflow T the two points match. To limit the number ofcomparisons, only the points having near coordinates (witha precision of 5 pixels) are compared. Figure 4 gives anexample of retrieval result (b) using the query (a).

Figure 4. Example of retrieval result of [6](a) query image (b) 2nd, 4th and 6th results

Like this, this method reduces the whole complexity ofcomparison by considering only some points of interest.However, it is not scale invariant. Local templates are usedat different steps of the process (with the Harris detector, tocompute the Zernike moments and to compare point coor-dinates). Another problem of this method is the complexityof comparison. When the number of points becomes im-portant, it could take time to match them together. At last,retrieval precision will depend a lot on the stability of de-tected points. As an example, smallest images will have

5The system can be tested from the Passe-Partout website4.

less stable points. This makes the method more adapted tobig ornament images, like emblems.

In [2] the authors propose an alternative approach tothe previous ones. In order to make their retrieval pro-cess accurate, they use the full pixel information to com-pare images. However, to address the complexity problemrelated to such a comparison, they employ a multi resolu-tion approach. The multi-resolution permits them to reducethe whole complexity of their process by reducing imagesizes. A scaling factor (from 1

1 to 116 ) is determined in

a semi-automatic way by an human expert, according tosome training results obtained on the image corpus. Thescaled images are next compared using a Hausdorff dis-tance computed locally. This local measure results in alocal-dissimilarity map including the dissimilarity spatiallayout, and can be achieved within a linear complexity. Ina last step, the authors employ a classification step basedon a Support Vector Machine classifier to compute similar-ity scores between images. Figure 5 gives an example ofcomputed local-dissimilarity map6 (c) from two images.

Figure 5. Local-dissimilarity map of [2](a) image 1 (b) image 2 (c) dissimilarity map

The main characteristic of the method is a comparisonof images without any feature extraction, a high precisionis obtained about image differences. On the other hand, themethod increases computation times and can’t be used on-line. Scalability of the method is unknown as it is not testedon large database. The last point concerns scale invariance.As scale variations are not taken into account, variations inresolution will mislead the classification.

The authors in [8] employ also pixel information to com-pare the images. However, they propose to address the com-plexity by using a Run Length Encoding (RLE) of images.RLE encodes successive pixels of same intensity into a sin-gle object as illustrated Figure 6. (a). It is a lossless com-pression technic, but working from binary versions of im-ages. In their experiments, they obtain a mean compressionrate of 0.88 on a database of initials. This reduces thereforethe needed time for retrieval. Figure 6. (b) gives some ex-amples of compression rates. RLE is used in a retrieval pro-cess working in two steps: centering and comparison. Thecentering computes dx dy offsets to align two images to-gether, to make more accurate their comparison. The com-parison next computes a distance between the two images

6The bright parts correspond to low distances.

12

Localized and quantified


Mesures dedissimilarites locales etglobales entre images,

symetriques etasymetriques

F. Morain-Nicolier

[email protected]

http://pixel-shaker.fr

Plan

Presentation de laCarte de DissimilariteLocales

Mesure locale⇒mesure globale

Formulations finale -Exemples

Bilan - discussion

Exemples et proprietes

Proprietes :localisation, quantification, robustesse aux petites variations.

6

13

Good for non strongly textured images


LDM as template matcher?

14


15

P reference shapeI binary image

find the position in of an instance of I P

use LDM as similarity measure


16

P

I

I(x,y)

x

y PI(x,y)

to be compared!

LDMA,B = BdtA +AdtB


17

PI(x,y)

MI,P (x, y) =�

k

�

l

LDMI(x,y),P (k, l)

MI,P (x, y) = NCSI,P (x, y) +NCSP,I(x, y)

matcher (first version)

Symetrical Chamfer Matching [Borgefors]


18

use quadratic sums to get less false positives [Borgefors]

LDMA,B = BdtA +AdtB

and

QMI,P = dt2I � P + I � dt2P

Quadratic Matcher(very fast computation via Fourier)

�( : cross-correlation)

matcher(final

version)


Generalization (not used here)

19

QMI,P = dt2I � P + I � dt2P

AQMI,P = αdt2I � P + βI � dt2P

Links to Tverksy results : human judgement of similarity is asymmetric

s(A,B) = F (A�

B)− αF (A−B)− βF (B −A)


An example

20


21

Fig. 1. (a) A test image. – (b) a reference pattern, the ideal location in (a)is labeled with G.

that belong to b but not to a. A comparable idea can beformulated from (10) by weighting the contribution of eachterm (Generalized Quadratic Matcher - GQM):

AQMI,P = αdt2I � P + βI � dt2P . (11)

A symmetric matcher is obtained with α = β. Asymmetricones are obtained with α �= β, such as α = 1,β = 0 (thechamfer matching) or α = 0,β = 1.

IV. THE LDM-MATCHER CAN PROVIDE LESS FALSEPOSITIVES

The final LDM-matcher (eq. (11)) (symmetric one, withα = β = 1) is compared in this section to the Borgeforschamfer matcher (purely asymmetric, with α = 1 and β = 0).The behavior of the two matchers is compared in a concreteexample.

Coast image edges are given in figure 1. A test pattern to belocalized in (a) is given in (b). The ideal response of a matcheris marked by label G in 1(a). The Borgefors chamfer matchingresponse is given in figure 2(c) and the LDM-matcher responsein figure 2(d). As these two matchers are based on dissimilaritymeasures, good matches between the reference pattern and thelocal information in the image are indicated by low values(dark tones in the figures).

At first, a good localization is achieved by the two matchers.In images of figures 2(c) and 2(d), the absolute minimumvalue corresponds to coordinates x = 88 and y = 398 (x-axisis horizontal and y-axis is vertical). Theses positions matchperfectly the ideal position labeled G in fig. 1(a).

Positive responses (i.e. coordinates where an instance ofP can be found) are given by low values (possibly at localminima). For example, low values are obtained with Borgeforschamfer matching for pixels in the area labeled N in fig.1(a), in the large dark area in fig. 2(c). A decision based onlow values could conclude that the pattern P can be found

Fig. 2. In (c), image response by Borgefors chamfer matcher. – In (d), imageobtained with symmetric LDM-matcher. – A good match with the referencepattern is reported by low values (with dark gray levels).

in the area N. The LDM-matcher is more selective. For thearea-N pixels, high values are obtained by the LDM-matcher.The area-N values can be interpreted as false-positives forthe Borgefors chamfer matching and true-negatives for theLDM-matcher. The LDM-matcher can thus provide less falsepositives in a pattern matching task than the Borgefors chamfermatching.

Here is an interpretation of these observations. The pixeldensity in area-N is higher than in other locations of theimage. When the pattern template P is somewhere in area-N,there is a high probability that a subset of pixels matching thepattern exists, providing low values by the Borgefors chamfermatching. High values are given by the LDM-matcher as it isa symmetric matcher. Low values are only obtained when Pmatches the images and the image matches the pattern. Let’stake a drastic example : figure an area completely filled withpixels (ie an area filled with foreground pixels) and a pattern tobe matched. The Borgefors chamfer matching output responseswould be 0 as distance transform values (dtI in eq. 7) are zero.Intuitively each pixel of the pattern is corresponding to a pixelin this area, leading to a positive response of the Borgeforschamfer matching. For the LDM-matcher, all the values ofdtP are selected, leading to a very high score. The Borgeforschamfer matching values thus strongly depends on the imagepixels density.

V. THE LDM-MATCHER IS ROBUST

The LDM-matcher and Borgefors chamfer-matcher arecompared here with respect to noise robustness. How goodare the responses when a noisy pattern is given? Threeimages taken from a lab’s project are given in figure 3 [4].The dimensions of these images are 256 × 256. Patternsare extracted from these images. Patterns are 51 × 51 sub-images cropped from random positions in the full image. Eachpattern is then modified by inverting a given rate of its pixels.Finally the LDM-matcher and the Borgefors chamfer-matcherare applied. The estimated pattern position is obtained byfinding the global minimum in the image response of the

P reference shape















I binary image
















22





























Chamfer Matching Quadratic LDM matcher

(less false positives)

low value match! →


Robust?

23


24

I

P

Pnoisy

a) extractpattern

b) invert some pixels

c) localize


Fig. 3. three test images and some examples of extracted patterns

Fig. 4. Noise robustness of the Borgefors chamfer-matcher and the LDM-matcher. The mislocation error (a distance in pixel) is given with respect tothe perturbation rate of the pattern (percentage of inverted pixels).

matchers. The position error is estimated by the euclideandistance between the real position and the computed patternposition. For these three images, one thousand patterns havebeen extracted and processed.

An illustration of the position error evolution with respectto the perturbation rate is given in figure 4. The curves are ob-tained from an average of the position errors returned for thesethree images with perturbation rates between 0% and 30%.For perturbation rates lower than 7%, the error of the LDM-matcher remains very low (1 pixel of mislocation) whereas theBorgefors chamfer-matcher error increases. Beyond 10%, theposition error increases for the two matchers, even if the LDM-matcher provides a better matching. From these curves, theLDM-matcher is significantly more robust than the Borgeforschamfer-matcher.

VI. CONCLUSION

A new pattern matcher from local dissimilarity measures(LDM-matcher) between an image and a template has beenproposed. This pattern matcher is a generalization of theBorgefors chamfer-matching algorithm. Some comparisons

showed that the LDM-matcher potentially returns less false-positives than the Borgefors chamfer-matcher. Moreover it ismore robust to noise.

Further investigations are actually conducted to take intoaccount a fine balance of the asymmetry in the generalizedquadratic matcher. More tests on more real tasks with groundtruth are also planned, for example on logo recognition.

REFERENCES

[1] A. Andreev, N. Kirov, ”Word Image Matching Based on HausdorffDistances”, Proc. of 10th International Conference on Document Analysisand Recognition, pp. 396-400, 2009.

[2] E. Baudrier, F. Nicolier, G. Millon, S. Ruan, ”Binary-image comparisonwith local-dissimilarity quantification”, Pattern Recognition, vol. 41, n.5, pp. 1461–1478, jan. 2008

[3] G. Borgefors, ”Hierarchical chamfer matching: a parametric edge match-ing algorithm”, IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 10, n. 6, pp. 849–865, 1988

[4] Calypod - graphiCs imAge anaLYsis from Printed Old Document,http://calypod.free.fr/

[5] A. Ghafoor, R. N. Iqbal, S. A. Khan, ”A Modified Chamfer MatchingAlgorithm”, IDEAL intelligent data engineering and automated learning,Hong Kong, LLNCS vol. 2690, pp. 1102-1106, 2003.

[6] D.P. Huttenlocher, W.J. Rucklidge, ”Comparing images using the haus-dorff distance”, IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 15, n. 9, pp. 850–863, 1993

[7] U. Montanari, ”A method for obtaining skeletons using a quasi-Euclideandistance”, Journal of the Association for Computing Machinery, vol. 15,pp. 600-624, 1968.

[8] E. Rosch, ”Cognitive reference points,” Cognitive Psychology, vol. 7, iss.4, pp. 532-547, 1975.

[9] S. Santini, R. Jain, ”Similarity Measures”, IEEE Trans. Pattern Anal.Mach. Intell., vol. 21, n. 9, pp. 871-883, 1999.

[10] A. Tversky, ”Features of similarity”, Psychological Review, vol. 84, n.4, pp. 327-352, 1977

[11] B. Zitova, J. Flusser, ”Image registration methods: a survey”, Image andVision Computing, vol. 21, pp. 977–1000, 2003

25

Texte

localization error vs pattern perturbation


Conclusion

26

• Template matching via Local Dissimilarity Measure (LDM)

• LDM = symmetrical Chamfer matching

• Less false positives

• More robust to noise


Main future works

• Playing with asymmetry weights

• Formalize and use links with Tversky’s constrast model

• Going to gray-level images similarity (soon via extended distance transform)

27


Ipta2010

Technology

Transcript of Ipta2010