Slide 1

Bart M. ter Haar Romeny Slide 2 Question: can top-points be used for object- retrieval tasks? Slide 3 The locations of particularly characteristic points are called the interest points or key points. These interest points have to be as invariant as possible, but at the same time they have to carry a lot of distinctive information. Slide 4 Information in interest points is defined by their neighborhood. But how big should we choose this neighborhood? Lets take the corners of the mouth as interest points. The red circles are the areas in which the information is gathered. If we make the picture bigger, the size of the neighborhood is too small. The neighborhood should scale with the image Slide 5 When the interest points are detected in scale space they do not only have spatial coordinates x and y, but also a scale . This scale tells us how big the neighborhood should be. Slide 6 Our interest points have to be detected in scale space. They also have to contain a lot of information be reproducible be stable be well understood Slide 7 Critical Points, Paths and Top-Points Maxima Minimum Saddles L=0 Critical Points Slide 8 Maxima Minimum Saddles L=0 Critical Points Det(H)=0 Top-Points Critical Points, Paths and Top-Points Slide 9 OriginalGradient Magnitude LaplacianDet(H) Slide 10 Since for a critical path L=0 Intersection of Level Surfaces L x =0 with L y =0 Will give the critical paths. Slide 11 Since for a top-point both L=0 and Det[H]=Lxx Lyy-Lxy 2 =0 We can find them by intersecting the paths with the level surface Det[H]=0 Slide 12 Slide 13 Top-points are invariant to certain transformations. By invariant we mean that they move according to the transformation. Allowed Trans. Slide 14 It is possible to make a reconstruction of the original image from its top-points. We can generate reconstructed images which give the same (plus more) top-points as the original image. This reconstruction resembles the original image. Slide 15 Original Image Top-Points and Features Reconstruction Slide 16 Original By adjusting boundary and smoothness constraints we can improve the visual performance. For this 300x300 picture 1000 top-points with 6 features were used. Slide 17 For points close to top-points it is possible to calculate a vector pointing towards the position of the top-point. x y Approximated Top- Points Displacement Vectors Real Locations Slide 18 For points close to top-points it is possible to calculate a vector pointing towards the position of the top-point. This enables us to use fast top-point detection algorithms which do not have to be very accurate. Slide 19 The locations of top-points change when noise is added to the image. Slide 20 Slide 21 We can calculate the variance of the displacement of top- points under noise. We need 4 th order derivatives in the top- points for that. Slide 22 Thresholding on Stability Stable Paths Unstable Paths Slide 23 123458796 Databas e Query Image Slide 24 A simple image retrieval task. Using a small version of the Olivetti Faces Database. Consisting of 200 images of 20 different people (10 p.p.) Slide 25 Look at all scales simultaneously Scale x y Slide 26 Critical Points, Paths and Top Points Maxima Minimum Saddles u=0 Critical Points Det(H)=0 Top Points Slide 27 Slide 28 CompareEMD Slide 29 Earth Movers Distance (EMD) wiwi f ij c ij AB ujuj [*] Rubner, Tomasi, Guibas, 1998, IEEE Conf. on Computer Vision PilesHoles Slide 30 123456789 a93%78%68%62%58%54%49%45%43% b93%82%90%73%70%68%63%59%56% c95%88%83%76%73%66%62%59%58% d100%97%96%95%92%88%86%84%81% a.Using Euclidean Distance b.Using Eberly Distance c.As b. including stability norm d.As c. including 2 nd order derivatives. Slide 31 To distinguish top-points from each other a set of distinctive features are needed in every top-point. These local features describe the neighborhood of the top-point. Slide 32 We use the complete set of irreducible 3 rd order differential invariants. These features are rotation and scaling invariant. Slide 33 The top-points and differential invariants are calculated for the query object and the scene. Slide 34 We now compare the differential invariant features. compare distance = 0.5distance = 0.2distance = 0.3 Slide 35 The vectors with the smallest distance are paired. smallest distance distance = 0.2 Slide 36 A set of coordinates is formed from the differences in scale (Log( o1 )- Log( s2 )) and in angles ( o1 - s2 ). ( 1, 1 ) Slide 37 Dq Important Clusters For these clusters we calculate the mean and If these coordinates are plotted in a scatter plot clusters can be identified. In this scatter plot we find two dense clusters Slide 38 The stability criterion removes much of the scatter Slide 39 Rotate and scale according to the cluster means. Slide 40 The translations we find correspond to the location of the objects in the scene. Slide 41 In this example we have two clusters of correctly matched points. C1 C2 Slide 42 We can transform the outline of the query object and project it on the scene image. Slide 43 Slide 44 Slide 45 Top-points have proved to be invariant interest points which are useful for matching. The differential invariants have shown to be very distinctive. Experiments show good results.