Applications of

Shape Similarity

ASR: Applications in Computer Vision

Robotics: Shape Screening(Movie: Robot2.avi)

• Straightforward Training Phase

• Recognition of Rough Differences

• Recognition of Differences in Detail

• Recognition of Parts

ASR: Applications in Computer Vision

Application 2:

View Invariant Human Activity Recognition

(Dr. Cen Rao and Mubarak Shah, School of Electrical Engineering and Computer Science, University of Central Florida)

Application: Human Activity Recognition

Human Action Defined by TrajectoryAction Recognition by Comparison of Trajectories

(Movie: Trajectories)

• Rao / Shah:• Extraction of ‘Dynamic Instants’ by Analysis of Spatiotemporal Curvature• Comparison of ‘Dynamic Instants’ (Sets of unconnected points !)

• ASR: •Simplification of Trajectories by Curve Evolution• Comparison of Trajectories

Application: Human Activity Recognition

Trajectory Simplification

Activity Recognition: Typical Set of Trajectories

Trajectories in Tangent Space

Trajectory Comparison by ASR: Results

Recognition of 3D Objects by Projection

Background: MPEG 7 uses fixed view anglesImprovement: Automatic Detection of Key Views

Automatic Detection of Key Views

(Pairwise) Comparison of Adjacent Views•Detects Appearance of Hidden Parts

Automatic Detection of Key Views

Result (work in progress):

Application: ASR

The Database Implementation

The Main Application: Back to ISS

Task:Create Image Database

Problem:Response TimeComparison of 2 Shapes: 23ms on Pentium1Ghz

ISS contains 15,000 images:Response Time about 6 min.

Clustering not possible:ASR failed on measuring dissimilarities !

Solution:

Full search on entire database using a simplercomparison

Vantage Objects (Vleugels / Veltkamp, 2000) provide a simple comparison of n- dimensional vectors (n typically < 100)

Vantage Objects

The Idea:Compare the query-shape q to a predefined

subset S of the shapes in the database D

The result is an n-dimensional Vantage Vector V,n = |S|

Vantage Objects

q

s1

s2

s3

sn

…

v1

v2

v3

vn

-- Each shape can be represented by a single Vantage Vector

-- The computation of the Vantage Vector calls theASR – comparison only n times

-- ISS uses 54 Vantage Objects, reducing the comparison time (needed to create the Vantage Vector) to < 1.5s

-- How to compare the query object to the database ?

Vantage Objects

-- Create the Vantage Vector vi for every shape di in the database D

-- Create the Vantage Vector vq for the query-shape q

-- compute the (euclidean) distance between vq and vi

-- best response is minimum distance

-Note: computing the Vantage Vectors for the database objects is an offline process !

Vantage Objects

-How to define the set S of Vantage Objects ?

Vantage Objects

-Algorithm 1 (Vleugels / Veltkamp 2000):

-Predefine the number n of Vantage Objects-S0 = { }-Iteratively add shapes di D\Si-1 to Si-1 such that

-Si = Si-1 di

-andk=1..i-1 e(di , sk) maximal. (e = eucl. dist.)

Stop if i = n.

Vantage Objects

-Result:

-Did not work for ISS.

Vantage Objects

-Algorithm 2 (Latecki / Henning / Lakaemper):

Def.: • A(s1,s2): ASR distance of shapes s1,s2

• q: query shape• ‘Vantage Query’ : determining the result r by minimizing e(vq , vi ) vi = Vantage Vector to si

• ‘ASR Query’: determining the result r by minimizing A(q,di )

Vantage Query has certain loss of retrieval quality compared to ASR query.

-Define a loss function l to model the extent of retrieval performance

Vantage Objects

Given a Database D and a set V of Vantage Vectors, the loss of retrieval performance for a single query by shape q is given by:

lV,D (q) = A(q,r),

Where r denotes the resulting shape of the vantage query to D using q.

Property:lV,D (q) is minimal if r is the result of the ASR-Query.

Vantage Objects

Now define retrieval error function L(S) of set S={s1 ,…, sn } D of Vantage Vectors of Database D:

L(S) = 1/n lS,D\{si} (si)

Task:Find subset S D such that L(S) is minimal.

Vantage Objects

Algorithm:

V0={ }iteratively determine sj in D\Sj-1 such that Sj =Sj-1 sj and L(Vj) minimal.

Stop if improvement is low

Vantage Objects

Result:Worked fine for ISS, though handpicked objects still performed better.

Vantage Objects

HandpickedAlgorithm 2

Number of Vantage Objects

L(S)

…some of the Vantage Objects used in ISS:

Vantage Objects

The Vantage Objects are used in the ASR in the first (handdrawn) query.

The query is compared to 54 Objects, then a vector comparison is computed with the whole database.

The first result, also called ‘first guess’, is the result of the vantage vector search.

Searching for a ‘grabbed’ a shape on the user interface leads to direct comparison with the ASR, these results are precomputed, since the query is a known shape !

Vantage Objects and ISS

A: the handdrawn sketchB: the result of the Vantage searchC: the result of the exact match

Vantage Objects and ISS