A Novel Scheme for Video Similarity Detection

Chu-Hong Hoi, Steven

March 5, 2003

OutlineIntroductionOverviewPhase I: Coarse Similarity Measure

Pyramid Partitioning & Density Histogram Naïve Pyramid Density Histogram (NPDH) Fuzzy Pyramid Density Histogram (FPDH)

Phase II: Fine Similarity Measure Near Feature Trajectory (NFT) Simplification Algorithm Similarity Measure Based on NFT

Experiments and ResultsConclusion & Future Work

Introduction Motivation

Huge volume of video data are distributed over the Web.

How to fast detect the similar video effectively? Applications

Copyright issues / watermarking Content-based video retrieval

Overview Challenging Issues

Efficiency Effectiveness

We propose a Two-Phase Similarity Detection Framework based on two kinds of signatures with different granularity.

Solutions by two kinds of signatures Coarse Signature

Pyramid Density Histogram Fine Signature

Nearest Feature Trajectory

Overview

A two-phase framework for video similarity detection

Phase I: Coarse Similarity Measure Video data indexing

A frame is considered as a feature point. A video sequence is formed by a series of feature points. It is hard to index and search the video dat in original data

space. Two partitions of data space

Regular partitioning (Fig.2 (a)) Pyramid partitioning (Fig.2 (b)) (S. Berchtold-SIGMOD 98)

Center Point at (0.5,0.5,…,0.5)

Phase I: Coarse Similarity Measure Pyramid Density Histogram (PDH)

Map the feature points to the pyramid data space, and statistically calculate the distribution of the feature points

Obtain a density histogram of feature points as the coarse signature

Two kinds of PDHs Naïve Pyramid Density Histogram Fuzzy Pyramid Density Histogram

Phase I: Coarse Similarity Measure Naïve Pyramid Density Histogram

2d-dimension NPDH vector u=(u1,u2,…,u2d) How to calculate the density histogram?

For a d-dimension feature point v=(v1,v2,…,vd)

Center Point at (0.5,0.5,…,0.5)

Phase I: Coarse Similarity Measure Fuzzy Pyramid Density Histogram

In NPDH, a given feature point is allocated to only 1 pyramid. It would loss the information of other dimensions.

In FPDH, we fuzzyly allocate a feature point v to d pyramids based on the value of its d dimensions.

Center Point at (0.5,0.5,…,0.5)j=1,2,…,d

Phase I: Coarse Similarity Measure Similarity Filtering Based on PDH

Given a query example q and a compared sample s from the video database.

Set a filtering threshold δ , then video s is filtered out if it satisfies the following condition:

Phase II: Fine Similarity Measure Conventional Similarity Measure

Nearest Neighbor (NN) or (k-NN) Nearest Center (NC) Disadvantage: ignore the temporal

information of video sequences Nearest Feature Trajectory (NFT)

A video sequence is considered as a series of feature trajectories rather than isolated key-frames.

Phase II: Fine Similarity Measure Nearest Feature Trajectory

A frame in a video sequence is considered as a feature point.

Two feature points form a feature line. A series of feature lines form a feature trajectory in a

video shot. A video sequence consists of a series of feature

trajectories. Each trajectory corresponds to a individual shot or a

gradual transition of shots. Similarity measure is based on the nearest average

distance of feature trajectories in two video sequences.

Phase II: Fine Similarity Measure Generation of Simplified Feature Trajectory

Formulate the procedure by Minimum Square Error approach

The minimum procedure of MSE is time-consuming!

Phase II: Fine Similarity Measure We propose an algorithm for efficient generate

the feature trajectories. Define a local similarity measure function to

approximate the deviation degree.

The larger the value of LR(vk) is, the larger the deviation degree at vk is.

Based on the LR(vk), we remove the point with the minimum value each time until there remains only feature points.

Phase II: Fine Similarity Measure Similarity Measure Based NFT

Phase II: Fine Similarity Measure

Distance Measure of Two Feature Trajectories

Similarity Measure of Two Video Sequences

Considering the boundary problem, if 0≦λ≦1, falls in the line segment; otherwise, it falls out of the line

Experiments and Results Ground Truth Data

About 300 video clips with different coding formats, resolutions and slight color modifications

Feature Extraction RGB Color Histogram 64 dimensions

Performance Evaluation Metric Average Precision Rate

Average Recall Rate

Experiments and Results Coarse Similarity Measure

FPDH vs. NPDH

Experiments and Results Fine Similarity Measure

NFT vs. NN

Conclusions We propose an effective two-phase framework

to achieve the video similarity detection. Different from the conventional way, our

similarity measurement scheme is based on different granular similarity measure.

In the coarse measurement phase, we suggest Fuzzy Pyramid Density Histogram.

In the fine measurement phase, we present the Nearest Feature Trajectory technique.

Experimental results show that our scheme is better than the conventional approach.

Future Work Engaging more effective features in our

scheme to improve the performance

Enlarging our database and testing more versatile data

Cost performance evaluation

Q & A

Thank you!