Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12 I9 CHAIR...
-
Upload
quinn-weatherhead -
Category
Documents
-
view
215 -
download
3
Transcript of Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12 I9 CHAIR...
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12I9CHAIR OF COMPUTER SCIENCE 9DATA MANAGEMENT AND EXPLORATION
Ranking Multimedia Databases via Relevance Feedback
with History and Foresight Support
DBRank 08, April 12th 2008, Cancún, Mexico
Marc Wichterich, Christian Beecks, Thomas Seidl
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12
Outline
Motivation Ranking DB according to Earth Mover’s Distance Search for suitable ground distance via user interaction
Relevance Feedback The MindReader approach Challenges in multimedia context History – Change of user preferences over time Foresight – Fast exploration
Conclusion and Outlook
2
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12
Transform object features to match those of other object Minimum work for transformation: EMD[1]
Feature signatures: {(center1, weight1), (c2,w2), …}
signature of object 1 signature of object 2 EMD weight assignment
Motivation: Ranking according to Earth Mover‘s Distance
3
[1] Rubner, Tomasi, Guibas, “A metric for distributions with applications to image databases,” in IEEE ICCV 1998.
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12
Requires ground distance gd in feature space gd(“blue/left”, “purple/right”) vs. gd(“blue/left”, “red/middle”) ?
gd?
gd?
Possibly complex gd: “Blue may move horizontally at low cost if at top of image (sky)”
Idea: Find gd according to user preferences
Motivation: Ranking according to Earth Mover‘s Distance
4
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12
Collecting preference information on feature space Utilize histogram-based Relevance Feedback system Histogram dimensions correspond to points in feature space
System has to deliver information on histogram dimension pairs Define gd on feature space
Rank DB according to EMDgd on signatures
Motivation: Ranking according to Earth Mover‘s Distance
5
feature spacehistogram
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12
1. MR shows candidate objects
2. User rates relevant objects
3. MindReader determines: new query point q similarity matrix S for
ellipsoid-shaped distance
4. Goto 1
Similarity matrix S is (pseudo) inverse covariance matrix S reflects user preferences w.r.t. histograms dimensions
Relevance Feedback: MindReader Approach [2]
6
[2] Ishikawa, Subramanya, Faloutsos, “MindReader: Querying databases through multiple examples,” VLDB 1998.
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12
MindReader: Challenges in multimedia context
Multimedia object histograms usually high-dimensional Number of rated candidates << histogram dimensionality Pseudo inverse results in open ellipsoid
search region MindReader implicitly assumes:
no info from user maximum preference
Solution: close the query ellipsoid Ask user for many more object ratings Replace assumption:
no info from user as preferred as least preferred direction [3]
Avoid assumptions by tackling “no info from user”
7
[3] Ye, Xu, “Similarity measure learning for image retrieval using feature subspace analysis,” ICCIMA 2003.
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12
“No information” only true within single iteration Idea: save information from previous rounds
+ =
iteration k-1 iteration k result
Technique: Incrementally compute weighted covariance matrix Exponential aging for ratings of previous iterations Include relevant points from all previous iterations
Relevance Feedback with History (1)
8
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12
Relevance Feedback with History (2)
9
= 0.1 = 0.3
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12
Relevance Feedback with History (Summary)
Feedback information crosses iteration boundaries Parameter sets aggregated weight for previous rounds Weighted covariance matrix is computed incrementally
No need to store or access old objects and weights Efficiently computable from aggregated information
Benefits: Guarantees closed query ellipsoids
Suitable for high-dimensional multimedia data
10
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12
Relevance Feedback with Foresight (1)
11
Framework can be reused to tackle another challenge
Exploratory search: user navigates through DB User picks objects to move query
point into preferred direction New search region might
be oriented contrary to intended movement
Slow or no advancement
Idea: Introduce heuristic direction matrix
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12
Relevance Feedback with Foresight (2)
Orientation of matrix D depends on direction of query point movement
Influence as a function of magnitude of movement
Adjust seamlessly to phases of exploration and stationary refinement
12
Ranking Multimedia Databases via Relevance Feedback with History and Foresight Support / 12
Observations and Outlook
Preliminary results Implemented prototype Relevance Feedback system History approach successfully extends MindReader to high
dimensions Foresight promising but naïve functions sometimes showed too
rapid or too slow a change in influence
Work in progress: Suitable function for Foresight parameter Heuristics for aggregating Relevance Feedback results into gd Find gd using signature-based Relevance Feedback
13