Iterative and Interactive Search for Objects
description
Transcript of Iterative and Interactive Search for Objects
Iterative and Interactive Search for Objects
Moty Golan & Oren Kerem
Instructor: Dr. Sigal Ar
Introduction
• Databases of objects– choose one working database– choose object from the database– arrange database by similarity to chosen object
• Interactive: results can be marked as “good” or “bad”
• Iterative: marked results used to refine the search
Search
Chooseobject
MarkResults
Databases
• 3D objects’ surface– 10000 sampling points (no color)– Uniform distribution– Location of point and normal to the surface
• 2D color pictures– Distinct families of pictures– HSV color representation
Objects’ Metric
• Object representation - feature vector
• Initial distance between objects - Euclidean distance: d(Dx , Dy) = || X - Y ||2
• Iterations: weighted Euclidean distance
• Weights
• Weighted distance : d(Dx , Dy) = [X - Y ]T W [X - Y] + b
Feature Vectors
• Data of a certain type is extracted from the object
• A vector of values is calculated based on the data
• Feature type – the method of the values calculation– Moments– Histograms
Data Types
• One aspect of the object is represented by a finite set of values
• Naturally numerous aspects exist• For graphic object use visual aspects:
– Pixels– Normals– Curvature– Color (HSV representation)
Implemented Data and Feature Types
3D
Moments Histograms
points normals points normals2D
Moments Histograms
curvature HSV points curvature HSV points
Goals
• Organize the databases
• Design an interface to enable searches on both databases using all parameters
• Conduct searches to explore behavior
Organizing the Databases
• Extracting the data types
• Calculation of feature vectors
• Create object icons
• Build directory hierarchy
Read_Graphic_Body Create_Icon
Compute_Moments
Organize_3dBase_Mom
Extract_MomFeatures
3D Example:
Calculating Features
• Normalize the data if needed – Ex. according to 1st & 2nd order moments
• Calculate moment vectors according to desired orders
• Calculate and flatten histograms
• Save in appropriate files
Icons
• “Flatten” and scale the 3D objects
• Sub-sample the 2D objects
Organized Databases
2d 3d sample
3d
images
2d
features
root
3d2d 3d2d
Interface Options
• Select database: 2D / 3D• Choose object• Set search parameters
– Feature kind and level
– Data type
• Conduct search• Database viewing & browsing
– Full data display
• Good and bad object marking
Program FlowDiagram
STARTING POINT
CHOOSE OBJECT
WAITING TO
SEARCH
WAITING FOR
MARKS
MARKING OBJECTS
Choose Object Button
Choose This Button
Search Button
Mark Results ButtonUpdate Search Button
Choose Object ButtonANYWHERE
Change search parameters
/ Change Database
Implementation: State Machine
• Each program stage is a state:– Program waiting stages– Program action stages
• User actions switch between states
• In each waiting state the relevant options are made available
while
switch
case a: ... case b: ... . . .
end
end
Cases Diagram
…
case 1
case 3
case 5case 23
case 25
30
10
100
Choose This pressed
Search pressed
50
case 2Mark Results
pressed
Update Search pressed
ANYWHERE
Good Object pressed
Bad Object pressed
Change Object / Database
INITIAL CASE
command case
user action
Legend
empty
Change Parameters
…
Demonstration
Test Runs
• Good results– Most tests converged within 3-4 iterations– Increase in “good” objects in each iteration– “Bad” objects removed from top 30 results
• Limitations– Database variety– Computation demands vs. response time– Human perception vs. object representation
Feature Explorations
• Higher feature levels do not guarantee better results
• Completeness of representation does not guarantee better results
• Some tests affected by feature type
Conclusions
• Database arrangement and interface were implemented as shown
• Various searches show that the algorithm successfully identifies classes of objects
• Further research can include:– The effect of specific high feature levels– The effect of computation constants– Combining different feature types and levels