Fast and Compact Retrieval Methods in Computer Vision Part II
Computer Vision, Part 1
description
Transcript of Computer Vision, Part 1
![Page 1: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/1.jpg)
Computer Vision, Part 1
![Page 2: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/2.jpg)
![Page 3: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/3.jpg)
![Page 4: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/4.jpg)
![Page 5: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/5.jpg)
![Page 6: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/6.jpg)
Topics for Vision Lectures
1. Content-Based Image Retrieval (CBIR)
2. Object recognition and scene “understanding”
![Page 7: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/7.jpg)
Content-Based Image Retrieval
![Page 9: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/9.jpg)
Query Image
Extract Features(Primitives)
Image Database
Features Database
SimilarityMeasure
MatchedResults
RelevanceFeedbackAlgorithm
From http://www.amrita.edu/cde/downloads/ACBIR.ppt
Basic technique
Each image in database is represented by a feature vector: x1, x2, ...xN, where xi = (xi1, xi2, …, xim)
Query is represented in terms of same features: Q =(Q1, Q2, …, Qm)
Goal: Find stored image with vector xi most similar to query vector Q
![Page 10: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/10.jpg)
• Distance measure:
• Possible distance measures d(Q, xi): – Inner (dot) product– Histogram distance (for histogram features)– Graph matching (for shape features)
.
.
.
€
x i⋅Q = x1iQ1 + x2
iQ2 + ...+ xmiQm
![Page 11: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/11.jpg)
Some issues in designing a CBIR system
• Query format, ease of querying
• Speed
• Crawling, preprocessing
• Interactivity, user relevance feedback
• Visual features — which to use? How to combine?
• Curse of dimensionality
• Indexing
• Evaluation of performance
![Page 12: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/12.jpg)
Types of Features Typically Used
• Intensities
• Color
• Texture
• Shape
• Layout
![Page 13: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/13.jpg)
Intensity histogramshttp://www.clear.rice.edu/elec301/Projects02/artSpy/intensity.html
![Page 14: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/14.jpg)
Color Features
Hue, saturation, value
![Page 15: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/15.jpg)
http://www.owlnet.rice.edu/~elec301/Projects02/artSpy/patmac/mcolhist.gif
Color Histograms (8 colors)
![Page 16: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/16.jpg)
http://www.owlnet.rice.edu/~elec301/Projects02/artSpy/patmac/mcolhist.gif
![Page 17: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/17.jpg)
Color auto-correlogram
• Pick any pixel p1 of color Ci in the image I.
• At distance k away from p1 pick another pixel p2.
• What is the probability that p2 is also of color Ci?
p1
p2k
Red ?
Image: I
From: http://www.cse.ucsc.edu/classes/ee264/Winter02/xgfeng.ppt
![Page 18: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/18.jpg)
• The auto-correlogram of image I for color Ci , distance k:
• Integrates both color information and space information.
]|,|Pr[|)( 1221)(
iii CCkC IpIpkppI
From: http://www.cse.ucsc.edu/classes/ee264/Winter02/xgfeng.ppt
![Page 19: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/19.jpg)
Two images with their autocorrelograms. Note that the change in spatial layout would be ignored by color histograms, but causes a significant difference in the autocorrelograms. From http://www.cs.cornell.edu/rdz/Papers/ecdl2/spatial.htm
![Page 20: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/20.jpg)
Color histogram rank: 411; Auto-correlogram rank: 1
Color histogram rank: 310; Auto-correlogram rank: 5
Color histogram rank: 367; Auto-correlogram rank: 1
From: http://www.cs.cornell.edu/rdz/Papers/ecdl2/spatial.htm
![Page 21: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/21.jpg)
Texture representations
• Gray-level co-occurrence
• Entropy
• Contrast
• Fourier and wavelet transforms
• Gabor filters
![Page 22: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/22.jpg)
Texture Representations
Each image has the same intensity distribution, but different textures
Can use auto-correlogram based on intensity (“gray-level co-occurrence”)
![Page 23: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/23.jpg)
Texture from entropy
Images filtered by entropy:
Each output pixel contains entropy value of 9x9 neighborhood around original pixel
From: http://www.siim2011.org/abstracts/advanced_visualization_tools_ss_pao.html
![Page 24: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/24.jpg)
Texture from fractal dimensionFrom: http://www.cs.washington.edu/homes/rahul/data/iccv07.pdf
![Page 25: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/25.jpg)
Texture from Contrast
• Example: http://www.clear.rice.edu/elec301/Projects02/artSpy/graininess.html
![Page 26: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/26.jpg)
Texture from Wavelets
http://www.clear.rice.edu/elec301/Projects02/artSpy/dwt.html
![Page 27: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/27.jpg)
http://www.clear.rice.edu/elec301/Projects02/artSpy/dwt.html
![Page 28: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/28.jpg)
Shape representations
Some of these need segmentation (another whole story!)
• Area, eccentricity, major axis orientation
• Skeletons, shock graphs
• Fourier transformation of boundary
• Histograms of edge orientations
![Page 29: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/29.jpg)
From: http://www.lems.brown.edu/vision/researchAreas/ShockMatching/shock-ed-match-results1.gif
![Page 30: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/30.jpg)
From: http://www.cs.ucl.ac.uk/staff/k.jacobs/teaching/prmv/Edge_histogramming.jpg
Histogram of edge orientations
![Page 31: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/31.jpg)
Visual abilities largely missing from current CBIR systems
• Object recognition
• Perceptual organization
• Similarity between semantic concepts
![Page 32: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/32.jpg)
Image 1 Image 2
Examples of “semantic” similarity
![Page 33: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/33.jpg)
Image 1 Image 2
Examples of “semantic” similarity
![Page 34: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/34.jpg)
Image 1 Image 2
Examples of “semantic” similarity
![Page 35: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/35.jpg)
“In general, current systems have not yet had significant impact on society due to an inability to bridge the semantic gap between computers and humans.”
![Page 36: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/36.jpg)
![Page 37: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/37.jpg)
Image Understanding and Analogy-Making
![Page 38: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/38.jpg)
Bongard problems as an idealized domain for exploring the “semantic gap”
![Page 39: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/39.jpg)
![Page 40: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/40.jpg)
![Page 41: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/41.jpg)
![Page 42: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/42.jpg)
![Page 43: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/43.jpg)
![Page 44: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/44.jpg)
![Page 45: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/45.jpg)
![Page 46: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/46.jpg)
![Page 47: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/47.jpg)
![Page 48: Computer Vision, Part 1](https://reader035.fdocuments.in/reader035/viewer/2022062323/568163c8550346895dd4ff86/html5/thumbnails/48.jpg)