Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li...
-
Upload
saul-headley -
Category
Documents
-
view
213 -
download
0
Transcript of Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li...
![Page 1: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/1.jpg)
Complex Networks for Representation and
Characterization of Images
For CS790g ProjectBingdong Li9/23/2009
![Page 2: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/2.jpg)
Outline• Background• Motivation• Current States (CS):
– Representation– Characterization
Using examples from – Backes, Casanova, and Bruno’s Approach using local information – Kim, Faloutsos and Hebert’s Approach using global information
• Comparison of Two Approaches• Summary• Questions and Comments
![Page 3: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/3.jpg)
Background: Complex Network
Source: cs790: complex network lecture
![Page 4: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/4.jpg)
Background: Image
Source: CS674 Image Processing Lecture
![Page 5: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/5.jpg)
Background: Image Processing
Source: CS674 Image Processing Lecture
![Page 6: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/6.jpg)
Background: Image Representation
Source: CS674 Image Processing Lecture
![Page 7: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/7.jpg)
Outline
• Background • Motivation
![Page 8: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/8.jpg)
Motivation
• Belief:
• Computer vision is one of the most difficult problem remains, how can we represent and characterize image in the way of complex network so that we analysis it?
For a given problem, if it can be described in the
way of mathematics, it is half way to solve
the problem.
![Page 9: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/9.jpg)
Outline
• Background • Motivation• Current States (CS):
– Representation– Characterization
Using examples from – Backes, Casanova, and Bruno’s Approach using
local information
![Page 10: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/10.jpg)
CS: Backes’ Approach
• Construction of graph, – Vertices: points of shape boundary are modeled as
fully connected network,– Weight: the Euclidean distance d– through a sequential thresholds Tl (d< Tl), the fully
connected network becomes a dynamic complex network, the topological features of the growth of the dynamic network are used as a shape descriptor (or signature)
![Page 11: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/11.jpg)
CS: Backes’ Approach
![Page 12: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/12.jpg)
CS: Backes’ Approach
• Properties of the complex network– High clustering coefficient– The small world property
![Page 13: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/13.jpg)
CS: Backes’ Approach
![Page 14: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/14.jpg)
CS: Backes’ Approach
• Dynamic evolution signature• F: T T where
Tini and TQ, respectively, the initial and final threshold
![Page 15: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/15.jpg)
CS: Backes’ Approach
• Characterization– Degree descriptor
kμ average degree, Kk max degree
![Page 16: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/16.jpg)
CS: Backes’ Approach
• Evolution by a threshold T=0.1, .15, .20
![Page 17: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/17.jpg)
CS: Backes’ Approach
Process of extraction of degree descriptor from an Image
![Page 18: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/18.jpg)
CS: Backes’ Approach
• Advantage of Degree Descriptors– Rotation and scale inveriance– Noise tolerance– Robustness
![Page 19: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/19.jpg)
CS: Backes’ Approach
Representation of rotate invariance
![Page 20: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/20.jpg)
CS: Backes’ Approach
Representation of scale invariance
![Page 21: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/21.jpg)
CS: Backes’ Approach
• Characterization– Joint Degree descriptor
Is the concatenation of the entropy(H), energy(E), and average joint degree(P) at each instant threshold T
![Page 22: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/22.jpg)
CS: Backes’ Approach
• Advantage of Joint Degree Descriptors– Rotation and scale inveriance– Noise tolerance– Robustness
– Normalization of vertex is irrelevant because the joint degree concerns the probability distribution P(ki,k’)i
![Page 23: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/23.jpg)
CS: Backes’ Approach
![Page 24: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/24.jpg)
CS: Backes’ Approach
![Page 25: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/25.jpg)
CS: Backes’ Approach
![Page 26: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/26.jpg)
CS: Backes’ Approach
![Page 27: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/27.jpg)
CS: Backes’ Approach
![Page 28: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/28.jpg)
CS: Backes’ Approach
• Weakness of Backe’s Approach:– Initial and final threshold
![Page 29: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/29.jpg)
Outline
• Background• Motivation• Current States (CS):
– Representation– Characterization
Using examples from – Backes, Casanova, and Bruno’s Approach using local
information – Kim, Faloutsos and Hebert’s Approach using global
information
![Page 30: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/30.jpg)
CS: Kim’s Approach
• Construct Visual Similarity Network (VSN)– Vertices (V): features of from training images– Edges (E): link features that matched across
images– Weights (W): consistence of correspondence with
all other correspondences in matching image Ia and Ib
VSN = (V, E, W)
![Page 31: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/31.jpg)
CS: Kim’s Approach
• Construction of VSN– Vertices: can be any unit of local visual
information. In this approach, features detected using Harris-Affine point detector and the SIFT descriptor
![Page 32: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/32.jpg)
CS: Kim’s Approach
• Construction of VSN– Edges: established between features in different
images. • Spectral matching algorithm is used to each pair of
image (Ia, Ib)
• A new edge is established between feature ai and bj
![Page 33: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/33.jpg)
CS: Kim’s Approach
![Page 34: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/34.jpg)
CS: Kim’s Approach
![Page 35: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/35.jpg)
CS: Kim’s Approach
• Construction of VSN– Edge weights
– M n*n is a spare weight matrix, M(ai , bj) is the weight value
![Page 36: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/36.jpg)
A small part of VSN
![Page 37: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/37.jpg)
CS: Kim’s Approach
• Characterization– Ranking of information
• Remove noisy• Measure the importance
P is the PageRank vector
![Page 38: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/38.jpg)
CS: Kim’s Approach
• Characterization– Structural similarity“similar nodes are highly likely to exhibit similar link
structures in the graph” p.4The similarity is computed by using link analysis
algorithm
![Page 39: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/39.jpg)
CS: Kim’s Approach
• CharacterizationLink analysis algorithmGiven a VSN G, a node ai , the neighborhood
subgraph Gai either pointed to ai or point to by ai M, the adjacency matrix of G ai.
![Page 40: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/40.jpg)
CS: Kim’s Approach
The left image is extracted features, the right image shows top20% high-ranked features
![Page 41: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/41.jpg)
CS: Kim’s Approach
• Weakness of Kim’s Approach– Using threshold in computing edge weights– Mystery constant α =0.1– Category partition to pre-determined K groups– The difference of objects appearance in the
training data set is too big, make the conclusion weak
![Page 42: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/42.jpg)
Outline
• Background• Motivation• Current States (CS):• Comparison of Two Approaches
![Page 43: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/43.jpg)
Comparison• Backes’s Approach
– Unsupervised approach– using local information – Dynamic complex network– More task on complex network, less work on image
processing• Kim’s Approach
– Supervised approach– using global information– Static complex network– More work on image processing, less work on complex
network• Both using threshold, but Backe’s approach based on
initial and final value,
![Page 44: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/44.jpg)
Outline
• Background• Motivation• Current States (CS):• Comparison of Two Approaches• Summary
![Page 45: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/45.jpg)
Summary
• In both approaches using complex network for representation and characterization of image,– provide a unique way for object classification and
analysis, – present better results than traditional and state-
of-art methods, – demonstrate the potential of complex network
analysis to computer vision.
![Page 46: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/46.jpg)
Questions and Comments
![Page 47: Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062620/5519d12f55034649768b48e7/html5/thumbnails/47.jpg)
Thanks