Unsupervised Learning of Hierarchical Spatial Structures
description
Transcript of Unsupervised Learning of Hierarchical Spatial Structures
![Page 1: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/1.jpg)
Unsupervised Learning of Hierarchical Spatial Structures
Devi Parikh, Larry Zitnick and Tsuhan Chen
![Page 2: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/2.jpg)
2… hierarchical spatial patterns
Our visual world…
What is an object?What is context?
Intro
Approach
Results
Conclusion
![Page 3: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/3.jpg)
3
Goal
Unsupervised!
Intro
Approach
Results
Conclusion
![Page 4: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/4.jpg)
4
Related work
[Todorovic 2008]
[Fidler 2007] [Zhu 2008]
[Sivic 2008]
Fully unsupervised
Structure and parameters learnt
From features to multiple objects
Intro
Approach
Results
Conclusion
![Page 5: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/5.jpg)
5
ModelRule based
c2
c4
c1
c2
c3
r1 0.9
0.1
0.60.7
0.6
Intro
Approach
Results
Conclusion
![Page 6: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/6.jpg)
6
c2
r2
c1
c2
c3
r1 0.9
0.1
0.60.7
0.6
ModelRule based
Intro
Approach
Results
Conclusion
![Page 7: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/7.jpg)
7
c2
r2
c1
c2
c3
r1 0.9
0.1
0.60.7
0.6
ModelHierarchical rule-based
Intro
Approach
Results
Conclusion
![Page 8: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/8.jpg)
8
Rules R
Image-parts V
Model
Codewords C
Features F
Intro
Approach
Results
Conclusion
![Page 9: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/9.jpg)
9
Model NotationV = {v} instantiated image-parts
rv rule corresponding to instantiated part v
Ch(rv) = {x} children of rule rv
includes instantiated children Ch(v) and un-instantiated children
Intro
Approach
Results
Conclusion
![Page 10: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/10.jpg)
10
Model
Intro
Approach
Results
Conclusion
![Page 11: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/11.jpg)
11
Inference
Intro
Approach
Results
Conclusion
![Page 12: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/12.jpg)
12
Inference
Intro
Approach
Results
Conclusion
![Page 13: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/13.jpg)
13
Inference
Intro
Approach
Results
Conclusion
![Page 14: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/14.jpg)
14
Inference
Intro
Approach
Results
Conclusion
![Page 15: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/15.jpg)
15
Inference
Intro
Approach
Results
Conclusion
![Page 16: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/16.jpg)
16
Inference
Intro
Approach
Results
Conclusion
![Page 17: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/17.jpg)
17
Inference
Intro
Approach
Results
Conclusion
![Page 18: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/18.jpg)
18
Inference
Intro
Approach
Results
Conclusion
![Page 19: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/19.jpg)
19
Inference
Intro
Approach
Results
Conclusion
![Page 20: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/20.jpg)
20
Inference
Intro
Approach
Results
Conclusion
![Page 21: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/21.jpg)
21Minimum Cost
Steiner TreeCharikar 1998
Inference
Intro
Approach
Results
Conclusion
![Page 22: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/22.jpg)
22
Inference
Intro
Approach
Results
Conclusion
![Page 23: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/23.jpg)
23
Generalized distance transformFelzenszwalb et al. 2001
Inference
Intro
Approach
Results
Conclusion
![Page 24: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/24.jpg)
24
EM style
Initialize rules
Infer rules Update parameters Modify rules
Learning
Intro
Approach
Results
Conclusion
![Page 25: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/25.jpg)
25
Initialize rules
…
Learning
Intro
Approach
Results
Conclusion
![Page 26: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/26.jpg)
26
Inference
…
Learning
Intro
Approach
Results
Conclusion
![Page 27: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/27.jpg)
27
Inference
…
Learning
Intro
Approach
Results
Conclusion
![Page 28: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/28.jpg)
28
Add children
…
Learning
Intro
Approach
Results
Conclusion
![Page 29: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/29.jpg)
29
Add children
Update parameters
Pruning children
Removing rules
…
Learning
Intro
Approach
Results
Conclusion
![Page 30: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/30.jpg)
30
Adding rules
Randomly add rules
…
…
Learning
Intro
Approach
Results
Conclusion
![Page 31: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/31.jpg)
31
Behavior Competition among rules Competition with root (noise)
Intro
Approach
Results
Conclusion
![Page 32: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/32.jpg)
32
Behavior Competition among rules Competition with root (noise) Dropping children and rules Number of children Structure of DAG and tree # rules, parameters, structure learnt automatically Multiple instantiations of rules Multiple children with same appearance
Intro
Approach
Results
Conclusion
![Page 33: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/33.jpg)
Experiment 1: Faces & MotorbikesIntro
Approach
Results
Conclusion
![Page 34: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/34.jpg)
34
Faces and Motorbikes SIFT (200 words)
Learnt 15 L1 rules, 2 L2 rules Each L1 rule average ~7 children Each L2 rule average ~4 children
Faces & Motorbikes
Intro
Approach
Results
Conclusion
![Page 35: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/35.jpg)
35
Example rules
Intro
Approach
Results
Conclusion
![Page 36: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/36.jpg)
36
Patches
Intro
Approach
Results
Conclusion
![Page 37: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/37.jpg)
37
Localization behavior
Intro
Approach
Results
Conclusion
![Page 38: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/38.jpg)
38
Categorization behavior
Faces Motorbikes Faces Motorbikes Faces Motorbikes
occu
rren
ce
code-words first level rules second level rules
Intro
Approach
Results
Conclusion
![Page 39: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/39.jpg)
39
Categorization behavior
Words Rules Tree
Words: 94 %
Tree: 100%
KmeansPLSASVM
Intro
Approach
Results
Conclusion
![Page 40: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/40.jpg)
40
Edge features
Words: 55 %
Tree: 82%
Intro
Approach
Results
Conclusion
![Page 41: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/41.jpg)
Experiment 2: Six categoriesIntro
Approach
Results
Conclusion
![Page 42: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/42.jpg)
42
Six categories
61 L1 rules (~9 children)12 L2 rules (~3 children)
Kim 2008: 95 %
Words: 87 %
Tree: 95 %
Intro
Approach
Results
Conclusion
![Page 43: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/43.jpg)
Experiment 3: Scene categoriesIntro
Approach
Results
Conclusion
![Page 44: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/44.jpg)
44
Scene categories
Image Segmentation
Mean color Codeword
Intro
Approach
Results
Conclusion
![Page 45: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/45.jpg)
45
Outdoor scenes
rule
s
images
Intro
Approach
Results
Conclusion
![Page 46: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/46.jpg)
Experiment 4: Structured street scenesIntro
Approach
Results
Conclusion
![Page 47: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/47.jpg)
47
Windows
Intro
Approach
Results
Conclusion
![Page 48: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/48.jpg)
48
Object categories
Intro
Approach
Results
Conclusion
![Page 49: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/49.jpg)
49
Object categories
Intro
Approach
Results
Conclusion
![Page 50: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/50.jpg)
50
Object categories
Intro
Approach
Results
Conclusion
![Page 51: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/51.jpg)
51
Parts of objects
Intro
Approach
Results
Conclusion
![Page 52: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/52.jpg)
52
Multiple objects
Intro
Approach
Results
Conclusion
![Page 53: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/53.jpg)
53
Street Scenes (PLSA)
Intro
Approach
Results
Conclusion
![Page 54: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/54.jpg)
54
Dataset specific rules
irrelevant
relevantIntro
Approach
Results
Conclusion
![Page 55: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/55.jpg)
55
Conclusion
Unsupervised learning of hierarchical spatial patterns Low level features, object parts, objects, regions in scene
Rule-based approach Learning: EM style Inference: Minimum cost Steiner tree
Features SIFT, edges, color segments
Intro
Approach
Results
Conclusion
![Page 56: Unsupervised Learning of Hierarchical Spatial Structures](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816140550346895dd0af8d/html5/thumbnails/56.jpg)
56
Summary
I
Root
Scene
Objects
Object Parts
Features
Intro
Approach
Results
Conclusion