2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 -...
Transcript of 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 -...
![Page 1: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/1.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Image WebsComputing and Exploiting Connectivity in Image Collections
![Page 2: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/2.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Challenges and opportunities in large image collections
• Challenges– Understanding what large collections contains– Finding the image or information you want
• Opportunities– Lots of visual information not searchable– Data-driven approach to hard problems
![Page 3: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/3.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Goal: Link images together like web documents
• Discover “visual hyperlinks” between images in the collection induced by shared objects
• Exploit these links to search, visualize, and mine data from large image collections
![Page 4: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/4.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Goal: Link images together like web documents
![Page 5: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/5.jpg)
Kyle Heath, Zixuan Wang – Stanford University
OverviewWhat is an Image Web?Efficient constructionApplications
– Explore photo collections– Auto-annotate Flickr images– Mobile collaborative annotation
![Page 6: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/6.jpg)
Kyle Heath, Zixuan Wang – Stanford University
What is an Image Web?An Image Web is a graph generated by
– Detecting corresponding regions in pairs of images– Forming links between these regions
![Page 7: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/7.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Affine cosegmentation
Local Feature Extraction
Shared Region Segmentation
Feature Matching & Verification
Goal: Detect the shared region between a pair of images
![Page 8: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/8.jpg)
Kyle Heath, Zixuan Wang – Stanford University
...
...
Ideal Image Web construction
Problem: Cosegmentation with all image pairs is not scalable
![Page 9: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/9.jpg)
Kyle Heath, Zixuan Wang – Stanford University
...
Phase 1: Discover connected components
![Page 10: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/10.jpg)
Kyle Heath, Zixuan Wang – Stanford University
...
...
Phase 1: Discover connected components
Sort candidate pairs by CBIR score
decreasing similarity
![Page 11: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/11.jpg)
Kyle Heath, Zixuan Wang – Stanford University
...
Phase 1: Discover connected components
decreasing similarity
![Page 12: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/12.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Recover component structure
![Page 13: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/13.jpg)
Kyle Heath, Zixuan Wang – Stanford University
is the degree of vertex
Phase 2: Boost component connectivity
• Use a notion of connectivity from spectral graph theory called algebraic connectivity
– Defined as the second smallest eigenvalue ( ) of the graph Laplacian matrix
– Eigenvector of corresponding to is ,called the Fiedler vector
Example*
* shamelessly lifted from Wikipedia
![Page 14: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/14.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Recover component structure
![Page 15: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/15.jpg)
Kyle Heath, Zixuan Wang – Stanford University
...
...
Phase 2: Boost component connectivity
Sort candidate pairs by connectivity score
decreasing connectivity
![Page 16: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/16.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 17: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/17.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 18: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/18.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 19: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/19.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 20: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/20.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 21: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/21.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 22: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/22.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 23: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/23.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 24: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/24.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 25: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/25.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 26: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/26.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 27: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/27.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 28: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/28.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 29: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/29.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 30: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/30.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 31: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/31.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 32: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/32.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 33: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/33.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 34: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/34.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 35: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/35.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Phase 2: Boost component connectivity
...decreasing connectivity
![Page 36: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/36.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Distributed computation• Construction pipeline easily distributed on a
computer cluster
![Page 37: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/37.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Image Web Construction Timings on Cluster with 500 Nodes
![Page 38: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/38.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Applications
![Page 39: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/39.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Auto-tagging Flickr Images
memorialchurch stanfordmemorialchurch stanford
goldengatebridge
![Page 40: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/40.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Tag cleaning by simulated ESP game
Lyndon Kennedy, Malcolm Slaney, Kilian Weinberger "Reliable Tags Using Image Similarity: Mining Specificity and Expertise from Large-Scale Multimedia Databases" In WSMC '09: Proceedings of the 1st workshop on Web-scale multimedia corpus (2009).
![Page 41: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/41.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Auto-tagging experiment• Dataset
– Flickr search for “Stanford” 195,268 images– Image Web 1,132,406 regions– Tag cleaning 525 tags
![Page 42: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/42.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Auto-tagging experiment• Results
![Page 43: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/43.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Mobile collaborative annotation
![Page 44: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/44.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Acknowledgments• Natasha Gelfand• Maks Ovsjanikov• Mridul Aanjaneya
![Page 45: 2010 04 27 - ImageWebs POMI Workshop Shared · 2015. 1. 21. · Microsoft PowerPoint - 2010 04 27 - ImageWebs POMI Workshop Shared Author: saylee Created Date: 5/3/2010 3:07:09 PM](https://reader036.fdocuments.in/reader036/viewer/2022071017/5fd047861c458d783f7118c3/html5/thumbnails/45.jpg)
Kyle Heath, Zixuan Wang – Stanford University
Thanks• Questions?