Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer...
-
Upload
martina-banks -
Category
Documents
-
view
218 -
download
0
Transcript of Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer...
![Page 1: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/1.jpg)
Classifying Visual Objects Regardless of Depictive Style
Qi Wu, Peter HallDepartment of Computer Science
University of Bath
![Page 2: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/2.jpg)
Summary
• Conventional Comp.Vis. classifiers do not generalise well across depictive styles.
• We propose new visual class model,one invariant to depictive style.
• Experiments validate our model.
![Page 3: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/3.jpg)
People can see objects in a wide variety of depictive styles.
Photos Artwork
![Page 4: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/4.jpg)
Literature Gap: BoW does not generalise across depictive styles
Photos Artwork47% (Dense SIFT)
Photos Artwork51% (Dense SIFT)
![Page 5: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/5.jpg)
Our solution: A new Visual Class Model that does generalise across styles.
Photos Artwork47% (Dense SIFT)
Photos Artwork51% (Dense SIFT)
64%
67%
![Page 6: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/6.jpg)
A New Visual Class Model• We assume an object class is characterised by:
– the qualitative shape of object parts,– the structural arrangement of those parts.
• A hierarchical graph model per image:– coarse-to-fine representation (layered),– nodes labelled by primitive shapes,
• abstracting region shape brings greater robustness.
– arcs labelled with displacement vectors
• Median graph models:– aggregates models from several instances,– single class model.
![Page 7: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/7.jpg)
Making a VCM
(a): An input collection. (b): Probability maps for each input image, and graph models for each map. (c): The median graph model for the whole class. (d): The refined median graph as the final class model
![Page 8: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/8.jpg)
A schematic VCM• A hierarchical description
• Berkeley segmentation• Filtering process using cLge
• A graph• Arcs at same level denote
touching neighbours.• Arcs between layers link
parent – children.
• Nodes label• A 6-elements
probability vector.• The probability that a
region belongs to a given prime shape class.
![Page 9: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/9.jpg)
Prime Shapes,BMVC 2012
![Page 10: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/10.jpg)
Prime Shapes• Does a set of elementary planar shapes
appear commonly in the world ?• Art provides strong anecdotal evidence “yes”– 20th century Western Art --- Cubism
![Page 11: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/11.jpg)
Determine Prime Shapes• A fully unsupervised framework
![Page 12: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/12.jpg)
Determine Prime Shapes
![Page 13: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/13.jpg)
Back to our model…
![Page 14: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/14.jpg)
Build graphs, one for each image
Left: graph model. Right: Object broken in primitive shapes
![Page 15: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/15.jpg)
Compute an initial Visual Class Model• Median Graph
• First compute the graph distance between each pair.
• Using the distance matrix to embed graph into a vector space
• Compute the Euclidean Median of all the data points.
• Transfer the median vector back to graph using a state-of-art method proposed in [Ferrer and Valveny, 2008]
![Page 16: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/16.jpg)
Refine the Visual Class ModelThe initial model contains nodes and arcs that derive from visual clutter in back ground of images in the training set
• Refine the model• Match the median back
into each training image.
• Count the number of times a given node in the model appears in the training data.
• Delete all nodes below a frequency threshold., which is computed via maximising matching score.
![Page 17: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/17.jpg)
Some Examples
![Page 18: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/18.jpg)
Experiments
• Compare with other two methods• PHOW features (Dense SIFT) [Bosch and Zisserman, ICCV
2007]• Local PAS features [Ferrari and Jurie, IJCV 2010]• Structure Only [Bai and Song, CVIU 2011]
![Page 19: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/19.jpg)
Results
![Page 20: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/20.jpg)
Conclusions
• It’s possible to learn models of objects classes that generalise across depictive styles.
• Many applications are promised.
• Just a first step– Simplify the model, still too much nodes and arcs.– Time consuming.– Additional labelling– Move to object localisation.
![Page 21: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/21.jpg)
Questions?
![Page 22: Classifying Visual Objects Regardless of Depictive Style Qi Wu, Peter Hall Department of Computer Science University of Bath.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649e935503460f94b98b35/html5/thumbnails/22.jpg)
One application of Prime shapes