Cataloguing moving images: describing artists' videos with MARC21
Describing Images Using Attributes
16
Describing Images Using Attributes
description
Describing Images Using Attributes. Describing Images. Farhadi et.al . CVPR 2009. Describing Objects by their Attributes. No examples from these object categories were seen during training. Farhadi et.al . CVPR 2009. Absence of typical attributes. 752 reports 68% are correct. - PowerPoint PPT Presentation
Transcript of Describing Images Using Attributes
Describing Images Using Attributes
Describing Images
Farhadi et.al. CVPR 2009
No examples from these object categories were seen during training
Describing Objects by their Attributes
Farhadi et.al. CVPR 2009
Absence of typical attributes
752 reports
68% are correct Farhadi et.al. CVPR 2009
Presence of atypical attributes
951 reports47% are correct Farhadi et.al. CVPR 2009
NormalitySaleh et. al. Object Centric Anomalty Detection by Attribute-Based Reasoning, CVPR13
Abnormal Object DatasetSaleh et. al. Object Centric Anomalty Detection by Attribute-Based Reasoning, CVPR13
Abnormality Prediction and RankingMethod AUC
One class SVM 0.5980Two class SVM 0.8657Graphical Model 0.8703Our Model with surprise score
0.9105
Less Abnormal High Abnormal
• Based on Abnormality Score, we can classify an object as Normal vs. Abnormal.
• Also, using this score we are able to rank images based on how strange they look like.
Saleh et. al. Object Centric Anomalty Detection by Attribute-Based Reasoning, CVPR13
Reasoning about Abnormality via Attributes
Saleh et. al. Object Centric Anomalty Detection by Attribute-Based Reasoning, CVPR13
Describing Objects
• Detector input– Strongest category response with good overlap– Strongest part response within each spatial bin
Farhadi et. al, Attribute-Centric Recognition for Cross-Category Generalization, CVPR10
Describing Objects
• Learn spatial correlations and co-occurrence
Detector Responses
True Value for Categories and Spatial
Parts
Has PartHas Function
Pose/Viewpoint
Latent “Root”
Learned by EM in training
Farhadi et. al, Attribute-Centric Recognition for Cross-Category Generalization, CVPR10
animal
function: can bitefunction: can fly
part: eyepart: footpart: headpart: legpart: mouthpart: tailpart: wing
Pose: objects_front
Animalblc: eaglefunction: can bitefunction: can flyfunction: is predatorfunction: is carnivorouspart: eyepart: footpart: headpart: legpart: mouthpart: wingPose: extended_wingsPose: objects_front
Describing Familiar Objects
Farhadi et. al, Attribute-Centric Recognition for Cross-Category Generalization, CVPR10
Using Localized Attributes
Vehicle
Wheel
Animal
Leg
HeadFour-leggedMammal
Can runCan JumpIs HerbivorousFacing right
Moves on roadFacing right
Farhadi et. al, Attribute-Centric Recognition for Cross-Category Generalization, CVPR10
Relative (ours):
More natural than insidecity Less natural than highway
More open than street Less open than coast
Has more perspective than highway Has less perspective than insidecity
Binary (existing):
Not natural
Not open
Has perspective
Using Relative Attributes
14Parikh, Grauman, Relative Attributes, ICCV 2011
Relative (ours):
More natural than tallbuilding Less natural than forest
More open than tallbuilding Less open than coast
Has more perspective than tallbuilding
Binary (existing):
Not natural
Not open
Has perspective
Using Relative Attributes
15Parikh, Grauman, Relative Attributes, ICCV 2011
Relative (ours):
More Young than CliveOwenLess Young than ScarlettJohansson
More BushyEyebrows than ZacEfron Less BushyEyebrows than AlexRodriguez
More RoundFace than CliveOwenLess RoundFace than ZacEfron
Binary (existing):
Not Young
BushyEyebrows
RoundFace
Using Relative Attributes
16
(Viggo)
Parikh, Grauman, Relative Attributes, ICCV 2011
