NATIONAL TECHNICAL UNIVERSITY OF ATHENS Image, Video And Multimedia Systems Laboratory Background .
9
-- NATIONAL TECHNICAL UNIVERSITY OF ATHENS Image, Video And Multimedia Systems Laboratory Background http://www.image.ntua.gr
-
Upload
alexandra-kelley -
Category
Documents
-
view
218 -
download
0
Transcript of NATIONAL TECHNICAL UNIVERSITY OF ATHENS Image, Video And Multimedia Systems Laboratory Background .
--
NA
TIO
NA
L TE
CH
NIC
AL U
NIV
ER
SIT
Y O
F
AT
HE
NS
--
NA
TIO
NA
L TE
CH
NIC
AL U
NIV
ER
SIT
Y O
F
AT
HE
NS
Image, Video And Multimedia Systems Laboratory
Background
http://www.image.ntua.gr
--
NA
TIO
NA
L TE
CH
NIC
AL U
NIV
ER
SIT
Y O
F
AT
HE
NS
Automatic Image Annotation
Sky
Sky
Fog
MountainMountain
Mountain
Field
Field
FieldField
FieldRoofWall
Sky
Mountain
Field
House
Input image
Automatic segmentation Desired result
--
NA
TIO
NA
L TE
CH
NIC
AL U
NIV
ER
SIT
Y O
F
AT
HE
NS
Tool for:Ground truth constructionSemi-automatic image annotation
Support of:Automatic segmentationManual, user driven region mergingExport of segmentation masks and textual annotation
Image Annotator Tool
--
NA
TIO
NA
L TE
CH
NIC
AL U
NIV
ER
SIT
Y O
F
AT
HE
NS
Visual Descriptor Ontology
MPEG-7(XML Schema) defines visual descriptors by specifying their componentsIn VDO (RDFS), descriptors are defined through relations with their components Descriptors related to higher – level concepts through inference rulesRules define spatio-temporal constraints
--
NA
TIO
NA
L TE
CH
NIC
AL U
NIV
ER
SIT
Y O
F
AT
HE
NS
Knowledge-Assisted Analysis Tool
Developed in collaboration with CERTH-ITI
--
NA
TIO
NA
L TE
CH
NIC
AL U
NIV
ER
SIT
Y O
F
AT
HE
NS
KAA Results
<?xml version="1.0" encoding="UTF-8"?>
<KAA>
<SpatialDecomposition id="KaaMask">
<Region>
<RegionNumber>0</RegionNumber>
<Concept>Sea</Concept>
<Confidence>0.81172</Confidence>
</Region>
<Region>
<RegionNumber>1</RegionNumber>
<Concept>Person</Concept>
<Confidence>0.948059</Confidence>
</Region>
<Region>
<RegionNumber>2</RegionNumber>
<Concept>Sea</Concept>
<Confidence>0.80658</Confidence>
</Region>
<Region>
<RegionNumber>3</RegionNumber>
<Concept>Sand</Concept>
<Confidence>0.885552</Confidence>
</Region>
<Region>
<RegionNumber>4</RegionNumber>
<Concept>Sky</Concept>
<Confidence>1</Confidence>
</Region>
</SpatialDecomposition>
…
</KAA>
--
NA
TIO
NA
L TE
CH
NIC
AL U
NIV
ER
SIT
Y O
F
AT
HE
NS
Approach:Graph-based representation of imagesSemantic vs Syntactic: regions are assigned fuzzy set of labels instead of low-level featuresModification of traditional segmentation algorithms to operate on labelled regionsSimultaneous image segmentation and region labeling
Target:Solve oversegmentation problemsAssign labels with confidence values to regionsLink labels with concepts existing in ontologies
Semantic Segmentation
--
NA
TIO
NA
L TE
CH
NIC
AL U
NIV
ER
SIT
Y O
F
AT
HE
NS
Sea is oversegmented
People have been incorrectly merged with the sand
RSST segmentation
Semantic RSST segmentation
Region is assigned to a fuzzy set of labels:
{rock/0.89,sand/0.46}
Sea segments
are merged correctly
Semantic Segmentation
--
NA
TIO
NA
L TE
CH
NIC
AL U
NIV
ER
SIT
Y O
F
AT
HE
NS
Visual Attention & Classification
Generate visual saliency mapsDetect foreground / backgroundSelect most representative regions for classification, based on saliency
Lower Classification errorOriginal Saliency MapBackground
Detection
