MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis...
Transcript of MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis...
![Page 1: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/1.jpg)
MPEG-7 Interoperability& the Semantic Web
Yiannis KompatsiarisMultimedia Knowledge Laboratory
CERTH - Informatics and Telematics Institute
Thanks toS. Dasiopoulou (ITI), V. Tzouvaras (NTUA)
![Page 2: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/2.jpg)
22
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Outline
• MM content annotation
• MPEG-7
• Semantic Web
• Interoperability issues
• MPEG-7 ontologies
• Towards Interoperable Semantic MultimediaDescriptions
• Conclusions
![Page 3: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/3.jpg)
33
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Evolution of Content• 1-2 exabytes (millions of
terabytes) of new informationproduced world-wide annually
• 80 billion of digital images arecaptured each year
• Over 1 billion images related tocommercial transactions areavailable through the Internet
• This number is estimated toincrease by ten times in the nexttwo years.
• 4 000 new films are producedeach year
• 300 000 world-wide availablefilms
• 33 000 television stations and 43000 radio stations
• 100 billions of hours ofaudiovisual content
Personal Content
Sport - News
Movies
Web
Mobile
![Page 4: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/4.jpg)
44
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Need for annotation + medatata
“The value of information depends on howeasily it can be found, retrieved,
accessed, filtered or managed in anactive, personalized way”
![Page 5: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/5.jpg)
55
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
MM Content Metadata
• Two levels within the annotation context fromanalysis, search and retrieval perspective
• conceptual (descriptive, domain specific)
• media (structural, decomposition specific)
(signal level: feature representation)
![Page 6: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/6.jpg)
66
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Conceptual and mediaannotation
Conceptual levelMedia level
(example from http://comm.semanticweb.org/examples)
![Page 7: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/7.jpg)
77
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
MM Content Metadata
• Domain specific ontologies (Semantic Web)provide the vocabulary & semantics fordescriptive-related annotations
• MPEG-7 provides (among others)standardised description tools for media-related annotations
• How can they be combined?
• What happens with interoperability?
![Page 8: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/8.jpg)
88
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
MPEG-7
• MPEG-7: greatest effort towards standardisedmultimedia content descriptions• rich set of broad coverage tools
• generic schema: usage flexibility
• Interoperability issues• XML Schema (lack of precise semantics)
• ambiguous interpretations
• incompatibility with proliferating SW technologies
![Page 9: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/9.jpg)
99
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Interoperability Issuesin MPEG-7
• XML Schemas define only the structure of well-formed descriptions
• e.g. MovingRegion DS vs VideoSegment DS
• Multiple representations per semantic notion
• e.g. an image can be represented using either ofStillRegion DS, VideoSegment DS, etc.
![Page 10: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/10.jpg)
1010
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
<Mpeg7>
<Description xsi:type="ContentEntityType">
<MultimediaContent xsi:type=„ImageType">
<Image>
<SpatialDecomposition>
<StillRegion id=„SR1“>
<TextAnnotation>
<KeywordAnnotation xml:lang="en">
<Keyword>Sky</Keyword>
</KeywordAnnotation>
</TextAnnotation>
</StillRegion>
<StillRegion id=„SR2“>
<Semantic>
<Label>
<Name>Sky</Name>
<Label>
</Semantic>
</StillRegion>
<StillRegion id=„SR3“>
<Semantic> <Definition> <!-- Also TextAnnotation!! -->
<StructuredAnnotation>
<WhatObject>
<Name xml:lang="en">Sky</Name>
</WhatObject>
</StructuredAnnotation>
</Definition>
</Semantic>
</StillRegion>
...
Segment 1
Segment 2
Segment 3
How do you formulate
a query to get all segments
that show „Sky“?
(syntactic interoperability)
What is „Sky“?
No ability to automatically
process the information in a
machine-understandable
manner
(semantic interoperability)
(example from Semantics in Multimedia Content for Multimedia Use,Steffen Staab, University of Koblenz - Landau, Germany,
http://mklab.iti.gr/ist2006)
![Page 11: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/11.jpg)
1111
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Semantic Web vision
• Technologies to capture, represent, exchange,and share metadata
• Logic-grounded formal semantics
• Explicit, machine processable semantics
challenge & prerequisite for realisticsemantic enabled applications
![Page 12: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/12.jpg)
1212
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
What is an ontology?
Objekt
Engine Gear Car
GasolineDiesel
isConnected
hasPart
Type
Power
TDI V6
V 6136
hasPart isIn
hasPartX hasPart Y hasPart Z X Z
YX X Y
Rules / Query Language
hasPart
Concept
Instance
(example from "Semantic Web: ontologies, applications and EU projects”, Professor Juergen Angele,http://www.multimine.gr)
![Page 13: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/13.jpg)
1313
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Semantic Web Architecture
Data Exchange
Entailment of the Implicit
Explicit Semantics
Relational Distributed Data
![Page 14: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/14.jpg)
1414
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
MultimediaSemantic
Interoperabilityin the SW
![Page 15: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/15.jpg)
1515
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Scenario 1
• Information exchange in predefined format
• XML-based ! “meaning” lies in strictconformance to exchanging format
• Achieves interoperability at functional level
However: it’s syntactic interoperability,not semantic interoperability
![Page 16: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/16.jpg)
1616
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Scenario 2
• Formal representation language• machine understandable interpretation
• Unified Resource Identifiers (URIs)
• Each MM repository adopts its ownrepresentation formalism and conceptualschema
Automated mapping and reasoningrequired to achieve information/meaningintegration
![Page 17: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/17.jpg)
1717
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Scenario 3
• “Semantic Version” of scenario 1
• Reference framework: uppermultimedia ontologies
• common understanding for semantic meaning
• All multimedia repositories use thecommon, upper multimedia ontologies
Requires mutualagreement
![Page 18: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/18.jpg)
1818
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Scenario 4
• An attempt to fulfill the SW vision
• Each multimedia repositoryuses its own ontologies
• Mappings are defined (manually/ automatically)to shared, upper multimedia ontologies
inability of semantic technologies to support asatisfactory level of automation, especially forhighly expressive semantics
![Page 19: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/19.jpg)
1919
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Semantic MultimediaDescriptions
• SW technologies: subject descriptive
• MPEG-7: media descriptive• structure (decomposition, localisation)
• signal descriptors (colour, motion)
Attach formal semantics to MPEG-7through its ontologicalrepresentation, overcoming itsinteroperability issues
![Page 20: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/20.jpg)
2020
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
State-of-the-Art in MPEG-7ontologies
• Different approaches• normative semantics allow for individual
interpretations of descriptions
• Lack of interoperability at conceptual level
• MPEG-7 ambiguities propagate
![Page 21: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/21.jpg)
2121
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Hunter’s (FUSION) approach
• Chronologically the first approach
• Addresses Multimedia DSs plus the Audio andVisual Parts
• Expressed in RDFS (plus OIL constructs),eventually ported to OWL
• Direct translation from the specifications• maintains MPEG-7 intended flexibility
• can be handy for handling part-whole semantics
• Conceptual differences are not addressed
![Page 22: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/22.jpg)
2222
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
MM Content
class hierarchy
Segment
class hierarchy
<owl:Class rdf:ID=“Video">
<rdfs:subClassOf rdf:resource="#MultimediaContent"/>
</owl:Class> ! no axiomatised definition
<owl:ObjectProperty rdf:ID="videoSegment_spatial_decomposition">
<rdfs:subPropertyOf rdf:resource="#spatial_decomposition"/>
<rdfs:domain rdf:resource="#VideoSegment"/>
<rdfs:range rdf:resource="#MovingRegion"/>
</owl:ObjectProperty>
Decomposition relations
defined per multimedia
content/segment pair
![Page 23: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/23.jpg)
2323
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
The aceMedia approach
• Extends the Hunter approach
• RDFS ontology• expressivity restricted to subclass and domain/range
• Introduction of additional concepts not includedin MPEG-7 to account for notions semanticallydistinct in humans (e.g. video frame)
• Two ontologies• MSO (MPEG-7 Multimedia Description Scheme)
• VDO (MPEG-7 Visual Part)
![Page 24: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/24.jpg)
2424
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Multimedia Content
class hierarchySegment
class hierarchy
Additional
Concepts
Additional Properties
•hasStartFrame
•hasEndFrame
•hasFrameIndex
![Page 25: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/25.jpg)
2525
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
The SmartWeb approach
• OWL DL ontology covering the ContentDescription and Content Management DSs
• MultimediaContent and Segment classes aredefined as disjoint
• Definition of properties for the representation ofdecomposition description tools
• Semantics “hidden” in the employed linguisticterms to a large extend
![Page 26: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/26.jpg)
2626
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Multimedia Content & Segment
class hierarchies
Semantics interrelated with naming conventions extensibility/alignment issues
![Page 27: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/27.jpg)
2727
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
The BOEMIE approach
• Goal: cleaner, explicit conceptualisation
• OWL DL
• Identification of different types of MM content• discrimination between single & composite
• Identification of respective segment types• grouped based on the applicable decomposition dimension
(spatial, temporal, spatiotemporal)
• MM content types are defined using the decompositionrelations in conjunction with the applicable segment types
• Two ontologies (MCO & MDO)
![Page 28: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/28.jpg)
2828
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Inferred Multimedia
Segment class hierarchy
Multimedia Content & Segment
Locator class hierarchy
![Page 29: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/29.jpg)
2929
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
OntologyInfrastructure
• Integration of multimedia anddomain specific ontologiesusing upper level ontologies• DOLCE (aceMedia, SmartWeb)
• SUMO (SmartWeb)
• ABC (Fusion)
• Examples of Scenario 2• upper ontologies serve as reference for the domain
specific ontologies
• provide means for linking to multimedia ones, notupper multimedia reference framework
![Page 30: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/30.jpg)
3030
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Garcia’s and Celma’s approach
• Automatic translation of entire MPEG-7 based ona generic XML Schema to OWL mapping
• The MM DS, the Audio and Visual Parts havebeen evaluated against the manually createdHunter’s ontology
• OWL Full ontology• limits practical applicability (tool support, sound
automated inference)
![Page 31: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/31.jpg)
3131
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
The DS-MIRF approach
• Methodology for the complete MPEG-7translation into OWL DL
• Focus on the Semantic DS
• Intended to serve as a core multimediaontology• domain specific ontologies attachment through
subclass/subproperty relations
![Page 32: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/32.jpg)
3232
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
OntologyInfrastructure
• Approaches within Scenario 3 usingMPEG-7 as the reference multimediaontology
• MPEG-7 as upper ontology for bothmultimedia and domain specificontologies• verbose to adapt (media oriented)
• MPEG-7 conceptual ambiguities aren’tresolved making infeasible the definition ofmappings from individual mm ontologies
![Page 33: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/33.jpg)
3333
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
OWL Application Ontologies
Fully captures the MPEG-7 MDS and
the MPEG-21 DIA...
...
OWL Upper Ontology
“Fill the blanks” left from the MPEG -7 MDS and the MPEG-21 DIA
for Application Support
Typed
Relationships
Semantic User Preferences
Application
Ontology X
...
OWL Domain Ontologies
Extend the Upper Ontology and the Application Ontologies with
Domain Knowledge
Soccer Domain Ontology
F1 Domain Ontology
Domain Ontology X
...
(example from http://www.w3.org/2005/Incubator/mmsem/meetings/f2f-athens/2006-12-08-
Tsinaraki-MMOntology.ppt )
The DS-MIRF OntologicalInfrastructure
![Page 34: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/34.jpg)
3434
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
The Core Ontology for Multimedia(COMM) initiative
• Extends the Descriptions & Situations (D&S) andOntology of Information Objects (OIO) designpatterns
• Models at semiotic level the aspects involved• aims to conceptualise the conceptualisation process
itself
The axiomatisation does not cover the media relatedrepresentations (i.e. the design patterns don’t propagate tothe MPEG-7 concept/properties subtrees)
![Page 35: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/35.jpg)
3535
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
(example from http://comm.semanticweb.org/examples)
![Page 36: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/36.jpg)
3636
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
The Core Ontology for Multimedia(COMM) initiative (cont’d)
• Does not resolve interoperability issues• two multimedia repositories using the COMM
in combination with the FUSION andSmartWeb approaches respectively cannotexchange meaning
• Systematic exploration towards aconceptual schema for structuringsemantically different multimedia relatedaspects
![Page 37: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/37.jpg)
3737
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Towards Interoperable SemanticMultimedia Descriptions
• MPEG-7 (multimedia) ontologies
• explicit semantics
• Semantic Web compliant
• Possible solution: manual definition of mappings
• relatively straightforward with upper ontologies
• hardly possible when considering conceptualdifferences in the individual multimedia ontologies
![Page 38: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/38.jpg)
3838
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Common Multimedia OntologyFramework
• Multimedia ontologies harmonisation
• Collaborative specification of aspects involved inmultimedia description
• Definition of “minimum set” for a multimediaontology to be used as a basis for all multimediarelated applications to build on and extend
![Page 39: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/39.jpg)
3939
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
W3c Multimedia SemanticsIncubator Group (XG)
• Mission: show how metadata interoperability can beachieved by using the Semantic Web technologies tointegrate existing multimedia metadata standards
• Selection of use cases demonstrating commoninteroperability problems
• Possible solutions roadmap
![Page 40: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/40.jpg)
4040
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Conclusion
• MPEG-7 is the greatest effort towards standardisedmultimedia descriptions
• The Semantic Web provides the means to represent,exchange and process semantics of information
• Explicit, unambiguous MPEG-7 semantics (richaxiomatisation)• definition of semantics mappings• automated inference services
• Upper multimedia ontologies (conceptualclarity)• reference framework for multimedia related aspects• Modularity & extensibility
![Page 41: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/41.jpg)
4141
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
References
1. http://www.acemedia.org/acemedia/reference/multimedia_ontology/
2. http://www.w3.org/2005/Incubator/mmsem/
3. http://www.acemedia.org/
4. http://www.boemie.org/
5. http://www.smartweb-project.de/
6. http://metadata.net/sunago/fusion.htm/
7. http://comm.semanticweb.org/
![Page 42: MPEG-7 Interoperability & the Semantic Web · MPEG-7 Interoperability & the Semantic Web Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute](https://reader033.fdocuments.in/reader033/viewer/2022052717/5f0477fc7e708231d40e1ee7/html5/thumbnails/42.jpg)
4242
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Thank you!
CERTH-ITI / Multimedia Knowledge Laboratory
http://mklab.iti.gr