Roberto GarcíaUniversitat de Lleida, Spain

Createasphere Digital Asset Management Conference (DAMNY’13)

October 7-8, 2013, New York

10.04.2023 Slide 2 of 25

MediaMixerCommunity set-up and networking for the reMIXing of online MEDIA fragments

• European Research Project– Started November 2012 – Facilitate a Web-wide market for

media fragments re-purposing and re-sellingA history of G8 violence (© Reuters) http://slideshare.net/troncy/a-semantic-multimedia-web-create-annotate-present-and-share-your-media

10.04.2023 Slide 3 of 25

Introduction

• MediaMixer promotes the use of semantic technologies for media mixing

• Apply them in real use cases and demos– Fragmenting media assets– Annotating them using semantic descriptions– Exposing descriptions for fragment level search

–Representing, integrating and checking copyright

10.04.2023 Slide 4 of 25

Use Case

• User Generated Content

10.04.2023 Slide 5 of 25

Use Case

• UGC from reused media

10.04.2023 Slide 6 of 25

Use Case – ContentID

ContentID ©

©ContentID

match

match

match

10.04.2023 Slide 7 of 25

Use Case

©Block

Choices:

Track

Monetize

“Gangnam Style”, plus +33,000 videos using it, more than $870,000 ad revenue - New York Times, Dec 7th 2012

10.04.2023 Slide 8 of 25

Use Case - Disputes

• What if more than one ownership claim?

• If disputed, no one can monetize

©© match match

10.04.2023 Slide 9 of 25

Use Case - Claim?

• Are we sure we can claim?• Do we own that particular copy?• In that territory?• Also streaming on YouTube?• Does the talent authorise YouTube?• …

10.04.2023 Slide 10 of 25

Use Case - Claim?

…1000s of pieces of registered content……1000s of videos on YouTube…• Decision Support System (DSS):– Disputes with Media on A and B, can we claim?

– Trade A for B with OtherMedia (both win, start to get revenue)

Can

Claim Cannot

ClaimA B

10.04.2023 Slide 11 of 25

Use Case - Claim?

• Rights DSS requirements:– Fine grained– Scalable (largely automatized)– Takes into account:

• Contracts“…all rights on the live version but studio version just in the US.”

• Policies“…artist does not want his music together with violent images”.

• Rights Expression LanguagesDDEX metadata: <UseType>OnDemandStream</UseType>

<TerritoryCode>Worldwide</TerritoryCode>

10.04.2023 Slide 12 of 25

MediaMixer TechnologiesProposed Architecture

10.04.2023 Slide 13 of 25

Media Fragments

13

• Goal: transform whole videos to sets of meaningful, indexable and re-usable video fragments

Person, Snow, Trees, Building, Ski, …

Baseball, Throwing, Sports, Plant, Running, …

Kitchen, Indoor, Cake, …

…

Concept and Event Detection

Semantic Media

Annotation

10.04.2023 Slide 14 of 25

Linked Data

• Globally unambiguous identifiers for any concept needed– Ideally more information

about the concept itself is available via the identifier

– Use of Web URIs → Linked Data concept space

– Would allow for inference of concept type and relationship to other concepts

Semantic Media

Annotation

dbpedia:Category:Whisky

dbpedia:Category:Distilled_beverages

broader

dbpedia:Category:Alcoholic_beverages

broader

Policy: “Avoid streaming mediashowing alcoholic beverages in countries prohibiting alcohol”

dbpedia:List_of_countries_with_alcohol_prohibition

dbpedia:Category:Member_states_of_the_Organisation_of_Islamic_Cooperation

dbpedia:Saudi_Arabia

subject

ddex:OnDemandStream iso3166-2:SA

same as

location

theme

Check: streaming in Saudi Arabia?

Media Fragment

Semantic Media

Annotation

10.04.2023 Slide 16 of 25

From Controlled Vocabularies …to Ontologies

INCREASING EXPRESSIVENESS

Features Controlled Vocabularies Synonyms Taxonomies Thesaurus Ontologies

Control Ambiguity X X X X

Control Synonym X X X X

Hierarchical Relations X X X

Associative Relations X X

Custom Relations X

10.04.2023 Slide 17 of 25

Copyright Ontology

Data & Ontologies Integration

RDF Store & Reasoner

Copyright Ontologyschema.org

DDEX Ontology

ISO 3166Country Codes

Place

Action

SA

locationrange

is a

CreationObject

VideoScreenCapture

TVEpisode subclass of

subclass of

Use

MakeAvailable

subclass of

OnDemandStream

subclass of

10.04.2023 Slide 18 of 25

Copyright Ontology

Lifecycle, Actions, Rights,…

RDF Store & Reasoner

10.04.2023 Slide 19 of 25

• Ontology provides building blocks to model and reason about contracts, policies, rights expression languages…

No, because it is matched to pattern

which prohibits it:“Artists does not want her music with violent images”

Can we claim asset X

Agree

Disagre

e

Copyright Ontology

Reasoning

RDF Store & Reasoner

dbpedia:Category:Whisky

dbpedia:Category:Distilled_beverages

broader

dbpedia:Category:Alcoholic_beverages

broader

Policy: avoid streaming mediashowing alcoholic beverages in countries prohibiting alcohol

dbpedia:List_of_countries_with_alcohol_prohibition

dbpedia:Category:Member_states_of_the_Organisation_of_Islamic_Cooperation

dbpedia:Saudi_Arabia

subject

iso3166-2:SA

same asREASONING

ddex:OnDemandStream

location

theme

10.04.2023 Slide 21 of 25

Semantic Data Exploration

http://rhizomik.net/mediamixer/

Rhizomer Media Explorer

10.04.2023 Slide 22 of 25

http://rhizomik.net/mediamixer/rightsbuilderui/

Assisted Policy and Agreement Modelling User Interface

Rights Builder User Interface

10.04.2023 Slide 23 of 25

Semantic Technologies for Rights Data Integration & Intelligence

MediaMixer Semantic Techs

Policy 1: The licensor disagrees the licensee adaptsmedia fragments containing…

<xml><right> <play/></right><content id=“frg1”/></xml>

Semantic Media

Annotation

Rights Builder User Interface

Rights Language Mapping

RDF Store & Reasoner

media fragments, semantic data, annotation, ontologies, reasoning,…

10.04.2023 Slide 24 of 25

Interested? Get involved• Join MediaMixer community to know more

(http://community.mediamixer.eu):– discussions, use cases, demonstrators,

tutorials, presentations, software,…

More details:http://community.mediamixer.eu@project_mmixer

Roberto García (@rogargon)Universitat de Lleida, Spain

http://rhizomik.net/~roberto

Thank you for your attention