IMF Mechanisms for carrying Metadata by Marquise Technologies

EBU Production Technology Seminar, January 2017

Dan Tatut, VP Business Strategy & Development

IMF Mechanisms for carryingMetadataCurrent Practices

IMF mechanisms for carrying metadata

• CPL structure• Intrinsic & Extrinsic metadata• Metadata in track files• Metadata processing via OPLs (Output Profile Lists)• About timecode• Conclusion / Topics for debate

OVERVIEW


• <Annotation> user-defined text (comments about/description of the composition)• <Creator> software/device that created the composition (usually not user-

editable)• <ContentOriginator> who created the composition (e.g. studio name)• <ContentTitle> title of the composition, often using a “naming convention”

• <ContentKind> defined in ST2067-3 or custom value (scope attribute should be

used)• <ContentVersionList> a list of ID/label pairs, references to external metadata• <LocaleList> a list of <Locale> structures for language(s), region(s) and content

maturity rating(s)

What is available in the “Prologue section” ?

• <ContentVersionList> is a list of <ContentVersion> elements• Each <ContentVersion> is a pair of <Id> and <LabelText> elements• Each Id/LabelText pair can be anything, e.g.:


External metadata through “<ContentVersionList>”

OWNER FORMAT

Disney <Id>tag:disney.com,2015:wmls-id:7000001</Id><LabelText>In & Out (US English)</LabelText>

Warner Bros <Id>tag:warnerbros.com/series/mpmNumber,2004582</Id><LabelText>series(GILLIGAN'S ISLAND - YR02 65/66)</LabelText>

EIDR <Id>urn:eidr:10.5240:4FEC-7003-F150-AD59-9D70-S</Id><LabelText>This is an EIDR reference</LabelText>

ISAN <Id>urn:isan:aef0-1000-6721-022-x-0000-9034-1</Id><LabelText>This is an ISAN reference</LabelText>

• <EssenceDescriptor> elemets are part of <EssenceDescriptorList>

• Each <EssenceDescriptor> exposes in the CPL asset metadata using SMPTE

RegXML• Some assets can carry user defined metadata, e.g. MCA in the audio tracks


A word on “<EssenceDescriptor>” elements

MCA Metadata

...<MCATitle>The Great Escape</MCATitle><MCATitleVersion>this is version 2</MCATitleVersion><MCATitleSubVersion>actually version 2.1b</MCATitleSubVersion><MCAEpisode>of the first episode<MCAEpisode>...

• <ExtensionProperties> is the weapon of choice for adding static metadata to

the CPL• One unique instance of any custom-defined element• Must be identified by their specific name spaces• Can contain any sub-structure (as defined by their XML schema)• Can be ignored by QC systems (unless they understand the schema)• Can be stored internally or externally to the CPL (linking is extension specific)• <ExtensionProperties> outside of the timeline, not well suited for time-based

metadata


Adding metadata through“<ExtensionProperties>”

• CPL-embedded EBU extension for linking EBUCore documents as presented at PTS 2016


Examples using “<ExtensionProperties>”<ExtensionProperties> <ebuext:EBUExtension xmlns:ebuext="http://tech.ebu.ch/schemas/XXXX"> <ebuext:EBUCoreList> <ebuext:EBUCore> <Id>urn:uuid:7a5656b3-64b0-4dc7-960b-5a819c4669e5</Id> <LabelText>France TV</LabelText> </ebuext:EBUCore> <ebuext:EBUCore> <Id>urn:uuid:2945c709-8f76-491e-8aaf-bccb97bf99d8</Id> <LabelText>UK DPP</LabelText> </ebuext:EBUCore> </ebuext:EBUCoreList> </ebuext:EBUExtension></ExtensionProperties>


Examples using “<ExtensionProperties>”<ExtensionProperties> <mt:ExtensionProperties xmlns:mt="http://www.marquise-tech.com/imf/2016/cpl-extension-properties"> <mt:DM_AS_11_UKDPP_Framework> <mt:UKDPP_Audio_Loudness_Standard>Loudness_None</mt:UKDPP_Audio_Loudness_Standard> <mt:UKDPP_Textless_Elements_Exist>False</mt:UKDPP_Textless_Elements_Exist> <mt:UKDPP_Picture_Ratio>16/9</mt:UKDPP_Picture_Ratio> <mt:UKDPP_Tertiary_Audio_Language>zxx</mt:UKDPP_Tertiary_Audio_Language> <mt:UKDPP_Audio_Comments>Mixed to PPM 6</mt:UKDPP_Audio_Comments> <mt:UKDPP_PSE_Pass>PSE_Not_tested</mt:UKDPP_PSE_Pass> ... <mt:UKDPP_Genre>Rock/Pop</mt:UKDPP_Genre> ... <mt:UKDPP_Originator>BBC S&PP DMS</mt:UKDPP_Originator> <mt:UKDPP_Product_Placement>False</mt:UKDPP_Product_Placement> ... </mt:DM_AS_11_UKDPP_Framework> </mtextprop:ExtensionProperties></ExtensionProperties>


Examples using “<ExtensionProperties>”<ExtensionProperties> <imf:ExtensionProperties xmlns:imf="http://dalet.com/api/imf/2015-12/imp-metadata-extension"> <WB:WBExtensionProperties xmlns:WB="http://warnerbros.com/schemas/imf/2016/extensionProperties"> <Series xmlns="http://warnerbros.com/schemas/imf/2016/extensionProperties"> <Name>GILLIGAN'S ISLAND - YR02 65/66</Name> <MpmNumber>2004582</MpmNumber> ... </Series> <Title xmlns="http://warnerbros.com/schemas/imf/2016/extensionProperties"> <MpmNumber>4034033</MpmNumber> ... <EpisodeNumber>001056</EpisodeNumber> <EpisodeSequenceNumber/> ... </Title> ... </WB:WBExtensionProperties> </imf:ExtensionProperties></ExtensionProperties>

• IMF allows metadata extension at the timeline level. This is done through virtual

tracks• Can be intrinsic or extrinsic to the CPL• Can be time varying or constant, contiuous or discrete.• Extensions are identified by their XML namespace/schema• Can be ignored by QC systems that do not understand them• Must follow the timeline rules (tracks, segments, sequence, etc.)


Adding timeline metadata

M1 & M2

A1

Soundfield 5.1

Metadata


SEGMENT 1 SEGMENT 2 SEGMENT 3

V1Sequence

Sequence

Sequence


Examples of intrinsic timeline metadata: markers<MarkerSequence>... <ResourceList> <Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" si:type="cpl:MarkerResourceType"> <Id>urn:uuid:ce398822-cd01-4eee-b725-965bc78ea3e1</cpl:Id> <Annotation/> <EditRate>24000 1001</EditRate> <IntrinsicDuration>0</IntrinsicDuration> <EntryPoint>0</EntryPoint> <SourceDuration>0</SourceDuration> <Marker> <Annotation>SMPTE_BARS</Annotation> <Label scope="http://vubiquity.com/dete/imf/2016/markerLabel">SMPTE_BARS</Label> <Offset>0</Offset> </Marker> </Resource> </ResourceList></MarkerSequence>


Examples of intrinsic timeline metadata: RDD6<rdd6:RDD6Sequence xmlns:rdd6="http://bbc.co.uk/imf/rdd6/2017">... <ResourceList> <Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" si:type="rdd6:RDD6ResourceType"> <Id>urn:uuid:ce366622-cd01-4e2e-1115-7622c48ea3e1</Id> <EditRate>25 1</EditRate> <IntrinsicDuration>0</IntrinsicDuration> <EntryPoint>0</EntryPoint> <SourceDuration>0</SourceDuration> <RDD6> <FirstSubFrame>... </FirstSubFrame> <SecondSubFrame>... </SecondSubFrame> </RDD6> </Resource> </ResourceList></rdd6:RDD6Sequence>


Examples of extrinsic timeline metadata: TT commentary<cc:CommentarySequence xmlns:cc="http://www.smpte-ra.org/schemas/2067-2/2016"> <Id>urn:uuid:0ea040c7-418f-4b3d-b27e-8b4b3da9bdfa</Id> <TrackId>urn:uuid:b13a9521-df5b-403a-8634-cacf14094835</TrackId> <ResourceList> <Resource xsi:type="TrackFileResourceType"> <Id>urn:uuid:53c5dd91-9b31-4a89-a63b-b2d8d7112627</Id> <EditRate>24000 1001</EditRate> <IntrinsicDuration>4208</IntrinsicDuration> <SourceEncoding>urn:uuid:3ef4dbcf-d4e1-a432-b4b0-38da2d96442f</SourceEncoding> <TrackFileId>urn:uuid:3ef4dbcf-d4e1-a432-b4b0-38da2d96442f</TrackFileId> <Hash>T8pvcuXA+GVQM/fc0DMlgoMwOSE=</Hash> </Resource> </ResourceList></cc:CommentarySequence>

• <SourceEncoding> references an <EssenceDescriptor> defined in the <EssenceDescriptorList>

• The actual “external metadata” is referenced by the <TrackFileId> element


Examples of extrinsic timeline metadata: ancillary data<cc:AncillaryDataSequence xmlns:cc="http://www.smpte-ra.org/schemas/2067-2/2016"> <Id>urn:uuid:cc64ab4c-d7bf-4453-85a6-40f372bc4f6c</Id> <TrackId>urn:uuid:cf4a166f-3110-4512-a3d7-8193102c0458</TrackId> <ResourceList> <Resource xsi:type="TrackFileResourceType"> <Id>urn:uuid:0739ffce-918b-496c-b2c7-f3d2ef06f3f2</Id> <EditRate>24000 1001</EditRate> <IntrinsicDuration>719</IntrinsicDuration> <SourceEncoding>urn:uuid:0d28fd36-8485-490e-a06d-b2bdcebc8fbf</SourceEncoding> <TrackFileId>urn:uuid:c1e8dfc3-495b-44ba-a436-35027d0db0a3</TrackFileId> </Resource> </ResourceList></cc:AncillaryDataSequence>

• <SourceEncoding> references an <EssenceDescriptor> defined in the <EssenceDescriptorList>

• The actual “external metadata” is referenced by the <TrackFileId> element

• Intrinsic timeline metadata is stored in plain XML in the CPL• Extrinsic timeline metadata is stored using “any” encoding method as long as

there is a MXF wrapping defined• Extrinsic timeline metadata can be transmitted as supplemental packages

• Extrinsic timeline metadata can have any size

• Extrinsic timeline metadata can be reused multiple times (multiple references)• Dynamics are defined by the encoding. No “generic animation mechanism”


Adding timeline metadata

• Product placement• Per-frame technical metadata (e.g. camera metadata, geo-localisation)• Creative changes (pan & scan), colorimetry (ST2094), etc.• Support for legacy essence formats (EBU STL, CEA 608/708, etc.)• New accessibility tracks• Live & post-live capture metadata• Etc.


Examples of timeline metadata

• Instructions on how to “process” CPLs• List of standard (SMPTE) & custom macros• Each OPL references a CPL• Can have <ExtensionProperties> too!• Metadata access in the CPL via CPL UUID• Metadata access in the CPL’s virtual tracks via UUID


Metadata processing via OPLs


Metadata processing via OPLs...<MacroList> <Macro type="mtio:FileOutMacroType"> <Name>macro_9313321f-7f2b-4ced-939d-efbf54ca8dff</Name> <mtio:InputImageSequence xmlns:mtio="http://www.marquise-tech.com/schemas/imf/2016/fileout-macro"> <Handle>cpl/virtual-tracks/a8bb2208-3915-43b1-ab51-1b8ea48ae8ff</Handle> </mtio:InputImageSequence> ... <mtio:RDD6Sequence xmlns:mtio="http://www.marquise-tech.com/schemas/imf/2016/fileout-macro"> <Handle>cpl/virtual-tracks/cff02228-faac-7543-dd23-cba6695aa3e2</Handle> </mtio:RDD6Sequence> ... <mtio:OutputFormat xmlns:mtfileout="http://www.marquise-tech.com/schemas/imf/2016/fileout-macro"> <mtio:Container>mxfas11ukdpp</mtfileout:Container> </mtio:OutputFormat> </Macro></MacroList>...


How to choose a strategy?PLACE ROLE GLOBAL TIMELINE

CPL-related Related to specific moments in the timeline

INTRINSIC Essential to CPL interpretation, XML based, smaller than the CPL

Use a child element in <ExtensionProperties>

Store as a resource in a virtual track (could be custom)

EXTRINSIC Opposite of intrinsic Stored as external asset (e.g. QC report, EBUCore XML documents, etc.

Stored as Track file, referenced by Virtual Track in CPL (e.g. ST2094)

• IMF does not make use of TIMECODE

• CPLs have an optional timecode metadata• MXF supports timecode• IMF MXFs can carry the timecode you want/like (e.g. original/source timecode)• However timecode is asset-bound, not composition-bound• Stop thinking tape

• Start thinking files, structured edit/playlists & assets


Last but not least: Timecode!

• Current practice: UNREADABLE & INCOMPETENT NAMING CONVENTIONS

• SMPTE 35PM50 working on a specification to “ease the attachement” of sidecar

files• EBU IMF-TV working on this subject too!• Send comments/feedback, participate!


Conclusion

contact information

www.marquise-tech.com

[email protected]

THANK YOU

IMF Mechanisms for carrying Metadata by Marquise Technologies

Technology

Transcript of IMF Mechanisms for carrying Metadata by Marquise Technologies