Navigating the transition from ONIX 2.1 to 3.0

Graham BellEDItEUR

BISG Metadata Committee & webcast16 July and 14th October 2014

Roots of ONIX• 1997 EPICS and BIC Basic

• 1998 <indecs> project, W3C XML specification

• 1999 ‘Online Information Exchange’ initiative from AAP Digital Issues working party

• ONIX developed by EDItEUR• 2000 ONIX 1.0 and 1.1 – obsolete

• 2001 ONIX 2.0 – retired

• 2003 ONIX 2.1 – revision 02 in 2004

• 2009 ONIX 3.0 – version 3.0.2 in 2014

Roots of ONIX• 1997 EPICS and BIC Basic

• 1998 <indecs> project, W3C XML specification

• 1999 ‘Online Information Exchange’ initiative from AAP Digital Issues working party

• ONIX developed by EDItEUR• 2000 ONIX 1.0 and 1.1 – obsolete

• 2001 ONIX 2.0 – retired

• 2003 ONIX 2.1 – revision 02 in 2004

• 2009 ONIX 3.0 – version 3.0.2 in 2014

Roots of ONIX• 1997 EPICS and BIC Basic

• 1998 <indecs> project, W3C XML specification

• 1999 ‘Online Information Exchange’ initiative from AAP Digital Issues working party

• ONIX developed by EDItEUR• 2000 ONIX 1.0 and 1.1 – obsolete

• 2001 ONIX 2.0 – retired

• 2003 ONIX 2.1 – revision 02 in 2004

• 2009 ONIX 3.0 – version 3.0.2 in 2014

Roots of ONIX• 1997 EPICS and BIC Basic

• 1998 <indecs> project, W3C XML specification

• 1999 ‘Online Information Exchange’ initiative from AAP Digital Issues working party

• ONIX developed by EDItEUR• 2000 ONIX 1.0 and 1.1 – obsolete

• 2001 ONIX 2.0 – retired

• 2003 ONIX 2.1 – revision 02 in 2004

• 2009 ONIX 3.0 – version 3.0.2 in 2014

Roots of ONIX

• widely used in North America, Western Europe, Eastern Europe and Russia,growing in Asia-Pacific

• used by small and large organisations alike• included in many off-the-shelf IT systems, but

also plausible for in-house developers

• current status• ONIX v2.1 – still most widely deployed in US

• ONIX v3.0 – growing in importance

ONIX governance• standard is managed by EDItEUR

• documentation and tools available free of charge, covered by permissive EDItEUR licence

• membership supports development, and members help identify new requirements

• all changes discussed by National Groups to ensure broad international applicability, and ratified by International Steering Committee to ensure stability

• BISG metadata committee is US national group

• ISC decided in 2011 to sunset ONIX 2.1

ONIX governance• standard is managed by EDItEUR

• documentation and tools available free of charge, covered by permissive EDItEUR licence

• membership supports development, and members help identify new requirements

• all changes discussed by National Groups to ensure broad international applicability, and ratified by International Steering Committee to ensure stability

• BISG metadata committee is US national group

• ISC decided in 2011 to sunset ONIX 2.1

support for 2.1 will be

reduced at end of 2014

ONIX 2.1 vs ONIX 3.0• two quite distinct messages

• block-level updates

• digital products

• sets and series

• sales rights

• marketing collateral

• parallel multi-lingual data

• related works

• international markets

ONIX 2.1 vs ONIX 3.0• two quite distinct messages

• block-level updates

• digital products

• sets and series

• sales rights

• marketing collateral

• parallel multi-lingual data

• related works

• international markets

10 years accumulated experience

extended schema support

global best practice

guide

so how do we gofrom 2.1 to 3.0 ?

ONIX 2.0 vs ONIX 2.1• not really distinct messages – ONIX 2.1 was

fully compatible with 2.0• deprecations of dedicated elements,

eg�<EAN13>, <BICMainSubject>

• encouragement to use composites instead, eg�<ProductIdentifier>, <MainSubject>

• optional new functionality, eg in <SalesRights>, <ProductFormDetail>, <MarketRepresentation>

• if your 2.1 has been brought ‘up-to-date’, then migration to 3.0 is much simpler than it is if you are still (in effect) using 2.0

ONIX 2.1 vs ONIX 3.0• these are distinct messages – there is no

backwards compatibility• removal of previously deprecated elements

(there are actually fewer tags in 3.0)

• continued development of composites, and the introduction of blocks

• new functionality, eg usage constraints, licensing, rentals, multi-lingual metadata, though again this is mostly optional

• but huge amount of continuity• at least half of the message remains unchanged

ONIX 3.0 data elements• message details

• identity and authority• record details

• product identifiers

• 1. descriptive details• product form

• special features

• packaging

• physical size

• drm, usage constraints

• trade classification

• product parts

• collection titles

• titles

• contributors

• conference

• edition

• language

• extent

• subject

• audience

ONIX 3.0 data elements• 2. collateral details

• supporting text

• cited material

• supporting resources

• prizes

• 3. content detail

• 4. publishing details• imprint and publisher

• lifecycle dates

• copyright details

• territorial rights

• 5. related material• related works

• related products

• 6. supply details• markets

• market details

• suppliers

• discounts

• prices and tax

• reissue details

1. spring cleaning

<ISBN>0007232837</ISBN><EAN13>9780007232833</EAN13>

1.0 1.1 2.12.01.2

<ISBN>0007232837</ISBN><EAN13>9780007232833</EAN13><ProductIdentifier> <ProductIDType>01</ProductIDType> <IDTypeName>HCP PID</IDTypeName> <IDValue>32032-4</IDValue></ProductIdentifier>

2.12.0

<ProductIdentifier> <ProductIDType>01</ProductIDType> <IDTypeName>HCP PID</IDTypeName> <IDValue>32032-4</IDValue></ProductIdentifier>

3.0<ProductIdentifier> <ProductIDType>03</ProductIDType> <IDValue>9780007232833</IDValue></ProductIdentifier><ProductIdentifier> <ProductIDType>15</ProductIDType> <IDValue>9780007232833</IDValue></ProductIdentifier>

2.12.0

<AudienceCode>03</AudienceCode>

<Audience> <AudienceCodeType>01</AudienceCodeType> <AudienceCodeValue>03</AudienceCodeValue></Audience>

3.02.12.0

2.1

<ProductAvailability>20</ProductAvailability>

<AvailabilityCode>IP</AvailabilityCode>

2.1 3.0

<ProductAvailability>20</ProductAvailability>

2. renaming and reordering

<Contributor> <SequenceNumber>1</SequenceNumber> <ContributorRole>A01</ContributorRole> <PersonName>Maj Sj&ouml;wall</PersonName> <PersonNameIdentifier> <PersonNameIDType>01</PersonNameIDType> <IDTypeName>HCP Author ID</IDTypeName> <IDValue>7421</IDValue> </PersonNameIdentifier> <BiographicalNote textformat="05"><p>Maj��������Sj&ouml;wall is a poet. She lives in Sweden.</p>��������</BiographicalNote></Contributor>

2.1

<Contributor> <SequenceNumber>1</SequenceNumber> <ContributorRole>A01</ContributorRole> <NameIdentifier> <NameIDType>01</NameIDType> <IDTypeName>HCP Author ID</IDTypeName> <IDValue>7421</IDValue> </NameIdentifier> <PersonName>Maj Sjöwall</PersonName> <BiographicalNote textformat="05"><p>Maj��������Sjöwall is a poet. She lives in Sweden.</p>��������</BiographicalNote></Contributor>

3.0

3. restructuring for greater consistency and flexibility

<PublicationDate>20110428</PublicationDate><YearFirstPublished>1811</YearFirstPublished>

<OnSaleDate>20110428</OnSaleDate>

2.1

<PublishingDate> <PublishingDateRole>11</PublishingDateRole> <Date dateformat="05">1811</Date></PublishingDate><PublishingDate> <PublishingDateRole>19</PublishingDateRole> <Date dateformat="01">198510</Date></PublishingDate><PublishingDate> <PublishingDateRole>01</PublishingDateRole> <Date>20110428</Date></PublishingDate><PublishingDate> <PublishingDateRole>02</PublishingDateRole> <Date>20110428</Date></PublishingDate>

3.0

<SalesRights> <SalesRightsType>01</SalesRightsType> <RightsCountry>AG AI AU BB BD BM BN BS BW ��������BZ CM CY DM EG FJ FK GB GD GH GI GM��������GY HK IE IL IN IO JM JO KE KI KN KW KY��������LC LK LS MT MU MV MW MY MZ NA NG��������NR NZ PG PK PN SB SC SD SG SH SL SO SZ��������TC TO TT TV TZ UG VC VG VU WS YE RS ME��������ZA ZM ZW</RightsCountry></SalesRights>

2.1

<SalesRights> <SalesRightsType>01</SalesRightsType> <RightsCountry>CA US</RightsCountry></SalesRights><SalesRights> <SalesRightsType>03</SalesRightsType> <RightsCountry>GB IE AU NZ</RightsCountry></SalesRight>

2.1

<SalesRights> <RightsCountry> * <RightsTerritory> * <RightsRegion> (dep.)<NotForSale> <RightsCountry> * <RightsTerritory> *

2.1

<MarketRepresentation> <MarketCountry> * <MarketTerritory> * <MarketCountryExcluded> *

<SupplyDetail> <SupplyToCountry> * <SupplyToTerritory> * <SupplyToRegion> (dep.) <SupplyToCountryExcluded> *

<Price> <CountryCode> <Territory> * <CountryExcluded> * <TerritoryExcluded> *

<SalesRights> <Territory> <CountriesIncluded> * <RegionsIncluded> * <CountriesExcluded> * <RegionsExcluded> *

3.0

<SalesRights> <SalesRightsType>01</SalesRightsType> <Territory> <CountriesIncluded>CA US</CountriesIncluded> <Territory></SalesRights><SalesRights> <SalesRightsType>03</SalesRightsType> <Territory> <CountriesIncluded>GB IE AU NZ</CountriesIncluded> <Territory></SalesRights><ROWSalesRightsType>02</ROWSalesRightsType>

3.0

<SalesRights> <SalesRightsType>01</SalesRightsType> <Territory> <CountriesIncluded>US</CountriesIncluded> <RegionsExcluded>US-HI US-AK</RegionsExcluded> </Territory></SalesRights><ROWSalesRightsType>03</ROWSalesRightsType>

3.0

<SalesRights> <SalesRightsType>01</SalesRightsType> <Territory> <RegionsIncluded>WORLD</RegionsIncluded> </Territory></SalesRights>

3.0

4. syntax updates

Basic syntax updates

• no <!DOCTYPE>, recomendation to use XSD schema for validation

• recommendation to use Unicode and UTF-8, but can still use Latin-1 if required

• named character entities (eg &ouml;) not valid – use ö instead, or if not using a rich character set, use &#246; or &#xf6;

• in line with broader XML developments

5. structural changes

…set, series and main titles,and markets

sets, series and main titles

<Series> <Title> <TitleType>01</TitleType> <TitleText textcase="02">The Chronicles of Narnia</TitleText> </Title></Series>

2.1

<Collection> <CollectionType>10</CollectionType> <TitleDetail> <TitleType>01</TitleType> <TitleElement> <TitleElementLevel>02</TitleElementLevel> <TitlePrefix>The</TitlePrefix> <TitleWithoutPrefix textcase="02">Chronicles of Narnia</TitleWithoutPrefix> </TitleElement> </TitleDetail></Collection>

3.0

<Collection> <CollectionType>10</CollectionType> <TitleDetail> <TitleType>01</TitleType> <TitleElement> <TitleElementLevel>02</TitleElementLevel> <TitlePrefix>The</TitlePrefix> <TitleWithoutPrefix textcase="02">Chronicles of Narnia</TitleWithoutPrefix> </TitleElement> </TitleDetail></Collection>

3.0

<TitleDetail> <TitleType>01</TitleType> <TitleElement> <TitleElementLevel>01</TitleElementLevel> <TitlePrefix>The</TitlePrefix> <TitleWithoutPrefix textcase="02">Lion, the Witch and the Wardrobe</TitleWithoutPrefix> </TitleElement></TitleDetail>

<TitleDetail> <TitleType>01</TitleType> <TitleElement> <TitleElementLevel>02</TitleElementLevel> <NoPrefix/> <TitleWithoutPrefix textcase="02">Focus on Maths</TitleWithoutPrefix> </TitleElement> <TitleElement> <TitleElementLevel>01</TitleElementLevel> <NoPrefix/> <TitleWithoutPrefix textcase="02">Workbook 6</TitleWithoutPrefix> </TitleElement></TitleDetail>

3.0.2

• two ways to provide collection details in ONIX 3.0• in P.5, as a title element in <TitleDetail>

within�<Collection>, or…

• in P.6, as just another title element within�<TitleDetail>

• not always a simple choice

• depends on whether collection title is an integral part of the identity of the product, or whether the product has an identity separate from the collection

• two ways to provide collection details in ONIX 3.0• in P.5, as a title element in <TitleDetail>

within�<Collection>, or…

• in P.6, as just another title element within�<TitleDetail>

• not always a simple choice

• depends on whether collection title is an integral part of the identity of the product, or whether the product has an identity separate from the collection

Focus on Physics:

Workbook 6

The Lion, the Witch and the Wardrobe

markets and prices

Markets and suppliers• ONIX 2.1 describes suppliers and the markets

they operate in• <SupplyDetail> describes one supplier

• within <SupplyDetail>, <SupplyToCountry> describes the geographical extent of the supplier’s market

• ONIX 3.0 describes markets and the suppliers that operate there

• <ProductSupply> describes one market, including <Market><Territory> for the geographical extent

• within one market, multiple <SupplyDetail> composites, for each of that market’s suppliers

migration project outline

2.1 to 3.0 migration outline

• bring ONIX 2.1 ‘up to date’• no deprecations

• use composites wherever possible

• delta updates, if you are not already using them

• add simple enhancements• use a rich native character set like UTF-8

• add XHTML tagging in richer textual elements

• this is all possible in 2.1

2.1 to 3.0 migration outline• deal with tags that change names, position

or context• eg <PersonNameIdentifier> to

<NameIdentifier> (but they’re not all that easy)

• tackle the tougher stuff• <Set> and <Series> become <Collection>

• <SalesRights>

• <ProductSupply> and <SupplyDetail>

• get to 3.0 that conveys the same data as 2.1

2.1 to 3.0 migration summary

• test and implement with initial recipients

• then add new functionality (maybe as ‘phase II’)• e-book licensing and restrictions

• richer collateral material

• block updates are optional (for senders)• further reduces amount of data to be processed

but what doesmigration get me?

Consistency and simplicity• greater consistency in the way data is

specified – eg with sales rights…• <Territory> always has the same structure

• requirement for a ROW statement (even if it is just ‘unknown’)

• sales restriction operates within a territory

• or with contributors…• contributor identifiers aren’t limited to

personal names – can be used with corporate authors too

Consistency and simplicity

• in turn, this means greater consistency in the way data is interpreted• one recommended way of doing things, not

three (eg 2.1 uses either <AvailabilityCode> or <ProductAvailability>, 3.0 uses only the latter)

• helped by greater precision in the Specification and advice in the Guide

• makes it simpler for recipients

Flexibility and extensibility

• consistent use of composites means new types, roles, rights, restrictions etc can be added via a codelist change, without adding specialised tags• eg added pre-order embargo to ONIX 3.0,

which would require new tags in ONIX 2.1

• less disruption as further functionality is added in future

• most textual metadata can be multi-lingual

New functionality (e-books)• e-book DRM (including watermarking)

• usage constraints, eg• preview 10% before purchase

• print only 10% per month

• lend only 24 times

• rent for 3 months

• links to product licences• vital for open access, useful for commercial

products too

New functionality (content)

• primary and secondary content types

• primary part in multi-component products

• multiple work relationships

New functionality (collections)

• collections remove anomalies• series cannot be multi-level

• sets cannot have contributors, or carry identifiers like ISSN

• collections can have multiple orders, eg publication order, narrative order

New functionality (contributors)

• contributors can be associated with multiple places• place of birth / death

• place of current residence / citizenship

• to city or town detail, not just country

• corporate contributors can carry contributor identifiers and dates

New functionality (collateral)

• content audience and ‘use from… until’ dates for descriptive text, cited content and other supporting resources

• pixel size replaces resolution for images

• file size, MD5 hash to validate downloads

New functionality (publishers)

• multiple publisher and imprint identifiers

• multiple product contacts for different purposes

• clarity on sales rights and restrictions

• clarity on product status and availability• clear distinction between ‘publisher view’ and

‘distributor view’

New functionality (markets)

• more flexible market-specific publishing details• eg local market ‘pub date’

• was originally added to 2.1 rev.02 in 2004 (important for Australian exports, but very rarely used because of patchy support)

New functionality (suppliers)• multiple supplier’s own coding

• proximity and velocity for stock holdings• gives ONIX compatibility with EDItX stock

report and X.12 / Tradacoms stock messages

• price identifiers

• price conditions (including rentals)

• coded / tiered prices

• comparison prices (strikethrough prices)

• price printed on product

• it’s not as difficult as most people assume• improvements in the message – particularly for

e-books, internationalization, multi-lingualism

• some tricky areas

• improvements in the tools (eg Schematron)

• block-level updates (eventually)

• and• loss of support from end of 2014

• risks of not upgrading

Why update?

graham@editeur.orghttp://www.editeur.org