The OAI and OAI-PMH: where to go from here?
description
Transcript of The OAI and OAI-PMH: where to go from here?
![Page 1: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/1.jpg)
The OAI and OAI-PMH:where to go from here?
Carl Lagoze – Cornell Information [email protected]
Herbert Van de Sompel – [email protected]
OAI3 – CERN – February 12, 2004
![Page 2: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/2.jpg)
Building on the base
• New infrastructure• Protocol extensions• Non-traditional uses• Research contexts
![Page 3: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/3.jpg)
New Infrastructure
Building blocks for cross-repository federation
![Page 4: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/4.jpg)
http://gita.grainger.uiuc.edu/registry/searchform.asp
![Page 5: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/5.jpg)
http://www.oclc.org/research/projects/oairesolver/default.htm
![Page 6: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/6.jpg)
Protocol Extensions
New functionality on a stable base
![Page 7: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/7.jpg)
OAI Static Repository
• OAI-PMH is low-barrier protocol
• nevertheless, implementation is
sometimes not trivial:
• size of collection does not justify the
investement
• ISP does not allow 3rd party software
• security considerations
![Page 8: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/8.jpg)
OAI Static Repository
• research on lowering barrier even further
• make metadata available in XML files
(not dbases)
• put XML file on web-server
• make XML file OAI-PMH harvestable
• 2 tracks:
• autonomous data provider
• dependent data provider
![Page 9: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/9.jpg)
OAI Static Repository
• autonomous data provider:
• XML file on web-server
• XSL style sheet to respond to OAI-PMH
requests on web-server
• requires:
• native XSLT support in web server
• XSL v.2 functionality
=> Not (yet) low barrier
![Page 10: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/10.jpg)
OAI Static Repository
• dependent data provider:
• XML file on web-server
• depend on Gateway to respond to OAI-
PMH requests
•requires:
• registration with Gateway
• Gateway implementation(s)
![Page 11: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/11.jpg)
static repository 1
http://an.oai.org/ma/mini.xml
static repository n
http:// site1.org/mini/file1
![Page 12: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/12.jpg)
static repository 1
http://an.oai.org/ma/mini.xml
static repository n
http:// site1.org/mini/file1
http://gateway.institution.org/oai/
staticrepository gateway
http://gateway.institution.org/oai/site1.org/mini/file1
http://gateway.institution.org/oai/an.oai.org/ma/mini.xml
![Page 13: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/13.jpg)
static repository 1
http://an.oai.org/ma/mini.xml
static repository n
http:// site1.org/mini/file1
http://gateway.institution.org/oai/
staticrepository gateway
http://gateway.institution.org/oai/site1.org/mini/file1
http://gateway.institution.org/oai/an.oai.org/ma/mini.xml
OAI-PMH harvester
OAI-PMH
HTTP
HTTP
![Page 14: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/14.jpg)
LANL Static Repository Gateway
• The OAI-PMH Static Repository and Static Repository Gateway - Patrick Hochstenbach, Henry Jerez, Herbert Van de Sompel http://lib-www.lanl.gov/~herbertv/papers/jcdl2003-submitted-draft.pdf
• Experimental registration system - http://libtest.lanl.gov/registry.htm
• Sourceforge download site - https://sourceforge.net/projects/srepod/
![Page 15: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/15.jpg)
OAI Rights
• Motivations– Distinction between data and metadata fuzzy,
especially regarding intellectual property– XML content already fits into protocol– Consumers of metadata are almost always
interested in access to underlying resource
• Scope – No new definition of a rights expression
language– Avoid restriction to any rights language
• Initial prototypes with Creative Commons licenses
![Page 16: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/16.jpg)
OAI rights issues
• Entity Association– Focus on rights
expressions for metadata and associated resources
• Aggregation association– OAI-PMH entities:
repository, resource, item, record, set
• Binding– Use about container for
metadata rights exp.– Designated metadata
prefix to contain resource rights exp.
![Page 17: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/17.jpg)
Non-traditional usage
Beyond metadata for resource discovery
![Page 18: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/18.jpg)
OAI-PMH-based access to DL usage logs
http://www.dlib.org/dlib/july03/young/07young.html
![Page 19: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/19.jpg)
OAI-PMH access to DL usage logs
• usage logs filtered and stored in MySQL
db
• accessible as 2 OAI-PMH repositories:• document oriented• agent oriented (user-proxy)• interlinked
• recommender system:• harvests logs• interpretes logs• exposes relationships (OpenURL access)
![Page 20: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/20.jpg)
agent
alog:IP:128.1.22.13
Repository 1
docs accessedby agentabout
agent
![Page 21: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/21.jpg)
document
dlog:ori:pmid:258471
Repository 2
agents accessingthe documentabout
document
![Page 22: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/22.jpg)
LANL Repository Architecture
• Problem: provide multiple service access to variety of locally hosted assets
• Assets include secondary assets (ISI, BIOSIS, Inspec, etc.) and primary feeds (Elsevier, Wiley, IOP, APS, etc.)
• Common representation of assets using MPEG-21 DIDL– Facility for multiple disseminations
• Components of architecture federated through OAI-PMH
![Page 23: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/23.jpg)
LANL Repository ArchitectureComponents
• Asset repositories – one per data feed with assets stored as DIDLs, harvestable by OAI-PMH
• Repository index – keeps track of creation and location of data repositories, harvestable by OAI-PMH
• Identifier resolver – single point resolution to get repository location of DIDL object.
• OAI-PMH federator – single point OAI access for service clients
![Page 24: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/24.jpg)
LANL Repository Architecture
![Page 25: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/25.jpg)
LANL Repository Architecture
• D-Lib nov 2003 : http://dx.doi.org/10.1045/november2003-bekaert (MPEG-21 DIDL use)
• D-Lib fed 2004 : http://dx.doi.org/10.1045/february2004-bekaert (MPEG-21 and OpenURL based dissemination architecture)
• Submission to JCDL 2004
![Page 26: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/26.jpg)
Experimentation
Exploration of new contexts
![Page 27: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/27.jpg)
OAI and P2P
Enabling a metadata refinement network that enables the creation of
document value chains
![Page 28: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/28.jpg)
Original OAI-PMH Model
R ep o s ito r y
O A I-P M HS erv er
R ep o s ito r y
O A I-P M HS erv er
R ep o s ito r y
O A I-P M HS erv er
R ep o s ito r y
O A I-P M HS erv er
L in k in gS er v ic e
O A I-P M HH arv es ter
Br o w s eS er v ic e
O A I-P M HH arv es ter
S ear c hS er v ic e
O A I-P M HH arv es ter
Service Providers
Data Providers
![Page 29: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/29.jpg)
Hybrid Model with Aggregator
O A I-P M HH arv es ter
OA
I-PMH
Server
Met
adat
aR
epos
itor
y
S ear c hS er v ic e
O A I-P M HH arv es ter
Br o w s eS er v ic e
O A I-P M HH arv es ter
C o llec tio n
O A I-P M HS erv er
C o llec tio n
O A I-P M HS erv er
C o llec tio n
O A I-P M HS erv er
C o llec tio n
O A I-P M HS erv er
![Page 30: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/30.jpg)
Metadata Exchange Graph
OA
I-P
MH
Har
vest
erOA
I-PM
HS
erverO
AI-P
MH
Server
OA
I-PM
HS
erver
Va lu eA d d edA g g re-g a to rO
AI-
PM
HH
arve
ster
OA
I-PM
HS
erver
P a ssTh ro u g hA g g re-g a to rO
AI-
PM
HH
arve
ster
OA
I-PM
HS
erver
![Page 31: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/31.jpg)
Implementation Questions
• Underlying framework– JXTA
• Metadata item/record location– Broadcast search– Distributed Hash Tables
• Provenance chains– Exploit provenance information in OAI-PMH– Logical joins based on provenance information
• Network Harvesting– Efficient range queries using P-trees
![Page 32: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/32.jpg)
OAI and RDF
Expressing relationships among metadata records
![Page 33: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/33.jpg)
NSDL Metadata Repository (1)
Relationship Metadata<hasItems> i1 i2 i3</hasItems>
Is “A” equivalent to “B”?
What resources fit standard “C”?
![Page 34: The OAI and OAI-PMH: where to go from here?](https://reader036.fdocuments.in/reader036/viewer/2022062305/5681584e550346895dc5aa14/html5/thumbnails/34.jpg)
NSDL Metadata Repository (2)
<rdf:Description about=”ID1”> <nsdlrel:hasMember>ID2</nsdlrel:hasMember> <nsdlrel:conformsTo>STD4</nsdlrel:conformsTo> </rdf:Description>
O AI s y n c h r o n iza tio n ?
Fe do ra C o n te n t /M e ta da ta S to re J e n a R e la t io n s h ip S to re
Issues:
• push/pull model?
• schema validation