PSIgate Knowledge Exchange: Using OAI to Share Information Paul Meehan, PSIgate Technical Manager...
-
Upload
marilynn-wilkerson -
Category
Documents
-
view
212 -
download
0
Transcript of PSIgate Knowledge Exchange: Using OAI to Share Information Paul Meehan, PSIgate Technical Manager...
PSIgate Knowledge Exchange:Using OAI to Share Information
Paul Meehan,PSIgate Technical Manager
UKSG Meeting. May 14, 2003
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Subjects to Cover
Google, PSIgate and the RDN
Database Services and Partners
Why OAI?
PSIgate Information Exchange via OAI
OAI Pros and Cons
Future Developments
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
The Trouble with Google
It’s too big!
e.g. copper – 5.15 million hitscopper + element – 518,000copper + element + properties – 163,000
relevancy and ranking
Only 7 out of first 25 “copper” hits refer to the metal
But as a starting point it’s still number 1!!
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
This is what happens …
Lost in (hyper)space!
You lose focus
You waste time
You miss information
FRUSTRATION!
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
The RDN
The Resource Discovery Network (RDN) is a collaboration of over seventy educational and research organisations, including the Natural History Museum and the British Library
It is a free service, funded by JISC
It is divided into a number of subject gateways or “hubs”, each of which selects and catalogues Internet resources for learning, teaching and research. Each hub provides search and browse interfaces to its records
The RDN additionally provides a central database of records
Subject gateways may also provide additional services
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
The RDN
EEVL: Engineering
BIOME: Biomedical sciences
Humbul:Humanities
SOSIG:Social sciences
PSIgate:Physical sciences
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
PSIgate
PSIgate is the physical sciences gateway of the RDN
Based in Manchester; team of 4
We offer a number of database services including:
Internet Resource Catalogue astronomy, chemistry, earth sciences, materials, physics, history and policy of science
Expanded “Web Catalogue”
“additional services”
We work with a number of professional bodies and organisationse.g. RSChem, IoP, BGS
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
PSIgate vs Google
PSIgate IRC* Web Catalogue* Google
Copper 56 255 5.15 million
Copper + element 10 21 518,000
Copper + element + properties 6 4 163,000
* IRC contains 7600 resources, Web catalogue ca. 80,000 at May 12, 2003
• Catalogued by subject specialists• Should all be relevant!
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
PSIgate – Additional Services
News channels Spotlight science magazine Reference data Science timelines Plug-ins Training materials Featured sites and topics
Databases provided by external sources Partnership with LTSNs
… all of these require different technologies … including OAI!
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Why OAI?
The Open Archives Initiative (OAI) “develops and promotes interoperability standards that aim to facilitate the efficient dissemination of content”
We should aim to use OAI because:
Promotes interoperability Provides standard tools Free! It is supported by an increasing number of data providers
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
PSIgate Information Exchange
We currently use OAI for three distinct purposes:
Exposure of our records for gathering by the RDN Harvesting of records from the IoP Exposure of records to our LTSN partners
Each of these services has its own quirks and methods!
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Using OAI to Share Records with the RDN
The RDN gathers resources from all the hubs to provide a central “ResourceFinder” service
We store our records in a MySQL database (1), and convert them to “OAI format” on a regular basis (2). The RDN gathers the records (3,4) and stores them in a Cheshire database (5), which is then cross-searched via Z39.50.
PSIgateDatabase(MySQL)
PSIgateRecords
(OAI)
RDNrepository
(OAI)
Stored in CheshireDatabase, searchusing Z39.50
resources
2 - weekly
3 - request
4 - retrieval
5
1
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Step 1 – Record Conversion
Firstly we need to create a series of OAI compliant records from our MySQL database. This is done by running a simple script and creating XML format records.
This process is done automatically once per week.
A typical record will contains the following core fields …
PSIgate Knowledge Exchange: using OAI to share information
IDENTIFIER - 1
TITLE
DESCRIPTION
RESOURCE TYPE
KEYWORD(S)
IDENTIFIER - 2
PUBLISHER
LANGUAGE
RIGHTS
<identifier>oai:psigate.ac.uk:1033748964-226</identifier><dc:title> Aroma Chemistry</dc:title><dc:description> A small set of overheads to accompany a lecture on aroma chemistry, by Terry Acree of Cornell University. The overheads provide theoretical summaries relating to: volatility, activity (Raoult's law), potency, odour units, complexity and mixtures.</dc:description><dc:type> Lecture notes and courses</dc:type><dc:subject> smells</dc:subject><dc:subject> odours</dc:subject><dc:identifier> http://www.nysaes.cornell.edu/fst/faculty/acree/fs430/lectures/tea18olfaction.html</dc:identifier> <dc:publisher> www.psigate.ac.uk </dc:publisher> <dc:language> en </dc:language> <dc:rights> This metadata record is for use by RDN partners only. </dc:rights>
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Key points
we create a bulk OAI repository once per week the RDN is granted access to this repository the RDN harvest this metadata once per week we use the latest version (2.0) of the OAI-PMH
OAI is efficient for this because:
the records are easy to generate the process is automated the RDN handles the retrieval and subsequent storage
The PSIgate database currently stands at around 7,600 resources. The central ResourceFinder service stores more than 50,000 records
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
The IoP Select Service
The Institute of Physics Publishing offers a service known as “IOP Select”.
“A FREE service from IoP Journals comprising articles chosen by our Editors for their novelty, significance and potential impact on future research”
free
updated weekly
normally requires username and password; direct access via PSIgate
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Using OAI with IoP Select
The IoP allow us to gather their “select” resources via OAI
we use a harvesting tool to gather records on a weekly basis (1,2) we extract relevant fields from each record eg title, abstract, URL (3) we create search and browse interfaces to the records (4,5)
IoPRepository
(OAI)
PSIgaterepository
(OAI)
1 - request
2 - retrieval3 - convert
Key fields(text file)
Search interface
Browseinterface
4
5
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Browse all papers
Search by keyword
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Option 1 – View all papers
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Option 2 – Keyword search
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Our experience of using OAI with IOP Select
Pros:
gathering is simple, using standard OAI-PMH conventions fast (c. 5 minutes to search and retrieve records from large database) we store records so we can adapt and create search interfaces easily maintenance of records occurs at IoP end building of search interface
Cons:
record storage uses disk space (not significant at present, < 1MB) we need to convert and store records in searchable format building of search interface (!)
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Using OAI to Share Records with the LTSNs
We are engaged in projects with four LTSN partners:
LTSN Physical SciencesUK Centre for Materials EducationLTSN Geography, Earth and Environmental Science (GEES)LTSN Built Environment
OAI allows us to expose our metadata records to these partners
… work to be undertaken Summer 2003
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Future Work
LTSN partnerships
Other OAI data providers may offer useful data
Additional work with professional bodies and the IoP
Sharing records with RDN partners and hubs
GEsource
PSIgate Knowledge Exchange: using OAI to share information
Paul Meehan UKSG Meeting. May 14, 2003
Conclusions
The Open Archives Initiative is already offering PSIgate a significant means of collaborating with external sources:
fast fairly simple methodology gaining increasingly widespread usage harvesting processes can easily be automated some work to be done with storage and searching