RSS feeds using Millennium data
29
www.london.ac.uk RSS feeds using Millennium data Andrew Preater, University of London Research Library Services Presented at EIUG 2010, 15 th June 2010
-
Upload
andrew-preater -
Category
Technology
-
view
762 -
download
1
description
I presented this talk on creating RSS feeds at the European Innovative Users Group (EIUG) 2010 conference held at Aston University, 15-16 June 2010. I describe a method of exporting and reusing metadata held in the Innovative Millennium LMS that enables reuse of the data and presentation as an RSS feed - in this case new books lists. This is achieved using Free / Open Sources software. I explain how the process can be generalised to export of other bibliographic data, for example to export reading lists information to a VLE (BlackBoard) as XML, or presenting lists of e-resources on a Web site using a PHP front-end.
Transcript of RSS feeds using Millennium data
www.london.ac.uk
RSS feeds using Millennium data
Andrew Preater, University of London Research Library Services
Presented at EIUG 2010, 15th June 2010
A short break in County Durham
I work for University of London Research Library Services, at Senate House
But I will talk about my previous development work at Durham University Library
Introduction
The problem is the new books list
We use these to list new items for readers as a current awareness tool
Various ways to do this...
A traditional approach
Durham’s new books featured list
Problems
High maintenance
Not split by subject; not easily ‘mashable’
Usage next to nothing by 2007-08
10 hits!
Puts our metadata where the reader is
Much less work for library staff
Standards-based XML data, can be reused elsewhere or mashed up
RSS feed improvements
RSS feed icon from www.feedicons.com
Low-risk pilot for automated export and processing of Millennium data
Demonstrates this utility for future projects
Quickest and easiest example using this approach
Project as proof of concept
Automated as much as possible
Minimal effort by non-systems staff to maintain
No special software – no budget!
Stable and reliable, ‘just works’
Desired outcomes
Other than Millennium...
1. Linux server with Perl installed
2. MySQL database
3. Web server running PHP
Software used
A featured list was created each week based on changing book item status to ‘d’
So a ‘new books’ review file was being made...
Basic idea
New step added: export the contents of the review file and reuse it
BIB MARC 245 $aBIB MARC 245BIB AUTHORBIB IMPRINTBIB SUBJECTBIB RECORD #ITEMFUND CODEITEMSHELFMARKITEMLOCATION
Export these fields
"Dead white men and other important people :"~"Dead white men and other important people : sociology's big ideas / Ralph Fevre and Angus Bancroft."~"Fevre, Ralph, 1955-"~"Basingstoke : Palgrave Macmillan, 2010."~"Social sciences -- Philosophy.";"Sociology."~"b25978974"~"bgsoc"~"300.1 FEV"~"main4"
Example single item
Perl script run every 15 minutes by cron:
1. Checks if there is a new file
2. Processes the data
3. Loads it into a MySQL database
4. Cleans up
Processing this list
1. Replace & with &
2. Insert RFC822-compliant date
3. Strip quotation marks around fields
4. Strip trailing non-alphanumeric character in 245 $a
5. Lowercase fund codes
Step 2: tidying up the data
Step 2: example single item
|Dead white men and other important people|Dead white men and other important people : sociology's big ideas / Ralph Fevre and Angus Bancroft.|Fevre, Ralph, 1955-|Basingstoke : Palgrave Macmillan, 2010.|Social sciences -- Philosophy.";"Sociology.|b25978974|bgsoc|300.1 FEV|main4|Mon, 07 Jun 10 12:31:01 BST|Mon, 07 Jun 10 12:31:01 BST|
Step 2: example single itemDead white men and other important people 245$aDead white men and other important people : sociology's big ideas / Ralph Fevre and Angus245 Bancroft.Fevre, Ralph, 1955- AuthorBasingstoke : Palgrave Macmillan, 2010. ImprintSocial sciences -- Philosophy.";"Sociology. Subjectb25978974 Record #bgsoc Fund code300.1 FEVShelfmarkmain4 LocationMon, 07 Jun 10 12:31:01 BST Date
Two tables are used:
items is refresh weekly: contains our books information
fundmap maps Millennium fund codes to subjects. Export is automated but doesn’t need to run weekly
Database
foobar
fundmap exampledeptcode
fundcode deptname site
ECON bcecoEconomics & Finance
DURHAM
HIST bchis History DURHAM
MEIS bbcmeGovt & Intl Affairs/IMEIS
DURHAM
MEIS bxabcGovt & Intl Affairs/IMEIS
DURHAM
CTV ctvl1Trevelyan College Library
DURHAM
Web front end
PHP script hosted on IT Service Web server will serve the feeds
http://www.dur.ac.uk/reading.list/newitems.php?dept=HIST
Parameter is ‘all’ or a subject code
1. Select items from database
2. Writes beautiful, valid RSS
3. Serves it up to the browser
What it does
A bit more detail...
Generating RSS feed XMLWrite <title>, <description> <link>, <image> once
Item <description> author, shelfmark and subjects hyperlinked to catalogue search.
Item <title> is 245 $a and links to catalogue bib record
Item <description> contains full item data. Can include encoded HTML
For each database line, write one RSS news <item> ...
Finished product - I
Shown in Akregator feed reader
Running happily since August 2007
Finished product - II
HTML version of RSS feeds on Library Web site
Also: in-house PC screensavers, plasma displays...
SummaryMillennium review file
Exported flat file
Display with Web front end
Process and load into database
Easiest to use Unicode everywhere
Write valid RSS 2.0 or Atom, use http://feedvalidator.org for hints
Few complaints; change uncovered a tiny hard core of featured lists fans
Lessons learned
That said...
You can automate much of it with Expect or AutoIt
Recommend Marc Dahl’s presentation on Expect for Innopac: http://bit.ly/dahl-expect
“Couldn’t you automate this?”
Automated export and processing used for:
Exporting Course Reserves to Blackboard
Display of e-resources data in CMS
Sending fines data to Oracle Financials
Following on from this...
