Post on 29-Nov-2014
description
Recommendations Improve the Search Experience
Alison Brock
www.open.ac.uk/blogs/rise
Outline
Open University (OU) Context
Why use activity data?
Scope of the project
What we did
Evaluation and next steps
OU context
“The search engine on the library is not very user friendly. I had to find a specific article recommended in the text and it took several attempts to locate it.”
“The search facility is poor and doesn’t find stuff that is supposed to be there”
http://www.flickr.com/photos/james_lumb/3921968993/sizes/z/in/photostream
New search system
http://www.flickr.com/photos/jiscimages/435135071/sizes/m/in/photostream/
New generation Discovery System from EBSCO (EDS)
http://www.flickr.com/photos/davepattern/5808712333/sizes/z/in/photostream/
Could we do more?
“That recommender systems can enhance the student experience in new
generation e-resource discovery services”
Recommendations Improve the Search Experience?
Do recommendations improve the search experience?
Can you use search data to make recommendations?
Are recommendations useful in Discovery systems?
http://www.flickr.com/photos/davepattern/3473326634/sizes/z/in/photostream/
JISC funded project
February – July 2011
One of eight projects [list at http://bit.ly/gwCmNS]
JISC Activity Data Programme
http://www.flickr.com/photos/zerimski/5215633183/sizes/z/in/photostream/
"Every day I wake up and ask, 'how can I flow data better, manage data better, analyse data better?"
Rollin Ford, the CIO of Wal-Mart
Why activity data?
Loans Holds
Computer bookings
Library access
Typical library activity data
Loans Holds
Computer bookings
Library access
xx xx
OU Library activity data
OU Library systems environment
Scope of our project
Search interface
Algorithms & recommender code
Activity data
EZProxy
SFX
EDS
VLE
websitebookmarklet
What data is RISE using?
• Remote host • Date/Time• OUCU• Request• Status• Size of response• Referrer• User agent• Session
So what is in the EZProxy logs?
http://www.flickr.com/photos/vixon/116447718/sizes/m/in/photostream/
\"0\"|||\"137.108.143.168\"|||20110115235421|||\“nn1234\"|||\"GET http://libezproxy.open.ac.uk:80/connect?Session=st3ShtizgtrS7tU5&url=http://search.ebscohost.com/login.aspx?direct=true&site=edslive&scope=site&type=0&cli0=FT&clv0=Y&cli1=FT1&clv1=Y&authtype=ip&group=VCStud&bquery=War%20Against%20the%20Panthers HTTP/1.1\“|||302|||0|||\http://library.open.ac.uk/\|||\"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13\"|||\"t3ShtizgtrS7tU5\"
So what is in the EZProxy logs?
\"0\"|||\"137.108.143.168\"|||20110115235421|||\“nn1234\"|||\"GET http://libezproxy.open.ac.uk:80/connect?Session=st3ShtizgtrS7tU5&url=http://search.ebscohost.com/login.aspx?direct=true&site=edslive&scope=site&type=0&cli0=FT&clv0=Y&cli1=FT1&clv1=Y&authtype=ip&group=VCStud&bquery=War%20Against%20the%20Panthers HTTP/1.1\“|||302|||0|||\http://library.open.ac.uk/\|||\"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13\"|||\"t3ShtizgtrS7tU5\"
date and time
So what is in the EZProxy logs?
\"0\"|||\"137.108.143.168\"|||20110115235421|||\“nn1234\"|||\"GET http://libezproxy.open.ac.uk:80/connect?Session=st3ShtizgtrS7tU5&url=http://search.ebscohost.com/login.aspx?direct=true&site=edslive&scope=site&type=0&cli0=FT&clv0=Y&cli1=FT1&clv1=Y&authtype=ip&group=VCStud&bquery=War%20Against%20the%20Panthers HTTP/1.1\“|||302|||0|||\http://library.open.ac.uk/\|||\"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13\"|||\"t3ShtizgtrS7tU5\"
User name
So what is in the EZProxy logs?
\"0\"|||\"137.108.143.168\"|||20110115235421|||\“nn1234\"|||\"GET http://libezproxy.open.ac.uk:80/connect?Session=st3ShtizgtrS7tU5&url=http://search.ebscohost.com/login.aspx?direct=true&site=edslive&scope=site&type=0&cli0=FT&clv0=Y&cli1=FT1&clv1=Y&authtype=ip&group=VCStud&bquery=War%20Against%20the%20Panthers HTTP/1.1\“|||302|||0|||\http://library.open.ac.uk/\|||\"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13\"|||\"t3ShtizgtrS7tU5\"
Request
So what is in the EZProxy logs?
RISE database
RISE database
user type | course code(s)
EZProxy
CIRCE
RISE database
People on course ‘A’ viewed resource ‘B’People who looked at resource ‘C’ also
looked at resource ‘D’
What can the data tell us?
Course recommendation
Relationship recommendation
Which are the most popular resources
This resource is being used by people studying this course
ISSNs DOI
Article information
Subject terms
But what isn’t there?
http://www.flickr.com/photos/kevharb/5466661946/sizes/z/in/photostream/
user type | course code(s)
Bibliographic data matching
EZProxy
CIRCE
So how do you improve your data?
EDS Crossref
MyRecommendations
http://www.google.com/ig/directory?type=gadgets&url=library.open.ac.uk/rise/google_gadget/risesearch.xml
user type | course code(s)
Bibliographic data matching
Searches in RISE
EZProxy
CIRCE
RISE
So how do you improve your data?
EDS Crossref
RISE database
People on course ‘A’ viewed resource ‘B’
People who looked at resource ‘C’ also looked at resource ‘D’
People who searched for subject ‘E’ looked at resource ‘F’
What can the data tell us?
Course recommendation
Search recommendation
Relationship recommendation
People are looking at resources on this subjectSubject data
This resource is being used by people studying this course
Resource management
Getting a recommendation
Getting a recommendation
User A Module A123
Resource BRV=14
Views
User C Module A123
Resource BRV=15
Recommended Resource BRV=16
Views +1
User C Module A123
Resource BRV=17
Rate Useful +1
User C Module A123
Resource BRV=14
Rate Not Useful -2
Resource BRV=15
Views +1
Added a privacy policy to RISE, EDS and SFX interfaces
Privacy and opt-out URL
http://library.open.ac.uk/rise/?page=privacy
Data Protection and privacy
Evaluation
Not useful22%
Quite useful22%
Very useful45%
Not sure11%
Related to records you have viewed
Survey results
Survey results
Undergraduates
Like ratings and reviews from other students
‘other people’s experiences valuable’
Which module studied?
How high a mark?
Postgraduates
Citation as a recommendation
Wary of provenance
Feed to module website
Want synonyms
Trust repository
Focus groups
First impressions of recommendations (course-related)
Asked to enter a search term. Results and recommendations explored.
Asked about relevance
Asked about preference for type of recommendation
Face to face interviews
Should we have a recommender system?
“I think it would be a very good useful feature. It would be definitely very very useful” postgraduate Maths student
I have just had a go, it was good with suggested papers that I had already found (which shows potential in my view) through Google.
“I'm afraid my first reaction is to be a bit sceptical - it presumably doesn't tell you if fellow students found the information/article useful or relevant to what they were looking for. I would hate to waste time following unproductive links laid down by others who might be failing students or think that any "lazy" students might develop poor practice by relying on what others had looked at. It sounds like a good idea but I think caution needs to be exercised. ”
Recommendations usage
1 2 3 40
200
400
600
800
1000
1200
1400
1600
1800
2000
Relationship
Course
Search
Findings and lessons learnt
•Users like recommendations ‘in principle’
•Recommendations provenance
•Interest in the search tools
•EZProxy data
•Use other data sources
•Search terms
•Need more data
•License restrictions on metadata
Findings and lessons learnt
Open Data
http://www.flickr.com/photos/okfn/6262973028/sizes/z/in/photostream/
What next?
• Google Gadget search tool• Recommendations database• MACON• Learning Analytics
http://www.flickr.com/photos/shandrew/2102808886/sizes/m/in/photostream/
What next?
Blog: www.open.ac.uk/blogs/RISECode: http://code.google.com/p/rise-project/source/browse/trunk/rise/
My thanks go to Richard Nurse and Liz Mallett of the Open University Library for giving me the use of their slides on the project for this presentation.
Any questions ???