Measuring Library Impact at DBS
-
Upload
dbslibrary -
Category
Education
-
view
19 -
download
3
Transcript of Measuring Library Impact at DBS
![Page 1: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/1.jpg)
Measuring Library Impact at
DBS
David Hughes
Systems Librarian
Dublin Business School
![Page 2: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/2.jpg)
“You can observe a lot by just watching”
Yogi Berra
“Aw, people can come up with statistics to prove anything, Kent. Forfty [sic] percent of all people know that.”
Homer Simpson
![Page 3: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/3.jpg)
http://www.artscouncil.org.uk/media/uploads/pdf/Evidence_review_economi
c_contribution_libraries.pdf
![Page 4: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/4.jpg)
Question
Does the library make a difference to
student attainment?
![Page 5: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/5.jpg)
Why?
The library does not exist in a vacuum
Traditional measurements of library ‘value’ not so appropriate these days
![Page 6: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/6.jpg)
Why: Culture
We live in an era of quantitative assessment (i.e.
an obsession with measurement)
![Page 7: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/7.jpg)
Why: The Next Big Thing
http://library.wiki.nmc.org/Learning+Analytics
![Page 8: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/8.jpg)
Why: Validation
Can help reassure that we are providing a
good (i.e. are we
doing the right
things) service to
our users
![Page 9: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/9.jpg)
Why: Marketing
Use the library and get more smart, duh.
![Page 10: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/10.jpg)
Antecedents
University of Wollongong Library: the Library Cube
Tailored database and reporting function joining library usage data with student data
Found a strong correlation between students' grades and use of information resources the library provides. http://www.educause.edu/ero/article/discovering-impact-library-use-and-student-performance
![Page 11: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/11.jpg)
Antecedents
JISC Library Impact Project:
Aimed to prove a statistically significant
correlation between library usage and
student attainment at 8 universities.
https://library3.hud.ac.uk/blogs/lidp/
![Page 12: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/12.jpg)
JISC Library Impact Project
E-resources usage, library borrowing
statistics, and library footfall
“The research successfully demonstrated a
statistically significant relationship between
library resource use and level of degree
result”
Stone, G., & Ramsden, B. (2013). Library Impact Data Project: Looking for
the Link between Library Usage and Student Attainment. College &
Research Libraries, 74(6), 546–559. http://doi.org/10.5860/crl12-406
![Page 13: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/13.jpg)
So why bother?
Reproducibility “What Kabir talks of is only what he has lived through. If you have not lived through something it is not true.”
Kabir
“..Recent studies indicate that up to 70% of research from academic labs cannot be reproduced, representing an enormous waste of money and effort”
Dr. Elizabeth Iorns CEO Science Exchange
http://blog.scienceexchange.com/2012/08/the-reproducibility-initiative/
![Page 14: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/14.jpg)
Caveat: Lots of limitations
Me vs 16+ Library Impact project members
Lack of statistical analysis
Access to data (e.g. library footfall, on-
campus electronic resource access)
Student demographics/external factors
Crude measurements
![Page 15: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/15.jpg)
Looked at
Final exam grade and
Library borrowing
Off-campus resource usage
Attending a library class
Printing (Yes, printing)
![Page 16: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/16.jpg)
Materials and Methods 1
Library Management System
(Heritage, Koha)
Circulation Reports
Patron Attributes
Slave labour
Final Year Students
Open Athens (electronic resource access management)
Papercut (Print Management System)
Student Management System
(Prestige, Agresso)
MS Excel
MS Access
![Page 17: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/17.jpg)
Materials + Methods 2
Run circulation/usage/attendance reports
Run exam grade report
Export reports to Excel
Import to Access
Run Access queries
![Page 18: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/18.jpg)
Materials + Methods 3
![Page 19: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/19.jpg)
Library Borrowing 2009-2010
0
10
20
30
40
50
60
70
1st Class Honours Upper Second Lower Second 3rd Class
Library Usage and Exam Success 2009-2010
Average Library Usage
![Page 20: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/20.jpg)
Library Borrowing 2010-2011
0
10
20
30
40
50
60
FIRST CLASS HONOURS
DEGREE
UPPER SECOND CLASS
HONOURS DEGREE
LOWER SECOND CLASS
HONOURS DEGREE
THIRD CLASS
HONOURS DEGREE
Library Usage and Exam Success 2010-2011
Average Library Usage
![Page 21: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/21.jpg)
Library Borrowing 2011-2012
0
10
20
30
40
50
60
FIRST CLASS HONOURS UPPER SECOND CLASS
HONOURS
LOWER SECOND CLASS
HONOURS
THIRD CLASS
HONOURS
Library Usage and Exam Success 2011-2012
Average Library Usage
![Page 22: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/22.jpg)
Library Borrowing 2012-2013
![Page 23: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/23.jpg)
Library Borrowing 2013-2014
0
10
20
30
40
50
60
70
80
90
1st Class Honours Upper 2nd Class Honours Lower 2nd Class Honours Third Class Honours
Library usage and exam success 2013-2014
Average borrowing per student
![Page 24: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/24.jpg)
Cumulatively
0
10
20
30
40
50
60
70
1st Class Honours Upper Second Lower Second 3rd Class
Library Borrowing and Exam Grade 2009/2014
Average Borrowing
![Page 25: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/25.jpg)
By counts, not average
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
Non-users 1 to 20 books 21+ books
Library Borrowing and Exam Success 2013-2014
H1
H21
H22
H3
![Page 26: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/26.jpg)
However…
![Page 27: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/27.jpg)
Mexican Lemon Imports Prevent US
Highway Deaths
http://scienceblogs.com/commonknowledge/20
09/04/01/mexican-lemons-prevent-highway/
![Page 28: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/28.jpg)
But
Would we really, REALLY expect to see
this?
0
10
20
30
40
50
60
70
FIRST CLASS
HONOURS
UPPER SECOND
CLASS
HONOURS
LOWER
SECOND CLASS
HONOURS
THIRD CLASS
HONOURS
Fantasy Graph
Average Borrowing
![Page 29: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/29.jpg)
Student Engagement
Engaged students borrow more from the
library
Engaged students get better grades
Increased borrowing = better grades???????
![Page 30: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/30.jpg)
Off-campus resource usage
0
5
10
15
20
25
30
1st Class Honours Upper 2nd Class
Honours
Lower 2nd Class
Honours
3rd Class Honours
Athens usage and exam success 2013-2014
Average Usage
![Page 31: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/31.jpg)
Off-campus resource usage
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
1st Class Honours Upper 2nd Class
Honours
Lower 2nd Class
Honours
3rd Class Honours
Athens Use and Exam Grade
Users
Non-users
![Page 32: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/32.jpg)
Student engagement (2)
Engaged students access electronic
resources more
Engaged students get better grades
More electronic resource logins = better
grades???????
![Page 33: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/33.jpg)
Library Classes
Drop-in/Sign-Up classes on e.g.,
Referencing
Zotero
Database Searching
Embedded classes
Part of module
Integrated classes
invited by lecturer
Do they have any impact?
![Page 34: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/34.jpg)
Yes! (1)
0.00
5.00
10.00
15.00
20.00
25.00
30.00
35.00
40.00
45.00
50.00
Non-Attendees Library Class Attendees
Library Borrowing vs Library Class Attendance
Average Borrowing
![Page 35: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/35.jpg)
Yes! (2)
0
2
4
6
8
10
12
14
16
18
Non-Attendees Library Class Attendees
Average Usage
![Page 36: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/36.jpg)
Student engagement (3)
Students who take drop-in library class are
engaged
Drop-in library classes encourage library
use
Engaged students make more use of the
library
![Page 37: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/37.jpg)
Although
Attendance at (Embedded/Integrated)
classes is voluntary
Embedded = part of module
Integrated = invited by lecturing staff
![Page 38: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/38.jpg)
More impact (1)
0
10
20
30
40
50
60
70
Not Attending Embedded/Integrated Sign-Up Classes
Library Borrowing vs Library Class Attendance
Average Borrowing
![Page 39: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/39.jpg)
More Impact (2)
0
2
4
6
8
10
12
14
16
18
Not Attending Embedded/Integrated Sign-Up Classes
Athens Usage vs Library Class Attendance
Average usage
![Page 40: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/40.jpg)
Conclusion: Library Classes…
will increase student borrowing and
off-campus resource [Athens] usage
Do they improve exam results?
![Page 41: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/41.jpg)
Remember
“Aw, people can come up with statistics to
prove anything, Kent. Forfty [sic] percent of all
people know that.”
Homer Simpson
![Page 42: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/42.jpg)
Library Classes and Exam Grade
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
1st Class Honours Upper 2nd Class Honours Lower 2nd Class Honours H3
Library Class Attendence and Exam Success
Non-Attending
Attending
![Page 43: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/43.jpg)
Embedded Library Classes
“Learning to Learn”
Commenced September 2013
Level 8 Business students
Subsequently rolled out to other classes
![Page 44: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/44.jpg)
Embedded Library Class Attendees
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Sit exams Score 70-
100% in
exams
Score 60-
69% in
exams
Pass exams
% of students
on course
Students on courses with learning to learn element are more likely to...
The effects of embedded library classes I
Learning to Learn
Non-Learning to Learn
![Page 45: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/45.jpg)
Embedded Library Class Attendees
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
Score 50-59%
in exams
Score 40-49%
in exams
Fail exams Be non-active
on course
% of students
on course
Students on courses with learning to learn element are less likely to...
The effects of embedded library classes II
Learning to Learn
Non-Learning to Learn
![Page 46: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/46.jpg)
https://xkcd.com/552/
![Page 47: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/47.jpg)
If nothing else
Advocacy & Marketing
We have some nice slides for
advocacy/marketing
purposes
![Page 48: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/48.jpg)
Printing
Free student printing from Moodle
6 (b/w) or 25 (colour) cents/sheet for
other printing
![Page 49: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/49.jpg)
The key to exam success … print
more…
0
200
400
600
800
1000
1200
1400
1600
1800
H1 H21 H22 H3
Pages Printed
Honours Class
Printing and Exam Success 2013-2014
Average Pages Printed
![Page 50: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/50.jpg)
… but don’t pay for it
0
10
20
30
40
50
60
70
H1 H21 H22 H3
Honours Class
Print spending and Exam Success 2013-2014
Money spent (Euro)
![Page 51: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/51.jpg)
Tentative conclusions/hypotheses
The library has the right books and right
electronic resources
Library classes might improve student
engagement and so their final exam grade
Link between free printing and exam grade
Moodle notes are effective
Students prefer reading print to reading on
screen (or information retained better
when read in print)
![Page 52: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/52.jpg)
Next
More data - more library class analysis
Statistical analysis
Library measurement as a tool not to measure library impact, but measure and monitor student engagement student progression?
Communication/collaboration
with other departments?
with other libraries?
![Page 53: Measuring Library Impact at DBS](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d16de9bb61ebc33b8b4721/html5/thumbnails/53.jpg)
Thank you for listening