Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self,...
-
date post
21-Dec-2015 -
Category
Documents
-
view
215 -
download
0
Transcript of Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self,...
![Page 1: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/1.jpg)
Two Decades of User Surveys
The Experience of Two Research Libraries from 1992 to 2011
Jim Self, University of VirginiaSteve Hiller, University of Washington
Northumbria Conference22 August 2011
![Page 2: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/2.jpg)
User Surveys
• Accepted methodology• Identify user issues, concerns and needs• Measure library performance from user perspective • Provide opportunities for open ended comments• Provide data for statistical analysis and generalization• Results can lead to changes and improvements• Useful for institutional assessment and accreditation
![Page 3: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/3.jpg)
User Surveys • Records of perceptions, not actual performance• Difficult to frame complicated questions• Results affected by wording of questions• Results often lack context• Can be difficult and costly to do properly • Survey fatigue can limit participation• Not the only tool in the assessment toolkit
• ...but longitudinal data can show graphic changes
![Page 4: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/4.jpg)
The University of Virginia
• 15,000 undergraduates– 67% in-state, 33% out – Most notable for liberal arts– Highly ranked by U.S. News
• 6,000 graduate students– Prominent for humanities,
law, business– Plans expansion in sciences
• Located in Charlottesville– Metro population of 160,000
U.Va. Library Assessment Information:http://www2.lib.virginia.edu/mis/
![Page 5: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/5.jpg)
University of Washington
• Comprehensive public research university located in Seattle– 28,000 undergraduate students– 12,000 graduate/professional
students (80 doctoral programs)– 4,000 faculty– 23 in Times World Uni Rankings
• $1.4 billion yearly in external research funds
UW Libraries Assessment Information:http://www.lib.washington.edu/assessment
![Page 6: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/6.jpg)
U.Va. Library Surveys 1.0
• Faculty • Sample of 500
– 1993 (paper)– 1996 (paper)– 2000 (Web)– 2004 (Web)
• Students• Separate tallies for grads and undergrads• Sample size varied
– 1994 (paper)– 1998 (Web)– 2001 (Web)– 2005 (Web)
![Page 7: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/7.jpg)
U.Va. Library Surveys 1.5LibQUAL+® 2006
• Why do LibQUAL?– Comparisons with peer libraries– Comparison of user ratings and desires– Ease of administration
• Why not?– Low response rates– Inability to customize
![Page 8: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/8.jpg)
U.Va. Library Surveys 2.0
• Starting in 2008• Annual surveys with smaller samples• One third of teaching faculty• 500 graduate students, 750 undergraduates• Separate tallies and analysis for each category• Brief report annually• Comprehensive report every three years
![Page 9: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/9.jpg)
UW Libraries Surveys
• Triennial surveys began in 1992 (7th survey in 2010)• Surveys initially focused on satisfaction and use, now
includes importance, impact, usefulness and priorities • All faculty surveyed; sample of students
– 1634 faculty respondents in 2010 • Switched from paper to Web in 2004• Groups surveyed at same time, but questions differ
between groups and may change over time• New in-library use survey began in 2002 on 3 year
cycle to better capture use of physical library
![Page 10: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/10.jpg)
Survey Response Rates U.Va. and UW, 1992-2011
92/94 95/98 00/01 04/05 07/09 10/110%
20%
40%
60%
80%
Fac, 28%
Ugrads, 43%
Ugrads, 41%
Fac, 46%44%
Fac, 39%
Fac, 70%
Ugrads, 16%
![Page 11: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/11.jpg)
the results …
University of Virginia• Faculty value online access
and ILL/Document Delivery• Grad students value online
collections and access• Undergrads value space and
comfort• Customer service high in
satisfaction and importance
University of Washington• Faculty value collections• Grad Students value access• Undergrads value place• Libraries get a lot of love• Moving beyond love and
satisfaction to importance and impact
![Page 12: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/12.jpg)
Weekly Visits to a Physical Library U.Va. and UW, 1993-2011
93/94 96/98 00/01 04/05 07/09 10/1110%
30%
50%
70%
90%
Undergrads
Grads
21%
67%
23%
43%
69%
84%
79%
67%
Faculty
38%
![Page 13: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/13.jpg)
UW In-Library Use Survey User Percentage by Group 2002-2011
2002n=3878
2005n=3266
2008n=3196
2011n=3379
Undergraduate Students
59% 64% 69% 74%
Graduate Students
25% 22% 20% 17%
Faculty/Staff 7% 5% 4% 4%
Not UW affiliated 8% 8% 5% 4%
![Page 14: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/14.jpg)
U.Va: Monitoring the Online CatalogSatisfaction with VIRGO since 1993
1993
/94
1996
/98
2000
/01
2004
/05
2008
2009
2010
2011
3.00
3.50
4.00
4.50
5.00
Ugrads
Grads
Faculty
![Page 15: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/15.jpg)
U.Va.: Monitoring the Online CatalogVirgo Visibility since 1993
1993
/94
1996
/98
2000
/01
2004
/05
2008
2009
2010
2011
60
70
80
90
100
Perc
ent
Ugrads
Faculty
Grads
70%
84%
86%
![Page 16: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/16.jpg)
Overall Library Satisfaction1993-2011
93/94 96/98 00/01 04/05 08/10 20113.75
4.00
4.25
4.50
4.75
Faculty
Grads
Undergrads
4.23
4.35
4.25
1995 1998 2001 2004 2007 20103.75
4.00
4.25
4.50
4.75
UW
Faculty
4.25
4.63
4.43
U.Va.
![Page 17: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/17.jpg)
UW Library Services and Resources: Overall Importance to Work by Group
(Scale of 1 “Not Important” to 5 “Very Important)
3
3.5
4
4.5
5
Collections Discovery tools Info services and instruction
Physical spaces
Undergrad Grad Faculty
![Page 18: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/18.jpg)
UW Libraries Contribution to Research Productivity (Based on Funding Source)
(Scale of 1 “Minor” to 5 “Major”)
4
4.25
4.5
4.75
5
All Soc Sci Sciences Med School
No federal research funds
Federal research funds
![Page 19: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/19.jpg)
UW Libraries Contribution to Recruiting Colleagues & Students to UW
(Faculty Response by College/School)
Humanities
Social Sci
Arts
Nursing
Public Hlth
Medicine
Science
Business
Engineering
2 2.25 2.5 2.75 3 3.25 3.5 3.75 4
Scale of 1 (Minor) to 5 (Major)
![Page 20: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/20.jpg)
UW Libraries Contribution to Enriching Student Learning Experiences
(Faculty Response by College/School)
Humanities
Social Sci
Arts
Nursing
Public Hlth
Medicine
Science
Business
Engineering
3 3.25 3.5 3.75 4 4.25 4.5 4.75 5
Scale of 1 (Minor) to 5 (Major)
![Page 21: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/21.jpg)
UW Subject Librarian Visibility and Satisfaction By Faculty College/School
4.90
4.30
4.48
3.85
4.73
3.924.00
4.76
4.00
4.334.28
4.63
4.26
4.88
4.724.71
4.34
3.00
4.25
30% 67%
Satisfaction
Visibility
Built Environments
Education
Business
EngineeringMedicine
I School
PharmacyDentistry
Nursing
Public Affairs
Public Health
Social Work
Environment
Fine Arts
Humanities
Social Sciences
Sciences
![Page 22: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/22.jpg)
Issues in Survey Implementation
• Customized vs Standardized• Web vs Paper• Frequency of administration• Longitudinal tracking vs new or changed
questions• Library only surveys
![Page 23: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/23.jpg)
Other Tools at U.Va. and UW
• Balanced Scorecard• Usability testing• Focus groups• Structured interviews• Staff surveys• Task surveys, with post-it notes• In-house user surveys• Observation• Data mining• Statistical compilations
![Page 24: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/24.jpg)
Putting Survey Data to Use
• Additional resources for the science & arts libraries• Undergraduate library open 24 hours • Support for transition from print to e-journals• New and improved study spaces• Reallocate budgets and staffing internally• Use externally in budget requests• Inform the strategic planning process, develop
performance measures, • Enhanced usability of discovery tools• Identify areas for further study
![Page 25: Two Decades of User Surveys The Experience of Two Research Libraries from 1992 to 2011 Jim Self, University of Virginia Steve Hiller, University of Washington.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d575503460f94a352d3/html5/thumbnails/25.jpg)
more info at…
• U.Va.http://www2.lib.virginia.edu/mis/
• UWhttp://www.lib.washington.edu/assessment/