Project web site – www.arl.org/libqual/

TM

Academic Library and Information Services:New Paradigms for the Digital Age

Bielefeld, GermanyFebruary 8, 2006

Presented by:

Dr. Colleen Cook, DeanTexas A&M University

&

Dr. Fred Heath, Vice Provost of General LibrariesUniversity of Texas

88thth International InternationalBielefeld ConferenceBielefeld Conference

Why Assess?Why Assess?

““In an In an age of accountabilityage of accountability, there is a pressing , there is a pressing need for an effective…process to evaluate and need for an effective…process to evaluate and compare research libraries.” compare research libraries.”

700 participants in LibQUAL+700 participants in LibQUAL+™™

123 Association of Research Libraries (ARL) 123 Association of Research Libraries (ARL) alone, over alone, over $3.4 billion dollars$3.4 billion dollars were expended in were expended in 2003/20042003/2004

Note. M. Kyrillidou and M. Young. (2005).ARL Statistics 2003-04. Washington, D.C.: ARL, p.5.

Libraries Remain a CredibleLibraries Remain a CredibleResource in 21Resource in 21stst Century Century

Note. Digital Library Federation and Council on Library and InformationResources. (2002). Dimensions and Use of the Scholarly Information Environment.

98% agree with statement98% agree with statement, “My … library , “My … library contains information from credible and contains information from credible and known sources.”known sources.”

Changing BehaviorsChanging Behaviors

Note. Digital Library Federation and Council on Library and InformationResources. (2002). Dimensions and Use of the Scholarly Information Environment.

Recent Survey:Recent Survey:Only Only 15.7% agreed with the statement15.7% agreed with the statement “The “The Internet has not changed the way I use the Internet has not changed the way I use the library.”library.”

Faculty: Dependence onElectronic Resources Will Increase

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2000 2003

Not WellSomewhatVery Well

“I will become increasingly dependent on electronic research resources in the future.”

http://www.arl.org/arl/proceedings/144/guthrie_files/guthrie.ppt

Research Behavior:Research Behavior:Personal ControlPersonal Control

When searching for When searching for printprint journals for journals for research:research:

Note. Digital Library Federation and Council on Library and InformationResources. (2002). Dimensions and Use of the Scholarly Information Environment.

• Only 13.9% ask a librarian for assistanceOnly 13.9% ask a librarian for assistance• Only 3.2% consider consulting a librarian a preferred Only 3.2% consider consulting a librarian a preferred way of identifying informationway of identifying information

Total Circulation

Note. M. Kyrillidou and M. Young. (2005). ARL Statistics 2003-04. Washington, D.C.: ARL, p.6.

Reference Transactions

Note. M. Kyrillidou and M. Young. (2005). ARL Statistics 2003-04. Washington, D.C.: ARL, p.6.

Web UsageWeb UsageTotal File Requests - UT Austin Libraries 2000-2003Total File Requests - UT Austin Libraries 2000-2003

0

100,000,000

200,000,000

300,000,000

400,000,000

500,000,000

600,000,000

700,000,000

800,000,000

900,000,000

2000 2001 2002 2003

Total Hits

Enter LibQUAL+™

The necessity of assessment

Rapid shifts in information-seeking behavior

The reallocation of resources from traditional services and functions

PERCEPTIONS SERVICE

“….only customers judge quality;

all other judgments are essentially

irrelevant”

Note. Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.

The LibQUAL+™ Premise

13 LibrariesEnglish LibQUAL+™ Version

4000 Respondents

QUAL

QUAN

QUAL

QUAL

QUAN

QUAL

PURPOSE DATA ANALYSIS PRODUCT/RESULTDescribe library environment;build theory of library service quality from user perspective

Test LibQUAL+™ instrument

Refine theoryof service quality

Refine LibQUAL+™ instrument

Test LibQUAL+™ instrument

Refine theory

Unstructured interviewsat 8 ARL institutions

Web-delivered survey

Unstructured interviews at Health Sciences and the Smithsonian libraries

E-mail to surveyadministrators

Web-delivered survey

Focus groups

Content analysis:(cards & Atlas TI)

Reliability/validityanalyses: CronbachsAlpha, factor analysis,SEM, descriptive statistics

Content analysis

Content analysis

Reliability/validity analyses including Cronbachs Alpha,factor analysis, SEM, descriptive statistics

Content analysis

VignetteRe-tooling

Iterative

Emergent2000

2005700 Libraries English, Dutch, Swedish,

German LibQUAL+™ Versions160,000 anticipated respondents

LibQUAL+LibQUAL+™ Project™ Project

Case studies1

Valid LibQUAL+™ protocol

Scalable process

Enhanced understanding of user-centered views of service quality in the library environment2

Cultural perspective3

Refined survey delivery process and theory of service quality4

Refined LibQUAL+™ instrument5

Local contextual understanding of LibQUAL+™ survey responses6

76 Interviews Conducted

York University University of Arizona Arizona State University of Connecticut University of Houston University of Kansas

University of Minnesota University of Pennsylvania University of Washington Smithsonian Northwestern Medical

LoadedPT:P1:01xxxxxxxxxxxxxx.txt,S:\Admin\Colleen\ServQual Interviews\TEXT Only\01xxxxxxxxx.txt (redirected: c:\zz\atlasti\fred

Dimensions ofLibrary Service Quality

Empathy

InformationControl

Responsiveness

Symbol

Utilitarian space

Assurance

Scope of Content

Ease of Navigation

Self-Reliance

Library as Place

LibraryServiceQuality

Model 3

Refuge

Affect of Service

Reliability

Convenience

Timeliness

Equipment

Affect of Service

“I want to be treated with respect. I want you to be courteous, to look like you know what you are doing and enjoy what you are doing. … Don’t get into personal conversations when I am at the desk.”

Faculty member

Library as Place

“One of the cherished rituals is going up the steps and through the gorgeous doors of the library and heading up to the fifth floor to my study. … I have my books and I have six million volumes downstairs that are readily available to me in an open stack library.”

Faculty member

Information Control

“…first of all, I would turn to the best search engines that are out there. That’s not a person so much as an entity. In this sense, librarians are search engines [ just ] with a different interface.”

Faculty member

Information Control

“By habit, I usually try to be self-sufficient. And I’ve found that I am actually fairly proficient. I usually find what I’m looking for eventually. So I personally tend to ask a librarian only as a last resort.”

Graduate student

Multiple Methodsof Listening to Customers Transactional surveys* Mystery shopping New, declining, and lost-customer surveys Focus group interviews Customer advisory panels Service reviews Customer complaint, comment, and inquiry capture Total market surveys* Employee field reporting Employee surveys Service operating data capture*A SERVQUAL-type instrument is most suitable for these methodsNote. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000).

Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.

LibQUAL+™ Resources

An ARL/Texas A&M University joint developmental effort based on SERVQUAL.

LibQUAL+™ initially supported by a 3-year grant from the U.S. Department of Education’s Fund for the Improvement of Post-Secondary Education (FIPSE)

Initial project established a expert team, re-grounded SERVQUAL concepts, and designed survey methodology

Survey conducted at over 700 libraries resulting in a data base of over half a million user responses

NSF funded project to refocus LibQUAL+™ on the National Science Digital Library (NSDL)

Participating Libraries

World LibQUAL+™ Survey 2005

LibQUAL+™ Languages

American English

Dutch EnglishFrench Canadian DutchSwedish

Swedish(British English)

Afrikaans

DanishFinnishGerman Norwegian

British English

Continental French

Rapid Growth in Other Areas

Languages American English British English French Dutch Swedish

In development Chinese Greek Spanish German

Consortia Each may create 5 local

questions to add to their survey

Types of Institutions Academic Health Sciences Academic Law Academic Military College or University Community College European Business Hospital Public State

Countries U.S., U.K., Canada, the

Netherlands, South Africa, Sweden, France, Australia, New Zealand, Malaysia

“22 items”

2000 2001 2002 200341-items 56-items 25-items 22-items

Affect of Service Affect of Service Service Affect Service Affect

Reliability Library as Place Library as Place Library as Place

Library as Place Reliability Personal ControlInformation Control

Provision of Physical Collections

Self-RelianceInformation Access

Access to Information

Access to Information

Survey Instrument

“And a Box”

Why the Box is so Important About 40% of participants provide open-

ended comments, and these are linked to demographics and quantitative data.

Users elaborate the details of their concerns.

Users feel the need to be constructive in their criticisms, and offer specific suggestions for action.

Understanding Understanding LibQUAL+™ ResultsResults

• Measures the distance between minimally Measures the distance between minimally acceptableacceptable and and desireddesired service quality ratings service quality ratings

• Perception ratings ideally fall within the Perception ratings ideally fall within the Zone Zone of Toleranceof Tolerance

Key to Bar Charts

LibQUAL+™ 2004 Summary Collegesor Universities American English

(n = 69,449)

LibQUAL+™

Adequacy GapThe difference between the minimum and perceived score

Score Norms

Norm Conversion Tables facilitate the interpretation of observed scores using norms created for a large and representative sample.

LibQUAL+™ norms have been created at both the individual and institutional level

LibQUAL+™ InteractiveInstitution Statistics

YEAR (REQ.) – defaults to current year of the survey; controls the Master List (dynamic)

Master List of Institutions

All

Clear

• OVERALL – defaults to OVERALL

• Dimension (3)

mindesperadeq sup

SUBMIT

VARIABLES - Default to perceived

Results Page: This page generates a graph, Summary Statistics, Your Statistics, and Norms (including users’ norm values); restates information entered into the form

4 items included in this

drop-down menu

Your List

INSTITUTION TYPE

SAVED LISTS

CONSORTIA

LANGUAGE ADD

ADD

ADD

ADDLanguage (s) will be determined based on

selection (s) from the Master List of Institutions; dynamic

User selects an institution from the Master List, the selection is then added to “Your List”. To avoid duplicate choices, the selection from the Master List will disappear once added to “Your List”.

Other parameters can be added below (institution type, language, consortia, and/or SAVED LISTS).

Consortia, based on

current year; dynamic

1)

2)

SAVEText here stating that this section is optional and may be added to “Your List” to narrow down results.

Text box for user to name and save search parameters

for future searches.

In Closing LibQUAL+™

Focuses on success from the users’ point of view (outcomes)

Demonstrates that a web-based survey can handle large numbers; users are willing to fill it out; and survey can be executed quickly with minimal expense

Requires limited local survey expertise and resources

Analysis available at local and inter-institutional levels

Offers many opportunities for using demographics to discern user behaviors

LibQUAL+™ Resources

LibQUAL+™ Website:http://www.libqual.org

Publications: http://www.libqual.org/publications

Events and Training: http://www.libqual.org/events

LibQUAL+™ Bibliography: http://www.coe.tamu.edu/~bthompson/servqbib

LibQUAL+™ Procedures Manual: http://www.libqual.org/Information/Manual/index.cfm

LibQUAL+™ Contact Information Amy Hoseth

LibQUAL+™ Communications Coordinator amyh@arl.org

Richard Groves Statistics Research Assistant richard@arl.org

MaShana Davis Junior Technical Applications Developer mashana@arl.org

Martha Kyrillidou Director, ARL Statistics and Measurement Program martha@arl.org

https://webspace.utexas.edu/fh355/www/

- 30 -