Evaluation of Reference Services

Dr. Dania Bilal

IS 530

Fall 2005

Why Evaluate?

Assess and improve the quality of existing services

Identify the need for new types of services or programs

Justify funding and support

Why Evaluate?

Manage human and physical resources more effectively and efficiently

Justify the need for maintaining services and need for additional programs

Levels of Evaluation

Lancaster’s scheme:– Inputs– Outputs– Outcomes

Levels of Evaluation: Inputs

Materials available to provide a service– Reference collection

• Print, CD-ROM, Web-based databases

• Virtual materials

Levels of Evaluation: Outputs

Measurement of quality of providing a service:– Answering reference questions

• accuracy

• completeness

– Assistance given in using sources

Levels of Evaluation: Outcomes

Meeting user information needs– Level of satisfaction– Level of knowledge gained or improved

Discussion Question: How should professionals assess user satisfaction?

Evaluation Guidelines

RUSA guidelines for reference services– http://www.ala.org/rusaTemplate.cfm?Section=

referenceguide&Template=/ContentManagement/ContentDisplay.cfm&ContentID=26884

Evaluation of Print Sources

Direct examination using criteria– Authority, currency, physical condition, format,

target audience, curriculum needs, objectiveness, etc.

Standardized lists Interlibrary Loan Requests Collection mapping software

Evaluation Electronic Sources

User Interface– Search features– Retrieval features

Help file Content vis-à-vis purpose and target

audience Currency

Evaluation Techniques: Electronic Sources Design

– Record structure– Navigation – Visual elements (Icons and their metaphors,

colors, etc.)– Hyperlinks– Overall user interaface

Authority

Evaluation of Reference Staff

Approachability Knowledge of reference collection Knowledge of local services and referral Skills in identifying user needs Communication with users

Evaluation of Reference Transactions Types:

– Obtrusive• Issues & challenges

– Unobtrusive• Issues & challenges

– Reference statistics

Evaluation of User Satisfaction

Assess satisfaction of all types of users, including specific populations

Techniques– interviews– via telephone– via the Web– via other means