Evaluation of Evaluation of Information Information Sources and Sources and Reference Reference ServicesServicesDr. Dania BilalDr. Dania Bilal

IS 530IS 530Fall 2006Fall 2006

Why Evaluate Services?Why Evaluate Services?

Assess and improve the quality of Assess and improve the quality of existing servicesexisting services

Identify the need for new types of Identify the need for new types of services or programsservices or programs

Justify funding and supportJustify funding and support

Why Evaluate Services?Why Evaluate Services?

Manage human and physical Manage human and physical resources more effectively and resources more effectively and efficientlyefficiently

Justify the need for maintaining Justify the need for maintaining services and need for additional services and need for additional programsprograms

Levels of EvaluationLevels of Evaluation

Lancaster’s scheme:Lancaster’s scheme: InputsInputs OutputsOutputs OutcomesOutcomes

Levels of Evaluation: Levels of Evaluation: InputsInputs

Materials available to provide a Materials available to provide a serviceservice Reference collectionReference collection

Print and electronicPrint and electronic External materials (virtual)External materials (virtual)

Levels of Evaluation: Levels of Evaluation: OutputsOutputs

Measurement of quality of providing Measurement of quality of providing a service:a service: Answering reference questionsAnswering reference questions

accuracyaccuracy completenesscompleteness

Assistance given in using sourcesAssistance given in using sources

Levels of Evaluation: Levels of Evaluation: OutcomesOutcomes

Meeting user information needsMeeting user information needs Level of satisfactionLevel of satisfaction Level of knowledge gained or improvedLevel of knowledge gained or improved

Discussion QuestionDiscussion Question: How should : How should professionals assess user professionals assess user satisfaction?satisfaction?

Evaluation GuidelinesEvaluation Guidelines

RUSA guidelines for reference RUSA guidelines for reference servicesservices http://http://

www.ala.org/rusaTemplate.cfm?Sectionwww.ala.org/rusaTemplate.cfm?Section==referenceguide&Templatereferenceguide&Template=/=/ContentManagement/ContentDisplay.cfContentManagement/ContentDisplay.cfm&ContentIDm&ContentID=26884=26884

Selection of Reference Selection of Reference SourcesSources

RUSA Reference CommitteeRUSA Reference Committee Outstanding Reference Sources

Annually in May issue of Annually in May issue of American LibrariesAmerican Libraries Print sourcesPrint sources Small- and medium-sized academic and Small- and medium-sized academic and

public librariespublic libraries Online via RUSA websiteOnline via RUSA website

Selection of Reference Selection of Reference SourcesSources

ARBA (American Reference Books ARBA (American Reference Books Annual)Annual)

GuidesGuides Guide to Reference Books Guide to Reference Books Walford’s Guide to Reference MaterialsWalford’s Guide to Reference Materials Recommended Reference Books for Small Recommended Reference Books for Small

and Medium-Sized Libraries and Media and Medium-Sized Libraries and Media CentersCenters

Guide to Reference Materials for School Guide to Reference Materials for School Library Media Centers Library Media Centers (Safford, Barbara R.) (Safford, Barbara R.)

Selection of Reference Selection of Reference SourcesSources

Booklist Booklist ((Reference Book BulletinReference Book Bulletin)) Library Journal Library Journal (print, websites, and databases)(print, websites, and databases) School Library JournalSchool Library Journal Reference & User Services QuarterlyReference & User Services Quarterly Books in Print With Book ReviewsBooks in Print With Book Reviews Choice Choice (reviews electronic databases in each (reviews electronic databases in each

issueissue

Many other sources are listed in your text (Vol. I, Many other sources are listed in your text (Vol. I, chap. 3). These Sources serve as selection and chap. 3). These Sources serve as selection and evaluation tools.evaluation tools.

Evaluation of Print Evaluation of Print SourcesSources

Examination using criteria:Examination using criteria: PurposePurpose AuthorityAuthority CurrencyCurrency Physical conditionPhysical condition FormatFormat Curriculum needsCurriculum needs Target audienceTarget audience Objectiveness of contentObjectiveness of content Online availabilityOnline availability CostCost

Evaluation Electronic Evaluation Electronic DatabasesDatabases

PurposePurpose DesignDesign

User InterfaceUser Interface Search and Retrieval featuresSearch and Retrieval features

Record structureRecord structure Navigation Navigation

Visual elements (icons, metaphors, colors, Visual elements (icons, metaphors, colors, etc.)etc.)

Ease of useEase of use Links to external resources (e.g., SFX) Links to external resources (e.g., SFX)

Evaluation of Electronic Evaluation of Electronic DatabasesDatabases

AuthorityAuthority Coverage (time period)Coverage (time period) Scope (type of publications Scope (type of publications

covered/indexed)covered/indexed) Full-text availability of articlesFull-text availability of articles Target audienceTarget audience Curriculum needsCurriculum needs CostCost Other criteria?Other criteria?

Evaluation of Reference Evaluation of Reference StaffStaff

ApproachabilityApproachability Knowledge of reference collectionKnowledge of reference collection Knowledge of local services and referralKnowledge of local services and referral Skills in identifying user needs Skills in identifying user needs

(mediation)(mediation) (we will cover the evaluation of the (we will cover the evaluation of the

reference process and mediators later in reference process and mediators later in the course) the course)

Communication with usersCommunication with users

Evaluation of Reference Evaluation of Reference TransactionsTransactions

Types:Types: ObtrusiveObtrusive

Issues & challengesIssues & challenges UnobtrusiveUnobtrusive

Issues & challengesIssues & challenges Reference statisticsReference statistics

Evaluation of User Evaluation of User SatisfactionSatisfaction

Assess satisfaction of all types of Assess satisfaction of all types of users, including specific populationsusers, including specific populations

TechniquesTechniques interviewsinterviews telephonetelephone WebWeb