P T. 1 : D E V E LO P I N G A N E VA LU AT I O N T O O L

CUNY E-BOOK TASK FORCE:

CITY UNIVERSITY OF NEW YORK (CUNY)

• 11 senior colleges, 7 community colleges, Macaulay Honors College, and 5 graduate and professional schools—all located throughout the 5 boroughs.

• 269,000 degree-credit students; 270,000 certificate and professional students.

• From certificate courses to Phd programs, CUNY offers postsecondary learning to a remarkably diverse population of students of all backgrounds from 208 countries.

AN INTRODUCTION TO OUR LIBRARIES

7Community Colleges

5Graduate

Center and Professional

Schools

Office of Library Services

11Senior

Colleges

Aleph catalogSFX link resolver

EZ proxy Cataloging

supportE-resources

StudentsExpertisePrint collections

ELECTRONIC RESOURCES ADVISORY COUNCIL (ERAC) OF CUNY

• 1 Representative from each campus library• Usually the Electronic Resources Librarian • Many consortium pricing arrangements are

discussed and decided upon at ERAC (some funded by the University; others funded by individual libraries)• From this Committee, the University Dean of

Libraries put together a task force.

TASK FORCE CHARGE:

• Task Force 1: Develop the evaluative criteria we should be using to determine which, if any, e-book collection would best suit the needs of the University.

TASK FORCE MEMBERS

• Task Force 1: • Nancy Egan, Chair, John

Jay College• Linda Dickinson, Hunter

College• John Drobnicki, York

College• Madeline Ford, Hostos CC • Maria Kiriakova, John Jay

College • Catherine Stern, LaGuardia

CC• Lisa Tappeiner, Hostos CC• Mike Waldman, Baruch

College

• Task Force 2:• Angela Sidman, Central Office,

Co-Chair• Nancy Egan, John Jay College,

Co-Chair• Jane Fitzpatrick, Graduate

Center, CUNY• Helen Georgas, Brooklyn College• Maria Kiriakova, John Jay College • Laroi Lawton, Bronx CC • Linda Roccos, College of Staten

Island• Catherine Stern, LaGuardia CC• Lisa Tappeiner, Hostos CC• Susan Vaughn, Brooklyn College

and Central Office

COMMITTEE MAKE-UP, TASK FORCE 1

Members by Functional Area

63%

38%

Members by Institution Type

Senior CollegeCommunity Col-lege

Members by Functional Area

29%

21%21%

7%

7%

7%

7%

Members by Functional Area

acquisitionselectronic resourcescollection man-agementcataloging mediapublic serviceschief

TASK FORCE 1

TASK FORCE 1INITIAL MEETING

• Fleshing out the mission • Determining broad categories:

Ease of ProcurementContentTechnical RequirementsLicensing TermsCollection BuildingCataloging and Catalog MaintenanceCompatibility with other Library ResourcesAdministrative FunctionsUser ExperienceProduct Support for LibrariansLibrary Culture

SHARING THE WORK

• Used categories to create outline• Every category had at least one person. • Shared articles, thoughts, glossary terms, etc. on

GoogleDocs• Used GoogleDoc information to fill in the outline• Second meeting & discussion to refine model

THE CHECKLIST

• URL: http://guides.lib.jjay.cuny.edu/ebooktaskforce• Categories and subcategories• Questions under each category and/or

subcategory• An accompanying glossary and report

WEIGHTING SYSTEM

• Making the system work:• Weighting system will be for this particular purpose• Categories and subcategories are assigned a weight• 1 is considered helpful, 5 is important, and 10 is very

important or vital to the needs of the University community

• Questions are used to flesh out each category (or subcategory) and determine how the vendor fares in each category

• The vendor gets points—all or none in each category—based on whether they’ve met the criteria

TASK FORCE CHARGE

• Task Force 2: Apply the model to two competing vendors’ Ebook packages and make a recommendation to the University Librarian.

P T. 2 : A P P LY I N G T H E E VA LU AT I O N T O O L

CUNY E-BOOK TASK FORCE:

TASK FORCE, PT. 2

• Goal: • To apply the

checklist and evaluate the two competing ebook packages

• To evaluate the effectiveness of the checklist

• Method:• 10 people on

committee

• Cross section of functional areas and institutions represented

• Each person evaluates two areas

COMMITTEE MAKE-UP, ROUND II

10%

50%

30%

10%

Members by institution type

Central Office

Senior College

Community Col-lege

Graduate Cen-ter

30%

20%20%

10%

10%

10%

Members by functional area

E-resources

Media

Collection management

Public services

Acquisitions

Cataloging

EVALUATION IN ACTION

• Assigned 2 sections each for review• Given a month to evaluate both ebook packages• Base conclusions on:• Own evaluation• Information provided by the vendor• Feedback from colleagues

• Regroup and discuss > Formally assign points

EVALUATING THE CHECK LIST

• Would the checklist help us evaluate each product fully?• Would the all or nothing points system prove

effective?• Are partial points needed?

• Would discussion be productive?• Would one package come out the clear winner?

WHAT WE FOUND: EBOOKS

• Both ebook packages were excellent• Use of the checklist teased out differences

between them and allowed us to weigh our own values and come to a clear decision

WHAT WE FOUND: CHECKLIST

Pros

• Elicited great discussion• The list covered a range

of scenarios but easy to pick only applicable ones

• Provided a neutral avenue for considering products, bypassing vendor prejudices

• The points worked!

Cons

• Written to cover any scenario, so quite long and time consuming to apply• For best results (and

most likely consensus) multiple participants are needed

OUTCOMES

• Based on evaluation, change made to checklist• Wrote findings up in a report, shared with

University Dean, E-resources Advisory Committee, and on internal listserv of 300+ librarians• Allowed for harmonious decision-making• Transparent process• Participation and buy-in from all major groups

• Basis of feedback provided to vendors• Shared survey instrument with other CUNY

librarians• Available via Support Site