E-Metrics: Lessons Learned from the ARL E-Metrics Project...
Transcript of E-Metrics: Lessons Learned from the ARL E-Metrics Project...
![Page 1: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/1.jpg)
www.arl.orgAssociation of Research Libraries
E-Metrics: Lessons Learned from the ARL E-Metrics Project, Challenges and Opportunities
Martha Kyrillidou226th American Chemical Society National MeetingCINF Section B, Jacob K. Javits Convention Center, Room 1E20September 8, 2003 • New York, NY
![Page 2: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/2.jpg)
www.arl.orgAssociation of Research Libraries
Association of Research Libraries
•Mission: Shaping and influencing factors affecting the future of Research Libraries in the process of scholarly communication
• Members: 124 major research libraries in North America
• Users: More than 3 million students and faculty served
• Service measures: 75 million circulations17 million reference transactions
•42,000 FTE staff employed
•Members’ total expenditures are over $3.2 billion
![Page 3: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/3.jpg)
www.arl.orgAssociation of Research Libraries
ARL Library Expenditures 2001-02114 ARL University Libraries
Contract Binding$25,168,718
1%
Total Salaries$1,143,942,869
46%
Library Materials$954,401,999
39%
Other Operating$353,755,527
14%
![Page 4: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/4.jpg)
www.arl.orgAssociation of Research Libraries
Graph 2
Monograph and Serial Costs in ARL Libraries, 1986-2002
-50%
0%
50%
100%
150%
200%
250%
1986 1988 1990 1992 1994 1996 1998 2000 2002Source: ARL Statistics 2001-02 , Association of Research Libraries, Washington, D.C.
% C
hang
e S
ince
198
6
Serial Unit Cost(+227%)
Serial Expenditures(+227%)
Monograph Unit Cost (+75%)
CPI (+64%)Monograph Expenditures(+62%)
Serials Purchased (+9%)
Monographs Purchased (-5%)
Avg. Annual % Change
Serial Unit Cost
7.7%
Serial Expenditures
7.7%
Serials Purchased
0.6%
Monograph Unit Cost
3.6%
Monograph Expenditures
3.1%
Monographs Purchased
-0.3%
CPI 3.2%
![Page 5: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/5.jpg)
www.arl.orgAssociation of Research Libraries
Graph 4 Expenditure Trends
in ARL Libraries, 1986-2002
-10%
20%
50%
80%
110%
140%
170%
200%
230%
1986 1988 1990 1992 1994 1996 1998 2000 2002Source: ARL Statistics 2001-02 , Association of Research Libraries, Washington, D.C.
% C
hang
e S
ince
198
6
Serial Expenditures(+227%)
Library Materials (+184%)
Operating Expenditures (+92%)
TOTAL (+120%)
Total Salaries (+108%)
CPI (+64%)Monograph Expenditures (+62%)
Avg. Annual % Change
Serial Expenditures
7.7%
MonographExpenditures
3.1%
Total Materials Expenditures
6.7%
Total Salary Expenditures
4.7%
Operating Expenditures
4.2%
Total Library Expenditures
5.1%
CPI 3.2%
![Page 6: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/6.jpg)
www.arl.orgAssociation of Research Libraries
LibQUAL+™ 2003 Colleges or Universities – Library Use
![Page 7: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/7.jpg)
www.arl.orgAssociation of Research Libraries
Expenditures for Electronic Resources
19.54%
of the library materials budget
![Page 8: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/8.jpg)
www.arl.orgAssociation of Research Libraries
Total Electronic Serials Expenditures, 1995-2002
-
20,000,000
40,000,000
60,000,000
80,000,000
100,000,000
120,000,000
140,000,000
160,000,000
180,000,000
1995 1996 1997 1998 1999 2000 2001 2002
Year
![Page 9: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/9.jpg)
www.arl.orgAssociation of Research Libraries
4.32%
37.28%
8.48%
47.46%
4.18%
12.86%
9.18%
23.22%
5.53%
22.5%
6.35%
24.81%
3.4%
27.06%
9.12%
32.69%
5.92%
24.29%
0
5
10
15
20
25
30
35
40
45
50
1992-93 1993-94 1994-95 1995-96 1996-97 1997-98 1998-99 1999-2000 2000-01 2001-02
Yearly Increases in Average Electronic Resources and Total Library Materials Expenditures
Increase in Average Library Materials Expenditures Increase in Average Electronic Resources Expenditures
![Page 10: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/10.jpg)
www.arl.orgAssociation of Research Libraries
(Carla Stoffle, University of Arizona)
Developing measures and evaluation techniques for networked services will take time, effort, and on-going learning on everyone’s part – but we must begin now.
We not only need to measure things in new ways but we also need to measure new things.
(Sherrie Schmidt, Arizona State University)
![Page 11: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/11.jpg)
www.arl.orgAssociation of Research Libraries
Data Collection
• How/What to Collect – Sources of Data– Systems Based – OPAC, Web, Vendor Data (Canned
Reports, Other Reports)– Electronic/Web Resource (costs & use)– Non-Systems (manual) Library Counts
o Reference Activities o ILL Transactionso Document Delivery o Turnstile Countso Other Manual Counts
– Qualitative Data Sources (Surveys, Focus Groups, Process Analysis)
![Page 12: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/12.jpg)
www.arl.orgAssociation of Research Libraries
Data Examples
Source: TUG (Tri-University Group of Libraries - University of Guelph, Wilfrid Laurier University & University of Waterloo) SilverPlatter ERL (Electronic Reference Library) Usage Statistics
![Page 13: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/13.jpg)
www.arl.orgAssociation of Research Libraries
Data ExamplesExample – Consortia CSA Database Use Data
Network Databases 1999/2000 - TUG CSA Use
Vendor DatabaseSearches -
UG % UGSearches -
UW % UWSearches -
WLU % WLUSearches - Remote
% Remote CSA Tolal
% Total Use
Biological SciencesCSA Biological Sciences 206,318 78.1% 46,398 17.6% 2,691 1.0% 8,701 3.3% 264,108 21.7%CSA Biological Digest 134,607 80.2% 26,175 15.6% 2,019 1.2% 5,021 3.0% 167,822 13.8%CSA MEDLINE 112,601 72.3% 32,489 20.9% 3,812 2.4% 6,814 4.4% 155,716 12.8%CSA Plant Science 46,245 76.8% 11,230 18.6% 907 1.5% 1,871 3.1% 60,253 4.9%
CSA Recent References Related to Your Search (New)213,480 74.7% 58,422 20.4% 5,310 1.9% 8,663 3.0% 285,875 23.4%
CSA Web Resources Related to Your Search (New) 213,578 74.7% 58,533 20.5% 5,317 1.9% 8,667 3.0% 286,095 23.5%Total 926,829 76.0% 233,247 19.1% 20,056 1.6% 39,737 3.3% 1,219,869 64.8%Environmental Sciences & Polution Mgmt. Collection
CSA Conference Papers Index 35,616 71.7% 11,241 22.6% 801 1.6% 2,000 4.0% 49,658 16.8%CSA Environmental Engineering Abstracts 17 60.7% 2 7.1% 0 0.0% 9 32.1% 28 0.0%CSA Environmental Sciences & Polution Mgmt. 123,006 67.4% 49,744 27.3% 2,572 1.4% 7,214 4.0% 182,536 61.7%CSA Risk Abstracts 0 0.0% 1 14.3% 0 0.0% 6 85.7% 7 0.0%CSA TOXLINE 47,809 75.0% 13,240 20.8% 728 1.1% 1,953 3.1% 63,730 21.5%
Total 206,448 69.8% 74,228 25.1% 4,101 1.4% 11,182 3.8% 295,959 15.7%Sociological Abstracts Collection
CSA ERIC 37,085 72.4% 10,361 20.2% 2,128 4.2% 1,678 3.3% 51,252 13.9%CSA Sociological Abstracts 36,169 59.6% 15,095 24.9% 7,408 12.2% 2,030 3.3% 60,702 16.5%CSA Social Services Abstracts 57,075 73.4% 8,954 11.5% 8,479 10.9% 3,285 4.2% 77,793 21.2%
CSARecent References Related to the Social Sciences (New) 57,075 64.1% 20,148 22.6% 8,479 9.5% 3,285 3.7% 88,987 24.2%
CSAWeb Resources Related to the Social Sciences (New) 57,075 64.1% 20,148 22.6% 8,479 9.5% 3,285 3.7% 88,987 24.2%Total 244,479 66.5% 74,706 20.3% 34,973 9.5% 13,563 3.7% 367,721 19.5%
CSA Grand Total 1,377,756 73.1% 382,181 20.3% 59,130 3.1% 64,482 3.4% 1,883,549
Source: TUG (Tri-University Group of Libraries - University of Guelph, Wilfrid Laurier University & University of Waterloo) CSA (Cambridge Scientific Abstracts) Usage Statistics
![Page 14: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/14.jpg)
www.arl.orgAssociation of Research Libraries
Recommended Statistics & Measures
Patron Accessible Electronic Resources (R1-3)
Use of Networked Resources & Services (U1-5)
Expenditures for Networked Resources & Related Infrastructure (C1-3)
Library Digitization Activities (D1-3)
![Page 15: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/15.jpg)
www.arl.orgAssociation of Research Libraries
Patron Accessible Electronic Resources
• R1 – Number of electronic full-text journals
• R2 – Number of electronic reference sources
• R3 – Number of electronic books
![Page 16: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/16.jpg)
www.arl.orgAssociation of Research Libraries
Use of Networked Resources & Related Infrastructure
• U1 – Number of electronic reference transactions
• U2 – Number of logins (sessions) to electronic databases
• U3 – Number of queries (searches) in electronic databases
• U4 – Items requested in electronic databases
• U5 – Virtual visits to library’s website and catalog
![Page 17: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/17.jpg)
www.arl.orgAssociation of Research Libraries
Expenditures for Networked Resources & Related Infrastructure
• C1 − Cost of electronic full-text journals• C2 − Cost of electronic reference sources• C3 − Cost of electronic books• C4 − Library expenditures for bibliographic
utilities, networks & consortia• C5 − External expenditures for
bibliographic utilities, networks & consortia
![Page 18: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/18.jpg)
www.arl.orgAssociation of Research Libraries
Library Digitization Activities
• D1 – Size of library digital collection• D2 – Use of library digital collection• D3 – Cost of digital collection construction &
management
(Collecting these data requires staff familiar with the digital environment.)
![Page 19: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/19.jpg)
The E-Metrics Assessment Implementation Loop
Mission/Purposes
Objectives
Identify Tasks, Data & Needs
Data Collection
Managing Information
Preparation
Reporting, Learning & Implementing
![Page 20: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/20.jpg)
www.arl.orgAssociation of Research Libraries
Preparation for Assessing Network Services is as Important as the Assessment Itself !
![Page 21: E-Metrics: Lessons Learned from the ARL E-Metrics Project ...acscinf.org/docs/meetings/226nm/presentations/226nm41.pdfAvg. Annual % Change Serial Unit Cost 7.7% Serial Expenditures](https://reader033.fdocuments.in/reader033/viewer/2022050223/5f684c7c7a5a8b69f97af790/html5/thumbnails/21.jpg)
www.arl.orgAssociation of Research Libraries
Mind the Gap
Mine the Gap