Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study

8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study

1/29


2/29

University of Colorado

Boulder | Denver | Anschutz Medical Campus

Presentation Outline

Pilot Goals

Definitions

Methodology The Colorado Model

Data Management & Analysis

Recommendations

2

UCB Libraries
http://ucblibraries.colorado.edu/http://ucblibraries.colorado.edu/https://www.cu.edu/content/explore-cu-system


3/29



Pilot Goals

Measure the impact of librarycollections on faculty researchproductivity

Find a metric to express libraryvalue to our administrators

Develop and test a costbenefit model for librarycollections and facultyresearch Office.com
http://office.microsoft.com/en-us/images/results.aspx?ex=2&qu=goals&AxInstalled=copy&Download=MP900387781&ext=JPG&c=0http://office.microsoft.com/en-us/images/results.aspx?ex=2&qu=goals&AxInstalled=copy&Download=MP900387781&ext=JPG&c=0


4/29



ROI Studies

2009 Outsell and Springer study measured: Time saved by library users Money saved by using the library Revenue generated with the assistance of the library

IMLS Lib-Value Project Phase I: University of Illinois Urbana-Champaign

measured the value of the library in the grant process Phase II: International study measured value of e-

journals in academic libraries focusing on grantincome

Phase III: Lib Value: Values, Outcomes, and Returnon Investment of Academic Libraries


5/29



Defining Value

Internal Focus

Use or Utility

Financial Value;

Return on Investment Commodity Production

External Focus

Impact Value

Alternative Comparison

Oakleaf, Value of Academic LibrariesOffice.com
http://office.microsoft.com/en-us/images/results.aspx?qu=fill&ctt=1http://office.microsoft.com/en-us/images/results.aspx?qu=fill&ctt=1


6/29



IRB Approval for Human Research

Librarians at eachcampus applied to theirInstitutional Review Board

(IRB) Requested and granted

Exempt Review

Office.com
http://office.microsoft.com/en-us/images/results.aspx?qu=approval&ctt=1http://office.microsoft.com/en-us/images/results.aspx?qu=approval&ctt=1


7/29



Recruiting Participants

Auraria - focused on a single departmentrecommendation by Dean and support fromDepartment Chair

Health Sciences -focused on a singledepartment

Boulder - recruited

faculty from severaldepartments; usedfaculty library committee

Office.com
http://office.microsoft.com/en-us/images/results.aspx?qu=stairway&ctt=1http://office.microsoft.com/en-us/images/results.aspx?qu=stairway&ctt=1


8/29



Methodology

Interviews

Curriculum vitaanalysis

Publication citationanalysis

Contingent Valuation

Office.com
http://office.microsoft.com/en-us/images/results.aspx?qu=meeting&ctt=1http://office.microsoft.com/en-us/images/results.aspx?qu=meeting&ctt=1


9/29



Faculty Interviews

Conducted in-person,phone, or emails

Described pilot project

Requested copy ofcurriculum vita

Asked 4 questions regarding

library resource usageOffice.com
http://office.microsoft.com/en-us/images/results.aspx?qu=interview&ctt=1http://office.microsoft.com/en-us/images/results.aspx?qu=interview&ctt=1


10/29



Interview Questions

1. In your most recently published article, forevery article you cited how many additionalarticles did you read but did not cite?

2. Approximately what percent of the articles citedor read were obtained from the following:

electronically from the library?

in print format from the library?

via Interlibrary loan from the library?

from non-library sources?


11/29



Curriculum Vita Analysis andPublication Citation Analysis

Gathered publications (chapters & articles)published in 2009-2010

Identified cited references in faculty publications

Determined full-text sources for cited references

Calculated:

% cited references from journals

% of cited references from online libraryresources


12/29




Determine market valuefor articles

Random sample of journals

Cost to purchase individual

article from publisher Calculate average cost per

article

Calculate Standard Deviation

to determine low and highestimates

Office.com
http://office.microsoft.com/en-us/images/results.aspx?qu=stackhttp://office.microsoft.com/en-us/images/results.aspx?qu=stack


13/29



Methodology

Interviews

Curriculum vitaanalysis

Publication citationanalysis


Office.com
http://office.microsoft.com/en-us/images/results.aspx?qu=meeting&ctt=1http://office.microsoft.com/en-us/images/results.aspx?qu=meeting&ctt=1


14/29



The Colorado Model

Based on Cost Benefit Analysis (CBA) andReturn on Investment (ROI)

CBAis the ratio showing dollar value of benefits

gained for dollar value of costs.Benefits Costs = CBA

ROI is the percentage showing the return orincrease in value on dollars spent to achieve a

benefit. ((BENEFITS COSTS) COSTS) x 100

14


15/29



University of Colorado Results

CU Boulder and CU Denver Auraria Library results

published in the proceedings for the 2011 Associationof College and Research Libraries Conference

available online at http://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdf

CU Denver Health Sciences Library results

forthcoming May 22 at the 2012 Medical LibrariesAssociation Annual Meeting

15
http://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.mlanet.org/am/am2012/http://www.mlanet.org/am/am2012/http://www.mlanet.org/am/am2012/http://www.mlanet.org/am/am2012/http://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdf


16/29



Redefining Library Resources

Benefits = estimated cost for faculty to buy articlesdirectly from publisher with pay-per-view

Costs = Library costs for online journals/databases cited

CBA = Benefits Costs

ROI = ((BENEFITS

COSTS)COSTS) x 100

16


17/29



Colorado Model with Hypothetical Data

Benefits = estimated cost for faculty to buy articlesdirectly from publisher with pay-per-view

Estimated value of articles: $15,441

Costs = Library costs for online journals/databases cited

Journal subscriptions: $15,164

CBA = Benefits Costs

$15,441 $15,164 = $1.02

ROI = ((BENEFITS

COSTS)COSTS) x 100

(($15,441-$15,164) $15,164) x 100 = 2%

17


18/29



Data Management & Analysis

University of Colorado Model

Sample implementation with hypothetical data

Handout summarizes CBA and ROI formulas, variables and values

Excel workbook demonstrates

citation analysis and implementing model

18


19/29



Handout

Defines formulas

Presents variables

Identifies data sources

Faculty CVs

Citation Analysis

Interview Answers

Cost Estimates

#buildROI 19


20/29



Excel worksheet: Colorado Model

Highlighted totalvalues for variableslink to corresponding

worksheet E.g. Variable A

(yellow) = 15 articlespublished in 1 year


21/29


22/29



Excel worksheet: Citations

22

Data elements for citation analysis Participants cited references perpublished articles

Cited references sources (Journal X)

Full-text source (Database Y)

Acquisition type (Online Subscriptionv. Open Access or No Access)


23/29



Excel worksheet: I

23

Estimated Cost per ArticleBased on sampling of randomlyselected titles Utilizes pivot table of citation

analysis data Includes formulas forrandomly selecting titles

Look up pay-per-view priceson publishers website

Calculates costs for average

and low/high standarddeviation


24/29



Excel worksheet: K

24

Estimated Library Costs forSubscriptions based on cost tosubscribe to referenced titles

Utilizes pivot table to identifyfull-text source

Look up subscription costs andtotal # of titles in full-text sourceto derive avg. price per title

Calculates library costs bymultiplying avg. price per titlewith # of journals referenced infull-text source.


25/29



Analysis of Article Price Estimates

25

Contingent valuationis market value ofarticle costs

More statistically

significant by Random sample

of titles

Standard deviation ofaverage article costs


26/29


27/29



Implementation Recommendations

Align project to mission and objectives ofthe library and institution

Demonstrate support of faculty research

Consider readily available data sources

Access to CVs, course syllabi, faculty report ofprofessional activities, grant funding

Define scope and focus of project

Specific department and number of participants

Identify manageable timeframe Multiple steps (IRB approval, interviews, citation analysis)

27


28/29



Resources

Association of College and Research Libraries. (2010). Value of Academic Libraries: AComprehensive Research Review and Report. Researched by Megan Oakleaf.Chicago: Association of College and Research Libraries. Available online at:www.acrl.ala.org/value

Barnes, S. & Blake, M. (2010). Measuring Your Impact: Using Evaluation for LibraryAdvocacy, A library evaluation workshop. National Network of Libraries of Medicine,http://nnlm.gov/evaluation/workshops/measuring_your_impact/.

Kaufman, P. (2008). The library as strategic investment: Results of the Illinois return on

investment study.Liber Quarterly, 18(3/4), 425. Luther, J. (2008). University investment in the library: Whats the return?: A case study at

the university of Illinois at UrbanaChampaign. Available online at:http://libraryconnect.elsevier.com/whitepapers/0108/lcwp0101.pdf.

Pan, D., Wiersma, G., & Fong, Y. (2011). Towards demonstrating value: Measuring thecontributions of library collections to university research and teaching goals.Proceedings of the 2011 ACRL Conference Proceedings. Available online at:http://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/

papers/towards_demonstratin.pdf Tenopir, C. (2012). Beyond usage: measuring library outcomes and value. Library

Management, 33(1/2), 5-13.

28
http://www.acrl.ala.org/valuehttp://nnlm.gov/evaluation/workshops/measuring_your_impact/http://libraryconnect.elsevier.com/whitepapers/0108/lcwp0101.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://libraryconnect.elsevier.com/whitepapers/0108/lcwp0101.pdfhttp://libraryconnect.elsevier.com/whitepapers/0108/lcwp0101.pdfhttp://nnlm.gov/evaluation/workshops/measuring_your_impact/http://nnlm.gov/evaluation/workshops/measuring_your_impact/http://www.acrl.ala.org/value


29/29



Questions? Thank You!

Gabrielle [email protected]

Leslie [email protected]

Denise [email protected]

Yem [email protected]
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]

Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study

Documents

Transcript of Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study