Selena Killick Library Quality Officer Leadership Seminar on Strategy, Assessment and Service Development. University of Lund 19th September 2012.

Strategic Metrics

Eras of Library Metrics

• Holdings

• Physical usage

• Customer satisfaction

• Electronic usage

• Value & Impact

Few libraries exist in a vacuum,

accountable only to themselves. There is

always a larger context for assessing

library quality, that is, what and how well

does the library contribute to achieving the

overall goals of the parent constituencies?

(S. Pritchard 1996)

Value & Impact Metrics

• Based on what matters the most to your key

stakeholders

• Identify where the Library impacts upon the goals of

the parent organisation

• Demonstrate how the library is contributing to the

mission of your institution

Strategic Metrics

• Start by defining your outcomes

• What do you want to demonstrate?

• What does your institution care about?

• SMART outcomes:

• Specific

• Measureable

• Attainable

• Relevant

• Time-bound

• If in doubt, KISS.

Good Strategic Metrics

• Linked to the University/Library mission & goals

• Provide useful information

• Reasonably accurate within the data limitations

• Ethical

• Cost effective

Mixed-Methods Approach

Quantitative

• Library Data

• Institutional Data

• National/International

Data

• Activity analysis

• Return on Investment

• Cost benefit

Qualitative

• Customer Surveys

• Focus groups

• Observational Studies

• Interviews

Quantitative Data

• Be prepared to take a fresh look at your data collection

• Look beyond the Library

• Talk to other data providers in your institution

• Combine data sets

• Local (Library & student data), national, international

• Check to see what is available before you start to count things

Case Study: Cornell University Library

Library Value Calculation

• Annual Cost to the institution $56,678,222

If CUL did not exist:

• Sourcing information and answering enquiries would

cost the institution $90,648,785

Source: Cornell University Library Research & Assessment Unit

Case Study: University of West Florida

UWF University Libraries calculated its Return on Investment (ROI) for eight of its services:

• students studying in the library;

• borrowing books, e-books, DVDs and laptops;

• students or faculty members asking reference questions or meeting with reference librarians for individual research consultations;

• conducting library instruction sessions; and,

• students or faculty members using subscription databases when off-campus.

Calculations based on the number of occurrences (service outputs) multiplied by an informal and conservative market value of the occurrence to calculate a summed value for the services. The summed value was then divided by sum of the University Libraries' personnel and operating expenditures.

• It was calculated that for every dollar expended by the University Libraries, at least $5.89 was returned for the services identified.

Source: University of West Florida

Case Study: University of Huddersfield

• Hypothesis: do the students who use the library the

most get the highest grades?

• Library Usage Data

• Book loans

• eResources usage

• Visits to the Library

• Student Attainment Data

• Final grade received when graduating

13

Non/Low Use Project Source: Stone, G. (2011)

Measuring Library Impact Source: Stone, G. (2011)

Analysis of the results consistently revealed a

correlation between e-resource use, book

borrowing and student attainment

This appears to be the case across all

disciplines

Thoughts

• Does correlation equal causation?

• Do our stakeholders like this impact data?

Qualitative Assessment

• Numbers are only ever half of the story

• Customer feedback can provide more insight that

data alone

• Customer quotes can have great impact on

stakeholders

When I started my distance learning

MSc it was the first time I had accessed

a digital library with such resources; you

have played a major positive role in my

effective learning.

The Library now has access

to 15,586 full text electronic

journals and 9,354 eBooks

Data Management

• Data Management is an important part of the

planning cycle

• Assessment management systems

• Qualitative coding software

• MS Excel / MS Access

Considerations

• Bring both the Qualitative and Quantitative together

• Collaborate with local experts

• Not all news will be good news

• Be realistic on what is achievable

How many staff-hours were spent

counting statistics this year?

Common Challenges

• Where do I start?!

• Finding the time

• Getting support from all Library staff

• Requesting data from other units

• Finding other data sets

• Difficulty in analysing and interpreting data

• Fear of change

• What if I fail?

Source: Megan Oakleaf, 2011,

Source: Megan Oakleaf, 2011,

Source: Megan Oakleaf, 2011,

Reporting Back

• Know your audience

• What do they care about?

• What is their level of understanding?

• Will they feel threatened by the data?

• Tell a story with the data

• Begin with the outcomes in mind

• Be mindful of the data limitations

Case Study: McMaster University

http://library.mcmaster.ca/library-scorecard

References

• Institutional Return on Investment. University of West Florida.

http://libguides.uwf.edu/content.php?pid=188487&sid=2184200

• Library Scorecard: Strategic Objectives, Measures & Initiatives 2011/2012. McMaster

University Library, revised April 23, 2012, http://library.mcmaster.ca/library-scorecard

• Library Value Calculations. Cornell University Library Research & Assessment Unit.

http://research.library.cornell.edu/value

• Oakleaf, M. and Matthews, J. (2011). Assessing the Impact of the Academic Library.

Presented at the 9th Northumbria International Conference on Performance

Measurement in Libraries and Information Services. York, England. 22nd – 26th August.

• Stone, G. (2011). The Library Impact Data Project: hit, miss or maybe. Presented at the

9th Northumbria International Conference on Performance Measurement in Libraries

and Information Services. York, England. 22nd – 26th August.

Thank you!

Selena Killick s.a.killick@cranfield.ac.uk Twitter: @SelenaKillick Tel: +44(0)1793 785561