What Do Alternative Metrics and the 'Flight of the Bumblebee'

Have in Common?

Martha KyrillidouARL Statistics and Service Quality Programs

Association of Research Libraries (ARL)NISO June Virtual Conference:

Transforming Assessment: Alternative Metrics and Other TrendsJune 18, 2014

e-Pubs downloads (May 16, 2014)

“I made the Libraries presentation to the Purdue Board of Trustees this morning, including a video of the downloads from Purdue e-Pubs, speeded up to show downloads in 2 hours in 30 seconds (nearly 3,000). The sound track was “Flight of the Bumblebee.” The greatest compliment I had from a Trustee after the presentation was when he said, his only complaint was “that the presentation didn’t go longer, he found it very interesting, totally unexpected!”

James L. Mullins, PhDDean of Libraries and

Esther Ellis Norton ProfessorPurdue University

ENABLING WORLD-CHANGING RESEARCH Libraries’ Purdue e-Pubs

As of May 8, 2014

37,580 documents

6,908,130 total downloads

2,596,249 past year

Transformation I• "Flight of the Bumblebee" is an orchestral

interlude written by Nikolai Rimsky-Korsakov for his opera The Tale of Tsar Saltan, composed in 1899–1900. The piece closes Act III, Tableau 1, during which the magic Swan-Bird changes Prince Gvidon Saltanovich (the Tsar's son) into an insect so that he can fly away to visit his father (who does not know that he is alive).

Association of Research Libraries

ARL Statistics and Assessment

…To describe and measure the

performance of research libraries and their contribution to teaching, research, scholarship and community service …

Association of Research Libraries

ARL Statistics and Assessment

…To describe and measure the performance of research libraries and their

contribution to

teaching, research, scholarship and community service …

The Printed Library Materials I Need for My Work

Print and/or Electronic Journal Collections I Require for My Work

www.libqual.org

A Library Website Enabling Me to Locate Information on My Own

StatsQUAL®A gateway to library assessment tools that describe the role, character, and impact of physical and digital libraries.

LibQUAL+® is a rigorously tested Web-

based survey that libraries use to solicit, track, understand, and

act upon users‘ opinions of service quality.

LibQUAL+®

MINES for Libraries®

Measuring the Impact of Networked Electronic

Resources (MINES) is an online transaction- based

survey that collects data on the purpose of use of

electronic resources and the demographics of users.

ClimateQUAL®

ClimateQUAL® :

Organizational Climate and Diversity Assessment is

an online survey that measures staff perceptions

about: (a) the library's commitment to the

principles of diversity, (b) organizational policies and

procedures, and (c) staff attitudes.

ARL Statistics®

ARL Statistics™ is a series of annual publications that

describe the collections, expenditures, staffing, and

service activities for Association of Research

Libraries (ARL) member libraries.

LibValue

LibValue is a series of tested methods that capture Return on

Investment and Value of library services.

The data and the models

• I hesitate to hang the future of libraries on a demonstrable effect size for the equation,

L = $$$

– Rachel Applegate , Indiana University SLIS

ARL Statistics®

ARL Statistics™ is a series of annual publications that

describe the collections, expenditures, staffing, and

service activities for Association of Research

Libraries (ARL) member libraries.

Why study value

• Faced with difficult economic times and university budget cuts, the value of the library to the wider goals of the university is increasingly questioned.

– Introduction to the Lib-Value Project

» www.libvalue.org

Association of Research Libraries

Association of Research Libraries

Use ……Value

MINES for Libraries – Scholars Portal @ OCUL

9th Northumbria Conference, 23rd August 201114

In the Library

Off-campus

On-campus but not in the library

In the Library Off-campus On-campus but not in the library

LibValue

Special Collections – Google Analytics application

• LibValue: Digitized Special Collections (video on YouTube)Thursday, August 15, 2013, 1:00–2:00 p.m. eastern

Presenters: Gayle Baker, Professor and Electronic Resources Coordinator, University of Tennessee Libraries Ken Wise, Associate Professor, University of Tennessee Libraries

www.arl.org

A Values scorecard

• The value which stakeholders seek from libraries is transcendent, in the sense that the impact demonstrated must be beyond the library and immediate satisfaction or fulfillment. Library assessment effort to date has been (mainly) about quality rather than value.

• J. Stephen Town, Martha Kyrillidou, (2013) "Developing a values scorecard", Performance Measurement and Metrics, Vol. 14 Iss: 1, pp.7 - 16http://www.emeraldinsight.com/journals.htm?articleid=17086961

Association of Research Libraries

A Value Scorecard

• Relationship & reputation capital• Organisational capital

– Tangible assets and resources– Intangible and meta-assets

• Library virtue– Impact and social capital

• Library momentum …. Leadership … Culture

Article-level metrics – Relation/Reputation

• Usage• Captures• Mentions• Social Media• Citations

– Tananbaum, Greg. (2013, April 16). Article-level metrics: a SPARC primer. Scholarly Publishing and Academic Research Coalition. http://www.sparc.arl.org/sites/default/files/sparc-alm-primer.pdf

– Bollen, J. et al. (2008). Towards alternative metrics of journal impact: A comparison of download and citation data. http://arxiv.org/pdf/cs.DL/0503007.pdf

ARL Strategic Thinking and Design

• By 2033 the research library will have shifted from its role as a knowledge service provider within the university to become a collaborative partner within a rich and diverse learning ecosystem

System of action components

– Coordinated management of collective collections– Scholarly publishing at scale– ARL Academy– Building a boundless symposium– A first suite of smart libraries– Innovation lab

Transformation II

Questioning assumptions

Shaping the experience

La danse (I) by Matisse, 1909, The Museum of Modern Art, New York City

Henri Matisse The Dance, 1910, Hermitage Museum, St. Petersburg, Russia

Dancing to the end of Love

Henri Matisse Music, 1910, Hermitage Museum, St. Petersburg, Russia

Why I Love ….

• Stay connected• Get coffee• Take a break• Check out my class book• Reserve a conference room for a meeting• Use my own computer and plug in• Use the library’s computers• Study with a friend or on my own• Share laughs and hang with friends• Studying

Final Blessings

• Indulge your curiosity• Influence your friends• Enable your organization.

– May your analytics always be good – and your judgment even better.

» Robert Morison, Analytics at Work