LIRG SCAN AWARD 2010

A REVIEW OF EXISTING QUANTITATIVE AND QUALITATIVE

EVALUATION METHODOLOGIES FOR DEMONSTRATING THE VALUE OF

PUBLIC LIBRARIES

Christine Rooney-BrownePhD Researcher, University of Strathclyde, Glasgow

Umbrella Conference 2011

To define value To present an overview of current methods

for measuring performance To discuss quantitative and qualitative

evaluations methodologies To identify examples of successful library

valuation projects To introduce potential methods for

measuring value from the non-profit sector

Objectives...

How do we define value?

 © Christine Rooney-Browne (2009).

“...defining value in the context of libraries is complex, individual

stakeholders are unique, performance measurement is

essentially spatial, and operating in an environment that is neither causal nor predictive creates

complications” (Cram, 1999, p. 1).

Value is not fixed...

Personal, educational, professional circumstances

Seven Ages of Library Use (Bohme and Spiller, 1999):

Times of crisis: War, Recession, Natural

Disasters (New Orleans)

“Libraries can be valued in many different ways, from hard dollars to intangibles like

community goodwill and historical significance”

(Elliott, 2005).

Quantitative Methodologies

Audits...

CIPFA Public Library Statistic Actuals Report Library managers complete a

report about expenditure and income, staff levels, service points, stock levels, issues, enquiries, visits, inter-library loans

Library & Information Statistics Unit (LISU) summarises key statistics from this report in their Libraries, Archives, Museums, Publishing, Online Statistics Table (LAMPOST)

(LISU, 2010)

Pros & Cons...

“...quantity of use and quality of performance do not yet prove that users benefitted from their interaction with a

library. Measuring impact or outcome means going a step further and trying to assess the affect of services on users

and on society” (Poll and Boekhorst, 2007, p. 31).

How well the library is performing statistically

Statistics can be invaluable (budgets, staffing, stock)

Enables comparisons (league tables)

Disparity in funding Geographical location Overlook contribution

to lives of individuals, local community and economy

Return on Investment Studies (ROIs)

Cost-Benefit Analysis (CBA) Enables quantifiable values, such as cost or purchase price,

to be applied to variables that are difficult to measure; used to measure direct benefits

Consumer Surplus the value that consumers place on the consumption of a good

or service in excess of what they must pay to get it Cost of Time and Effort

measures time and effort expended by users Contingent Valuation (CV)

measures the value of use and non-use of non-priced goods and services (e.g. public libraries)

Willingness to Pay (WTP) Willingness to Accept (WTA)

Input-Output Models (IOMs)

Evaluating indirect benefits (e.g. impact that library has on the local economy)

Benefits measured using mathematical software models which looks at cause and effect relationships (e.g. Regional Input-Output Modelling System II and the Regional Economic Models, Inc (REMI)).

Have been used in the past to measure the value of beaches and parks.

“...if we can demonstrate our worth, with numbers, our budget numbers will

be all the more justifiable”

(Finch and Warner, 1998, p.158).

Combining CBAs and IOMs...

Suffolk Cooperative Library System (SCLS)

• Benefit-to-cost-ratio methods to measure direct benefits:

Total Value of library services ÷ Tax dollars supporting service:$509,415,038 ÷ $131,647,566 = $3.87 : 1 benefit/cost ratio

• For every $1 invested the library returned $3.87 in direct benefits

• Using input-output model called RIMS II to measure indirect benefits:

• Library generated $26 million in goods and services• Library enabled local earnings to increase by more than $50 million• Created more than 1,200 jobs for the local economy• Total multiplier effect of SCLS spending = $232 million.

(Imholz and Arns, 2007, p. 19).

Library Use Valuation Calculator...

(Chelmsford Public Library, 2010)

But what about social value...?

(Rooney-Browne, 2009b).

Qualitative Methodologies

Social Impact Audit...

Ethnography...

“Ethnography is the art and science of describing a group or culture......the ethnographer is both storyteller and

scientist; the closer the reader of an ethnography comes to understanding the native’s point of view, the better the

story and the better the science” (Fetterman, 1998, pp. 1-2).

Tracking Value...The Engaged Library: Chicago Stories of Community Building

•Prove that public libraries build social capital

•Identify & connect the library’s assets to the community

•Assess & strengthen the library’s connections with and use of community assets

•Produce a toolkit for other libraries to adopt to

•Mapping tools to perform an inventory services, identify areas for improvement and highlight library’s contribution to the community’s wider social, educational, cultural and economic goals.

Tracking Value...

(UKOC, 2008, p.4).

People behind the Numbers: Janet

(UKOK, 2010, p.38).

Profiling...

• Profiling has enabled UKOC to translate faceless data into outcomes that are “tangible, practicable and workable” (UKOC, 2010, p.5).

• Stakeholders and funders might be able to relate easier and have empathy for ‘Janet’ rather than a ‘C2DE female’.

Multiple Method Approaches...

Seattle Central Library Economics Benefits Assessment...

(SCL, 2005, p.10).

Methodology & Results... CBA methods to estimate economic impact Case studies, visitor and user surveys Interviews with local businesses, developers and

representatives from tourism Analysis of visitor and usage statistics.

Generated $16 million in net new economic activity in its first year of operation

Projections next twenty years: $80m (5yrs), $155m (10 years) or $310m (20 yrs)

improves desirability of downtown area & Seattle as tourist destination

An icon Findings used as an advocacy tool

Social Return on Investment...the future?

“...whether the capital provided is generating meaningful, real returns--returns for the manager, the investor and society as a whole” (Emerson, 2004).

Roberts Enterprise Development Fund (REDF) venture philanthropists working with non profits

to create job opportunities for disenfranchised Created SROI Model (2000):

Measure success...achieving goals...inform decisions...convince others of our impact

(Javitis, 2008, p.1).

Measuring SROI...

Compares net benefits of a project to investment (financial & time) Social Return Ratio (SRR): by combining net social benefits with

cash flow of the business then dividing by the total value of the philanthropic investment

SROI Rate: by carrying out an Internal Rate of Return (IRR) derived from total socio-economic value and total costs (Emerson and Cabaj, 2000, p. 11)

The New Economics Foundation, London Business School & Small Business Service produced a SROI Primer: Helps organisations identify and measure social outputs, outcomes

and impacts Provides advice to help organisations use SROI Model to translate

impacts into values (NEF, 2004).

Conclusion...

Perfect methodology does not exist but there are many possibilities

No general consensus as to the ideal model for measuring value

Competition for funding is fierce Cuts are inevitable Find the methodology that best fits your project and your

objectives “In today’s climate of accountability, a better understanding of the value of

public libraries is becoming essential to preserving and encouraging public and

private investment” (Imholz and Arns, 2007,

p.12).

CONTACT ME:

E-MAIL: christine.rooney-browne@cis.strath.ac.uk

BLOG: www.libraryofdigress.co.uk

Twitter: www.twitter.com/librarydigress

Voices for the Library: http://www.voicesforthelibrary.org.uk/wordpress/

THANK YOU