Telling the impact story defining new metrics for library success - 2014
-
Upload
stephen-abram -
Category
Career
-
view
727 -
download
0
description
Transcript of Telling the impact story defining new metrics for library success - 2014
witnessed @ Defining New Metrics for Library Success04.22.2014
presented by Moe Hosseini-Ara
Underlying assumptionsUnderlying assumptionsUnderlying assumptionsUnderlying assumptions
Few libraries have a “culture of assessment”…
• Tend to focus on quantitative “performance”
• Most measures indicate past performance
• Need to do more qualitative performance measures
• Difficult and complex (is the excuse given)
• Have we stopped and asked why we do what we Have we stopped and asked why we do what we Have we stopped and asked why we do what we Have we stopped and asked why we do what we do?do?do?do?
Underlying assumptionsUnderlying assumptionsUnderlying assumptionsUnderlying assumptions
Identifying & illustrating value depends on conversations with your stakeholders
…the first conversation shouldn’t be when measures are presented – if they don’t know you exist…
“More” isn’t necessarily better
…in fact, “more” clouds the issue and the message
Underlying assumptionsUnderlying assumptionsUnderlying assumptionsUnderlying assumptions
Not everything that counts can be counted AND not everything that can be counted counts…
…Wait! What??
- Einstein
Questions driving measuresQuestions driving measuresQuestions driving measuresQuestions driving measures
• What is your library or information centre doing?
• How much is it doing?
• How well is it doing it?
• Who is it doing it for?
• What positive impact is the What positive impact is the What positive impact is the What positive impact is the
service having for them?service having for them?service having for them?service having for them?
• Most important question Most important question Most important question Most important question ----
Are you adding VALUE?Are you adding VALUE?Are you adding VALUE?Are you adding VALUE?
Value
SatisfactionOperational
Three types of measuresThree types of measuresThree types of measuresThree types of measures
Operational/Usage MeasuresOperational/Usage MeasuresOperational/Usage MeasuresOperational/Usage Measures
• We’ve long tracked “how much”• We also need to look at differences:
• Peaks? Dips? Switches?• What are these statistics really telling us? Or, not?
• Who needs to see thesestatistics?
• Are they telling a story that you want to tell?
Customer Satisfaction MeasuresCustomer Satisfaction MeasuresCustomer Satisfaction MeasuresCustomer Satisfaction Measures
• Tends to focus on existing products and/or services
• Usually measured through surveys
• Are we also measuring
importance?
• Survey burnout, plus the
expectation is to score all
“excellent” or “10 out of 10”
The VALUE proposition…Measures?The VALUE proposition…Measures?The VALUE proposition…Measures?The VALUE proposition…Measures?
• Are we adding value?
• What difference are we making?
• What is the impact of our services and/or programs?
• How are we contributing
to the success of….?
Meaningful measuresMeaningful measuresMeaningful measuresMeaningful measures• Matter to you AND your stakeholders• Are conveyed in the stakeholders’ language• More effective when defined using all three types of measures• Demonstrate that your programs and services add value
• Focus attention on what is mostimportant for the organization
• Are critical for managing, planning & decision-making
• Are organization-dependent
The Logic ModelThe Logic ModelThe Logic ModelThe Logic Model
…goes beyond documenting what you did and measures what difference you made in the life of your target audience…how has your audience changed…specifically change in the human condition…
Measuring Impact/ValueMeasuring Impact/ValueMeasuring Impact/ValueMeasuring Impact/ValueChange in the human condition?
• Behavior• Skill• Knowledge• Attitude• Circumstances
• Awareness• Motivation• Condition • Status
IMPACT
Equipment
INPUTINPUTINPUTINPUT
resource resource resource resource perspectiveperspectiveperspectiveperspective
OUTPUTOUTPUTOUTPUTOUTPUT
operational operational operational operational perspectiveperspectiveperspectiveperspective
OUTCOMEOUTCOMEOUTCOMEOUTCOME
user user user user perspectiveperspectiveperspectiveperspective
IMPACTIMPACTIMPACTIMPACT
stakeholder stakeholder stakeholder stakeholder perspectiveperspectiveperspectiveperspective
It’s logical that:It’s logical that:It’s logical that:It’s logical that:
IMPACT
Equipment
INPUTINPUTINPUTINPUTResource Resource Resource Resource PerspectivePerspectivePerspectivePerspective
• Equipment
• Space
• Supplies
• Funding
• Staff
IMPACT
Equipment
OUTPUTOUTPUTOUTPUTOUTPUTOOOOperational perational perational perational PPPPerspectiveerspectiveerspectiveerspective
• Training module
• Program
• Report
• # of Program Attendees
• # of Programs Held
IMPACT
OUTCOMEOUTCOMEOUTCOMEOUTCOMEUser User User User
PerspectivePerspectivePerspectivePerspective
• Increased skill
• Know-how or know-that
• Change in status
• Change in behavior or
attitude
IMPACT
IMPACTIMPACTIMPACTIMPACTStakeholder Stakeholder Stakeholder Stakeholder PerspectivePerspectivePerspectivePerspective
• Lower dropout rates
• Increased employment
levels
• Increased recovery rates
Job Skills Job Skills Job Skills Job Skills TrainingTrainingTrainingTrainingInput(resources)
• 10 hours of staff time• $100 budget required to create the program (supplies and
handouts)
Output(operations)
• 2 hour job skills program created for student job seekers• 4 programs offered over a period of a month • total of 107 participants in attendance
Outcome(user)
• Students gained resume and cover letter writing skills• Knowledge of where to look for jobs and submit applications• Ability to give interviews, proficiency in answering interview
questions
Impact(stakeholder)
• 80% of the participants who attended the program reported that they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following graduation
Job Skills Job Skills Job Skills Job Skills TrainingTrainingTrainingTrainingInput(resources)
• 10 hours of staff time• $100 budget required to create the program (supplies and
handouts)
Output(operations)
• 2 hour job skills program created for student job seekers• 4 programs offered over a period of a month • total of 107 participants in attendance
Outcome(user)
• Students gained resume and cover letter writing skills• Knowledge of where to look for jobs and submit applications• Ability to give interviews, proficiency in answering interview
questions
Impact(stakeholder)
• 80% of the participants who attended the program reported that they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following graduation
Job Skills Job Skills Job Skills Job Skills TrainingTrainingTrainingTrainingInput(resources)
• 10 hours of staff time• $100 budget required to create the program (supplies and
handouts)
Output(operations)
• 2 hour job skills program created for job seekers• 4 programs offered over a period of a month • total of 107 participants in attendance
Outcome(user)
• Students gained resume and cover letter writing skills• Knowledge of where to look for jobs and submit applications• Ability to give interviews, proficiency in answering interview
questions
Impact(stakeholder)
• 80% of the participants who attended the program reported that they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following graduation
Job Skills Job Skills Job Skills Job Skills TrainingTrainingTrainingTrainingInput(resources)
• 10 hours of staff time• $100 budget required to create the program (supplies and
handouts)
Output(operations)
• 2 hour job skills program created for job seekers• 4 programs offered over a period of a month • total of 107 participants in attendance
Outcome(user)
• Participants gained resume and cover letter writing skills• Knowledge of where to look for jobs and submit applications• Ability to give interviews, proficiency in answering interview
questions
Impact(stakeholder)
• 80% of the participants who attended the program reported that they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following graduation
Job Skills Job Skills Job Skills Job Skills TrainingTrainingTrainingTrainingInput(resources)
• 10 hours of staff time• $100 budget required to create the program (supplies and
handouts)
Output(operations)
• 2 hour job skills program created for job seekers• 4 programs offered over a period of a month • total of 107 participants in attendance
Outcome(user)
• Participants gained resume and cover letter writing skills• Knowledge of where to look for jobs and submit applications• Ability to give interviews, proficiency in answering interview
questions
Impact(stakeholder)
• 80% of the participants who attended the program reported that they were either able to get interviews or actual job placements
• Library can claim that as a result of their program residents are able to get jobs, contribute to economy and lower unemployment
BEGINBEGINBEGINBEGINWITHWITHWITHWITH
THETHETHETHEIN MINDIN MINDIN MINDIN MINDENDENDENDEND
Covey 1989Covey 1989Covey 1989Covey 1989
80% of the participants who attended the program reported that they 80% of the participants who attended the program reported that they 80% of the participants who attended the program reported that they 80% of the participants who attended the program reported that they were either able to get interviews or actual job were either able to get interviews or actual job were either able to get interviews or actual job were either able to get interviews or actual job placementsplacementsplacementsplacements
Library can claim that as a result of their program residents are able to get Library can claim that as a result of their program residents are able to get Library can claim that as a result of their program residents are able to get Library can claim that as a result of their program residents are able to get jobs, contribute to economy and lower unemployment jobs, contribute to economy and lower unemployment jobs, contribute to economy and lower unemployment jobs, contribute to economy and lower unemployment
Measurement Measurement Measurement Measurement FrameworkFrameworkFrameworkFramework
Understand Understand Understand Understand the Contextthe Contextthe Contextthe Context
Align Align Align Align Strategies & Strategies & Strategies & Strategies & ObjectivesObjectivesObjectivesObjectives
Identify Identify Identify Identify Services & Services & Services & Services & ProgramsProgramsProgramsPrograms
Define Define Define Define MeasuresMeasuresMeasuresMeasures
Manage Manage Manage Manage Measurement Measurement Measurement Measurement
DataDataDataData
Translate Translate Translate Translate Data into Data into Data into Data into
Outcomes & Outcomes & Outcomes & Outcomes & ImpactsImpactsImpactsImpacts
Communicate Communicate Communicate Communicate ResultsResultsResultsResults
Understand the Context
Know the Context & the Know the Context & the Know the Context & the Know the Context & the CCCContextersontextersontextersontexters
How do theyHow do theyHow do theyHow do they
measure theirmeasure theirmeasure theirmeasure their
Progress/success?Progress/success?Progress/success?Progress/success?
Know your Know your Know your Know your Stakeholders/InflueStakeholders/InflueStakeholders/InflueStakeholders/Influencers ncers ncers ncers –––– who are who are who are who are they?they?they?they?
How is valueHow is valueHow is valueHow is valuedefined,defined,defined,defined,measured &measured &measured &measured &communicated?communicated?communicated?communicated?
• Libraries days are numbered
• Libraries aren’t relevant
• Everyone has computers and internet
• Need information? It’s all online
• Research happens online not in libraries
• With eBooks and research online believes facilities can be made smaller – no need for print material
Councillor Ural Doomed
Dean of HumanitiesDr. Kee Paper
• Influential in university & broader academic circles
• Focus on research
• Has deep seated beliefs in traditional library services
• Quiet
• No technology
• Libraries are institutions for reading & research
Align Strategies
& Objectives
Align Strategies & ObjectivesAlign Strategies & ObjectivesAlign Strategies & ObjectivesAlign Strategies & Objectives
Stakeholder’s Goals & Objectives Your Goals & Objectives: How they Contribute
Accelerate newcomers contribution to the community & economy
Offer programs and services that connect newcomers with relevant agencies and services
Increased research grants Support the research process and assist with submission and completion of grants
Align objectivesAlign objectivesAlign objectivesAlign objectives
• Articulate & document your goals & objectives
What are you trying to accomplish?
• Clarify how these contribute towards your stakeholders’ objectives & desired outcomes
Identify Services & Programs
Identify ServicesIdentify ServicesIdentify ServicesIdentify Services
What services, programs or products contribute to your realizing an objective?
Stakeholder Goals & Objectives
Your Goals & Objectives
Your Services
Identify ServicesIdentify ServicesIdentify ServicesIdentify Services
Library and Library and Library and Library and University University University University PrioritiesPrioritiesPrioritiesPriorities
What we do…What we do…What we do…What we do…
Define Measures
Define successDefine successDefine successDefine success• What will success look like?
• How will you track progress towards meeting goals?
• What indicators or measures will you use?
• Qualitative? Quantitative?
• What data needs to be collected?
• Keep it key
• Keep it simple
• Be practical
• Don’t get mired – or
overwork frontline staff
Manage Measurement
Data
Manage collectionManage collectionManage collectionManage collection• Determine a collection schedule• Work out agreements with other departments that have needed data
• Are you currently collecting data or indicators that are no longer relevant?
• How long do you need to keep data?• Who is responsible?
Translate Data into
Outcomes & Impacts
Define outcomes & impactDefine outcomes & impactDefine outcomes & impactDefine outcomes & impact
• What will success look like?
• And how will you “get” there?
OUTCOMEOUTCOMEOUTCOMEOUTCOME IMPACTIMPACTIMPACTIMPACT
INPUTINPUTINPUTINPUT OUTPUTOUTPUTOUTPUTOUTPUT
Define outcomes & impactDefine outcomes & impactDefine outcomes & impactDefine outcomes & impact
OUTCOMEOUTCOMEOUTCOMEOUTCOME
User User User User PerspectivePerspectivePerspectivePerspective
• Increased skill
• Know-how or know-that
• Change in status
• Change in behaviour or
attitude
IMPACTIMPACTIMPACTIMPACT
Stakeholder Stakeholder Stakeholder Stakeholder PerspectivePerspectivePerspectivePerspective
• Lower dropout rates
• Increased employment
levels
• Increased recovery rates
Communicate Results
Communicate resultsCommunicate resultsCommunicate resultsCommunicate results
• Actually begins when you are understanding the context
• Your message: “we have contributed towards your goals by……”
• If your goals & objectives are meaningful for your planning & decision-making, your measures will be indispensable
What do you need to stop doing?What do you need to stop doing?What do you need to stop doing?What do you need to stop doing?
Value
SatisfactionOperational
The Sweet SpotThe Sweet SpotThe Sweet SpotThe Sweet Spot
ResourcesResourcesResourcesResources
IMLS IMLS IMLS IMLS WebographyWebographyWebographyWebography
http://www.imls.gov/applicants/webography.aspx
Shaping Shaping Shaping Shaping Outcomes Outcomes Outcomes Outcomes
http://www.shapingoutcomes.org/index.htm
Overcoming Overcoming Overcoming Overcoming Our Habits and Learning to Measure Our Habits and Learning to Measure Our Habits and Learning to Measure Our Habits and Learning to Measure Impact Impact Impact Impact by Moe Hosseini-Ara and Rebecca Jones. Information Today, Inc. June 2013.
http://www.infotoday.com/cilmag/jun13/Hosseini-Ara_Jones--Overcoming-Our-Habits-and-Learning-to-Measure-Impact.shtml
Seven Seven Seven Seven Easy Steps to Help You Measure Easy Steps to Help You Measure Easy Steps to Help You Measure Easy Steps to Help You Measure Up Up Up Up by Moe Hosseini-Ara and Rebecca Jones. Information Today, Inc. July/August 2013.
http://www.infotoday.com/cilmag/jul13/Hosseini-Ara_Jones--Seven-Easy-Steps-to-Help-You-Measure-Up.shtml
Thank you!
Special thanks to Rebecca Jones (Dysart & Jones) for introducing me to the logic model and allowing
me the honor to teach this material with her at numerous conferences…