Integrating social media into the marketing mix to achieve tangible results
Assessment for Impact: Turning Data into Tangible Results
description
Transcript of Assessment for Impact: Turning Data into Tangible Results
Assessment for Impact:
Turning Data into Tangible Results
Laura MillerTim MortonPaul Rittelmeyer
Library Assessment ConferenceSeattle, WA
August 5, 2008
• Balanced Scorecard
• LibQUAL+
• Surveys• Users • Staff
• Focus Groups
• Follow-up Interviews
• Statistics (circ, ILL, gate counts, purchase requests,
etc.)
Where Does Data Come From?
• Improve Library Services
• Inform Collections Decisions
• Influence Satisfaction in the
Workplace
• Support Budget Requests
• Contribute to Staff Development
What Does Data Do?
Metric U.4.b. Turnaround time for new book and ILL requests (external resources).
Target1: Satisfy the turnaround targets 75% of the time.Target2: Satisfy the turnaround targets 50% of the time.
Method: New Books: Fill requests in 7 days. A sample of in-print US titles requested by University-affiliated patrons will be drawn from the request database by Management Information Services. Turnaround time from patron request to availability will be measured.ILL: Fill requests in 7 days. Turnaround time will be measured by ILS from patron request to notification that the book or article is available.
Improve Library ServiceRush Order (aka Purchase Express)
Improve Library Service
Acquisitions(order to receipt)
2-4 days
AUL
OK
Improve Library Service
Acquisitions(order to receipt)
2-4 days
Cataloging(receipt to shelf prep to
notifying patron)1 day
AUL
OK OK
Improve Library Service
Acquisitions(order to receipt)
2-4 days
Cataloging(receipt to shelf prep to
notifying patron)1 day
AUL
OK OK
Balanced Scorecard
Improve Library Service
Acquisitions(order to receipt)
2-4 days
Cataloging(receipt to shelf prep to
notifying patron)1 day
Patron request/selector review/placing order
?
AUL
OK OK
Balanced Scorecard
Improve Library Service
Acquisitions(order to receipt)
2-4 days
Cataloging(receipt to shelf prep to
notifying patron)1 day
Patron request/selector review/placing order
?
AUL
OK OK
Balanced Scorecard• Selector training• Back ups for email/order placement
• Funding for expedited shipping
Inform Collections DecisionsLibQUAL+ and satisfaction with journals
Inform Collections DecisionsLibQUAL+ and satisfaction with journals
LibQUAL+ 2006UVa Faculty and Graduate Student Ratings of Journal Collections
5.00
6.00
7.00
8.00
9.00
ArchitectureFaculty
EducationFaculty
EngineeringFaculty
HumanitiesFaculty
Science/MathFaculty
SocialScienceFaculty
ArchitectureGrads
EducationGrads
EngineeringGrads
HumanitiesGrads
Science/MathGrads
SocialScienceGrads
• Focused on low-scoring areas
• Approached a diverse group of faculty
• Asked for specific needs, wants and
suggestions
• Kept the interviews brief
Targeted InterviewsFollowing up with faculty
Inform Collections Decisions
• Dissatisfaction was not caused by single factor – no ‘smoking
gun’
• Perceived access was a problem – in one area we already
subscribed
to all but one of the requested journals
• Lack of foreign titles
• Missing or incomplete backfiles
What We Learned?
Inform Collections Decisions
• To address access, we instituted a major, ongoing re-design of our
website
• We invested in a electronic resource management system (Serials
Solutions 360)
• We prioritized the purchase of science backfiles (Wiley, Elsevier) and
new publisher packages
• In the arts, we put significant funds into the purchase of quality
images, or, if they could not be bought, we digitized them ourselves
The Results
Inform Collections Decisions
Influence Staff Satisfaction
Library Worklife Survey• Conducted in even-numbered years
• Available to all staff via online form with email reminders
• Response Rates: 2004 – 60%, 2006 – 52.2%
Two-Step Analysis/Response Process
1.Online survey• Concerns identified
2.Focus groups, one-on-one meetings• What can be done to address concerns?
Influence Staff Satisfaction
Problems Identified Solutions Implemented
• Staff don’t know about other departments
• Open Houses for departments
• Staff don’t know what’s going on system-wide
• “Did You Know?”, Job/Position Announcements
• Administration not available to hear concerns
• Brown Bag Lunches, Open Door Policies
Impacts Seen 2006 2004 Change
• Library staff in other departments are familiar with what I do. 28.3% 17.3%
11.0%
• The Library administration is willing and available to listen to 52.3% 37.3%
15.0% my concerns, comments, and recommendations.
• The Library administration effectively communicates to staff. 36.9% 36.6%
0.3%
• Collaboration is encouraged and supported in my job. 69.0% 75.8% -
6.8%
• I get cooperation from other departments when we work together. 71.2% 74.0% -
2.8%
Influence Staff Satisfaction
Problems Identified Solutions Implemented
• Need more staff and better distribution of work
• 24 new positions created
• Inadequate facilities, technology and support resources
• Library IT involved with allocations• Facilities staff increased• Balanced scorecard metric for facilities improvement
Impacts Seen 2006 2004 Change
• I have adequate resources and materials to complete assignments. 59.5% 49.2%
10.3%
• I feel safe in my work environment. 87.9% 81.5% 6.4%
• The Library is concerned about and addresses my ergonomic needs 67.0% 76.3% -
9.3%
• I am physically comfortable in my work environment (temperature, light, noise). 56.4% 44.6% 11.8%
Influence Staff Satisfaction
Problems Identified Solutions Implemented
• Like Administration, HR not always available
• New HR staff hired• HR moves from Admin area into library
• Salaries are inadequate • Extra allocations towards salary adjustments
• No opportunity for growth • Better advertising of existing policies• Mentoring programImpacts Seen 2006 2004 Change
• My salary and benefits are reasonable for the work I do. 43.9% 21.5% 22.4%
• The salary I receive is equitable when compared to co-workers in similar grade levels or ranks. 49.0% 30.9% 18.1%
• My work at the Library helps me achieve my career goals. 55.4% 35.9% 19.5%
• There are opportunities for me to advance at the Library. 36.8% 28.1% 8.7%
• There are adequate staff development opportunities provided by the Library. 64.2% 57.3% 6.9%
Influence Staff Satisfaction
The Overall Impact
2006 2004Change
• My work at the Library helps me achieve my career goals. 55.4% 35.9%
19.5%
• I am satisfied with my job. 63.2% 58.5%
4.7%
• Overall Job Satisfaction Category 65.5% 58.0% 7.5%
Support Budget Requests
• 80% of Music faculty responded to the 2004 Faculty Survey. Of these respondents, 88% chose sound recordings as their highest priority, making it the highest priority item for the department.
• While every Music faculty respondent rated the resource “sound recordings” on a 1-5 scale, the category received one of the lower ratings for satisfaction, a 3.88.
• The rating for sound recordings among faculty as a whole was 3.89.
Support Budget RequestsMusic Library Budget
• Raising awareness of traditional practices as assessment tools (ref counts, gate counts, circ data, ILL data)
• Need to review, revise, reassess our BSC metrics and other tools. Assessment is ongoing.
• Managers and all staff need to learn how to better use data and assessment results in daily operations.
• Continue to move assessment out of MIS only. Position MIS as consultant/coordinator or assessment done throughout the organization.
• Assessment is for everyone!
Challenges AheadPromoting a Culture of Assessment