Making Library Assessment Work The Role of Organizational Culture(s)
description
Transcript of Making Library Assessment Work The Role of Organizational Culture(s)
Making Library Assessment Work
The Role of Organizational Culture(s)Martha Kyrillidou
Association of Research Libraries
Steve Hiller University of Washington
Jim Self University of Virginia
Living the FutureTuscon, AZ
April 7, 2006
• Bangor University considers removing librarians posted by Blake on Thursday January 27, @07:30AM -753 hits Ms Information writes "News from the University of Wales Bangor in the UK. senior management no longer feel that subject librarians / academic liaison librarians are needed in the modern academic library. They have made restructuring proposals which include removing all but one of the subject librarians and a tier of the library management, including the Head of Bibliographic Services. The university management thinks that technology has 'deskilled' literature searching. As far as I know, this proposal is unprecedented in the United Kingdom. In essence, there will remain 4 professional librarians serving a 'research-led' university of 8,000 plus FTEs and with 8 library sites. These will be the university librarian, cataloguing librarian, acquisitions librarian and Law librarian.
• Has anything like this happened anywhere that you know of? If so, what have been the effects?
Making Library Assessment Work: Practical Approaches for
Developing and Sustaining Effective Assessment
• Association of Research Libraries Project– Under the aegis of Statistics and Measurement Program
• Funded by participating libraries• Site visits by Jim and Steve
– Pre-visit survey– Presentation– Interviews and meetings – Written report for each library
• Phase I: 7 libraries in Winter/Spring 2005 – Preliminary report at Northumbria Conference 2005
• Phase II: 18 libraries Autumn 2005-Autumn 2006 • Final report to ARL in late 2006
Why “Making Library Assessment Work”?Using Data Effectively in Libraries
• Library leadership• Organizational culture• Library priorities• Sufficiency of resources• Data infrastructure• Assessment skills and expertise • Sustainability • Analyzing and presenting results• Using results to improve libraries
MLAW Process
• Idea discussed with Martha, June 2004• Final proposal written with Martha, August 2004• Invitation to participate sent by ARL in Sept. 2005• 17 libraries express interest; 7 chosen for Phase I,
remainder for Phase II• Pilot site visit November 2005• Second call for participants in June 2005; 8 more
libraries express interest• Phase II revised to include longer site visit, follow-up
project, and community building
The Geographic Distribution of Participants(Jim and Steve Earn Frequent Assessment Miles)
Phase IIParticipant
Phase I Participant
Other ARL Libraries
Steve’s HomeUniversity of Washington
Jim’s HomeUniversity of Virginia
Canada
USA
Distribution Of Participants by ARL Index Ranking (113 Academic Libraries)
24 Participating Libraries: Mean 51.5 Median 49.5
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
3
0 10 20 30 40 50 60 70 80 90 100 110
Index Rank
Ind
ex
Sc
ore Non-participants Participants UVA UW
Phase I (At time of visit) Assessment Organization Structure
Assessment Position
Assessment Committee
Contact Person Admin. Area
Other Related Groups
Arizona Administration Distributed within team structure
Arizona State
Services
Connecticut Collection Services User Team, Network Svcs,
Design & Usability
Illinois Services Services Advisory
User Ed, Coll Development
New York U Established 2005
Public Services
Notre Dame Public Services Usability Group
Libraries Task Force
Oregon Administration
Phase II: Autumn 2005 (At time of visit)
Assessment Organization StructureAssessment
PositionAssessment Committee
AdministrativeArea
Other OngoingGroups
Emory 2005 2005 Administration
Louisville 1997 1997 Administration
Kansas 2004 2003 Administration
Massachusetts 2004 2004 Administration
Wayne 2005 2005 Administration
Pre-Visit Survey
• Summary of recent assessment activity • Inventory of statistics kept • Important assessment motivators• Organizational structure for assessment • What has worked well• Problems or sticking points • Specific areas to address• Expectations for this effort• Follow-up project (Phase II)
Phase I:Library-Identified Assessment Needs
Data Collection
Data Analysis
DataUse
Skills & Abilities
Perform. Measures
Data Warehouse
Sustain Assessment Culture
Library 1
Library 2
Library 3
Library 4
Library 5
Library 6
Library 7
Phase II: Autumn 2005Library-Identified Assessment Needs
Data Collection
Data Analysis
DataUse
Skills & Abilities
Perform. Measures
Learning Outcomes
Sustain Assessment Culture
Library 1
Library 2
Library 3
Library 4
Library 5
Sample Site Visit Schedule
• Meet with University Librarian/Contact person• Presentation on effective assessment
– 90 minutes to 2 hours with Q&A• Concepts and best practices• Examples from UVA and UW Libraries
• Group Meetings– with management/administrative group
– assessment-related group (if formed)
– different departments
– functional areas/groups (e.g. info literacy)
• Wrap-up session
Presentation Slide That Fostered Most Discussion
University of Virginia Balanced Scorecard
Metric U.3.A – Circulation of New Monographs
• Target1: 60% of newly cataloged cataloged monographs should circulate within two years
.• Target2: 50% of new monographs should circulate
within two years.
Our Perception of Visits
• Diverse organizational cultures offers opportunities and challenges for successful assessment– Every library is unique
• Reception overwhelmingly positive– Spirited and engaged discussions
• More assessment work going on than being reported– Internally and externally
• Important assessment catalysts include:– accreditation, facilities renovation, student learning, data driven
administrations, LibQUAL+™ results, “should be doing this” • Increase in interest reflected in creation of new assessment
positions and groups
Sample Report Format
• Introduction• Current Assessment Environment and Activities• Identified Issues and Concerns• Suggestions and Options for Moving Forward • Conclusion
Typical Recommendations
• Coordination of assessment– Involve library staff, better communication
• Prioritize assessment activities• Move from projects to sustainable assessment• Share assessment results• Allocate sufficient resources • Review internal statistics • Incorporate data into library management
– Management information systems important
• More knowledge of on-campus activities – Data warehousing
Feedback on Phase I ProcessChanges for Phase II
• One day is too short– Site visit increased to 1.5 days
• Resource materials would be helpful– Web site strengthened
– Assessment “reader” provided
• Follow-up activity would maintain momentum– Consult on follow-up activity (plan, specific effort)
• Establish community of assessment practitioners– Meeting at ALA
– Assessment Conference, September 2006
– ARL Assess list serv
What’s Next? Building the Library Assessment Community
• Meetings– ALA New Orleans, June 2006– Assessment Conference, Charlottesville VA
September 25-27, 2006
• Web Site– http://www.arl.org/stats/Hiller_Self.html
• Final Report– Late 2006
Icon WomanThe world is going through more fundamental change than it has in hundreds, perhaps
thousands of years … Do you want to be a player, a full-scale participant who embraces change? Here is the opportunity to participate in the lovely, messy playground called ‘Let’s reinvent the world.’
Here’s a new model I call Icon Woman:She is turned on by her work!The work matters!She is ‘in your face!’She is an adventurer!She is the CEO of her life!She is not God. She is not a Bionic Woman. She is determined to make a difference!My Icon Woman, of course, embraces and exploits the Web. She submits her resume on
the Web and keeps it perpetually active there …She creates and conducts scintillating projects on the Web via a far-flung ‘virtual’ stable
of teammates (most of whom she’s never met) …She is deeply committed to her self-designed, do-it-from-anywhere-with-anybody ‘career’
path … Her only security is her personal commitment to constant growth and her global (virtual) rep for great work.
Tom Peters ‘‘Visions 21: Our Work, Our World”
Assessment as a Bold Library Expression
• Where creative exploration happens with zeal and with purpose, without restraint or presumption, there will thrive a society that not only sustains its people but challenges, connects and enlightens them through the power of bold human expression. – Kenan Institute of the Arts