Measuring the impact of information literacy instruction: A starting point for moving forward

Lorna DoddUser Services Manager, UCD LibraryLorna.dodd@ucd.ieTwitter:@LornaDodd

Why?

…snapshot of impact to date & best path forward

Looking back

Large University with no single strategy for

the development of Information Literacy

Deliver a heavy load of information skills,

driven by individuals

Sometimes we evaluate but we rarely measure. with no clear indication in meeting learning outcomes

Looking back

Looking ForwardNew Library Strategic Plan 2010-2014 aims to “monitor, measure and evaluate the Library’s teaching and learning strategy and activities.” (UCD Library, 2010)

As a result there has been a realignment of

staffing structure.

We have shifted from module to programme approach

And from a responsive to consultative role

A key goal is to measure our impact on student learning

Focus on process &

outputs

Limitations of traditional

approach in measuring success

Important to use results of any study which measures

the impact of Library Service

Focus on Library use rather

than instruction

Common tools include

surveys, behavioural observation and pre/post tests

Literature Review

Challenges

Data protection rules

Measuring long-term impact

It takes time to measure impact of Library Instruction

Promote new service

Identify what has been successful

Gather feedback from those not engaged

Identify useful methods going forward

Measuring impact retrospectively

Perceived impact is subjective

External factors influence skills development

Opt for the familiar

Survey all module coordinators

Promotion

Survey Results

Yes 43%

No 35%

Don’t know 22%

Has the Library ever delivered instruction in your module?

35%

Lecture

33%

Workshop

25%Tour

7%Other

What kind of instruction did your students have?

Impact of Library Instruction

“Students have always received instruction to

date and so difficult to answer the

comparison questions below”

Should ALL students have Library Instruction?

Yes86% Don’t

Know9%

No 5%

Should ALL students have Library Instruction?

Yes96% Don’t

Know3%

No 2%

Those who had Library Instruction

Should ALL students have Library Instruction?

Yes77% N0

14%

Don’t Know 10%

Those who did NOT have Library Instruction

Should Library Instruction happen in every year?

“Instruction in Year one should be

built on in

subsequent years to

enhance skills by year”

Preferred Services

120

Subject specific

workshops

62

Library Tours by

peer mentors

58

Generic workshops

50

Online tutorials

48

Online instructional

videos

45

Tutorials

First choice for undergraduates

66

Generic workshops

52

Tutorials

50

Online instructional

videos

45

Online tutorials

42

Subject specific

workshops

36

Library tours with peer mentors

Second choice for undergraduates

First choice for taught postgraduates

154

Subject specific

workshops

62

Library Tours by peer mentors

57

Generic workshops

53

Online instructional

videos

51

Online

tutorials

Influence of instruction on preference?

Influence of instruction on preference?

Influence of instruction on preference?

“…create materials that are reusable and for

any student, regardless of level. Just because a student moves up a level

doesn't mean they've acquired all of the

learning they should have”

“…really good online video

materials, available on demand is the way

to go”.

Qualitative Feedback 20% of respondents

Themes

In line with

strategy

Online Options

Current & Future

Services

Comments on service

Comments on survey

“…same 'introductory' talks

by librarian using lecture slots of

various modules for at least three times

(the third time he skipped

as he found it ridiculous)

“Students just skip the

guest lecture.”

“…requires personal contact with a friendly face

early on.

“we really love our librarians; thank you”

“…it makes a noticeable difference in the

quality of assignment students produce. The

Library Tutorial is typically rated as one of the

most important lectures they have in that Stage 1 module.”

Implications for the Future

Impact of Instruction

Definite benefit in Library instruction

Stronger evidence in discovery and use of resources

Active learning tailored to specific subject needs

Tours perceived as ineffective

Preferred delivery of instruction

Asked to identify how

effective instruction was

at improving students’ information literacy skills.

When they request instruction, do they usually consider this?

Questions

More QuestionsDid respondents choose workshop environment over online version due to familiarity?

Then why did those who had no instruction choose workshop?

If delivering workshops on this scale is unsustainable,

What can we do ensure effectiveness?

What work do we need to do in terms of marketing?

We have a shared understanding…

…how do we move forward?

Conclusion

Lorna.dodd@ucd.ie

@LornaDodd

Blagden, P. (2005) ‘The LIRG/SCONUL Measuring Impact Initiative: Overview of phase 1 impact projects’, Library & Information Research (LIR) vol. 29 (91)

Dodd, L. & Kendlin, V. (2010) ‘Damned if we do and Damned if we don’t: How to address sustainability in the delivery of information literacy components in UCD’: Librarians' Information Literacy Annual Conference (LILAC), March 29-31, Limerick, Ireland.

Markless, S & Streatfield D. (2006) ‘Gathering and applying evidence of the impact of UK university libraries on student learning and research: A facilitated action research approach’, Journal of Information Management vol. 26 pp. 3-15

Poll, R & Payne, P. (2006) ‘Impact measures for libraries and information services’, Library Hi Tech vol. 24 (4) pp. 547-562

Schilling, K. & Applegate, R. (2012) ‘Best methods for evaluating education impact: a comparison of the efficacy of commonly used measures in library instruction’, Journal of the Medical Library Association, vol. 100 (4) pp. 258-269

Stone, G. et al (2011) ‘Does Library use affect student attainment? A preliminary report on the library impact data project’ Liber Quarterly vo. 21(1) pp. 5-22

UCD Library (2010) ‘UCD Library Strategic Plan 2010-2014’

http://www.ucd.ie/library/news_publicity/showcase_strategy/

Lorna Dodd: ANLTC 2013

References

Off the starting blocks http://www.flickr.com/photos/nickuzma/2570878552/

why can't i breathe whenever i think about you? http://www.flickr.com/photos/breannajosephine/5098047755/

Looking Into the Past - Custom House http://www.flickr.com/photos/coreytempleton/5893927056/

To go to http://www.flickr.com/photos/rubensmj/5399226001/

Literature review http://www.flickr.com/photos/carowallis1/4297460415/

Nissan Qashqai Challenge London http://www.flickr.com/photos/parmiter/2521273890/

Playing chess http://www.flickr.com/photos/bburt33/5930699148/

Surveying for Central Station, 1900 http://www.flickr.com/photos/state-records-nsw/4010230662

Shouting kid http://www.flickr.com/photos/jamessarmiento/4353936770/

Results of elections http://www.flickr.com/photos/ntr23/4650249185/

Four stages of the moon http://www.flickr.com/photos/leehaywood/6607585765/

Meteor crater http://www.flickr.com/photos/homer4k/113472442/

Writing http://www.flickr.com/photos/jjpacres/3293117576/

Used books http://www.flickr.com/photos/babblingdweeb/23816128/

Quality Barnstorm http://commons.wikimedia.org/wiki/File:Quality_Barnstar_v2.0.png

Apples & Oranges - They Don't Compare http://www.flickr.com/photos/thebusybrain/2492945625/

Lorna Dodd: ANLTC 2013

Images

Watermelon http://www.dreamstime.com/stock-photography-watermelon-image22521832

Orange http://www.dreamstime.com/royalty-free-stock-photo-orange-image23480475

Strawberry on white background http://www.dreamstime.com/royalty-free-stock-photo-one-strawberry-white-background-image28694065

One gooseberry fruits isolated http://www.dreamstime.com/stock-photography-one-gooseberry-fruits-isolated-image20607952

Construction Tunnel http://www.flickr.com/photos/joshuarothhaas/2250657031 Online http://www.flickr.com/photos/orkomedix/5398470372/

Conversation http://www.flickr.com/photos/stijnnieuwendijk/3098445189/

Backyard fresh http://www.flickr.com/photos/jim_belford/8544552591/

Abandoned amphitheatre http://www.flickr.com/photos/pixarman/5456444943/

Lego love http://www.flickr.com/photos/evaysucamara/5419530024/

Student working http://www.flickr.com/photos/92399131@N07/8399835572/

Future http://www.flickr.com/photos/maniya/4020026753/

Punch http://www.flickr.com/photos/timcaynes/148110505/

teach me http://www.flickr.com/photos/giovannamatarazzo/182108786

Question proposal http://www.flickr.com/photos/eleaf/2536358399/

Light Bulb http://www.flickr.com/photos/polifemus/561986434/

walk ahead and cross the sky http://www.flickr.com/photos/ldawei/4920681478/

Lorna Dodd: ANLTC 2013

Images