Post on 19-Jan-2016
ONE OR MANY?ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER.A WORK IN PROGRESS
June 11, 2012
Amy S. Van Epps and Megan R. Sapp NelsonEngineering Librarians, Purdue University
INTRODUCTION
OPPORTUNITY
3
Information Literacy opportunity with different sections of the same first year course
• Fundamentals of Speech Communication • Associated with engineering learning communities
Different delivery models • a single, 50 minute lecture early in the semester• four, approximately 12 minute lectures to be offered just before
each assignment was given
Material presented was coordinated • LibGuide to minimize variation
RESEARCH QUESTION
4
Is there a noticeable difference in the quality of citations in student assignments when “just-in-time” instruction is used as opposed to a “one-shot” session?
Hypothesis
Sections which received the just-in-time instruction would have better citations, both in quantity and quality than the section which received the one-shot session.
METHODS AND SAMPLE
METHODS
6
Study GroupAll students in 3 sections of the course
DataOutlines with references required for each
assignment
Data received by librarians after anonymized
Reference AnalysisDone using a framework to identify resource type and quality of intended audience and purpose
SAMPLE SELECTION
7
Sampling
Random sample of outlines for coding
Analysis Set
5 bibliographies for each assignment (1-3) per team
3 bibliographies for final assignment per team
36 bibliographies
233 references
124 references from just-in-time set; 109 from one-shot
Based on the work of Wertz et al (2011) Resource type Quality
BookPeriodicalWebFacts & FiguresUnknown
Inter-rater reliability, consensus estimate • 84.1 percent agreement
CODING FRAMEWORK
8
Scholarly Popular
Informative
Biased
Medium
Medium
High
Low
RESULTS ANDDISCUSSION
RESULTS
10
Average number of references per outline
All assignments required a minimum of 3 references.
Just-in-time (n=124) One-Shot (n=109) Total (n=233)
Individual Assignments (#1-3)* 4.93 3.86 4.4
Group Assignment (#4) 16.67 17.00 16.8
All assignments 6.88 6.05 6.5
* Statistically significant difference (p<.05)
QUALITY OF RESOURCES USED
11
High* Medium* Low* Unable to classify0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
mini-lectures one-shot
* Statistically significant difference (p<.05)
Quality Embeddedn=124
One-Shotn=109
Significance
High* 43.6% 22.9% (Z= 3.31, p< .001)Medium* 51.6% 65.2% (Z= -2.06, p< .05)Low* 2.4% 9.2% (Z= 2.24, p< .05)Unknown 2.4% 2.7% (Z= -0.16, p= .873)
High, medium, and low quality sources all show statistically significant differences between sections.
COMPLETENESS OF CITATIONS
12
Difference for IncompleteMay reflect differences between raters more than differences in student abilities.
APA format was not taught by the librarians
complete reference = presence of all elements
Complete Improper Incomplete* Unknown0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
mini-lectures one-shot
* Statistically significant difference (p<.05)
Completeness Embed. One-Shot SignificanceComplete 55.7% 60.6% (Z= -0.756, p= .453)Improper 29.8% 32.1% (Z= -0.37, p= .712)Incomplete* 12.1% 4.6% (Z= 2.03, p< .05)Unknown 2.4% 2.7% (Z= -0.16, p= .873)
TYPE OF RESOURCE
13
Significantly different distribution of resources between sections.
One-shot students use many more web resources.
Just-in-time students split between journals and web resources.
Monograph Periodical* Web* Interpersonal Unknown Facts & Figures0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
mini-lectures one-shot
* Statistically significant difference (p<.05)
Type Embed. One-Shot SignificanceMonograph 6.5% 4.6% (Z= 0.617, p= .535)Periodical* 48.4% 9.2% (Z= 6.52, p< .001)Web* 41.9% 83.5% (Z= -6.50, p< .001)Interpersonal 0% 0% Unknown 2.4% 2.7% (Z= -0.16, p= .873)Fact & Fig. 0.8% 0%
DISCUSSION
14
An apparent benefit can be see with just-in-time teaching• Use of more high quality resources• Use of more periodicals • Frequently visits = opportunity for follow-up
Overall use of popular and informative resources • Medium quality; 59.9% not surprising to the authors• Speech communication is well suited to popular resources• Less requirement of research or scholarly publications than
design assignments or research papers • 93.4% of the resources used were classified as informative
CONCLUSION
CONCLUSION
16
• A correlation exists embedded just-in-time teaching approach and the quality of reference sources used can be seen.
• Still unknown…if the library instruction model is responsible for the difference
QUESTIONS?