ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS...

17
ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012 Amy S. Van Epps and Megan R. Sapp Nelson Engineering Librarians, Purdue University

Transcript of ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS...

Page 1: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

ONE OR MANY?ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER.A WORK IN PROGRESS

June 11, 2012

Amy S. Van Epps and Megan R. Sapp NelsonEngineering Librarians, Purdue University

Page 2: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

INTRODUCTION

Page 3: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

OPPORTUNITY

3

Information Literacy opportunity with different sections of the same first year course

• Fundamentals of Speech Communication • Associated with engineering learning communities

Different delivery models • a single, 50 minute lecture early in the semester• four, approximately 12 minute lectures to be offered just before

each assignment was given

Material presented was coordinated • LibGuide to minimize variation

Page 4: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

RESEARCH QUESTION

4

Is there a noticeable difference in the quality of citations in student assignments when “just-in-time” instruction is used as opposed to a “one-shot” session?

Hypothesis

Sections which received the just-in-time instruction would have better citations, both in quantity and quality than the section which received the one-shot session.

Page 5: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

METHODS AND SAMPLE

Page 6: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

METHODS

6

Study GroupAll students in 3 sections of the course

DataOutlines with references required for each

assignment

Data received by librarians after anonymized

Reference AnalysisDone using a framework to identify resource type and quality of intended audience and purpose

Page 7: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

SAMPLE SELECTION

7

Sampling

Random sample of outlines for coding

Analysis Set

5 bibliographies for each assignment (1-3) per team

3 bibliographies for final assignment per team

36 bibliographies

233 references

124 references from just-in-time set; 109 from one-shot

Page 8: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

Based on the work of Wertz et al (2011) Resource type Quality

BookPeriodicalWebFacts & FiguresUnknown

Inter-rater reliability, consensus estimate • 84.1 percent agreement

CODING FRAMEWORK

8

Scholarly Popular

Informative

Biased

Medium

Medium

High

Low

Page 9: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

RESULTS ANDDISCUSSION

Page 10: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

RESULTS

10

Average number of references per outline

All assignments required a minimum of 3 references.

  Just-in-time (n=124) One-Shot (n=109) Total (n=233)

Individual Assignments (#1-3)* 4.93 3.86 4.4

Group Assignment (#4) 16.67 17.00 16.8

All assignments 6.88 6.05 6.5

* Statistically significant difference (p<.05)

Page 11: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

QUALITY OF RESOURCES USED

11

High* Medium* Low* Unable to classify0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

mini-lectures one-shot

* Statistically significant difference (p<.05)

Quality Embeddedn=124

One-Shotn=109

Significance

High* 43.6% 22.9% (Z= 3.31, p< .001)Medium* 51.6% 65.2% (Z= -2.06, p< .05)Low* 2.4% 9.2% (Z= 2.24, p< .05)Unknown 2.4% 2.7% (Z= -0.16, p= .873)

High, medium, and low quality sources all show statistically significant differences between sections.

Page 12: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

COMPLETENESS OF CITATIONS

12

Difference for IncompleteMay reflect differences between raters more than differences in student abilities.

APA format was not taught by the librarians

complete reference = presence of all elements

Complete Improper Incomplete* Unknown0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

mini-lectures one-shot

* Statistically significant difference (p<.05)

Completeness Embed. One-Shot SignificanceComplete 55.7% 60.6% (Z= -0.756, p= .453)Improper 29.8% 32.1% (Z= -0.37, p= .712)Incomplete* 12.1% 4.6% (Z= 2.03, p< .05)Unknown 2.4% 2.7% (Z= -0.16, p= .873)

Page 13: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

TYPE OF RESOURCE

13

Significantly different distribution of resources between sections.

One-shot students use many more web resources.

Just-in-time students split between journals and web resources.

Monograph Periodical* Web* Interpersonal Unknown Facts & Figures0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

mini-lectures one-shot

* Statistically significant difference (p<.05)

Type Embed. One-Shot SignificanceMonograph 6.5% 4.6% (Z= 0.617, p= .535)Periodical* 48.4% 9.2% (Z= 6.52, p< .001)Web* 41.9% 83.5% (Z= -6.50, p< .001)Interpersonal 0% 0% Unknown 2.4% 2.7% (Z= -0.16, p= .873)Fact & Fig. 0.8% 0%

Page 14: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

DISCUSSION

14

An apparent benefit can be see with just-in-time teaching• Use of more high quality resources• Use of more periodicals • Frequently visits = opportunity for follow-up

Overall use of popular and informative resources • Medium quality; 59.9% not surprising to the authors• Speech communication is well suited to popular resources• Less requirement of research or scholarly publications than

design assignments or research papers • 93.4% of the resources used were classified as informative

Page 15: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

CONCLUSION

Page 16: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

CONCLUSION

16

• A correlation exists embedded just-in-time teaching approach and the quality of reference sources used can be seen.

• Still unknown…if the library instruction model is responsible for the difference

Page 17: ONE OR MANY? ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER. A WORK IN PROGRESS June 11, 2012.

QUESTIONS?