KNOWING WHAT WE DON'T KNOW: STUDENT SELF-ASSESSMENT AND LIBRARY USE

Denyse RodriguesLiaison Librarian, E-learning LibrarianMount Saint Vincent University Library

Thursday May 9, 201381st ABQLA Annual Conference (2013)Assessment: Gaining insight, improving servicesÉvaluer : mieux connaître pour mieux servir

The ContextMount Saint Vincent University

• Located in Halifax, NS

• Approximately 4000 students, primarily undergraduate

• Local students: 75% from NS

• 80% female student body

What we know so far(from the literature & observation)• Students rate their own research skill quite high (Raven, 2012)

• Faculty and librarians rate students as having low research skills (Raven, 2012)

• To support students, the Mount Library provides a combination of:

• help guides/online videos via the library website

• reference assistance & research appointments

• one-shot workshops in collaboration with faculty

• credit course LIBR 2100: Introduction to Research Skill in the Information Age

What I want to know• A bench mark of students information literacy/research

readiness skills using an assessment tool

• If there is a gap between actual skill and perception of skill, would knowing about the gap motivate students to utilize library supports.

Method1. Self assessment: undergraduate students invited to complete an

online self assessment at the beginning of the term.

• Group 1 • Invited to completed the survey prior to or during a one-shot workshop. (with option to opt out of

research project)• 126 invited - 102 self assessments

• Group 2 • Invited to complete the survey separate from the offering or provision of a workshop• 307 invited - 58 self assessments

• Response rate: .36

2. Semi-structured interview: At the end of the term, participants invited to a semi structured interview regarding their research skills and the library services and resources made available to them during the term.

• 8 interviews

Participating Students• 93% female

• 42% first-year; 37% second-year; 21% third-year plus

• 60% age 20 or less; 27% age 21-30; 13% age 30+

• 46% only campus34% blended campus and distance courses20% only distance

• All disciplines represented

Research Readiness Self Assessment• Developed by Casey & Ivanitskaya at Central Michigan University, RRSA

consists of ~50 quiz type questions to objectively measure information literacy competencies

• Assess student performance in the knowledge and skill areas of the ACRL Informational Literacy Competency Standards for Higher Education• Obtaining Information• Evaluating Information• Understanding Plagiarism

• Survey student beliefs and behaviors pertaining to information seeking activities• Research experience• Perceived skills• Browsing the Internet

• Provides direct feedback to students with suggestions for improvement of skills

Sample feedbackThe following categories measure your experience, attitudes and beliefs:

Research and library experience - 17 / 34 - 50%Browsing the Internet - 25 / 50 - 50%Perceived research skills - 20 / 40 - 50%

The following categories measure your knowledge and skills:

Obtaining information - 15/ 30 - 50%Evaluating information - 6 / 12 - 50%Understanding of plagiarism - 7 / 14 - 50%

Overall score, an indicator of your research readiness - 28 / 56 - 50%

Sample feedbackYour score has indicated that you tend to use general search engines on the Internet for information. Surfing the Web can sometimes give you relevant information but should not be trusted for the majority of research you do because it is not always accurate or reliable.

The Mount Library provide access to books and articles through licensed databases. These items are paid for by the Library and are available to students, faculty, and staff. Licensed databases offer access to scholarly journal articles and other sources of information that students should be reading when working on university level papers.

* Visit our subject guides for databases in your area of study -http://bit.ly/msvulibraryguides* For more information on evaluating websites, please seehttp://bit.ly/web-site-evaluation

UndergraduatesNumber of Completed Records: 16095% of students above 50% in skill levels

Category Min Score Max Score Average Score

Points Possible %

Obtaining information 5 24 15.78 24 66

Evaluating information 1 10 6.13 12 51

Understanding of plagiarism 6 16 13.67 18 76

Browsing the Internet 0 45.2 18.5 50 37

Perceived research skills 8 35 25.32 35 72

Research and library experience 0 31 15.56 40 39

First year studentsNumber of Completed Records: 6790% of students above 50% in skill levels

Category Min Score Max Score Average Score

Points Possible %

Obtaining information 7 23 15.48 24 64

Evaluating information 1 10 5.78 12 48

Understanding of plagiarism 6 16 13.58 18 75

Browsing the Internet 0 42 18.39 50 37

Perceived research skills 8 32 25.32 35 72

Research and library experience 0 25 12.85 40 33

• One can retrieve the most documents in an online library catalog by searching for:

• Training and turnover • Training not turnover • Training or turnover • Not training turnover • Or turnover and training

69%

31%

Chart Title

incorrect correct

Faculty feedback • WOMS 1110 – 60 students• Feedback on bibliographies compared to previous years

• Decreased use of open Internet websites • Room for improvement with evaluating information

sources• Citation and bibliographies completed but contained

errors in APA style

Interviews• Research Readiness Self Assessment

• Did you read the feedback that was sent to you by the self assessment earlier in the term?

• Did you follow any of the links?• Have you ever watched any of the help videos on the Library website?• Did you make any changes to the way you conduct research after

receiving the self assessment report?• Were the suggested resources helpful?

• Research What are your research strategies?• What are the challenges you encounter when conducting research for

your course assignments?

• Library • How can the Library better support you with your research?

Interview groupNumber of Completed Records: 84 on campus/ 4 distance

Category Min Score Max Score Average Score

Points Possible %

Obtaining information 10 22 17.63 24 73

Evaluating information 2 9 6.63 12 55

Understanding of plagiarism 8 16 13.38 18 74

Browsing the Internet 2.58 38.38 22.22 50 44

Perceived research skills 8 32.88 24.8 35 62

Research and library experience 3 24 11 40 31

UndergraduatesNumber of Completed Records: 160

Category Min Score Max Score Average Score

Points Possible %

Obtaining information 5 24 15.78 24 66

Evaluating information 1 10 6.13 12 51

Understanding of plagiarism 6 16 13.67 18 76

Browsing the Internet 0 45.2 18.5 50 37

Perceived research skills 8 35 25.32 35 72

Research and library experience 0 31 15.56 40 39

Interview groupNumber of Completed Records: 84 on campus/ 4 distance

Category Min Score Max Score Average Score

Points Possible %

Obtaining information 10 22 17.63 24 73

Evaluating information 2 9 6.63 12 55

Understanding of plagiarism 8 16 13.38 18 74

Browsing the Internet 2.58 38.38 22.22 50 44

Perceived research skills 8 32.88 24.8 35 62

Research and library experience 3 24 11 40 31

RRSA feedback• Students recalled receiving the feedback but no one

reported following any of the links for further information in the RRSA feedback

I vaguely remember it. I know I did receive it and I did read it over, but it’s been a couple months so it is not fresh in my mind, but I do remember and I remember I don’t think I scored too poorly on it. I haven’t really ventured out to use any of the tips to help me with my research.

“I remember I checked it and I looked at it and I kind of like went through it really fast but I didn’t really do any more . . . There was something I did absolutely terrible on but I can’t even remember what it was [laughs] That’s not much help I guess. What was it? [uhhhhhm]. I don’t know but I got a 1 out of however much.

• “Not really, that was a long time. I remember it was very big. I remember receiving some criticism on one section . . . and I knew it was true anyway. . . and I can’t remember what it was.”

• “it was... was a really long email I scanned it and moved on. I like just one or two lines. That’s it. I really need to know the gist of what I’m reading in one or two lines at the top. I do not like to be inundated with information. Unless I go searching for it I do not want to be confronted by it. “

Challenges: research strategy• Finding Articles/Narrowing Results“Finding articles. Usually that’s kind of the part that I am kind of not very good at. So usually like if there is a huge number of articles and I find something that I like I usually look under those words, you know at what words they show or whatever, and then I’ll type those in so I can get smaller results.”

• Evaluating information was not mentioned explicitly“Well, … a lot of the time … the articles are really, really long and I don’t have the time to like read the whole thing so ones that are kind of shorter I tend to like go towards I guess, and the ones that seem more relevant to the topic that I want to focus on really, I guess.”

Positive Faculty Influence• My psychology professor is very specific. . . there these

certain articles that she wants us to actually find on the database so she actually tells me that this is where and what I want you look up on the library database and do research on it so [laughs] that’s really it.

• My history professor suggested about going on and finding the particular reading that we needed for an assignment and that is when I basically discovered the library. So, I’ve used it probably, if not every day then every other day I’ve been on the website.

Challenges - citation• Yeah I don’t know how to cite properly I don’t think.

• I cite my sources but with a less formal setup, like I would just list them at the end.

Challenges: school/work/life balance• I am usually working and going to school and all of this

homework and assignments you have to do and then doing the research on top of it is sometimes unbearable. I keep a calendar and I budget my time. If I allow myself 2 hours this afternoon to get materials I need, then I have to stick within that time to ensure that I use those hours efficiently to get what I need for my assignments. I wouldn’t discount looking further but I would definitely would choose the quicker options when it comes to searching.

Promotion of the library to distance students• Distance students less aware of library resources and

services“I did not realize what the library had to offer until I did this [self-assessment and interview]. Good to know that it is just not a ‘resource’ but that it is actually going to help me succeed”

Conclusions• Students rate their research skills higher than they

actually are, but most possess basic research skills

• The evaluation of information needs to be better integrated into research instruction.

• Self-assessment was most useful during in-class instruction for student engagement.

• The personalized self-assessment information does not appear to have motivated any of the undergraduate students to explore the Library’s help resources.

Food for thoughtStudents do not see research as a process but think of it as the tools they use to find articles.Resnis, E., Gibson, K., Hartsell-Gundy, A., & Misco, M. (2010). Information literacy assessment: A case study at Miami university. New Library World, 111, 7, 287-301.

There are discrepancies between what students know and what they do regarding source selection. Kim, K.-S., & Sin, S.-C. (2011). Selecting quality sources: Bridging the gap between the perception and use of information sources. Journal of Information Science,37, 2, 178-188.