Actual Perceived Skills in Pre Educatorslibraries.ucsd.edu/bmcl/dturnbow/finalpaper_brown... ·...

Actual & Perceived Information Literacy Skills in

Pre‐Service Educators

Prepared by

Marcy Brown Dominique Turnbow

Educational Technology 690 San Diego State University

May 25, 2009

Actual & Perceived Information Literacy Skills in Pre‐Service Educators

EDTEC 690: Brown/Turnbow 2

Table of Contents Overview ....................................................................................................................................................... 4

Literature Review .......................................................................................................................................... 5

Information Literacy within Higher Education .......................................................................................... 5

Libraries’ role in information literacy ................................................................................................... 5

Instruments to Assess Information Literacy Skills and Perception ........................................................... 6

Instruments to assess information literacy skills .................................................................................. 6

Assessing perceptions using Competency Theory ................................................................................ 6

Study Instrument Design ........................................................................................................................... 7

Questions to test actual skills ............................................................................................................... 7

Questions to test perceptions of knowledge ........................................................................................ 8

Information Literacy within Teacher Education ....................................................................................... 8

Information literacy standards and accreditation ................................................................................ 9

Information literacy instruction within teacher education coursework............................................. 10

Recommendations for change ............................................................................................................ 10

Methodology ............................................................................................................................................... 10

Participants ............................................................................................................................................. 11

Instrument Development ........................................................................................................................ 11

Administration ........................................................................................................................................ 12

Contextual Factors ...................................................................................................................................... 12

Study Sample .......................................................................................................................................... 12

Instrument .............................................................................................................................................. 13

Findings ....................................................................................................................................................... 13

Participant Demographics ....................................................................................................................... 13

Actual Information Literacy Skills – Whole Number Scores ................................................................... 14

Skills Perception – Estimates of Performance ........................................................................................ 14

Correlation of Actual and Perceived Skills .............................................................................................. 15

Actual Information Literacy Skills – Score Quartile ................................................................................. 16

Skills Perception – Estimates Relative to Other Respondents ................................................................ 17

Correlation of Actual and Perceived Quartiles ....................................................................................... 17



Comfort with Research‐Related Skills ..................................................................................................... 18

Correlation of Actual Skills and Research Comfort ................................................................................. 19

Library Use .............................................................................................................................................. 20

Correlation of Actual Skills and Library Use ............................................................................................ 20

Discussion ................................................................................................................................................... 21

Conclusion ................................................................................................................................................... 22

Appendix A: Results for Question 7 ............................................................................................................ 23

Appendix B: Results for Question 4 ............................................................................................................ 24

References .................................................................................................................................................. 25

Instrument…………………………………………………………………………………………………………………………………………….27



Overview

Today’s college graduates must be information literate; in fact, most accrediting agencies for higher

education—among them the Western Association of Schools and Colleges (WASC) – emphasize the

criticality of this competence, regardless of the degree students earn.

But what does it mean to be information literate? Researchers offer many definitions–some abstract,

some concrete. Following is the characterization provided by the Association for College &Research

Libraries (ACRL):

Information literacy is a set of abilities requiring individuals to recognize when information is

needed and have the ability to locate, evaluate, and use effectively the needed information

(2000).

ACRL offers a detailed list of student competencies for librarians and other educators to use in planning

instruction. They are organized into five “standards”; each includes several performance indicators and

outcomes.

The researchers explored whether a pre‐service educator’s perception of his or her ability to identify

potential information sources relates to his or her actual ability to identify information sources.

Specifically, this study focused on ACRL’s Standard 2: The information literate student accesses needed

information effectively and efficiently. The specific performance indicators for this standard are below

while the indicators to which this study attended are shown in italics.

1. The information literate student selects the most appropriate investigative methods or

information retrieval systems for accessing the needed information.

2. The information literate student constructs and implements effectively‐designed search

strategies.

3. The information literate student retrieves information online or in person using a variety of

methods.

4. The information literate student refines the search strategy if necessary.

5. The information literate student extracts, records, and manages the information and its sources

information literate student reevaluates the nature and extent of the information need.



In practice, many librarians aim interventions toward students who express lower levels of confidence in

their information literacy skills. The results of this study addressed whether or not this is a worthwhile

strategy. An alternative might be to more broadly target students—including those who appear

confident in their information literacy and library skills.

Literature Review

The researchers drew heavily from the Library &Information Science (LIS) and Education disciplines in

preparing this review. It provides an overview of how information literacy is taught within higher

education, instruments that have been used to assess skills and perception, and information literacy

within teacher education.

Information Literacy within Higher Education

Universities are turning their attention to assessing student information literacy skills on their campuses

largely due to accrediting agencies requirements (Hernon & Dugan, 2004). The methods used to deliver

instruction and assessment may be institutionally driven or instructor driven. In most cases, librarians –

as recognized experts on information finding and use – provide information literacy instruction.

Libraries’ role in information literacy

The way information literacy instruction is delivered on academic campuses is as varied as the campuses

themselves. Here are some examples of librarian‐led information literacy instruction:

• One‐Shot. “One‐shot” instruction is provided to a class as requested by a faculty member. These

sessions can be general overviews of how to use the library or focused on a specific paper or

project students need to complete for their course.

• “Just in Time.” Librarians sometimes provide several workshops to students as requested by a

faculty member. The workshops are dispersed throughout the course and cover topics that

students need when they need them.

• “First‐year experience.” This one‐unit course is usually a semester long and targets new and

transfer students. The goal is to provide students with essential skills that will help them

succeed in college, such as note taking, research, and critical thinking.

• Reference & consultations. These are the services with which most library users are familiar.

Through the reference desk or a consultation, librarians provide individual instruction with

users.



• Online instruction. Online instruction is a new, growing area for many libraries. It takes many

forms, from a small, one‐minute video about how to search a database (Turnbow, 2009b) to a

full‐length online workshop, including assessment (Turnbow, 2009a).

Whether or not Information Literacy instruction is required in the curriculum varies across college and

university campuses. Regardless, formalized “testing” or assessment of these skills is typically not

required by the campuses. Without support at a campus‐wide level, librarians will continue to struggle

to provide an accurate picture of student information literacy skills. Despite this, many librarians have

developed and administered instruments in order to illustrate student information literacy skills – or

lack thereof – to their campus administrations. These will be discussed below.

Instruments to Assess Information Literacy Skills and Perception

Instruments to assess information literacy skills

Many instruments have been developed to measure information literacy competencies (i.e. actual skills)

based on the ACRL standards. Neely (2006) discusses the use of many of these instruments in detail. In

contrast, only a handful of studies have examined the correlation between "actual" performance on

information literacy tasks with students' perceived performance (Cameron, Wise, & Lottridge, 2007;

Geffert & Christensen, 1998; Ivanitskaya, Laus, & Marie Casey, 2004; Maughan, 2001; Ren, 2000).

Unfortunately, most of these instruments have not been validated. As far as the researchers are aware,

none of these studies included correlation analysis of participant results with perceived level of skill.

Assessing perceptions using Competency Theory

The lean research base focused on the correlation between student perceptions of their information

literacy skills and actual performance on tasks tends to be grounded in competency theory; Kruger and

Dunning (1999) are largely credited with developing this train of thought. They suggest that people who

perform at a low skill level lack the metacognitive abilities to recognize their own incompetence.

Therefore, they cannot accurately assess their own or others’ skills and tend to overestimate their

abilities. Further, people that perform at a very high skill level tend to underestimate their performance.

Kruger and Dunning detail four studies that they designed to test their predictions in regard to

competency theory. Three of those studies directly relate to the relationships explored in the current

study of perceived versus actual information literacy.



1. In the first, 65 undergraduate students were asked to rate how funny given jokes were, as well

as rate their own ability to recognize what’s funny. The researchers found that participants were

moderately able to rate their own identification of humor, but overestimated their ability

relative to their peers (p. 1123).

2. The second study explored the logical reasoning skills of 45 undergraduate students. The

students completed 20 questions from the Law School Admissions Test, and then made three

estimates about their abilities and test performance. Participants overestimated their logical

reasoning abilities relative to their peers (pp. 1124‐1125).

3. Study 3a asked 84 undergraduates to estimate their knowledge of American standard written

grammar, to estimate their knowledge relative to their peers, and then to complete a 20‐item

grammar test. “As in studies 1 and 2, participants overestimated their ability and performance

relative to objective criteria” (p. 1125).

Kruger and Dunning conclude, “Indeed, across the four studies, participants in the bottom quartile not

only overestimated themselves, but thought they were above average” (p. 1130). They go on to say,

“Across the four sets of studies, participants in the top quartile tended to underestimate their ability

and test performance relative to their peers” (p. 1131).

In 2007, Gross and Latham used competency theory to explain the significant correlation they

discovered between incoming freshmen students’ actual performance on information literacy (IL) tasks

compared to their perceptions. Participants were 51 incoming freshman who were either in the top 25%

or bottom 25% of their high school graduating classes based in GPA and SAT scores. In addition to

testing the participants’ information literacy skills, Gross and Latham also asked them to estimate their

performance both before and after taking the test and their performance relative to other participants.

They analyzed IL scores and performance estimates by percentage, by estimated number of correct

answers, and by expectations of performance compared to others. “In all cases these variables were

positively correlated, but not to a significant degree” (343). These findings mirror those of Kruger and

Dunning as they relate to information literacy.

Study Instrument Design

Questions to test actual skills

The researchers discovered four validated instruments to test information literacy skills based on the

ACRL competencies.



1. Information Literacy Test (ILT) developed at James Madison University (Cameron et al., 2007)

2. Research Readiness Self‐Assessment (RRSA) from Central Michigan University (Ivanitskaya et al.,

2004)

3. Project SAILS from Kent State University (Kent State University, 2000‐2009)

4. ETS iSkills – Information and Communication Technology Literacy Test (ETS, 2009)

Due to high costs and access issues, the researchers were not able to use any of these instruments to

test actual information literacy skills in this study. They instead developed an instrument largely based

on validated information literacy test items from Morner (1993) and from other instruments that were

compiled by Neely (2006).

Questions to test perceptions of knowledge

The questions that were used to test perceived skills were largely based on studies conducted by Kruger

and Dunning (1999), Ehrlinger and Dunning (2003), and Gross and Latham (2007).

Gross and Latham (2007) used a simple method to test student perception of information literacy skills.

After completing the questions that tested actual information literacy skills, 58 incoming freshman at a

large state university were asked to:

1. Give a percentile estimate of their performance.

2. Estimate how many questions they thought they got right.

3. Give a percentile estimate of their performance relative to other students in the study.

For this study, the researchers used a similar method to this one.

Information Literacy within Teacher Education

Information literacy is a core component of teacher preparation—not surprising given the growing

number of content‐based professional organizations (e.g., the National Council of Teachers of English,

the National Council of the Social Studies and the National Council of Teachers of Mathematics) that

advocate for information literacy integration within all aspects of the K‐12 enterprise. The need for pre‐

service educators to demonstrate information literacy has been formalized by the nation’s primary

accrediting body, the National Council for Accreditation of Teacher Education (NCATE).

Cited in a 2008 study by Birch, Greenfield, Janke, Schaeffer, and Woods were the results of an informal

survey of College of Education faculty at the University of Arizona. Ten faculty members who teach pre‐



service educators were asked the question: “What information literacy skills do your students need to

prepare them to teach?” The following skills were identified (p. 370):

• Finding resources

• Organizing information

• Establishing priorities

• Maintaining research skills

• Keeping current

• Evaluating information quality

Results from this very small informal survey cannot be generalized, but do point to a general awareness

of information literacy among education faculty.

Information literacy standards and accreditation

In the 1990s, several education‐related professional organizations revised program standards for

teacher education in order to include information literacy competencies; among them were the National

Council for the Social Studies, the National Council of Teachers of English, and the International Reading

Association (Henderson & Scheffler, 2003). During the same time period, many university accrediting

bodies incorporated information literacy into accreditation standards. Agencies recognizing information

literacy included the Southern Association of Colleges and Schools, the New England Association of

Colleges and Schools, the North Central Accrediting Agency, and the Middle States Association

(Henderson & Scheffler, 2003).

The publication of the 2000 NCATE standards was a watershed for information literacy education, with

competence in information and technology literacy highly promoted/emphasized. In particular,

Standard 1 (Candidate Knowledge, Skills, and Dispositions) features the following behaviors that pre‐

service educators must display (Birch et al., 2008, p.370):

• Demonstrated ability to use tools and processes of inquiry

• Critical analysis

• Reflective practice

• Data collection

• Integration of technology and information literacy in instruction to support student learning



These behaviors closely mirror the ACRL standards described above. Existing information literacy

instruction, if closely tied to those ACRL standards, may be sufficient to meet the needs of candidates in

teacher education programs.

Information literacy instruction within teacher education coursework

Details of specific information literacy instruction within teacher preparation programs are scarce in the

professional literature. Henderson and Scheffler (2003) describe a few such programs within teacher

education curricula, most of which were conducted under the leadership of academic libraries.

A few years later, Birch and her colleagues (2008)identified several partnerships between teacher

education departments and college or university libraries designed to meet NCATE information literacy

and technology standards: “Partnerships between teacher education faculty and education librarians

improve the likelihood that both NCATE and ACRL competencies will be integrated with and reinforced

by content instruction and performance” (p. 370). Examples include the following:

1. The Information Literacy Project, University of British Columbia. This ongoing program allows

pre‐service educators to observe both the planning and implementation of an information

literacy class before planning one independently (Asselin & Lee, 2002).

2. Westfield State College. An education librarian and two professors of education collaborated

with 200 students to create the Pre‐Service Information Literacy Model (Birch et al., 2008).

3. Wayne State University. Librarians are paired with education faculty to team teach a research

methods course for early child education graduate students (Bhavnagri & Bielat, 2005).

Recommendations for change

Even the collaborative models described above appear librarian‐led, and many are published as case

studies within library science literature. Teacher education programs must ultimately become

responsible for integrating information literacy competencies into the curricula. These programs must

design robust assessment strategies to ensure mastery of information literacy skills, and offer ongoing

professional development opportunities related to information literacy to both pre‐service educators

and faculty alike.

Methodology

The study used a closed‐question survey to explore the possible correlation between actual and

perceived information literacy skills in pre‐service educators. The survey was designed to measure both



variables as well as collect demographic and other basic data that could be used to perform secondary

analyses if warranted. To recap, items measuring actual information literacy skills were derived from the

ACRL Information Literacy Competency Standards for Higher Education, with each mapped to a specific

performance indicator within Standard Two.

Participants

The sample for this correlation study was a convenience sample composed of students in a single, online

section of EDTEC 470: Technologies for Teaching, at San Diego State University. All are enrolled in the 5th

year credential program, seeking a variety of credentials. They complete EDTEC 470 to meet state

credentialing requirements associated with technology use and instructional integration. Sixteen (n=16)

respondents completed the survey and were included in the analysis.

Instrument Development

The instrument was developed and administered using SurveyMonkey, a subscription‐based survey‐

generation service for the web. The survey contained a total of 20 numbered items, although some were

multi‐part and/or required multiple responses. The entire survey instrument is available for review in

Appendix C.

One cluster of items was organized around demography and background. For example, respondents

were asked to indicate their educational background, undergraduate major, year bachelor’s degree was

earned (selected from a range), and the highest degree obtained. Three multi‐part items called for

respondents to categorize the frequency of their interactions with various library services such as in‐

person reference, email reference, and online tutorials. These items supported a secondary analysis of

correlation between depth and frequency of prior library interaction with actual information literacy

skills. The final background item required respondents to rate their comfort with various research tasks

such as identifying a relevant library database, using Boolean operators, or creating a reference list. The

results from this item were also useful for a secondary analysis of correlation between comfort with

research and either perceived or actual skills.

Eleven numbered items (some with multiple parts) measured each participant’s performance on one or

more ACRL performance indicators. Respondents could score a total of 29 points on this section. The

researchers divided the scores into three categories: Not Proficient (NP), Proficient (P) and Advanced

Proficiency (AP). Scores that reflected 0‐65% correct (0‐18) were NP, those that fell within 66‐90%



correct (19‐25 points) were P, and those from 91‐100% correct (26‐29 points) were AP. This

methodology was used in a similar study by Gross and Latham (2007, p. 338).

Similar to the Gross and Latham study mentioned above, the final two items on the survey measured

each respondent’s perception of how s/he performed on the skills questions. The first asked the

respondent to estimate how many questions were answered correctly, and the second item asked the

respondent to estimate percentile compared to other classmates completing the survey.

Administration

The EDTEC 470 course instructor sent an email to all enrolled students (n=41) inviting them to complete

the survey. Included in the email message were a brief overview of the study, information about the

course for which this study was designed, dates during which the survey would be active, and an email

link to the instrument in SurveyMonkey. The instructor also informed the email recipients that any

student who completed the survey could be entered into a drawing for a $10 Starbucks gift card.

Students were given two weeks in which to submit their responses. Only five people had completed the

survey by the conclusion of week one, so a reminder email was sent by the section instructor. By this

point, the number of enrolled students had by this time dropped to 38.

Contextual Factors

This section discusses the limitations the researchers encountered with the sample and instrumentation

for this study.

Study Sample

The researchers used a convenience sample of students enrolled in a single section of EDTEC 470,

Technologies for Teaching at San Diego State University. At the time that the survey was distributed,

enrollment in the section stood at 41 students, although several students withdrew from the class

before the data collection period ended. Additionally, there was a low response rate (n=16) to the

survey. While the researchers did not perform a sample size calculation for this analysis, they estimate

that the study was underpowered. Fraenkel and Wallen suggest that “the minimum acceptable sample

size for a correlational study is considered by most researchers to be no less than 30” (2009, p. 335).

Because of the small sample size in this study, correlations may not accurately describe relationships

between variables.



Instrument

Few instruments have been developed to study the correlation between actual and perceived

information literacy skills; thus, the researchers had to develop this instrument largely from scratch.

Preliminary analysis of the data suggests that a correlation might have been more accurately measured

if the scales for the two variables were the same. Actual information literacy was scored between 0 and

29, representing the actual number of items answered correctly by a participant on the skills portion of

the survey. Perceived information literacy was scored between 1 and 4, representing the participant’s

selection of a statement that most described how s/he thought he did on the skills portion. While a

correlation coefficient may be generated for any paired scores regardless of scale, the researchers

theorize that the coefficient for this study may have been different if participants were asked to guess

their actual score on the skills portion using the same (0‐29) scale. Similarly, participants were asked to

select a quartile when estimating how their scores compared to those of classmates completing the

survey. The correlation between perceived and actual percentile might have been more accurate if

participants were asked to guess their actual percentile.

Findings

This section details participant demographics as they relate to this study as well as results of correlation

analyses that compared participant test scores with perceptions of test performance.

Participant Demographics

Sixteen students enrolled in the section completed the survey by the end of the two‐week period. The

charts below illustrate the distribution of the undergraduate degrees earned by the participants by

broad subject area and year. Respondents were fairly well distributed between social sciences (n=6, or

38%), liberal studies (n=5, or 31%), and the humanities (n=4, or 25%). Eleven (69%) were recent

graduates, receiving their undergraduate degrees in 2004 or later. Only two respondents (13%)

completed any education beyond a bachelor’s degree (both indicated that they held a master’s or

professional degree).



Actual Information Literacy Skills – Whole Number Scores

Scores on the skills questions (items 8 through 18) were totaled for each survey respondent, with 29

points possible. Scores ranged from a low of 14 points to a high of 24 points. The median score was 19,

falling just into the range of points considered Proficient. The mode, or most commonly occurring score,

was 17—which for this study was considered Not Proficient.

Results suggest that pre‐service educators at the post‐baccalaureate level are not necessarily

information literate. No one scored in the range for Advanced Proficiency although nine participants

(56%) scored between 66% and 90%, placing them in the Proficient category. Seven respondents (44%);

a fairly high number, scored 65% or lower, making them Not Proficient.

Skills Perception – Estimates of Performance

The final two questions on the survey (items 19 and 20) elicited the respondents’ perceptions of how

well they performed on the skills portion of the survey. Item 19 asked the respondent to estimate the

6

5

4

1

Participants: Undergraduate Degree by Subject Area

Social Sciences

Liberal Studies

Humanities

Natural Sciences

11

4

10

2

4

6

8

10

12

after 2004 2001‐2004 1986‐1990

Participants: Year Bachelor's Degree Obtained



percentage of skills‐based questions s/he answered correctly. This closed question offered four answer

choices, scored as follows:

I think I answered 0‐25% correctly (0‐2 questions) = 1 point

26‐50% correctly (3‐5 questions) = 2 points



Only one respondent (n=1; 6%) selected 0‐25%, scoring 1 point; one respondent (6%) selected 76‐100%,

scoring 4 points. The rest of the participants (n=14; 88%) selected one of the two middle choices. The

answer selected most frequently was the third choice, with 10 respondents (63%) estimating that they

answered 51% to 75% of the skills questions correctly. Four respondents (25%) selected 26‐50%. That

88% of respondents placed themselves in one of the middle two quartiles might indicate that the

students in this sample neither underestimate nor overestimate their information literacy skills.

Correlation of Actual and Perceived Skills Each actual skills score (14 points through 24 points) was paired with the corresponding perception

score (1 through 4). The paired scores were entered into Excel so that a correlation coefficient

(Pearson’s r) could be generated.

Rounding to three decimal places, the correlation coefficient was 0.296. Fraenkel and Wallen state that

correlation coefficients below 0.35 show only a slight relationship between variables (p. 337).The two‐

tailed p‐value for this correlation is 0.27. Although the correlation is slightly positive, this p‐value

indicates a non‐significant relationship between actual and perceived information literacy skills in this

sample.



Kruger and Dunning’s competency theory, described in the literature review, hypothesizes that

competent individuals tend to underrate their abilities, while less competent people tend to overrate

their abilities when performing specific tasks. Competency theory thus suggests that any correlation of

perceived and actual skills would likely be negative. This analysis did not confirm that competency

theory was at work when pre‐service educators were asked about their perceptions of their information

literacy skills.

Actual Information Literacy Skills – Score Quartile

To explore performance more systematically, scores were assigned to quartiles. Because the final score

was a whole number and there were multiple values at several of the data points, the scores did not fall

into four evenly divided groups. The Excel Quartile function was used to determine cutoff points for

each quartile.

There were six respondents (38%) who fell into the first quartile: one score at 14 points and the five

respondents who scored 17. Three respondents (19%) scored 18 or 19 points and landed in the second

quartile. There was one score each at 20, 21, and 22 points, meaning 19% fell into the third quartile.

Three respondents received 23 total points and one scored 24, placing these four respondents (25%)

into the fourth and final quartile.

00.51

1.52

2.53

3.54

4.5

0 5 10 15 20 25 30

Perceived Inform

ation Literacy

Actual Information Literacy Skills Scores

Correlation: Actual & Perceived Skillsr=0.296; p=0.27



Skills Perception – Estimates Relative to Other Respondents

Respondents were also asked how well they thought they performed on the skills questions in

comparison to their classmates who also completed the survey. This question (item 20) was very similar

in construction and scoring to item 19:

I think I did as well as 0‐25% of the students = 1 point

26‐50% of the students = 2 points



Three respondents (19%) rated themselves in the highest quartile, scoring 4 points. Seven respondents

(44%) felt that they performed as well as 51‐75% of the students, scoring 3 points. The choice for the

second quartile – 26‐50% of the students – was selected by 5 respondents, or 31%. Only one person

(6%) estimated that they fell into the lowest quartile. The majority of participants again estimated their

relative performance in the middle two quartiles, possibly indicating a reluctance to overestimate or

underestimate performance.

Correlation of Actual and Perceived Quartiles The researchers next paired the actual quartile into which each respondent’s score fell with that

respondent’s perceived quartile. The correlation coefficient for these sets of variables was 0.514. This

can be considered a moderately positive correlation. Fraenkel and Wallen state that while correlations

of at least 0.50 allow “crude predictions…such predictions will be subject to sizable errors” (p. 337). The

p‐value for this correlation coefficient was 0.04, indicating a significant probability of a positive

6

3 3

438%

19% 19%

25%

0

1

2

3

4

5

6

7

Quartile 1 Quartile 2 Quartile 3 Quartile 4

Participant Score Quartiles (n=16)



relationship between these variables. It appears that participants were able to positively predict into

which quartiles their scores would fall.

The fact that this correlation was stronger than the prior one suggests that students were better at

judging their skills and abilities in relation to others than they were at judging their own skills

independent of any comparison standard.

Comfort with ResearchRelated Skills

One multi‐part question on the survey (Item 7) called for respondents to rate their comfort with 10

specific research‐related activities. Each activity corresponded to at least one question on the skills

portion of the instrument. Responses were scored according to degree of comfort:

Not at all [comfortable] = 1 point

Somewhat = 2 points

Moderately = 3 points

Very = 4 points

Extremely = 5 points

With 10 activities scored at a maximum of five points each, respondents could score a composite 50

points on this research comfort scale. Scores ranged from a low of 23 to a high of 50, with the median

score at 36. The mode was also 36.

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0 1 2 3 4 5

Perceived Qua

rtile

Actual Quartile

Correlation: Actual & Perceived Quartiler=0.514; p=0.04



Respondents indicated the greatest degree of comfort with basic library‐related tasks, such as finding

out whether the library had a specific book. Eight people (50%) indicated that they were “extremely

comfortable” with this task. The research activity that was least comfortable for this sample asked about

the respondent’s ability to use wildcards in a database search. Nine people (56%) were “not at all

comfortable” with this task, and only one person was “extremely comfortable.”

Appendix A includes a chart that illustrates how participants’ rated their comfort with various activities.

Correlation of Actual Skills and Research Comfort The researchers paired the actual skills scores (14 points through 24 points) with the corresponding

research comfort scores, which ranged from 23 to 50. This correlation was the weakest in this study

(Pearson’s r = 0.090). The two‐tailed p‐value for this r value is 0.74.

The lack of correlation between a respondent’s information literacy skills and his or her comfort with

various research tasks has many implications for information literacy instruction. Many campuses make

information literacy instruction available through optional tutorials or workshops on specific research

tasks. When instruction is optional, students might select only those tutorials or workshops for research

tasks with which they feel uncomfortable. This analysis hints that a student’s comfort level with a task

may not indicate a level of proficiency or competence with that same task. Mandatory instruction is one

solution to the lack of correlation.

0

10

20

30

40

50

60

0 5 10 15 20 25 30

Research Com

fort Score

Actual Information Literacy Skills

Correlation: Actual Score & Research Comfortr=0.09; p=0.74



Library Use

Library use was measured in a single question (Item 4) asking respondents to rate the frequency of

usage of three types of libraries: public, university, or specialized. For each of the three library types,

scoring was as follows:

At least one time per week = 3 points

At least one or two times per month = 2 points

At least one or two times per semester = 1 point

Can’t recall = 0 points

Never = 0 points

Appendix B includes a chart that illustrates how often participants used these types of libraries.

Points for each library type were totaled to give a composite library use score. Scores for the 16

respondents ranged from 0 to 6, with a median of 2.5 points.

Correlation of Actual Skills and Library Use For this final correlation analysis, the researchers paired each actual skills score with the respondent’s

library use score. As described above, this score ranged from 0 to 6 in the sample. Pearson’s r for this

correlation was 0.443, indicating a moderately positive correlation. For two‐tailed probability, p=0.09.

The researchers hypothesized that the frequency of library visits would have a positive impact on

information literacy skills. This correlation coefficient r indicates there is some positive relationship

0

1

2

3

4

5

6

7

0 5 10 15 20 25 30

Library Use

Actual Information Literacy Skills

Correlation: Actual Score & Library User=0.443; p=0.09



between library usage and information literacy, although the p value implies that the significance of this

relationship may be no stronger than random.

Discussion

Overall, the results of this study do not support the results found in other correlation studies completed

by Kruger and Dunning (1999) and Gross and Latham (2007). As mentioned previously, this may be in

large part due to a low response rate and the instrument design. A study that includes a larger sample

(preferably over 30) may yield more accurate correlations. In addition, an instrument that asks

participants to rate their performance based on percentages instead of quartiles would allow the data to

be correlated more accurately.

The researchers were surprised by the number of participants (n=8) who correctly perceived the

percentage of questions they answered correctly, and the number of participants (n=7) who accurately

estimated the quartile into which they fell. Prior research related to competency theory suggests that

this number would be lower compared to the total included in the study sample. The researchers

hypothesize that participants’ perceptions might be different if the “metatopic”—in this case,

information literacy – was one in which students had more working knowledge. For example, the Kruger

and Dunning (1999) study asked participants to rate their abilities to identify humor and grammar. Many

people perceive themselves as funny and having command of the English language. This study shows

that the participants feel “somewhat” or “moderately” comfortable to perform information literacy

tasks (Appendix A). It is possible that there would be a greater difference in the actual versus perceived

performance if participants were tested on a more common topic or one in which knowledge is

considered desirable.

The research comfort score was derived from responses to the research tasks listed in Item 7. Each of

these tasks maps to at least one skills question in the survey. Further analysis of the correlation

between the tasks in Item 7 and the corresponding skills questions might also prove interesting. For

example, can a participant who feels “extremely comfortable” finding a book in a library catalog actually

do so? This research question was not within the scope for this study, but may yield interesting results.



Conclusion

Higher education is increasingly interested in understanding students’ ability to attain information

literacy skills. Major associations in librarianship and teacher education already recognize the need for

students to learn these skills. For the past decade, librarians have been at the center of teaching and

assessing such skills. Librarians and faculty must partner to provide information literacy instruction that

is integrated into the curriculum. Identifying students’ actual information literacy skills and perceived

ability to complete such tasks will provide a place for librarians and faculty to address specific

information literacy skills in their own institutions.

EDTEC690: Brown/Turnbow 23

Appendix A: Results for Question 7

13

9

1 12 31

5

2

3 22 1

4

6 5

5

3

4

44

5

3

3

5 5

4

2

5 7 36

8

3 36

31

4 35 5

How comfortable are you doing the following things when you do research? (n=16)

extremely

very

moderately

somewhat

not at all


Appendix B: Results for Question 4

1

32

44

4

1

4

3

5 5

12

0

2

4

6

8

10

12

14

16

18

Public Library Academic Library Specialized Library

During this academic year, how often have you visited one or more of the libraries listed below? (n=16)

Never

Can't recall

At least 1x or 2x/semester

At least 1x or 2x/month

At least 1x/week


References

Asselin, M. M., & Lee, E. A. (2002). "I wish someone had taught me": Information literacy in a teacher education program. Teacher Librarian, 30(2), 10‐17.

Association of College & Research Libraries. (2000). Information literacy competency standards for higher education. Chicago, IL: American Library Association.

Bhavnagri, N. P., & Bielat, V. (2005). Faculty‐librarian collaboration to teach research skills: Electronic symbiosis. Reference Librarian, 43(89), 121‐138.

Birch, T., Greenfield, L., Janke, K., Schaeffer, D., & Woods, A. (2008). Partnering with librarians to meet NCATE standards in teacher education. Education, 128(3), 369‐379.

Cameron, L., Wise, S. L., & Lottridge, S. M. (2007). The development and validation of the information literacy test. College & Research Libraries, 68(3), 229‐236.

Ehrlinger, J., & Dunning, D. (2003). How chronic self‐views influence (and potentially mislead) estimates of performance. Journal of Personality and Social Psychology, 84(1), 5‐17.

ETS. (2009). iSkills assessment ‐ information and communication technology literacy test. Retrieved March 13, 2009, from http://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=159f0e3c27a85110VgnVCM10000022f95190RCRD&vgnextchannel=e5b2a79898a85110VgnVCM10000022f95190RCRD

Fraenkel, J. R., & Wallen, N. E. (2009). How to design and evaluate research in education (7th ed.). New York: McGraw‐Hill.

Geffert, B., & Christensen, B. (1998). Things they carry: Attitudes toward, opinions about, and knowledge of libraries and research among incoming college students. Reference & User Services Quarterly, 37(3), 279‐289.

Gross, M., & Latham, D. (2007). Attaining information literacy: An investigation of the relationship between skill level, self‐estimates of skill, and library anxiety. Library & Information Science Research, 29(3), 332‐353.

Henderson, M. V., & Scheffler, A. J. (2003). New literacies, standards, and teacher education. Education, 124(2), 390‐395.

Hernon, P., & Dugan, R. E. (2004). Outcomes assessment in higher education : Views and perspectives. Westport, CT: Libraries Unlimited.

Ivanitskaya, L., Laus, R., & Marie Casey, A. (2004). Research readiness self‐assessment: Assessing students' research skills and attitudes. Journal of Library Administration, 41(1), 167‐183.


Kent State University. (2000‐2009). Project SAILS (standardized assessment of information literacy skills). Retrieved March 13, 2009, from https://www.projectsails.org

Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self‐assessments. Journal of Personality and Social Psychology, 77(6), 1121‐1134.

Maughan, P. D. (2001). Assessing information literacy among undergraduates: A discussion of the literature and the University of California‐Berkeley assessment experience. College & Research Libraries, 62(1), 71.

Morner, C. J. (1993). A test of library research skills for education doctoral students. (Doctoral dissertation, Boston College, 1993).

Neely, T. Y. (2006). Information literacy assessment: Standards‐based tools and assignments. Chicago: American Library Association.

Ren, W. (2000). Library instruction and college student self‐efficacy in electronic information searching. Journal of Academic Librarianship, 26(5), 323.

Turnbow, D. (2009a). BENG1 tutorial. Retrieved March 13, 2009, from http://biomed.ucsd.edu/dturnbow/bioeng1_aftersessions/beng1_tutorial_aftersessions/player.html

Turnbow, D. (2009b). Getting started: PubMed. Retrieved March 13, 2009, from http://biomed.ucsd.edu/dturnbow/beng1_pubmed/

EDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 Survey

We are EDTEC graduate students enrolled this semester in a research class. Both the study we're conducting and this survey focus on

the information literacy skills of preservice educators like you. We hope you're willing to participate; you'll need about 10 to 15

minutes to answer all the questions--and both your anonymity and confidentiality are assured.

We are using a definition of information literacy provided by the Association for College & Research Libraries (ACRL):

"Information literacy is a set of abilities requiring individuals to recognize when information is needed and have the ability to locate, evaluate,

and use effectively the needed information." Basically this means that you know how and where to find and use information when you

have a specific question or research need. This survey is focused on your ability to find information.

We would like you to answer each question. However, it is never a problem and you shouldn't worry if you don't know or aren't sure of

an answer. Just select "I don't know" and move to the next question.

Everyone who completes the survey by April 17 will be entered into a drawing for a $10 Starbucks gift card. There are instructions at

the end of the survey for submitting your name for the drawing.

If you have questions about this study you may contact one of the researchers via e-mail - Dominique Turnbow or Marcy Brown.


1. Please select the category that best describes your undergraduate major.

2. When did you graduate from college with your bachelor's degree?

3. Please indicate your highest degree obtained?

4. During this academic year, how often have you visited one or more of the libraries listed below?

5. Think about your interactions with librarians while you were getting your undergraduate degree. Below, there are a variety of ways that you may have had contact with a librarian or library staff member. For each one, tell us whether or notit applies to you (choices=y/n). Then tell us how often you did this. [Choose Not applicable if your answer to the first question was No.]

Getting to know you

Please tell us a little about yourself and how you use libraries.

*

*

*

*

At least 1x/weekAt least 1x or

2x/month

At least 1x or

2x/semesterCan't recall Never

Public library nmlkj nmlkj nmlkj nmlkj nmlkj

University library nmlkj nmlkj nmlkj nmlkj nmlkj

Specialized library nmlkj nmlkj nmlkj nmlkj nmlkj

*

Did you do this? If yes, how often?

Asked a question at a service desk at the library

Asked a question via e-mail, chat, or txt message

A librarian came to my class and did a workshop

Went to the library for a workshop

Used an online tutorial on the library's web page to

learn how to search the catalog or a database

Arts (visual and performing)

nmlkj

Humanities (e.g. languages, literature, history)

nmlkj

Liberal Studies

nmlkj

Physical or Life Sciences (e.g. biology, chemistry, engineering)

nmlkj

Social Sciences (e.g. psychology, geography)

nmlkj

After 2004

nmlkj

Between 2001 and 2004

nmlkj


nmlkj


nmlkj


nmlkj

Before 1986

nmlkj

Bachelor's degree

nmlkj

Post-graduate certificate

nmlkj

Professional/Master's degree

nmlkj

Other

nmlkj

EDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 Survey6. Think about your experiences using a library "service" for a college-level project, paper or other assignment you had to complete. Several library services (and reasons for using them) are listed below. For each one, first indicate whether or notit applies to you (choices=yes/no). Then tell us how often you took advantage of it. [Choose Not applicable if your answer to the first question was no.]

*

Did you do this? If yes, how often?

Used the library catalog to find books,

encyclopedias and handbooks

Used library databases to find scholarly journal

articles

Went to the library for help identifying relevant

research articles or books

Went to the library to get help with a reference list

or bibliography


7. How comfortable are you doing the following things when you do research?

Comfort level with research skills

* not at all somewhat moderately very extremely

Finding out if your library has a

particular booknmlkj nmlkj nmlkj nmlkj nmlkj

Understanding when to use a library

catalog or article databasenmlkj nmlkj nmlkj nmlkj nmlkj

Identifying the online databases that

match your topic of interestnmlkj nmlkj nmlkj nmlkj nmlkj

Identifying keywords/phrases for your

research topicnmlkj nmlkj nmlkj nmlkj nmlkj

Using Boolean operators (AND,OR,NOT)

as part of a database searchnmlkj nmlkj nmlkj nmlkj nmlkj

Using truncation or wildcards (*) as

part a database searchnmlkj nmlkj nmlkj nmlkj nmlkj

Modifying a database search when you

get too many or too few resultsnmlkj nmlkj nmlkj nmlkj nmlkj

Identifying the most relevant results

for your research topic after searching

a database

nmlkj nmlkj nmlkj nmlkj nmlkj

Getting the full-text of an article for

free or a small fee from a librarynmlkj nmlkj nmlkj nmlkj nmlkj

Creating a reference list or bibliography

for your paper using an appropriate

style (e.g. APA, MLA)

nmlkj nmlkj nmlkj nmlkj nmlkj


8. The service offered in the Library that allows you to get almost any publication you need is called:

9. If you are looking for a specific journal article and have the publication information, you can use:

10. What types of information can you find in the SDSU Library subscription databases? (Select all that apply.)

Doing research at SDSU

The questions on this page will ask you about doing research at San Diego State University. Remember, it is OK to select "I don't

know" if you are unsure of the answer.

*

*

*

Interlibrary loan

nmlkj

Full text

nmlkj

Reference

nmlkj

Reserves

nmlkj

I don't know

nmlkj

Citation linker

nmlkj

Digital projects page

nmlkj

Electronic course reserves (ECR)

nmlkj

None of the above

nmlkj

I don't know

nmlkj

conference proceedings

gfedc

dissertations

gfedc

full text of journal articles

gfedc

journal article citations

gfedc

all of the above

gfedc

I don't know

gfedc


11. Below are pieces of information needed for various class assignments. Match each specific information need to the type of library source most likely to meet that need. Note: you will not use every source type.

12. Which database would be most likely to contain articles about bilingual education?

Research tasks

The next two questions will ask you about different sources and why you might use them for research. Remember, it is OK to select

"I don't know" if you are unsure of the answer.

*

Sources

Magazine article about vitamin supplements

Titles of books written by B. F. Skinner available in

the library

Census data for San Diego county

Daily accounts of the Wall Street bankruptcies

*

Academic Search Premier (EBSCO)

nmlkj

Applied Science and Technology Abstracts

nmlkj

ERIC

nmlkj

Historical Abstracts

nmlkj

PubMed

nmlkj

I don't know

nmlkj


13. You searched for articles about distance education and don’t think you found enough information. What synonyms might you use to search further? (Select all that apply.)

14. Which of the following search statements would you use in a database to find records that would include either of these terms: leadership; managerial skills

15. What is your next logical step when your search yields 2,000 citations?

16. The following is a list of strategies a researcher can use to narrow or broaden a search. For each strategy, indicate which would be used to find fewer results (narrow) or find more results (broaden).

Search strategies

The questions on this page will ask you about creating search strategies to find information about a topic. Remember, it is OK to

select "I don't know" if you are unsure of the answer.

*

*

*

*

narrow (fewer results) broaden (more results) I don't know

add more search terms with OR nmlkj nmlkj nmlkj

limit to a specific format (i.e. journal articles, book

chapters, etc.nmlkj nmlkj nmlkj

limit to date published nmlkj nmlkj nmlkj

use synonyms in the same search nmlkj nmlkj nmlkj

add specific author name(s) nmlkj nmlkj nmlkj

use truncation (when you substitute the last letter(s)

of a word with a symbol like "*" or "$", e.g.

manage*)

nmlkj nmlkj nmlkj

Distance training

gfedc

Online education

gfedc

Online training

gfedc

Web-based training

gfedc

All of the above

gfedc

None of the above

gfedc

I don't know

gfedc

Leadership AND managerial skills

nmlkj

Leadership OR managerial skills

nmlkj

Leadership NOT managerial skills

nmlkj

I don't know

nmlkj

Add another term to the search using AND

nmlkj

Add another term to the search using OR

nmlkj

Add another term to the search using NOT

nmlkj

Look at all of the citations in order to be thorough

nmlkj

Look at the first few citations and just use them if they are relevant

nmlkj

I don't know

nmlkj


17. The image below is a citation from a library database. Select the "element" from the drop-down menu that best describes the part illustrated in the citation.

Citation from a library database

18. The following are citations for different types of publications. Mark which publication type matches each citation. Note: you will not use every publication type.

Managing information

The next two questions will ask you about how to create reference lists and analyze the information you find. Remember, it is OK to

select "I don't know" if you are unsure of the answer.

*

Citation element

author

title of journal

volume

pages

issue

date

title of article

*

bookbook

chapter

conference

proceeding

journal

article

magazine

articlenewspaper

I don't

know

Koepke, M. (1991). Expertise for sale: Countless

organizations use teachers as consultants. Teacher

Magazine, 3(1), 30.

nmlkj nmlkj nmlkj nmlkj nmlkj nmlkj nmlkj

Foster, N. F., & Gibbons, S. (2007). Studying Students:

The Undergraduate Research Project at the University of

Rochester. Chicago: Association of College and

Research Libraries.


Kierniesky, N. C. (2005). Undergraduate research in

small psychology departments: Two decades later.

Teaching of Psychology, 32(2), 84-91.


Lopatto, D. (2004). Survey of undergraduate research

experiences: First findings. Cell Biology Education, 3(4),

270-275.


Duignan, P. A. (2006). Key challenges for educational

leaders. In M. L. Brown, Educational Leadership: Key

Challenges and Ethical Tensions (pp. 21-40). Cambridge,

UK: Cambridge University Press.



19. There were 11 questions on this survey focused on information literacy skills. How many do you think you answered correctly?

20. Now, how well do you think you did compared to other students enrolled in yourEDTEC 470 course?

How do you think you did?

Now, we are interested in how you think you did on the research questions.

*

*

I think I answered 0-25% correctly (0-2 questions)

nmlkj


nmlkj


nmlkj


nmlkj

I think I did as well as 0-25% of the students.

nmlkj


nmlkj


nmlkj


nmlkj

Actual Perceived Skills in Pre Educatorslibraries.ucsd.edu/bmcl/dturnbow/finalpaper_brown... ·...

Documents

Transcript of Actual Perceived Skills in Pre Educatorslibraries.ucsd.edu/bmcl/dturnbow/finalpaper_brown... ·...