Using Classroom Assessment to Determine Student Cognitive

Development LevelsLia Vella

Colorado School of MinesJune 28, 2015

STS Research Forum, ALA Annual Conference

Hypothesis

• Anonymous written feedback collected during classroom assessment activities can be used to characterize students’ cognitive development levels.

Background

– Senior Survey– Feedback from

Senior Design faculty

– Assessment feedback from various IL sessions

• Information literacy instruction at Mines is not as effective as we would like

Cognitive Development ResearchCognitive Development PositionBased on Perry (1970), Jackson (2008), Rapaport (2013)

Characteristic Beliefs

Positions 1-2 Simple Dualism

The world is black and white. There is a right answer to every question. Authorities know the right answer, and they are supposed to teach what the answer is.

Position 3 Complex Dualism

Authorities may not agree about the correct answer, or it may be that no one knows the correct answer, but a “right answer” does exist. Therefore, “everyone is entitled to their own opinion.”

Position 4 Late Multiplicity

Ideas can be better or worse rather than right or wrong. Students may think critically because they perceive that is “what the professors want.”

Positions 5-6 Relativism

Solutions to problems can be shades of gray rather than black and white; philosophical or speculative questions can be asked and understood

Positions 7-9 Commitment in Relativism

Commitments to answers may be made based on one’s experiences and choices; understanding that commitment implies continued pursuit of knowledge despite tensions around unknowns or conflicts

• William Perry (1970)– longitudinal study of Harvard students

• Mary Belenky, et al. (1987)• Patricia King and Karen

Kitchener (1994)

• Michael Pavelich, et al. (1995-2000)– engineering student cognitive development• Marra & Palmer (2000)

• Marlene Schommer (1995)– developed revised cognitive development model and questionnaire

• Jeffrey Gatten (2003), Rebecca Jackson (2008)-- applied cognitive development theories to Information Literacy instruction

“Minute Papers” Classroom Assessment

Methods

• Develop Codebook based on previous semesters’ feedback

• Collect feedback from test subjects• Code responses to “Left wondering” questions

based on Codebook• Record and visualize data

Levels Based on CodebookLevel Characteristic beliefs Sample questions

Low (simple dualism)

black and white answers and authorities who are always correct--there must be a “right” answer for every problem or a correct way to search for the right answer

-What is the minimum ranking in either category [scholarly, authoritative] that can be used as a valid source?-Where exactly am I going to be able to find exact resources?-I wonder if it is better to always use an article instead of a website?

Medium (complex dualism)

acceptance of disagreement between authorities, but correct answers exist; want to know how determine which of the multiple answers are to be believed

-What is the most efficient way to find all of these scholarly articles?-I am still worried about conflicting information and which to trust more, and … how to compare reputabilities.-How can I better determine if an article or website is … giving me information I want?

High (Multiplicity- Relativism?)

see solutions to problems in shades of gray rather than in black/white or good/bad terms-- more likely to ask questions that are philosophical or speculative in nature

-Can articles be partially scholarly in the sense that one part of an article is extremely useful but the rest is inaccurate?-…non-published sources. How should we consider asking a professional weighted against published works in credibility?

Results

Sum of Low25%

Sum of Medium22%

Sum of High10%Sum of Procedural

4%

Sum of No Response39%

Cognitive Levels Breakdown

Sum of LowSum of MediumSum of HighSum of ProceduralSum of No Response

Results, ctd

Sum of Low42%

Sum of Med35%

Sum of High16%

Sum of Procedural7%

Cognitive Levels Breakdown without No-Question Responses

Sum of LowSum of MedSum of HighSum of Procedural

A-B C E-F G-H I-J K-L M-N O-P S-T W-X Y-Z0

5

10

15

20

25

30

35

40

45

50

Sum of No QuestionSum of ProceduralSum of HighSum of MedSum of Low

Results by class section

What I Learned

• Use checklists and ranking exercises with caution

• “Correct way”/”Better Way” questions present a learning opportunity

• Identify Procedural and other common questions for planning purposes

• Extend learning by collecting student feedback and returning written answers

Other Considerations

• How does this research relate to ACRL’s new Information Literacy Framework? – IL Frames provide guidelines for approaching

instruction with lower-level students– The threshhold concept idea may “map” more

smoothly to cognitive development positions (a là Perry)

Conclusions

• General cognitive development levels were determined for test population

• Minimal time commitment compared to other researchers’ methods

• Librarians would not have to complete all the steps in this research study in order to benefit from this methodology

Questions for future work

• Are engineering students different from students in other fields, in terms of college-level cognitive development?

• Is there a gendered difference in cognitive development for STEM students?

• How does/can information literacy instruction affect the development of cognitive levels?

• Can the new Information Literacy Framework be used in concert with cognitive development models to teach IL more effectively? How?

Questions?

lvella@mines.edu • http://inside.mines.edu/~lvella/