Understanding our First Years
description
Transcript of Understanding our First Years
![Page 1: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/1.jpg)
Understanding our First YearsThree studies and a comparison
![Page 2: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/2.jpg)
Understanding our First Years
Karen Brodsky, Instruction and Outreach Coordinator, Sonoma State University
Linda J. Goff, Head of Instructional Services,California State University, Sacramento
Gabriela Sonntag, Coordinator, Information Literacy Program, California State University, San Marcos
Sharon Hamill, Professor of Psychology, California State University, San Marcos
CARL Conference, Friday April 4, 2008, Irvine CA
![Page 3: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/3.jpg)
Agenda – Part One
• Overview -handouts
• Definition of iSkills
• Demographics
• Similarities and Differences in campus projects
![Page 4: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/4.jpg)
Our Three Projects - Similarities
• Focused on Entering Freshman
• Each a 2 year project
• Common Goal – establish baseline under-standing of first-year students’ knowledge of information competence/literacy
• Used iSkills as pre and post test
![Page 5: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/5.jpg)
Sacramento - Differences• 132 F’06 & 107in F’07 (239)
did both pre & post test • Faculty groups worked on
InfoLit lesson plans• Intervention lesson was one
class session• Student incentive of $50 gift
certificateTesting at Sacramento
![Page 6: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/6.jpg)
San Marcos - Differences• Freshman Seminars and
Oral Communication• 3 weeks IL vs. 1 hour IL• Institutional data
gathered included analysis of retention factors
![Page 7: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/7.jpg)
Sonoma - Differences
• Year-long class over 2 pilot years (A2, A3)
• ~ 8 sessions per class over the year
• Librarians integral in curriculum development
• Faculty Development Instruction at Sonoma
![Page 8: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/8.jpg)
Agenda – Part Two
• Data review
• Test scores
• ETS reports
![Page 9: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/9.jpg)
Sacramento Pre/Post-test ScoresFall2006 Fall2007
![Page 10: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/10.jpg)
Sacramento Scores• 2006 Aggregate ETS reports indicate
strongest need for instruction in skill areas: Evaluate, Create and Communicate
• 2007 scores included Honors classes
• 2007 Aggregate ETS reports therefore slightly higher
![Page 11: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/11.jpg)
Sac - Comparison of Average ScoresEOP – HonorsHonors- GE classes
0
50
100
150
200
250
300
2006
Pre
2006
Post
2007
Pre
2007
Post
EOP (1+1)
Honors(0+2)
GE (5+2)
![Page 12: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/12.jpg)
San Marcos Pre/Post-test ScoresGEL(2006) GEO (2006)
![Page 13: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/13.jpg)
San Marcos Scores• Lesson plans were not matched to test
content• All students show considerable improvement.• GEO students outperformed GEL• Students not needing remediation take GEO
in Fall
![Page 14: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/14.jpg)
0
10
20
30
40
50
60
70
80
90
Pe
rce
nt
Co
rre
ct
Pre Test
Post Test
GEL pre and post test scores
Access
0
20
40
60
80
100
120
Chosecorrect
database onfirst search
Selected allof the
appropriateitems for the
customer
Used searchterms precise& useful in
Websearches
Rankedpotential
alternativesearches
accurately
Skills
% co
rrec
t Pre-Test
Post-Test
Evaluate
0
10
20
30
40
50
60
70
80
Chose bestWeb pages
as mostuseful forresearchproject
Judgedrelevance
Judgedauthority
Judgedpoint of
view
Judgedcurrency
Selectedbest sites
for researchtopic
Judged authority
Judged bias
Skill
% C
orre
ct
GEO pre and post test scores
0102030405060708090
Skill
perc
ent c
orre
ct
pre-test
post-test
![Page 15: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/15.jpg)
Comparing GEL and non-GEL
Fall 2000 Entrants Fall 2001 Entrants Fall 2002 Entrants Fall 2003 Entrants Fall 2004 Entrants
Percent Number Percent Number Percent Number Percent Number Percent Number
Cumulative GPA
All GEL 101 Students 2.73 (305) 2.70 (307) 2.59 (487) 2.66 (486) 2.73 (344)
Non-GEL 101 2.75 (204) 3.14 (176) 2.73 (258) 2.73 (331) 2.73 (317)
One Year Continuation Rates
All GEL 101 Students 72.0 (321) 69.7 (323) 76.7 (520) 76.2 (509) 71.9 (358)
All Freshmen* 60.0 (575) 62.0 (545) 70.7 (837) 71.0 (890) 69.5 (722)
Two Year Continuation Rates
All GEL 101 Students 65.1 (321) 59.8 (323) 63.5 (520) 66.2 (509) n/a n/a
All Freshmen* 51.3 (575) 52.7 (545) 58.4 (837) 60.6 (890) n/a n/a
![Page 16: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/16.jpg)
Pre - Fall 2006• 103 participants (68%)• Scheduled outside class• Required for “participation
points”• Slightly above average –
except for “Define”
Post - Spring 2007• 44 students participated
– 33 students repeat• Results not statistically
relevant• Scheduled out class• Required for “participation
points”
Sonoma Year Fall 2006 pre/post test
![Page 17: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/17.jpg)
Sonoma – Year 2 pre/post testFall 2007• 144 participated (84%)• Scheduled during Class• Slightly above average
- better on “Define”
Spring 2008
• No post-test
![Page 18: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/18.jpg)
Sonoma – pre-test comparison
Fall 2006– Slightly lower in “Define”– Slightly higher in “Access,” and “Create”
Fall 2007– Slightly higher in “Define” and “Integrate”
Both groups equivalent for “Evaluate”
![Page 19: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/19.jpg)
Agenda – Part Three• Have you assessed first
years? • Have you used the
iSkills test?• What other instruments
were used? • What were your
experiences?
![Page 20: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/20.jpg)
Sacramento conclusions
![Page 21: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/21.jpg)
iSkills is not based on ACRL-IL
• Lesson plan objectives based on ACRL Standards: (5.2.f- plagiarism) and (3.2.a evaluation of bias)
• Mismatch with iSkills test content – lessons and Standards
![Page 22: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/22.jpg)
Problem with Data-driven Decisions
• Aggregate data available from ETS only for 100+ users
• Disconnect – class-level data not available for faculty
• Statistical significance vs. practical significance?• One lesson intervention isn’t enough
![Page 23: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/23.jpg)
Faculty Feedback
“ETS needs to give us a results section that includes students weaknesses and strengths so that we can apply it to the development of lesson plans that will adequately address areas that need remediation.” RG, Sacramento
![Page 24: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/24.jpg)
Outcomes (Sacramento)• Freshman Programs Director and FS
faculty have developed stronger working rapport with librarians and positive influence on information literacy program.
• Data useful for Freshman Program assessment and re-design
![Page 25: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/25.jpg)
San Marcos Conclusions
![Page 26: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/26.jpg)
One Year Analysis• Sample as a whole scored higher on post-test.• GEO students outperformed GEL students but
most variance accounted for by the pre-test• Only prior difference between groups was HS
GPA (not EPT, ELM, or SAT)• No difference in college GPA, units completed,
units enrolled in or likelihood of enrolling
![Page 27: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/27.jpg)
Further Analysis• GEL students more likely to
– Go to the career center and math lab– Go see a professor during office hours and outside of
class• GEO students more likely to
– Have a job• No difference in going to the writing center,
visiting with an academic advisor or going to the academic advising web page
![Page 28: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/28.jpg)
Sonoma Conclusions
![Page 29: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/29.jpg)
iSkills - What we learned• SSU students scored near average or above in
all categories • Can’t assume entering classes will score the
same• Pre-test used to scaffold assignments, faculty
training, etc.• “Evaluation” was focus of several sessions• Other assessment confirmed some of iSkills
results – “Define”
![Page 30: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/30.jpg)
iSkills - What else we learned• iSkills could be one more indicator – must
have multiple assessment tools
• Question of correlation of categories
• Pre/post test didn’t work for us
• Could be more useful tied to entire freshmen program -- not this particular class
![Page 31: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/31.jpg)
Sonoma -- Outcomes• Commitment to working with first year
students.
Fall planning . . .
• What do we really want our students to learn?
• What do we really want our faculty to learn?
![Page 32: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/32.jpg)
Our Combined Conclusions
![Page 33: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/33.jpg)
Overall Outcomes
• We have established a baseline measure of freshman ICT/iSkill, but not of InfoLit
• All 3 Libraries have developed stronger working relationships with Freshman Programs and grants have had positive influence on information literacy programs
• Data useful for Freshman Program assessment and re-design
![Page 34: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/34.jpg)
Overall Conclusions
• Programmatic data not individual or by class
• Best used as a diagnostic
• Advanced iSkills test could be given to Major in upper division
• Faculty and students have greater awareness of Information Literacy
• Timing of testing critical
• Student motivation for assessment is lacking
About iSkills About First Years
![Page 35: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/35.jpg)
Overall Conclusions
• More appropriate to test through Assessment Office or Testing Center
• Multiple measures are needed• Recommitted to working with
freshmen programs
![Page 36: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/36.jpg)
Agenda – Part Four
• Next steps
• Your questions
![Page 37: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/37.jpg)
Sacramento -- Next Steps• Will add new First Year/Outreach
Librarian position• Will look at what worked at San Marcos
and Sonoma IL programs• Will invite other CSU’s to attend our Fall
2008 Freshman Seminar convocation, with sessions on FS/IL
![Page 38: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/38.jpg)
San Marcos -- Next Steps• Reviewing our homegrown Computer
Competence Requirement exam• Continue to analyze and review multiple
sources of data• Continue to revisit our curriculum for FY
students throughout GE
![Page 39: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/39.jpg)
Sonoma -- Next Steps• FYE class approved by Academic Senate
• Developing online modules for basics
• iSkills won’t be administered through the library
• Continue integrating IL with freshmen curriculum across campus
![Page 40: Understanding our First Years](https://reader035.fdocuments.in/reader035/viewer/2022081516/568145b1550346895db2b081/html5/thumbnails/40.jpg)
Understanding our First Years
Karen Brodsky, Instruction and Outreach Coordinator, Sonoma State University
Linda J. Goff, Head of Instructional Services,California State University, Sacramento
Gabriela Sonntag, Coordinator, Information Literacy Program, California State University, San Marcos
Sharon Hamill, Professor of Psychology, California State University, San Marcos
CARL Conference, Friday April 4, 2008, Irvine CA