NSSE 2013 – Overview of Results
description
Transcript of NSSE 2013 – Overview of Results
NSSE 2013 – Overview of Results
Jerre Cline, Reporting Analyst,Office of Institutional Research
Joshua Powers, Associate Vice Presidentfor Student Success
NSSE overview Engagement research & NSSE as an
independent variable NSSE at ISU
◦ 2013 NSSE Administration◦ ISU Results and in Comparison Context◦ Implications and Discussion
Goals of Session
Product of Indiana University’s Center for Postsecondary Research
in the IU-Bloomington School of Education. Part of a larger survey group that includes
the BCSSE and the FSSE. Measures empirically confirmed “good
practices” in undergraduate education related to achieving positive outcomes.
Not a tool that measures student learning.
What is the NSSE?
StudentSuccess Conference 2013
First administered to colleges and universities in 2000.
Indiana State University began participating in the NSSE in 2003.
Revised Survey in 2013 – NSSE 2.0. 4 million college students and 1500+
institutions of higher education have utilized the NSSE since it’s inception. In 2013, 613 institutions and 364,000
students participated in NSSE 2.0.
NSSE History
StudentSuccess Conference 2013
Amount of time and effort that students put towards their studies.
Effective institutional practices provided to students.
Related to student participation in extracurricular activities.
Affected by instructional strategies.
Student Engagement
StudentSuccess Conference 2013
10 official peers participated in NSSE 2.0. DRU - Basic Carnegie Classification. 613 institutions participated in NSSE 2.0. Accepted by Voluntary System of
Accountability and used in College Portraits; aimed at assisting students and parents in selecting a college.
Institutional Comparisons
StudentSuccess Conference 2013
Devoting substantial time and effort, caring about work quality, feeling the work has significance = engagement (Newmann, 1986).
Collaborative learning and experiential learning + accessible & relevant curriculum increase student engagement (Akey, 2006; Heller et al., 2003).
High quality instruction (rigorous; aligned with content standards; using instructional strategies to meet academic needs of all students – key factor in promoting culture of engagement and achievement in the classroom (Weiss & Paisley, 2004).
Engagement Research Findings
StudentSuccess Conference 2013
NSSE results good proxy measures for growth in important educational outcomes (Pascarella et al., 2010).
No relationship between total NSSE scores & GPA or graduation rates (DiRamio & Shannon, 2010).
NSSE Benchmarks failed to meet validity or reliability standards (Campbell & Cabrera, 2010; Porter, 2009).
NSSE benchmarks valid & reliable indicators for quality of student experience for historically underserved students; engagement had stronger effects on 1st year grades and persistence to the 2nd year for underprepared students (Kuh, et al., 2007, 2008).
NSSE 2.0 developed partly in response to research findings; some indicating students didn’t fully understand questions being asked (ambiguous verbiage; unknown contextual interpretation impact).
NSSE Research
StudentSuccess Conference 2013
NSSE 2013 Administration at ISU
Submitted population
Total respondents
2,789
First-year Senior
1,763
633637
Survey completions
Response Rate: 23% 36%
Message schedule
First-yearInvitationReminder 1Reminder 2Reminder 3Final reminder
31%36%
20%23%
02/27/201303/05/2013
DateCumulative response rate
9%18%27%
7%13%17%02/21/2013
02/05/201302/13/2013
Senior
StudentSuccess Conference 2013
Engagement Indicators: Overview
▲ Your students’ average was significantly higher (p<.05) with an effect size at least .3 in magnitude.
△ Your students’ average was significantly higher (p<.05) with an effect size less than .3 in magnitude.
-- No significant difference.
▽ Your students’ average was significantly lower (p<.05) with an effect size less than .3 in magnitude.
▼ Your students’ average was significantly lower (p<.05) with an effect size at least .3 in magnitude.
First-Year (FY) Students
Theme Engagement Indicator
Higher-Order Learning
Reflective and Integrative Learning
Learning Strategies
Quantitative Reasoning
Collaborative Learning
Discussions with Diverse Others
Student-Faculty Interaction
Effective Teaching Practices
Quality of Interactions
Supportive Environment
△--△
--
△ △▽△▽ ▽----
Campus Environment --
Experiences with Faculty
-- --
Engagement Indicators are summary measures based on sets of NSSE questions examining key dimensions of student engagement. The ten indicators are organized within four themes: Academic Challenge, Learning with Peers, Experiences with Faculty, and Campus Environment. The tables below compare average scores for your students with those in your comparison groups.
Learning with Peers
ISU Peer Group Carnegie Class
--
NSSE 2013▽
----
Your FY students compared with
Your FY students compared with
Your FY students compared with
▽▽--
Academic Challenge
▽--
--▽---- ----
37.133.840.028.0
32.140.4
22.939.1
40.936.1
ISU Results
Seniors
Theme Engagement Indicator
Higher-Order Learning
Reflective and Integrative Learning
Learning Strategies
Quantitative Reasoning
Collaborative Learning
Discussions with Diverse Others
Student-Faculty Interaction
Effective Teaching Practices
Quality of Interactions
Supportive Environment
--▽
▽ --
----
▼▽
--
-- ▽ ▽
▽
Carnegie Class
▽--
NSSE 2013
--
▽Campus Environment
Your seniors compared with
Your seniors compared with
Your seniors compared with
▽
-- ▽--
▽
------
Learning with Peers
--
▽Academic Challenge
--△Experiences
with Faculty
ISU Peer Group
▽
40.736.740.828.8
28.540.5
22.839.2
41.729.9
StudentSuccess Conference 2013
High Impact PracticesFirst-Year Students %
11c. Learning community 10 18 *** -.23 17 *** -.19 15 ** -.14
12. Service-learning 69 52 *** .37 56 *** .28 52 *** .36
11e. Research with faculty 5 5 .01 5 .01 5 .00
Participated in at least one 72 58 *** .29 61 *** .23 58 *** .29
Participated in two or more 10 14 * -.11 14 * -.11 12 -.05
Seniors11c. Learning community 31 25 *** .14 21 *** .23 24 *** .16
12. Service-learning 78 58 *** .43 59 *** .41 60 *** .38
11e. Research with faculty 20 20 .00 15 ** .12 23 -.08
11a. Internship or field exp. 59 46 *** .25 36 *** .46 48 *** .22
11d. Study abroad 10 8 * .08 8 .07 13 -.08
11f. Culminating senior exp. 43 36 *** .14 36 ** .13 45 -.03
Participated in at least one 91 82 *** .26 77 *** .39 84 *** .21
Participated in two or more 68 55 *** .27 48 *** .40 60 *** .17
Note: All results weighted by gender and enrollment status (and by institution size for comparison groups).
Indiana State NSSE 2013ISU Peer Group Carnegie Class
% b
a. Percentage of students who responded "Done or in progress" except for service-learning which is the percentage who responded that at least "Some" courses included a community-based project.b. *p<.05, **p<.01, ***p<.001 (z-test comparing participation rates).c. Cohen's h: The standardized difference between two proportions. Effect size indicates the practical importance of an observed difference. An effect size of .2 is generally considered small, .5 medium, and .8 large.
% bEffect size c
Effect size c
Effect size c% b
First-yearHighest Performing Relative to ISU Peer Group
About how many…courses have included a community-based project (service-learning)?e (HIP)
Inst. emphasizes… Helping you manage your non-academic responsibilities (…)c (SE)
Worked with a faculty member on activities other than coursework (…)b (SF)
Discussions with… People of a race or ethnicity other than your ownb (DD)
Discussed your academic performance with a faculty memberb (SF)
SeniorHighest Performing Relative to ISU Peer Group
About how many…courses have included a community-based project (service-learning)?e (HIP)
Participated in an internship, co-op, field exp., student teach., clinical placemt. (HIP)
Instructors… Provided feedback on a draft or work in progress c (ET)
Used numerical information to examine a real-world problem or issue (…)b (QR)
Completed a culminating senior experience (…) (HIP)
Points of Pride
+18+12+7+6+6
+20+12+7+7+7
Areas of ConcernLowest Performing Relative to ISU Peer Group
Tried to better understand someone else's views by imagining…his or her perspective b (RI)
Learned something that changed the way you understand an issue or concept b (RI)
Connected ideas from your courses to your prior experiences and knowledge b (RI)
Inst. emphasizes… Using learning support services (…)c (SE)
Participated in a learning community or some other formal program where… (HIP)
Lowest Performing Relative to ISU Peer GroupInstructors… Used examples or illustrations to explain difficult points c (ET)
Asked another student to help you understand course material b (CL)
Inst. emphasizes… Using learning support services (…)c (SE)
Prepared for exams by discussing or working through course material w/other students b (CL)
Explained course material to one or more students b (CL)
-5-5-6-8-8
-4-5-6-6-7
StudentSuccess Conference 2013
First-year
Senior
Do We Provide Enough Challenge?Challenging Courses
First-year Senior
To what extent did your students' courses challenge them to do their best work? Response options ranged from 1 = "Not at all" to 7 = "Very much."
52% 44% 44% 39%
47%54% 53% 58%
0%
25%
50%
75%
100%
IndianaState
ISU PeerGroup
IndianaState
ISU PeerGroup
StudentSuccess Conference 2013
Freshmen: Statistically below other institutions in all 9 categories, although effect size largest1 (.19 to .21) for:◦ Been available when needed.◦ Listened closely to your concerns and questions.◦ Informed you of academic support options.
Seniors: Statistically below other institutions in 4/9 categories (and none statistically higher), although effect size largest (.20) for:◦ Been available when needed.
Academic Advising Results
1Defined as .19 or above for both academic advising and writing.
StudentSuccess Conference 2013
Freshmen: Statistically above other institutions in 4/13 categories, although effect size largest (.25) for:◦ Gave feedback to a classmate about a draft or outline
he or she had written. Statistically below other institutions in 2/9 categories, although effect size largest for:
◦ Argued a position using evidence and reasoning. Seniors: Statistically below other institutions in
6/13 categories (and none statistically higher), although effect size largest (-.28) for:◦ Argued a position using evidence and reasoning.
Experience with Writing Results
StudentSuccess Conference 2013
Are there any findings that surprised you? How might we optimally disseminate the
results and engage the campus? What should be priority attention areas?
Discussion & Implications