NSSE:NSSE:Retrospective Retrospective

and and ProspectiveProspective

George KuhGeorge KuhAAC&U General Education and AAC&U General Education and

Assessment ConferenceAssessment ConferenceMarch 2, 2007March 2, 2007

National Survey of National Survey of Student EngagementStudent Engagement(pronounced “nessie”)

Community College Community College Survey of Student Survey of Student EngagementEngagement(pronounced “cessie”)

College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development

We value what we measureWe value what we measure

Wise decisions are Wise decisions are needed about needed about whatwhat to to measure in the context measure in the context of campus mission, of campus mission, values, and desired values, and desired outcomes. outcomes.

OverviewOverview

1.1. Why student engagement Why student engagement mattersmatters

2.2. What we’ve learnedWhat we’ve learned

3.3. ConversationConversation

What Really Matters in College: Student Engagement

Because individual effort and involvement are the critical determinants of impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement.

Pascarella & Terenzini, Pascarella & Terenzini, How College How College Affects StudentsAffects Students, 2005, p. 602, 2005, p. 602

Foundations of Student EngagementFoundations of Student Engagement

Time on task Time on task (Tyler, 1930s)(Tyler, 1930s)

Quality of effort Quality of effort (Pace, 1960-70s)(Pace, 1960-70s)

Student involvement Student involvement (Astin, 1984)(Astin, 1984)

Social, academic integration Social, academic integration (Tinto,1987, 1993)(Tinto,1987, 1993)

Good practices in Good practices in undergraduate education undergraduate education (Chickering & Gamson, 1987)(Chickering & Gamson, 1987)

College impact College impact (Pascarella, 1985)(Pascarella, 1985)

Student engagement Student engagement (Kuh, 1991, (Kuh, 1991, 2005)2005)

Good Practices in Good Practices in Undergraduate EducationUndergraduate Education

(Chickering & Gamson, 1987; Pascarella & Terenzini, 2005(Chickering & Gamson, 1987; Pascarella & Terenzini, 2005))

Student-faculty contactStudent-faculty contact Active learningActive learning Prompt feedbackPrompt feedback Time on taskTime on task High expectationsHigh expectations Experiences with diversityExperiences with diversity Cooperation among studentsCooperation among students

The Student Engagement TrinityThe Student Engagement Trinity

What students What students dodo -- time and energy -- time and energy devoted to educationally purposeful devoted to educationally purposeful activitiesactivities

What institutions What institutions dodo -- using -- using effective educational practices to effective educational practices to induce students to do the right induce students to do the right thingsthings

Educationally effective institutions Educationally effective institutions channel student energy toward the channel student energy toward the right activitiesright activities

NSSE’s Survey InstrumentThe College Student ReportThe College Student Report

Student Behaviors

Institutional Actions & Requirements

Reactions to College

Student BackgroundInformation

Student Learning &

Development

Effective Educational PracticesEffective Educational Practices

Level of Level of Academic Academic ChallengeChallenge

Active & Active & Collaborative Collaborative

LearningLearning

EnrichingEnrichingEducational Educational ExperiencesExperiences

SupportiveSupportiveCampusCampus

EnvironmentEnvironment

StudentStudentFaculty Faculty

InteractionInteraction

NSSE Project ScopeNSSE Project Scope

1,000,000+ students from 1,100+ 1,000,000+ students from 1,100+ different schools different schools

80% of 4-yr U.S. undergraduate 80% of 4-yr U.S. undergraduate FTE FTE

50 states, Puerto Rico, Canada50 states, Puerto Rico, Canada

70+ consortia70+ consortia

Documenting Documenting Good PracticeGood Practice

Institutional Institutional ImprovementImprovement

Public Public AdvocacyAdvocacy

NSSE Core PurposesNSSE Core Purposes

Customized Institutional ReportCustomized Institutional Report

OverviewOverviewInstitutional dataInstitutional dataMeans and frequenciesMeans and frequencies1st year students and 1st year students and

seniorsseniorsComparisons by peers, Comparisons by peers,

Carnegie, nationalCarnegie, nationalNational benchmarks National benchmarks Data use tipsData use tipsCD with raw data, etc.CD with raw data, etc.And more!And more!

What have weWhat have welearned so far?learned so far?

Student engagement varies Student engagement varies more more withinwithin than between than between institutions.institutions.

Supportive Campus Environment:Supportive Campus Environment: Seniors at Master's InstitutionsSeniors at Master's Institutions

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Master's Institutions

Percentile 10

Percentile 50

Percentile 90

Academic Challenge by Institutional TypeAcademic Challenge by Institutional Type

 

Seniors

25

30

35

40

45

50

55

60

65

70

75

DocExt

DocInt

MABacLA

BacGen Nation

First-Year Students

25

30

35

40

45

50

55

60

65

70

75

DocExt

DocInt

MA BacLA

Bac Gen

Nation

Ben

chm

ark

Sco

res

Behold the Behold the compensatorycompensatory effects of engagementeffects of engagement

Association of American Colleges and Universities

Narrow Learning is Not EnoughNarrow Learning is Not Enough——The Essential Learning OutcomesThe Essential Learning Outcomes

Knowledge of Human Cultures Knowledge of Human Cultures and the and the Physical & Natural WorldPhysical & Natural World

Intellectual and Practical SkillsIntellectual and Practical Skills

Personal and Social ResponsibilityPersonal and Social Responsibility

Integrative LearningIntegrative Learning

Integrating ideas or information from various sources

Included diverse perspectives in class discussions/writing

Put together ideas from different courses

Discussed ideas with faculty members outside of class

Discussed ideas with others outside of class

Analyzing the basic elements of an idea, experience, or theory

Essential Learning Outcome: Integrative Learning

Synthesizing & organizing ideas, info., or experiences

Making judgments about the value of information

Applying theories to practical problems or in new situations

Examined the strengths and weaknesses of your own views

Tried to better understand someone else's views

Learned something that changed how you understand an issue

NSSE DEEP LEARNING SCALE

Essential Learning Outcome: Intellectual and Practical Skills

NSSE SELF-REPORTED GAINS Acquiring job or work-related knowledge and skills

Writing clearly and effectively

Speaking clearly and effectively

Thinking critically and analytically

Analyzing quantitative problems

Using computing and information technology

Working effectively with others

Solving complex real-world problems

NSSE SELF-REPORTED GAINS Voting in local, state, or national elections

Learning effectively on your own

Understanding yourself

Understanding people of other racial and ethnic backgrounds

Developing a personal code of values and ethics

Contributing to the welfare of your community

Developing a deepened sense of spirituality

Essential Learning Outcome: Personal & Social Responsibilities

Gains in Intellectual and Practical Skills by Deep Learning Quartiles

lowest second third highest

Deep Learning Quartiles

First-Year

Senior

Very much

Quite a bit

Some

Very little

Gains in Personal and Social Responsibilities by Deep Learning Quartiles

lowest second third highest

Deep Learning Quartiles

First-Year

Senior

Very much

Quite a bit

Some

Very little

Effective Educational PracticesEffective Educational Practices

First-Year Seminars and Experiences First-Year Seminars and Experiences  Common Intellectual ExperiencesCommon Intellectual Experiences Learning CommunitiesLearning Communities Writing-Intensive CoursesWriting-Intensive Courses Collaborative Assignments and ProjectsCollaborative Assignments and Projects “ “Science as Science Is Done”; Science as Science Is Done”;

Undergraduate Research Undergraduate Research Diversity/Global LearningDiversity/Global Learning Service Learning, Community-Based Service Learning, Community-Based

Learning Learning InternshipsInternships Capstone Courses and ProjectsCapstone Courses and Projects

Effects of Learning Communities on EngagementEffects of Learning Communities on Engagement

Standardized Standardized Regression Regression CoefficientCoefficient Sig.Sig.

Y-Y-Standardized Standardized

Effect sizeEffect size

Standardized Standardized Regression Regression CoefficientCoefficient Sig.Sig.

Y-Y-Standardized Standardized

Effect sizeEffect size

Engagement ActivitiesEngagement Activities

Academic EffortsAcademic Efforts .16.16 ****** .32.32 .12.12 ****** .28.28

Higher Order ThinkingHigher Order Thinking .20.20 ****** .40.40 .15.15 ****** .35.35

Academic IntegrationAcademic Integration .19.19 ****** .39.39 .16.16 ****** .38.38

Active and Collaborative LearningActive and Collaborative Learning .26.26 ****** .53.53 .24.24 ****** .54.54

Interactions with FacultyInteractions with Faculty .30.30 ****** .60.60 .22.22 ****** .51.51

Diversity ExperiencesDiversity Experiences .21.21 ****** .41.41 .16.16 ****** .36.36

Perception of Campus EnvironmentPerception of Campus Environment

Quality of Academic AdvisingQuality of Academic Advising .12.12 ****** .23.23 .08.08 ****** .17.17

Supportive Campus EnvironmentSupportive Campus Environment .19.19 ****** .37.37 .14.14 ****** .32.32

SatisfactionSatisfaction .13.13 ****** .25.25 .10.10 ****** .23.23

Learning OutcomesLearning Outcomes

Gains in Personal and Social Gains in Personal and Social .24.24 ****** .48.48 .18.18 ****** .40.40

Gains in Practical CompetenceGains in Practical Competence .22.22 ****** .45.45 .16.16 ****** .36.36General Education GainsGeneral Education Gains .18.18 ****** .36.36 .11.11 ****** .24.24

First-yearFirst-year SeniorSenior

Diversity Diversity Diversity Diversity

Density Climate In Course- Diversity Density Climate In Course- Diversity

Dependent variable Index Diversity Work Press Index Diversity Work Press

Student Engagement

Academic challenge + + + Higher order thinking + + + + + +Active and Collaborative + + + + + +Diversity-related activities + + + + + + + +Supportive Campus Env.

Supportive Campus Env. + + Interpersonal - + Support for learning + + + + + +Satisfaction - + +Gains-Interpers. Dev.

Gains - Personal/social + + + + + + +Gains-Social Awareness

Gains-Understanding div. + + + + + + + +Gains -Cont. to community + + + + + +Gains-Understand self + + + + + +

First-year students Seniors

Faculty Time Devoted to Research and Faculty Time Devoted to Research and Probability of Students Doing ResearchProbability of Students Doing Research

0

0.13

0.25

0.38

0.50

Pro

bab

ilit

y o

f U

nd

erg

rad

uate

Researc

h

2.02 6.04 10.07 14.09 18.11

Faculty Research (hours)

Faculty View of Importance of Doing Research Faculty View of Importance of Doing Research with Students and Probability of Students Doing with Students and Probability of Students Doing

ResearchResearch

0.13

0.25

0.38

0.50

0

Pro

bab

ilit

y o

f U

nd

erg

rad

uate

Researc

h

1.87 2.25 2.62 2.99 3.37

Importance of Undergraduate Research

Faculty Priorities and Faculty Priorities and Student EngagementStudent Engagement

AVGAVG STUDENTSTUDENT

AVGAVG FACULTYFACULTY AcadAcad emicemic cchallhallengeenge

ActActiveive --ccollabollab

Diversity Diversity eexperiencesxperiences

StuStudentdent --ffacacultyulty

AAccadademicemic challchall engeenge emphemphasisasis

ActActiveive --collabcollab practicespractices

EmphEmph asisasis on diversityon diversity experiencesexperiences

EmphEmph asisasis on on hhigherigher orderorder thinkingthinking

ImpImportanceortance enrichingenriching ededucuc eexpxp erienceseriences

What to Make of This? What to Make of This?

1.1. When faculty members When faculty members emphasize certain educational emphasize certain educational practices, students engage in practices, students engage in them to a greater extent than them to a greater extent than their peers elsewhere.their peers elsewhere.

2.2. Good things go togetherGood things go together

Effective Educational PracticesEffective Educational Practices

First-Year Seminars and Experiences First-Year Seminars and Experiences  Common Intellectual ExperiencesCommon Intellectual Experiences Learning CommunitiesLearning Communities Writing-Intensive CoursesWriting-Intensive Courses Collaborative Assignments and ProjectsCollaborative Assignments and Projects “ “Science as Science Is Done”; Science as Science Is Done”;

Undergraduate Research Undergraduate Research Diversity/Global LearningDiversity/Global Learning Service Learning, Community-Based Service Learning, Community-Based

Learning Learning InternshipsInternships Capstone Courses and ProjectsCapstone Courses and Projects

Effective Educational Practices Increase Effective Educational Practices Increase Odds That Students Will:Odds That Students Will:

Invest time and effort Invest time and effort Interact with faculty and peers Interact with faculty and peers

about substantive mattersabout substantive matters Experience diversityExperience diversity Get more frequent feedbackGet more frequent feedback Discover relevance of their Discover relevance of their

learning through real-world learning through real-world applicationsapplications

Institutional ReflectionInstitutional Reflection

Areas of Areas of EffectiveEffective

EducationalEducationalPracticePractice

Areas of Areas of Question or Question or ImprovementImprovement

PossibilitiesPossibilities

Merge NSSE data Merge NSSE data with school recordswith school records

Identify engagement Identify engagement patterns by student patterns by student characteristicscharacteristics

Predict retention, Predict retention, degree attainment, degree attainment, grades, other grades, other outcomesoutcomes

Track student Track student engagement year to engagement year to yearyear

Compare vs. peer, Compare vs. peer, aspirational, same-aspirational, same-region schools region schools

Program evaluation Program evaluation Accreditation Accreditation Consortium and Consortium and

system data sharingsystem data sharing Student outcomes Student outcomes

researchresearch Public reportingPublic reporting A national reporting A national reporting

template?!?template?!?

NSSE VirtuesNSSE Virtues

Research basedFocus on educationally purposeful

activitiesResults point to areas where Results point to areas where

improvement is desirableimprovement is desirableCompelling face validityEstablished psychometrics Transparent operationsThird party administration

NSSE VirtuesNSSE Virtues

Random samplingTargeted samplingFlexibility: consortium question,

modulesBenchmarks for peer, national Benchmarks for peer, national

and other comparisons and other comparisons Link/integrate with other Link/integrate with other

measures -- BCSSE, FSSE, CLA, measures -- BCSSE, FSSE, CLA, MAPP, portfolios, graduation MAPP, portfolios, graduation rates etc. rates etc.

Other Data SourcesOther Data Sources

• Locally-developed measures • National instruments

– Cooperative Institutional Research Program (CIRP)

– Your First College Year (YFCY)– College Student Experiences Questionnaire

(CSEQ)– Noel Levitz Student Satisfaction Inventory– ETS MAPP and Major Field Tests– ACT Collegiate Assessment of Academic

Proficiency – Collegiate Learning Assessment (CLA)

• Institutional data -- GPA, financial aid, transcripts, retention, certification tests, alumni surveys, satisfaction surveys…

• Faculty Survey of Student Engagement (FSSE) and Beginning College Survey of Student Engagement (BCSSE)

NSSE Accreditation ToolkitNSSE Accreditation Toolkit

• Links Between NSSE & Accreditation

• Mapping NSSE to Accreditation Standards

• Vignettes of Institutional Usage

NSSE Questions

1.l. l. Used an electronic medium (list-serv, chat group, Internet, etc.) to discuss or complete an assignment

1.m. Used e-mail to communicate with an instructor

10.g. Using computers in academic work

11.g. Using computing and information technology

SACS Standard

3.4.14 The institution’s use of technology enhances student learning, is appropriate for meeting the objectives of its programs, and ensures that students have access to and training in the use of technology.

NSSE CautionsNSSE Cautions

Only one source of information Corroborate resultsEngagement skeptics Not an outcomes measureDenial in face of less-than-

desirable resultsEschew rankings

DiscussionDiscussion

www.iub.edu/~nsse