NSSE:NSSE:A Window into Student A Window into Student

and Institutional and Institutional PerformancePerformance

George KuhGeorge KuhUniversity of Maine TrusteesUniversity of Maine Trustees

Farmington MEFarmington MEMarch 19, 2007March 19, 2007

Commission on Future of Higher EducationCommission on Future of Higher Education

1.1. Improve student academic preparation and Improve student academic preparation and make more financial aid availablemake more financial aid available

2.2. Simplify & restructure student financial aid Simplify & restructure student financial aid system using incentives to control costs system using incentives to control costs

3.3. Cultivate a "robust culture of Cultivate a "robust culture of accountability & transparency“: invent accountability & transparency“: invent new student learning measures, make new student learning measures, make comparable information public, focus comparable information public, focus accreditation on outcomesaccreditation on outcomes

4.4. Embrace continuous innovation and Embrace continuous innovation and quality improvement. quality improvement.

5.5. Target federal investments to critical areas such Target federal investments to critical areas such as math, science, and foreign languages. as math, science, and foreign languages.

6.6. Develop a national strategy for lifelong learningDevelop a national strategy for lifelong learning

Association of American Colleges and Universities

We value what we measureWe value what we measure

Wise decisions are Wise decisions are needed about needed about whatwhat to to measure in the context measure in the context of campus mission, of campus mission, values, and desired values, and desired outcomes. outcomes.

OverviewOverview

1.1. Why student engagement Why student engagement mattersmatters

2.2. What we’ve learned from NSSEWhat we’ve learned from NSSE

3.3. Unsolicited advice for trusteesUnsolicited advice for trustees

4.4. ConversationConversation

Student Success in CollegeStudent Success in College

Academic achievement, Academic achievement, engagement in engagement in educationally purposeful educationally purposeful activities, satisfaction, activities, satisfaction, acquisition of desired acquisition of desired knowledge, skills and knowledge, skills and competencies, competencies, persistence, attainment persistence, attainment of educational of educational objectives, and post-objectives, and post-college performancecollege performance

Pre-college Characteristics Pre-college Characteristics Associated with Student SuccessAssociated with Student Success

Academic preparationAcademic preparationAbility and college-level skillsAbility and college-level skillsFamily education and supportFamily education and supportFinancial wherewithalFinancial wherewithal

32.N162FG12

Postsecondary Education OPPORTUNITY Association for Institutional Research – May 16, 2006 – Chicago, Illinois

Early College Indicators of Early College Indicators of Persistence and SuccessPersistence and Success

Goal realizationGoal realization Psycho-social fitPsycho-social fit Credit hours completedCredit hours completed Academic and social supportAcademic and social support Involvement in the “right” kinds Involvement in the “right” kinds

of activitiesof activities

Factors That Threaten Persistence and Factors That Threaten Persistence and Graduation from College Graduation from College

academically underprepared for academically underprepared for college-level workcollege-level work

first-generation college studentfirst-generation college student gap between high school and college gap between high school and college 30+ hours working per week30+ hours working per week part-time enrollmentpart-time enrollment single parentsingle parent financially independentfinancially independent children at homechildren at home

What Really Matters in College: Student Engagement

Because individual effort and involvement are the critical determinants of impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement.

Pascarella & Terenzini, Pascarella & Terenzini, How College How College Affects StudentsAffects Students, 2005, p. 602, 2005, p. 602

Foundations of Student EngagementFoundations of Student Engagement

Time on task Time on task (Tyler, 1930s)(Tyler, 1930s)

Quality of effort Quality of effort (Pace, 1960-70s)(Pace, 1960-70s)

Student involvement Student involvement (Astin, 1984)(Astin, 1984)

Social, academic integration Social, academic integration (Tinto,1987, 1993)(Tinto,1987, 1993)

Good practices in Good practices in undergraduate education undergraduate education (Chickering & Gamson, 1987)(Chickering & Gamson, 1987)

College impact College impact (Pascarella, 1985)(Pascarella, 1985)

Student engagement Student engagement (Kuh, 1991, (Kuh, 1991, 2005)2005)

Good Practices in Good Practices in Undergraduate EducationUndergraduate Education

(Chickering & Gamson, 1987; Pascarella & Terenzini, 2005(Chickering & Gamson, 1987; Pascarella & Terenzini, 2005))

Student-faculty contactStudent-faculty contact Active learningActive learning Prompt feedbackPrompt feedback Time on taskTime on task High expectationsHigh expectations Experiences with diversityExperiences with diversity Cooperation among studentsCooperation among students

The Student Engagement TrinityThe Student Engagement Trinity

What students What students dodo -- time and energy -- time and energy devoted to educationally purposeful devoted to educationally purposeful activitiesactivities

What institutions What institutions dodo -- using -- using effective educational practices to effective educational practices to induce students to do the right induce students to do the right thingsthings

Educationally effective institutions Educationally effective institutions channel student energy toward the channel student energy toward the right activitiesright activities

NSSE’s Survey InstrumentThe College Student ReportThe College Student Report

Student Behaviors

Institutional Actions & Requirements

Reactions to College

Student BackgroundInformation

Student Learning &

Development

National Survey of National Survey of Student EngagementStudent Engagement(pronounced “nessie”)

Community College Community College Survey of Student Survey of Student EngagementEngagement(pronounced “cessie”)

College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development

Effective Educational PracticesEffective Educational Practices

Level of Level of Academic Academic ChallengeChallenge

Active & Active & Collaborative Collaborative

LearningLearning

EnrichingEnrichingEducational Educational ExperiencesExperiences

SupportiveSupportiveCampusCampus

EnvironmentEnvironment

StudentStudentFaculty Faculty

InteractionInteraction

NSSE Project ScopeNSSE Project Scope

1,000,000+ students from 1,100+ 1,000,000+ students from 1,100+ different schools different schools

80% of 4-yr U.S. undergraduate 80% of 4-yr U.S. undergraduate FTE FTE

50 states, Puerto Rico, Canada50 states, Puerto Rico, Canada

70+ consortia70+ consortia

Documenting Documenting Good PracticeGood Practice

Institutional Institutional ImprovementImprovement

Public Public AdvocacyAdvocacy

NSSE Core PurposesNSSE Core Purposes

Customized Institutional ReportCustomized Institutional Report

OverviewOverviewInstitutional dataInstitutional dataMeans and frequenciesMeans and frequencies1st year students and 1st year students and

seniorsseniorsComparisons by peers, Comparisons by peers,

Carnegie, nationalCarnegie, nationalNational benchmarks National benchmarks Data use tipsData use tipsCD with raw data, etc.CD with raw data, etc.And more!And more!

What have weWhat have welearned so far?learned so far?

Grades, persistence, Grades, persistence, student satisfaction, student satisfaction, and engagement go and engagement go hand in handhand in hand

Does institutional size matter to Does institutional size matter to engagement?engagement?

Yes, Yes, size matters.size matters.

Smaller Smaller isis generally generally betterbetter..

Benchmark Scores for All Students by Undergraduate Enrollment

Benchmark Scores for All Students by Undergraduate Enrollment Intervals

25

35

45

55

65

0 -1000

1001 -1500

1501 -2000

2001 -2500

2501 -3000

3001 -4000

4001 -5000

5001 -7500

7501 -10000

10001 -15000

15001 -20000

20001 -25000

25001 -highest

Enrollment Intervals

Level ofAcademicChallenge

Active andCollaborativeLearning

StudentInteractionswith FacultyMembersEnrichingEducationalExperiences

SupportiveCampusEnvironment

Academic Challenge, Active Learning, Student-Faculty Interaction by Enrollment

20

30

40

50

60

186 1238 1921 3060 6337 11343 29426Enrollment

Level of Academic Challenge

Active and Collaborative Learning

Student Interactions with Faculty Members

Student engagement varies Student engagement varies more more withinwithin than between than between institutions.institutions.

Supportive Campus Environment:Supportive Campus Environment: Seniors at Master's InstitutionsSeniors at Master's Institutions

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Master's Institutions

Percentile 10

Percentile 50

Percentile 90

Academic Challenge by Institutional TypeAcademic Challenge by Institutional Type

 

Seniors

25

30

35

40

45

50

55

60

65

70

75

DocExt

DocInt

MABacLA

BacGen Nation

First-Year Students

25

30

35

40

45

50

55

60

65

70

75

DocExt

DocInt

MA BacLA

Bac Gen

Nation

Ben

chm

ark

Sco

res

Behold the Behold the compensatorycompensatory effects of engagementeffects of engagement

Association of American Colleges and Universities

Narrow Learning is Not EnoughNarrow Learning is Not Enough——The Essential Learning OutcomesThe Essential Learning Outcomes

Knowledge of Human Cultures Knowledge of Human Cultures and the and the Physical & Natural WorldPhysical & Natural World

Intellectual and Practical SkillsIntellectual and Practical Skills

Personal and Social ResponsibilityPersonal and Social Responsibility

Integrative LearningIntegrative Learning

Integrating ideas or information from various sources

Included diverse perspectives in class discussions/writing

Put together ideas from different courses

Discussed ideas with faculty members outside of class

Discussed ideas with others outside of class

Analyzing the basic elements of an idea, experience, or theory

Essential Learning Outcome: Integrative Learning

Synthesizing & organizing ideas, info., or experiences

Making judgments about the value of information

Applying theories to practical problems or in new situations

Examined the strengths and weaknesses of your own views

Tried to better understand someone else's views

Learned something that changed how you understand an issue

NSSE DEEP LEARNING SCALE

Essential Learning Outcome: Intellectual and Practical Skills

NSSE SELF-REPORTED GAINS Acquiring job or work-related knowledge and skills

Writing clearly and effectively

Speaking clearly and effectively

Thinking critically and analytically

Analyzing quantitative problems

Using computing and information technology

Working effectively with others

Solving complex real-world problems

NSSE SELF-REPORTED GAINS Voting in local, state, or national elections

Learning effectively on your own

Understanding yourself

Understanding people of other racial and ethnic backgrounds

Developing a personal code of values and ethics

Contributing to the welfare of your community

Developing a deepened sense of spirituality

Essential Learning Outcome: Personal & Social Responsibilities

Gains in Intellectual and Practical Skills by Deep Learning Quartiles

lowest second third highest

Deep Learning Quartiles

First-Year

Senior

Very much

Quite a bit

Some

Very little

Gains in Personal and Social Responsibilities by Deep Learning Quartiles

lowest second third highest

Deep Learning Quartiles

First-Year

Senior

Very much

Quite a bit

Some

Very little

Effective Educational PracticesEffective Educational Practices

First-Year Seminars and Experiences First-Year Seminars and Experiences  Common Intellectual ExperiencesCommon Intellectual Experiences Learning CommunitiesLearning Communities Writing-Intensive CoursesWriting-Intensive Courses Collaborative Assignments and ProjectsCollaborative Assignments and Projects “ “Science as Science Is Done”; Science as Science Is Done”;

Undergraduate Research Undergraduate Research Diversity/Global LearningDiversity/Global Learning Service Learning, Community-Based Service Learning, Community-Based

Learning Learning InternshipsInternships Capstone Courses and ProjectsCapstone Courses and Projects

Effects of Learning Communities on EngagementEffects of Learning Communities on Engagement

Standardized Standardized Regression Regression CoefficientCoefficient Sig.Sig.

Y-Y-Standardized Standardized

Effect sizeEffect size

Standardized Standardized Regression Regression CoefficientCoefficient Sig.Sig.

Y-Y-Standardized Standardized

Effect sizeEffect size

Engagement ActivitiesEngagement Activities

Academic EffortsAcademic Efforts .16.16 ****** .32.32 .12.12 ****** .28.28

Higher Order ThinkingHigher Order Thinking .20.20 ****** .40.40 .15.15 ****** .35.35

Academic IntegrationAcademic Integration .19.19 ****** .39.39 .16.16 ****** .38.38

Active and Collaborative LearningActive and Collaborative Learning .26.26 ****** .53.53 .24.24 ****** .54.54

Interactions with FacultyInteractions with Faculty .30.30 ****** .60.60 .22.22 ****** .51.51

Diversity ExperiencesDiversity Experiences .21.21 ****** .41.41 .16.16 ****** .36.36

Perception of Campus EnvironmentPerception of Campus Environment

Quality of Academic AdvisingQuality of Academic Advising .12.12 ****** .23.23 .08.08 ****** .17.17

Supportive Campus EnvironmentSupportive Campus Environment .19.19 ****** .37.37 .14.14 ****** .32.32

SatisfactionSatisfaction .13.13 ****** .25.25 .10.10 ****** .23.23

Learning OutcomesLearning Outcomes

Gains in Personal and Social Gains in Personal and Social .24.24 ****** .48.48 .18.18 ****** .40.40

Gains in Practical CompetenceGains in Practical Competence .22.22 ****** .45.45 .16.16 ****** .36.36General Education GainsGeneral Education Gains .18.18 ****** .36.36 .11.11 ****** .24.24

First-yearFirst-year SeniorSenior

Diversity Diversity Diversity Diversity

Density Climate In Course- Diversity Density Climate In Course- Diversity

Dependent variable Index Diversity Work Press Index Diversity Work Press

Student Engagement

Academic challenge + + + Higher order thinking + + + + + +Active and Collaborative + + + + + +Diversity-related activities + + + + + + + +Supportive Campus Env.

Supportive Campus Env. + + Interpersonal - + Support for learning + + + + + +Satisfaction - + +Gains-Interpers. Dev.

Gains - Personal/social + + + + + + +Gains-Social Awareness

Gains-Understanding div. + + + + + + + +Gains -Cont. to community + + + + + +Gains-Understand self + + + + + +

First-year students Seniors

Faculty Time Devoted to Research and Faculty Time Devoted to Research and Probability of Students Doing ResearchProbability of Students Doing Research

0

0.13

0.25

0.38

0.50

Pro

bab

ilit

y o

f U

nd

erg

rad

uate

Researc

h

2.02 6.04 10.07 14.09 18.11

Faculty Research (hours)

Faculty View of Importance of Doing Research Faculty View of Importance of Doing Research with Students and Probability of Students Doing with Students and Probability of Students Doing

ResearchResearch

0.13

0.25

0.38

0.50

0

Pro

bab

ilit

y o

f U

nd

erg

rad

uate

Researc

h

1.87 2.25 2.62 2.99 3.37

Importance of Undergraduate Research

Faculty Priorities and Faculty Priorities and Student EngagementStudent Engagement

AVGAVG STUDENTSTUDENT

AVGAVG FACULTYFACULTY AcadAcad emicemic cchallhallengeenge

ActActiveive --ccollabollab

Diversity Diversity eexperiencesxperiences

StuStudentdent --ffacacultyulty

AAccadademicemic challchall engeenge emphemphasisasis

ActActiveive --collabcollab practicespractices

EmphEmph asisasis on diversityon diversity experiencesexperiences

EmphEmph asisasis on on hhigherigher orderorder thinkingthinking

ImpImportanceortance enrichingenriching ededucuc eexpxp erienceseriences

What to Make of This? What to Make of This?

1.1. When faculty members When faculty members emphasize certain educational emphasize certain educational practices, students engage in practices, students engage in them to a greater extent than them to a greater extent than their peers elsewhere.their peers elsewhere.

2.2. Good things go togetherGood things go together

Effective Educational PracticesEffective Educational Practices

First-Year Seminars and Experiences First-Year Seminars and Experiences  Common Intellectual ExperiencesCommon Intellectual Experiences Learning CommunitiesLearning Communities Writing-Intensive CoursesWriting-Intensive Courses Collaborative Assignments and ProjectsCollaborative Assignments and Projects “ “Science as Science Is Done”; Science as Science Is Done”;

Undergraduate Research Undergraduate Research Diversity/Global LearningDiversity/Global Learning Service Learning, Community-Based Service Learning, Community-Based

Learning Learning InternshipsInternships Capstone Courses and ProjectsCapstone Courses and Projects

Effective Educational Practices Increase Effective Educational Practices Increase Odds That Students Will:Odds That Students Will:

Invest time and effort Invest time and effort Interact with faculty and peers Interact with faculty and peers

about substantive mattersabout substantive matters Experience diversityExperience diversity Get more frequent feedbackGet more frequent feedback Discover relevance of their Discover relevance of their

learning through real-world learning through real-world applicationsapplications

Institutional ReflectionInstitutional Reflection

Areas of Areas of EffectiveEffective

EducationalEducationalPracticePractice

Areas of Areas of Question or Question or ImprovementImprovement

PossibilitiesPossibilities

Merge NSSE data Merge NSSE data with school recordswith school records

Identify engagement Identify engagement patterns by student patterns by student characteristicscharacteristics

Predict retention, Predict retention, degree attainment, degree attainment, grades, other grades, other outcomesoutcomes

Track student Track student engagement year to engagement year to yearyear

Compare vs. peer, Compare vs. peer, aspirational, same-aspirational, same-region schools region schools

Program evaluation Program evaluation Accreditation Accreditation Consortium and Consortium and

system data sharingsystem data sharing Student outcomes Student outcomes

researchresearch Public reportingPublic reporting A national reporting A national reporting

template?!?template?!?

NSSE Accreditation ToolkitNSSE Accreditation Toolkit

• Links Between NSSE & Accreditation

• Mapping NSSE to Accreditation Standards

• Vignettes of Institutional Usage

NSSE Questions

1.l. l. Used an electronic medium (list-serv, chat group, Internet, etc.) to discuss or complete an assignment

1.m. Used e-mail to communicate with an instructor

10.g. Using computers in academic work

11.g. Using computing and information technology

SACS Standard

3.4.14 The institution’s use of technology enhances student learning, is appropriate for meeting the objectives of its programs, and ensures that students have access to and training in the use of technology.

Questions Trustees Should AskQuestions Trustees Should Ask

• To what extent does the To what extent does the institutional mission—espoused institutional mission—espoused and enacted—unmistakably and enacted—unmistakably emphasize student success? emphasize student success?

• How does the institution’s How does the institution’s spending plan enhance student spending plan enhance student learning and success? learning and success?

• What resources are available to What resources are available to support trying out good ideas? support trying out good ideas?

Put money where it will make a Put money where it will make a difference in student engagementdifference in student engagement

“…“…in professional in professional baseball it still baseball it still matters less how matters less how much you have much you have than how well you than how well you spend it”spend it”

Endowments & Undergraduate Enrollments Endowments & Undergraduate Enrollments of Selected Public DEEP Institutionsof Selected Public DEEP Institutions

Evergreen

$2,395,656

4,171 students

Fayetteville State

$8,465,541

5,029 students

UMF

$9,391,000

2,452 students

George Mason

$40,812,720

18,091 students

Source: US News America’s Best Colleges 2007

• What institutional and student-What institutional and student-performance indicators are used performance indicators are used to inform board decisions? to inform board decisions?

• Who is responsible for Who is responsible for maintaining an institutional focus maintaining an institutional focus on student success and reporting on student success and reporting these key indicators to the these key indicators to the board? board?

• Does the institution participate in Does the institution participate in NSSE and how are the results NSSE and how are the results being used (Bok, 2006)? being used (Bok, 2006)?

• Where do students and their Where do students and their learning fall among the board’s learning fall among the board’s interests and priorities? interests and priorities?

• Is a standing committee dedicated Is a standing committee dedicated to student learning and campus to student learning and campus life?life?

• Does the culture of the board Does the culture of the board encourage encourage or or discouragediscourage open, open, productive dialogue about matters productive dialogue about matters related to student success?related to student success?

•

Why stay the course?Why stay the course?

The good-to-great-transformations The good-to-great-transformations never happened in one fell swoop. never happened in one fell swoop. There was no single defining There was no single defining action, no grand program, no one action, no grand program, no one killer innovation, no solitary lucky killer innovation, no solitary lucky break, no miracle moment. break, no miracle moment. Sustainable transformations Sustainable transformations follow a predictable pattern of follow a predictable pattern of buildup and breakthroughbuildup and breakthrough……

(Collins, 2001, p. 186)

Last WordLast Word

A college or university cannot change A college or university cannot change the lineage of its students. Campus the lineage of its students. Campus cultures do not change easily or cultures do not change easily or willingly. But we can do far more to willingly. But we can do far more to shape the way students approach shape the way students approach college and what they do after they college and what they do after they arrive. arrive.

Do we have the Do we have the willwill to use promising to use promising policies and educational practices to policies and educational practices to increase the odds that more students increase the odds that more students “get ready,” “get in,” and “get “get ready,” “get in,” and “get through?”through?”

Questions & Questions & DiscussionDiscussion