SHOCK EFFECT 2001

Moving From Data to Goals

Dr. Gary Steward, Associate Dean & Professor of Sociology

Dr. Stacy Southerland, Professor of Modern Languages, Literature & Culture

Dr. David Macey, Professor of English

Ms. Karen Henderson, Director of Assessment

UCO’s Experience with NSSE

Administered Spring 2001 with a 33% response rate (42% for all NSSE 2001 institutions)

Administered Spring 2003 with a 34% response rate (43% for all NSSE 2001 institutions)

Next administration is Spring 2006

In 2001 NSSE Reported

Would They Start Over at UCO? Students with a good

or excellent experience would attend UCO if they were starting their educational career over again. 0%

20%

40%

60%

80%

100%

GoodExperience

SameInstitution

Fresh Senior

College Satisfaction

2001 Benchmarks

41 questions are assigned to five clusters of similar activities and

conditions to make up the national benchmark of effective

educational practices

Comparison of UCO to NSSE Schools on Benchmarks 2001

BenchmarkLevel of Student

**Standardized Residual (SR)

SR greater than approximately ___ % of NSSE schools

Level of Academic Challenge First year -2.2 1%Senior -0.3 38%

Active and Collaborative Learning First year -0.1 46%Senior 0.2 58%

Student Interactions with Faculty Members First year -0.6 27%Senior 1.5 93%

Enriching Educational Experiences First year -0.2 42%Senior 1.5 93%

Supportive Campus Environment First year -1.4 9%Senior 0.5 69%

*Actual benchmark scores statistically adjusted for the types of students at UCO and other institutional characteristics.

**Standardized residual is an estimate of the degree to which UCO exceeded or fell short of its predicted score relative to all other NSSE institutions.

Level of Academic Challenge Time spent preparing for class, amount of reading

and writing, and institutional expectations for academic performance

0.0

20.0

40.0

60.0

First-Year Senior

UCO Master's National (NSSE)

College of Liberal Arts (LA) Said

Academic Challenge Items focused on in the local LA Instruments

• Used Local Survey Items and Focus Groups to study

• Preparing for class• Worked harder than you thought you would to

meet an instructor’s standards or expectations

• Bloom’s taxonomy items• Campus environment emphasizes spending

significant amounts of time studying and on academic work

Academic Challenge Items focused on in the local LA Instruments

• Syllabi Review was used to study• Number of assigned textbooks, books, or

course readings• Number of written papers or reports of 20

pages or more….between 5 and 19 pages ….fewer than 5 pages

• Indication of significant expectations for work outside of the classroom.

Locally Developed Surveys & Focus Groups

Student Engagement Academic Standards Foundation Courses

Survey response averages 275 forms.

Focus Groups averages 14 participants.

TOPICS

College Level Faculty Input

Locally developed Surveys and Focus Groups Address issues tied to the College and Academic

Challenge Provides feedback regarding satisfaction with the

college administration

So We Took ACTION To Change!

Action Taken

Set College Goals Based on NSSE Data Writing Initiative First Day Engagement

Reinforce Positive Faculty Behavior Teaching Awards Strategic Plans Perception of Instructional Effectiveness Forms

Action Taken

Explored Options for Improving Academic Challenge Asked students to define Academic Challenge Asked faculty how to enhance Academic

Challenge

LA Academic Goals

2002 – Every class will have a writing requirement

2004 – First Day Engagement Initiative

Academic Standards

2001-02 54% claimed that ¾ or all of their LA courses were easier

than expected.

2005 Only 50% reported that ¾ or all of their upper division LA

courses were easier than expected. Only 36% reported ¾ or all of their lower division courses

were easier than expected.

“Overall, my classes are easier than I expected.”

Academic Standards

2001-02 11% reported none of their LA courses were

easier than expected. 2005

20% reported none of their LA courses were easier than expected.

“Overall, my classes are easier than I expected.”

Academic Standards

2001-02 Only 15% reported that they were intellectually

challenged in ¾ or all of their LA courses. 2005

65% (lower division) and 72% (upper division) claimed that they were intellectually challenged in ¾ or all of their LA courses.

Academic Standards

2001-02 42% claimed none of their courses challenged

them intellectually. 2005

Only 7% (lower division) and 8% (upper division) reported that none of their LA courses were intellectually challenging.

Academic Standards

“I take LA courses because it is a nice respite from biology.” Many members commented that LA courses allowed them to take “a breather.”

Students did comment on more oral presentations and some claimed that LA courses require more writing than courses in other colleges.

Student focus groups in 2001-02 reported:

Students have reported an increase in writing requirements in the last year.

“I don’t like it when teachers are giving writing assignments because it’s expected now.”

“I get aggravated with taking 12 hours and having instructors who are acting as if I’m only taking their class. I can’t have four hard teachers and make it through, so I look for easier classes.”

Student focus groups in 2005 reported:

Our first day engagement data (2005, 738 students) revealed the following

Their interest increased (68%) They felt the class would be more

challenging than they expected (50%) They believed they could be successful

in the course (93.5%)

Students reported that after the first day of class:

Writing Initiative

2001-2002: 13% stated ¾ or all of their LA courses. 48% reported that they were not required to write a

paper in any of their courses.

2005: 74% (lower division) and 86% (upper division)

reported required writing in ¾ or all of their LA courses.

Only 8% (lower division) and 4% (upper division) claimed that none of their classes required a paper.

“I am required to write a paper”

Writing Initiative

In 2001-02, 28% reported taking exams that included essays in ¾ or all of their courses.

In 2005, 66% (lower division) and 75% (upper division) reported taking exams that included essays in ¾ or all of their courses.

Writing Initiative

In 2001-02, 29% claimed that none of their LA courses included essays in their exams.

In 2005, only 9% (lower division) and 3% (upper division) reported that none of their LA courses included essays in their exams.

Faculty Reaction

“Our department has made a conscious effort to raise standards and rigor at the general education and survey level as well as in upper-division and graduate courses. Our cornerstone course offers students an opportunity to spend a semester researching and writing a fifteen-page paper based . . . on primary sources . . . . Students who are engaged in the course produce scholarly papers that compete well at regional conferences against students from OU and OSU.”

- History Faculty Member

Faculty Reaction

“Our department has begun producing custom readers for our general education courses because we found that commercially available readers did not respond effectively to the range of writing challenges that our first-year students face. Expanding the scope of our first-year writing curriculum has enabled us to increase the number and variety of writing assignments, including both shorter essays and longer research papers, that our first-year students complete.”

- English Faculty Member

Faculty Reaction

“I have, as a result of assessment activities, reevaluated how and what I communicate to the students with regard to my expectations for their achievement on writing assignments. I also give more detailed information about specific evaluation criteria by providing the form that I use to break down the score into areas of thesis statement, development, grammar, MLA style, and so on.”

- Modern Languages Faculty Member

Faculty Reaction

“In each of my junior- and senior-level courses, students complete at least one major research paper . . . . I emphasize throughout the semester that utilizing a grounded style of writing is preferable to a ‘my way of writing’ paper or a disorganized one.”

- Mass Communication Faculty Member

Faculty Reaction

“In the senior-level seminar, I assign three types of readings for discussion at meetings and writing about two of them for almost every meeting . . . . In the sophomore-level course, students are required to submit an essay on Turnitin.com . . . and a minimum three-page paper following a required visit to an art museum”

- Humanities & Philosophy Faculty Member

Faculty Reaction

“In 2001, we introduced a new, sophomore-level ‘cornerstone’ class focusing on writing and research strategies specific to our discipline. Our majors now receive a systematic introduction to academic writing in English Studies before advancing to junior- and senior-level courses . . . . Because instructors are able to count on students having this foundation, they are able to assign more varied writing projects and to require more writing without having to ‘backtrack’ in class to fill in gaps in students’ academic preparation.”

- English Faculty Member

Faculty Reaction

My junior-level course has two group work activities, four exams, and six quizzes. This year I added a written component. Now they summarize one of the chapters not covered in class and set up an interview with a practitioner working in the area. They then write up that interview in essay style. . . . . I also added a written component to my senior-level course. The best papers were presented in class. In researching these essays, a number of students changed their mind about the issues in question.”

- Mass Communication Faculty Member

New College Goal

2005 – Every class will hold students accountable for assigned reading.

Academic Affairs Mission

“Helping students learn so that they may become productive, creative, ethical, and engaged citizens.” Academic Values

Integrity Student Centered Academic Excellence Continuous Improvement Collegiality Community

Strategic Plan Process Implemented

Process has Four Criteria Helping Students Learn Accomplishing Other Distinctive Objectives Understanding Students and Other

Stakeholders’ Needs Valuing People

Perception of Instructional Effectiveness Form Revised

Based on 7 Principles of Good Undergraduate Education

Piloted in Summer 2002 Partially implemented in Fall 2002 Instrument administered on-line

Changes in Benchmarks

2001 2003

BenchmarkLevel of Student

**Standardized Residual (SR)

SR greater than approximately ___ % of NSSE schools

**StandardizedResidual (SR)

SR greater than approximately ___ % of NSSE schools

Level of Academic Challenge First year -2.2 1% -1.2 14%

  Senior -0.3 38% -0.5 31%

Active and Collaborative Learning First year -0.1 46% -0.2 44%

  Senior 0.2 58% -0.4 38%

Student Interactions with Faculty Members First year -0.6 27% 0.1 52%

  Senior 1.5 93% 0.1 52%

Enriching Educational Experiences First year -0.2 42% -0.8 20%

  Senior 1.5 93% 0.1 52%

Supportive Campus Environment First year -1.4 9% -0.7 24%

  Senior 0.5 69% 0.0 50%

Can’t Wait!

Spring 2006 UCO is conducting the National NSSE again.

Will the benchmarks go up?

Thank You!

Dr. Gary Steward, Associate Dean & Professor of Sociology, gsteward@ucok.edu

Dr. Stacy Southerland, Professor of Modern Languages, Literature & Culture, ssoutherland@ucok.edu

Dr. David Macey, Professor of English, dmacey@ucok.edu

Ms. Karen Henderson, Director of Assessment, khenderson@ucok.edu