Best Practices

Recommendations for using ETS Criterion® in Your College

Course

Based on data collected by the ISWI Assessment Team from the Spring 2010 Pilot

Presentation by Kim Morin 2010

Effective Uses of Criterion®

• Data from Spring 2010 indicates the following practices as more effective:o Multiple assignments o Multiple revisions over timeo Students correct “fixable errors” as identified by

Criterion®o Early orientation for students so they practice

using the program and gain a better understanding of how it works

o Students are given ample time between first and final submissions to make revisions

 Effective Uses of Criterion® (cont)  • Instructor uses the tools in Criterion® to give

additional feedback. • Students are motivated to use the Criterion®

program as part of the revision process by correcting "all fixable errors" ("0-0-0")

• Criterion® is used as an editing tool to help students think critically about their writing

• Students and faculty understand that the computer program will make mistakes; it is one part of the editing process

• Scoring is set at the appropriate level (College- 1st year or College 2nd year)

Less Effective Uses of Criterion®

• Data from last spring indicates the following practices as LESS effective:o A single paper at the end of the semestero Single submissions o Instructor gives little or no training or feedback regarding

Criterion® o Particular stylistic requirements within a subject (technical

writing, business formats, field specific terminology)o Using ETS Criterion as "Extra Credit" 

Less Effective Uses of Criterion® (cont)

• A long term paper that serves as a final assignment for the semester

• Assignments that incorporate lots of citations • The expectation that Criterion® will catch all

writing errors• The expectation that Criterion® can accurately

grade or evaluate any writing assignment• Students submit assignments through Criterion®

but do not know how to retrieve or use feedback.

So What About Holistic Scoring?

Should we even use the Holistic Score?

It is true that many faculty found the holistic score feature inaccurate or confusing and preferred using only the “text editor” feature.

More Effective Uses of Holistic Scoring:

• Writing assignments based on the 5-paragraph persuasive essay model

• Topics available on ETS Criterion® web site• Writing assignments 500-1000 words in length• Writing assignments based on stating a case or

expressing personal opinion• Holistic Score is regarded as just one piece of

information within the writing process

Less Effective Uses of Holistic Scoring  (When “text editor” may be a better option!)

• Assignments designed by faculty as “Scored Instructor Topics” (unless using Criterion® guides)

• Research Papers that require lots of citations• Single paragraphs • Assignments longer than 1000 words• Writing that requires tables and graphs• Holistic Score is used as a final grade or goal for an

essay (for example, must achieve level 5)

• Warning: A score of 6/6 does NOT mean that a paper is perfect!

Recommendations for Fall 2010 Pilot

• There should be a minimum of three writing assignments spaced throughout the semester.

• Each assignment should require multiple submissions (at least two) including a final draft that is submitted through Criterion®.

• Faculty should provide students with an on-screen, in-class demonstration of Criterion® early on in the semester o Students should submit a “practice assignment.”

Recommendations for Fall 2010 Pilot (cont)

• Faculty should seek ways to provide instructor comments that help students understand the feedback from Criterion®

• Faculty should let students know that the Learning Center has tutors trained to use Criterion®

• Faculty who wish to use Criterion® primarily as a proofreading/editing tool are encouraged to limit the features viewed by students o “Text Editor” o Grammar, Usage, and Mechanics

Recommendations for Fall 2010 Pilot (cont)

• Faculty need to become familiar with the program so they can be selective about which features to use: (ie Holistic Scoring, Make a Plan, etc)

• Students should be given a goal to reach before submitting the final revisiono Correct all “fixable errors”o Show improvement between first and last submissionso Journal reflection about their revision/editing process

Next Time, We Will Know Even More!

Thanks for you participation!

Acknowledgements

Bill Covino, ProvostEllen Junn, Associate ProvostChristina Leimer, Director, IRAPISWI Assessment Team:Kim Morin, Faculty, Theatre ArtsJohn Beynon, Faculty, English Henry Delcore, Faculty, AnthropologyBradley Hufft, Faculty, MusicJennifer Ivie, Faculty, PsychologyEileen Walsh, Faculty, HistoryAsao Inoue, Faculty, EnglishJohn Farrell, Faculty, HistoryDimitri Rogulkin, IRAP, StaffDiane Robinson, ISWI, CSALT, Staff