FOAI-9 @AnkaraSpeaking Rubrics

Focus Group A

Discussion Question #1

Does CEFR play an important role in your speaking rubric design? If yes, how?

It’s everywhere!!!

CEFR• “the backbone”• lots of things are “based on” it

To be precise..

CEFR (can&do) Objectives Syllabus

Assessment Tools

Speaking Rubrics

BUT!!

STEALING ADAPTING

Why adapting?• different objectives• different materials > syllabi• different levels• different exam types e.g. Proficiency, Mid-

TermSo,- adapting is essential!

Discussion Question #2

Should rubrics for external exams be used as a model?

“YES” because

• practical• efficient• already been piloted• reliable

Should rubrics for external exams be used as a model?

ADAPTATION

BUT• How realistic are they?• Is the wording appropriate?• Does it match with the exam format,

objectives etc?

Discussion Question #3

Who should be involved in rubric design?

Ideally..everyone involved in the programme- instructors- speaking examiners- course book writers- students

In reality..• assessment team• skill groups• supervisors, coordinators, course leaders

(constant feedback)• speaking examiners• experienced teachers

No Child Left BehindStudents;- do not take part in the “designing phase”

BUT!!- their feedback is valued (sharing the rubric

> focus groups)

Discussion Question #4

Which type of rubric outweighs the other in terms of advantages? Holistic or Analytic?

AnalyticAdvantages

secure

more accurate

justifiable

standard

more efficient after familiarity

easier to give feedback

lay out

Disadvantages

difficulties in scoring the “best” and “worst” practices

sometimes students get higher than they deserve

Holistic

Advantages

practical / time saving

teacher preferability

more efficient /requires less training

Disadvantages

Teachers attempt to compare students

“outliers” > good at 3 criteria but bad at 1

difficult for feedback

difficult to justify and not secure

not practical, lots of notetaking

BUT!• it depends on the exam type

Hybrid Practices• Using an analytic rubric but scoring

holistically • Using them both and crosschecking

Discussion Question #5

There’s a myth that analytic rubrics are much stricter than holistic ones. Do you think it is correct? Or is it vice versa?

YES but• it depends on the descriptors and their

wording• also depends on how rubrics are

interpreted / standardization

Discussion Question #6

What discrete elements/bands should be included in speaking rubrics?

Just all of them J

It actually depends on

• student profiles• nature of the test• coverage of objectives in speaking classes• institutional valuesa new item- listener’s effortshared (common) descriptors- 1response to task, 2lexis, 3grammar, 4fluency, 5accuracy, 6range

Discussion Question #7

How detailed should the descriptors be?

!!!• depends on the task• should have a simple wording• in addition to student versions, a detailed

version for teachers to refer• use of quantifiers should be carefully

analyzed and defined