The testing process

Decision-making in test creation

Golden Rule

• No testing without thinking!

• Decide what it is that you want to know (measure) and keep that in mind throughout the test development process

Three Key Concepts

• Validity

• Reliability

• Practicality

Define the domain

• What am I trying to test?– Speaking ability

• What aspect of speaking ability?– Discrete skills versus integrative?– General sample of language?– Performance on specific task?

• Order a meal at a restaurant• Give a presentation in class

Consider possible tasks

• Goal: to assess general speaking ability• Five-minute interview

– Spontaneous versus guided– Use of prompts

• Giving directions• Picture description• Picture sequence (“What happened?”, make a

story)• Topical discourse (recent events, vegetarianism)• Situations (Here is a menu. Order a pizza.)

Consider evaluation criteria

• Pass / Fail• Impressionistic scoring• Holistic scoring based on rubric

– Good = accent not distracting, uses appropriate language and vocabulary, fairly fluent delivery

• Analytic scoring– Fluency = 10 pts.– Accuracy = 10 pts.– Appropriate vocabulary = 10 pts.– Appropriate politeness = 10 pts. – Willingness to communicate = 10 pts.

Administration procedures

• When and where?– During class in the back of the room? – After school in the library?

• How?– Warm-up / cool-down period?– Tape or video recorded?– Checklist for grading?

Reporting procedures

• What will the student receive?– A grade– a comment about their performance– Extra homework

Criterion

The authenticity question

• There is always a gap between the test and the criterion

• Direct versus indirect versus real