Stages of test development and common test techniques (1)
-
Upload
maury-martinez -
Category
Education
-
view
1.199 -
download
0
description
Transcript of Stages of test development and common test techniques (1)
![Page 1: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/1.jpg)
STAGES OF TEST DEVELOPMENT
&COMMON TEST TECHNIQUES
DANIEL F. CALI. O.
![Page 2: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/2.jpg)
STAGES OF TEST
DEVELOPMENT
![Page 3: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/3.jpg)
STATING THE PROBLEM
SPECIFICATIONS
WRITING AND MODERATING
ITEMS
ITEMS ON NATIVE
SPEAKERS
ON NON-NATIVE
SPEAKERS
ANALYSIS OF RESULTS
CALIBRATION OF SCALESVALIDATION
HANDBOOKS FOR TEST TAKERS
TRAINING STAFF
OUTLINESTAGES OF TEST DEVELOPMENT
![Page 4: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/4.jpg)
• SHOULD BE A TEST DESIGNED BY A SINGLE PERSON OR A TEAM?
TEAM!
• IT IS DIFFICULT TO DESIGN A TEST BECAUSE OF…
…OBJECTIVITY
…CRITICISMS
…NATIVE COMMAND OF THE LANGUAGE
![Page 5: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/5.jpg)
STEPS…
![Page 6: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/6.jpg)
STATING THE PROBLEM
QUESTIONS
KIND?
PURPOSE?
ABILITIES?
HOW DETAILED?
HOW ACCURATE?
WASHBACK?
CONSTRAINTS?
FINAL
PROFICIENCY
PROGRESS
DIAGNOSTICPLACEMENT
RESULTS
EXPERTISE
FACILITIES
TIME
![Page 7: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/7.jpg)
• ONCE THE PROBLEM IS CLEAR, IT IS ALSO IMPORTANT TO:
GATHER INFORMATION ON ALREADY EXISITING TEST DESIGNED FOR SIMILAR SITUATIONS!
STATING THE PROBLEM
![Page 8: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/8.jpg)
WRITING SPECIFICATIONS FOR THE TEST
• A) CONTENT
• B) STRUCTURE, TIMING, MEDIUM/CHANNEL AND TECHNIQUES
• C) CRITERIAL LEVELS OF PERFORMANCE
• D) SCORING PROCEDURES
![Page 9: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/9.jpg)
A) CONTENT
• INCLUDE CLEAR SPECIFICATIONS REGARDING:
• SKILLS (SUB-SKILLS)
• TYPE OF TEXTS
• ADDRESSEES
• LENGTH OF TEXTS
• TOPICS
• (READABILITY)
• STRUCTURAL RANGE
• VOCABULARY RANGE
• DIALECT, ACCENT AND STYLE
• SPEED OF PROCESSING
![Page 10: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/10.jpg)
B) STRUCTURE, TIMING, MEDIUM/CHANNEL AND TECHNIQUES
• TEST STRUCTURE
• NUMBER OF ITEMS
• NUMBER OF PASSAGES
• TIMING
• MEDIUM
• KIND OF TEST TECHNIQUE(S)
![Page 11: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/11.jpg)
C) CRITERIAL LEVELS OF PERFORMANCE
• ACCURACY
• APPROPRIACY
• RANGE
• FLEXIBILITY
• SIZE
![Page 12: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/12.jpg)
D) SCORING PROCEDURES
• WHAT RATING SCALE WILL BE USED?
• HOW MANY PEOPLE WILL RATE EACH PEACE OF WORK?
• WHAT HAPPENS IF TWO OR MORE RATERS DISAGREE ABOUT A PIECE OF WORK?
![Page 13: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/13.jpg)
WRITING AND MODERATING ITEMS
• A) SAMPLING
• B) WRITING ITEMS
• C) MODERATING ITEMS
![Page 14: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/14.jpg)
A) SAMPLING
HOW THE TEXTS ARE GOING TO BE CHOSEN?
![Page 15: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/15.jpg)
B) WRITING ITEMS
• TRY TO LOOK AT THE TEST THROUGH THE EYES OF TEST TAKERS!
• AN ITEM WITHOUT A KEY IS INCOMPLETE!
• “THE BEST WAY TO IDENTIFY ITEMS THAT HAVE TO BE IMPROVED OR ABANDONED IS THROUGH THE PROCESS OF MODERATION”.
![Page 16: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/16.jpg)
C) MODERATING ITEMS• INTERVATION OF TWO COLLEAGUES:
![Page 17: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/17.jpg)
INFORMAL TRIALLING OF ITEMS ON NATIVE SPEAKERS
• 20 OR MORE…
• THEY SHOULD BE SIMILAR TO THE GROUP BEING TESTED IN TERMS OF:
AGE
EDUCATION
GENERAL BACKGROUND
![Page 18: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/18.jpg)
TRIALLING OF THE TEST ON A GROUP OF NON-NATIVE SPEAKERS
SIMILAR TO THOSE FOR WHOM THE TEST IS INTENDED
• PROBLEMS IN ADMINISTRATION AND SCORING CAN BE NOTED.
![Page 19: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/19.jpg)
ANALYSIS OF RESULTS OF THE TRIAL; MAKING OF ANY NECESSARY CHANGES
• STATISTICAL AND QUALITATIVE ANALYSIS.
HOW DIFFICULT ARE THE ITEMS?
DISCOVER MISINTERPRETATIONS (TO BE MODIFIED OR DROPPED)
![Page 20: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/20.jpg)
CALIBRATION OF SCALES
• IT IS IMPORTANT TO COLLECT SAMPLES OF PERFORMANCE AND ASSIGN EACH OF THEM TO A POINT ON THE RELEVANT SCALE.
![Page 21: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/21.jpg)
VALIDATION
LOW STAKES
TEST
HIGH STAKES
TEST
TOEFL
POP QUIZE
S
![Page 22: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/22.jpg)
WRITING HANDBOOKS FOR TEST TAKERS, TEST USERS AND STAFF
RATIONALE DEVELOPMENT DESCRIPTION SAMPLE ITEMS
ADVICES ON PREPARING FOR
TAKING THE TEST
TEST SCORES INTERPRETATIO
NS
TRAINING MATERIALS
TEST ADMINISTRATIO
N
![Page 23: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/23.jpg)
TRAINING STAFF
NEED TO BE TRAINE
D
INTERVIEWERS
RATERS
SCORERSCOMPUTER OPERATORS
PROCTORS
![Page 24: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/24.jpg)
COMMON TEST
TECHNIQUES
![Page 25: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/25.jpg)
OUTLINE
COMMON TEST
TECHNIQUES
GAP FILLING
MULTIPLE CHOICE
TRUE OR
FALSE
SHORT-ANSWERS
DEFINITION
![Page 26: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/26.jpg)
WHAT ARE TEST TECHNIQUES?
• MEANS OF ELICITING BEHAVIOR
RELIABLE AND VALIDBEHAVIOR
BEHAVIOR WHICH CAN BE RELIABLY
SCORED
LESS TIME AND
EFFORT
BENEFITIAL WASHBACK
TECHNIQUES
![Page 27: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/27.jpg)
MULTIPLE CHOICE ITEMS
• STEM
• DISTRACTOR
![Page 28: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/28.jpg)
MULTIPLE CHOICE ITEMS
ADVANTAGES DISADVANTAGES
WASHBACK
LACK OF DISTRACTORS
GUESSING
PERFORMANCE
TESTING OF RECEPTIVE
SKILLS
RELIABLE SCORING
DIFFICULT TO WRITE ITEMS
GREAT DEMAND ON TIME AND EXPERTISE
CHEATING
![Page 29: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/29.jpg)
YES/NO AND TRUE/FALSE ITEMS
MULTIPLE CHOICE (2 ITEMS)
TOO INFORMAL
REASONS?Reliability and validity
![Page 30: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/30.jpg)
SHORT-ANSWER ITEMS
READING TEST
S
LISTENIN
G TEST
S
![Page 31: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/31.jpg)
GAP FILLING ITEMS
• IT IS A VALUABLE TECHNIQUE AS LONG AS THE CONTEXT IS PROVIDED!
• A: WHAT WILL HE DO?
• B: I THINK HE ______ RESIGN.
WILL, MIGHT, COULD MAY, ETC.
BUT:
A: I WONDER WHO THAT IS.B: IT ______ BE THE DOCTOR.A: HOW CAN YOU BE SO CERTAIN?
![Page 32: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/32.jpg)
• Now you know what a teacher should do when preparing tests, what advices would you give?
• (Each one should only mention only 1 advice)
Names Advices
Carolina Think of the applicability to real situations ar real life situations.
Paola Put in your students´ shoes.
Mauricio Consider multiple intelligences.
![Page 33: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/33.jpg)
THANK
YOU!
![Page 34: Stages of test development and common test techniques (1)](https://reader035.fdocuments.in/reader035/viewer/2022081717/5549d527b4c905a97a8b4e80/html5/thumbnails/34.jpg)
REFERENCES
• Huges, Arthur. Testing for Language Teachers. New York: Cambridge UP, 2003.