Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri...
-
Upload
charlene-gregory -
Category
Documents
-
view
217 -
download
1
Transcript of Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri...
![Page 1: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/1.jpg)
Asking the Right Questions
Assessing Language Skills
2008 Presentation to ATESL Central Local Sheri Rhodes,
Mount Royal College
![Page 2: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/2.jpg)
What is the first thing you think of when you hear the word
“evaluation”?“assessment”?
“test”?
![Page 3: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/3.jpg)
![Page 4: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/4.jpg)
![Page 5: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/5.jpg)
![Page 6: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/6.jpg)
![Page 7: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/7.jpg)
![Page 8: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/8.jpg)
Example of a Speaking Test
• One item• Single-word repetition• Testing a discrete point • Stimulus is the prompt, “Say now Shibboleth.”• Scoring criteria – right / wrong• Consequences – potentially dire!!
![Page 9: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/9.jpg)
Purpose of Assessment
Why?
![Page 10: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/10.jpg)
Formative vs. Summative
• How will we use the test?
![Page 11: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/11.jpg)
Reasons for Testing
• Placement Tests• Diagnostic Tests• Progress or Achievement Tests• Proficiency Tests• Portfolio Assessments
![Page 12: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/12.jpg)
What makes a good test?
• Validity• Reliability
![Page 13: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/13.jpg)
Test Design
Discrete
•A verb tense•A word
Integrative
•Use a variety of language and skills for a task
![Page 14: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/14.jpg)
Indirect
•Knowledge•Study
Direct •Activation•Integrative
Test Design
![Page 15: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/15.jpg)
Question Types - Indirect
•Multiple Choice
![Page 17: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/17.jpg)
Question Types - Indirect
•Transformation•Sentence re-ordering•Edit•Match
![Page 18: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/18.jpg)
Question Types - Direct
• May use items that look like indirect items • Others are specific tasks to the skills• Replicate real-life interaction
– Mirror an authentic purpose
![Page 19: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/19.jpg)
Question Types - Direct
• Remember– Items should look like the kind of tasks students
have been practicing in their lessons.– Create a “level playing field”.– Marking direct test items will be more
difficult than indirect items.
![Page 20: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/20.jpg)
Constructed-response assessments
• Fill-in assessment
![Page 21: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/21.jpg)
Performance-based assessments
• Simulates real-life language use• Offers open-ended tasks• Allows for creative and divergent responses• Often employs strategies• Requires students to synthesize information• Promotes recursive learning • Evaluates performance based on
well-defined critera
![Page 22: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/22.jpg)
Testing Reading and Listening
• Choose the best summary• Put pictures in order• Complete a phone message form (listening)• Fill out a summary form (reading)
![Page 23: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/23.jpg)
Testing Reading and ListeningGlobal understanding
![Page 24: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/24.jpg)
Testing Reading
• Many skills at different levels of reading from word recognition, reading speed, skimming techniques, etc.
• Primary evaluation is looking at reading comprehension at every level.
![Page 25: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/25.jpg)
Testing Listening
• Use a recorded audio track to ensure test relability
• Ensure the recording is clearly and easily audible
![Page 26: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/26.jpg)
Testing Writing
• Produce written texts – Leaflets based on information in a text– Compositions (narrative or discursive essays)– Transactional letters (reply to advertisement)
![Page 27: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/27.jpg)
Testing Speaking
• Seen as the most challenging of exams to prepare, administer and score.
![Page 28: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/28.jpg)
Testing Speaking
• Interview students• Pair tasks
– Compare two pictures– Solve a problem– Discuss a topic– Role-play a situation
![Page 29: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/29.jpg)
Grammar and Vocabulary
• Knowledge versus Ability• Isolated (discrete points) or integrated
![Page 30: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/30.jpg)
Marking
• Backwash– Positive– Negative
![Page 31: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/31.jpg)
Marking Writing
![Page 32: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/32.jpg)
A + ?
Are you ever tempted to do this?
![Page 33: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/33.jpg)
Marking Writing
• Holistic Scoring– Rubrics assigning a single score
• Analytic Scoring– 5 aspects of writing weighted and combined
(Jacob)– 7 scales (TEEP by Weir)– 3 scales (Michigan Writing Assessment)
![Page 34: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/34.jpg)
Other Applications of Rubrics and Scales
• Speaking• Performance of Tasks• Portfolio evaluation
![Page 35: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/35.jpg)
1. List the things you want to test.2. Decide how important each item is.
• Give them more time or space on the test
• Weight the marks to reflect the importance
3. Consider the test situation.4. Write the test. 5. Consult and share.6. Test the test. (the trial)7. Reflect and refine.
Steps in Test Design
![Page 36: Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.](https://reader038.fdocuments.in/reader038/viewer/2022110103/56649e715503460f94b6fd30/html5/thumbnails/36.jpg)
Another list (Brown and Hudson)• Require students to perform, create, produce, or do something• Use real-world contexts or simulations• Allow students to be assessed on what they normally do in class every day• Use tasks that present meaningful instructional activities• Focus on processes as well as products• Tap into higher-level thinking and problem-solving skills• Provide information about both the strength and weaknesses of students• Are multi-culturally sensitive when properly administered• Ensure that people, not machines, do the scoring using human judgment• Encourage open disclosure of standards and rating criteria• Call upon teachers to perform new instructional and assessment roles