Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity...
-
Upload
jake-heyer -
Category
Documents
-
view
214 -
download
2
Transcript of Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity...
![Page 1: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/1.jpg)
Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test
Tianfei (Faye) Jiang & Peipei (Christina) DongCTL 3799 Communication and L2 Learning in WorkplaceProf. Julie KerekesFebruary 11, 2014
![Page 2: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/2.jpg)
OUTLINE
• Background & Context• Characteristics of the PhonePass test– Procedure– Setting
• Frameworks• Analysis of the PhonePass test• Conclusion• Critique & Implications• Questions & Comments
![Page 3: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/3.jpg)
BACKGROUND
• Global technology has quickly progressed to provide more accessible business modes of telecommunication, like telephone, video phone, emails and other internet/computer-mediated system in workplace.
• Tests over telecommunication networks are usually highly commercialized..
![Page 4: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/4.jpg)
CONTEXT
PhonePass Spoken English Test-10 (SET-10)• WHAT IS IT? An expedient assessment tool for screening job candidates’ ability
in spoken English• WHAT IS IT FOR? Evaluating employees for promotions and measuring service
employees with an accurate and objective test of their spoken English spoken capabilities.
• HOW DOES IT WORK? This test is a 10 minutes spoken English test for non-native English speakers, which is administered over a language-line telephone and immediately scored by computer. It consists of 5 sections, based on four diagnostic subscores which includes sentence mastery, vocabulary, fluency, and pronunciation. The cost ranges from $25 to $40.
• WHEN & WHERE IS IT? Anytime and Anywhere with a land-line phone system available.
• HOW IS IT SCORED? The first four sections are recorded and scored by computers, while the last section is not scored, which is open for review.
![Page 5: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/5.jpg)
CONTEXT
Purpose:to see if the characteristics of the
domain of the test tasks found in the PhonePass SET 10 test correspond to real life domains of nontest tasks.
• Key Words: telecommunication; spoken language
test; authenticity;
![Page 6: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/6.jpg)
CHARATERISTICS IN PHONEPASS TEST
Access to an
Internet-connected Computer
Download and Print out a Test
Paper
Access to a Land-line
Telephone
Follow Further
Instruction in the Phone
Testing
• PROCEDURE (requires employee’s computer technology & telephone technology capabilities)
![Page 7: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/7.jpg)
CHARATERISTICS OF PHONEPASS TEST• SETTING
• “Please read No. 3.”• “The winter in Toronto is freezing.”
A: Reading Aloud
• “War broke out.”• “There are three basic ways in which a story
might be told to someone.”
B: Repeating Sentence
• “What season comes before spring?”• “Does a blanket go on the bed or the wall?”
C: Giving Short Answers to Question
• “in/ bed/ stay”• “we wondered/ would fit in here/ whether the
new piano”
D: Building Sentences
• “Which would you prefer to live, in the big city or a small town? And why?”
E: Responding to Open Questions
![Page 8: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/8.jpg)
CHARATERISTICS OF PHONEPASS TEST
• Any location & time • High degree of speed• 10 minutes
• visual and aural channels
• Grammatical knowledge
• Textual cohesion
• Sentence mastery• Vocabulary• Fluency• Pronunciation• No reciprocal tasks
Flexibility & Short Length Input
ExpectationsTasks
![Page 9: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/9.jpg)
FRAMEWORKAuthenticity• “the degree of correspondence of the characteristics of a given
language test task to the features of a target language use task.” (Bachman & Palmer, 1996, p.23)
• “the developers need to show that the content of the test tasks is representative of the demands of the corresponding task outside the test situation, and that the scoring reflects this.” (Luoma, 2004)
• “authenticity of input, purpose, and outcome” (Lewkowicz, 2000)
Test Fairness Framework “access, administration, validity, absence of bias, and social
consequence.” (Kunnan, 2004)
![Page 10: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/10.jpg)
ANALYSIS: Authenticity
Rationale PhonePass Test (PART A)
Created real-life discourse Structured speaking task
Grammatical Knowledge Without correct sentence production and comprehension
Textual Knowledge Read in random order of items
Functional and Sociolinguistic Knowledge
Standard pronunciation
![Page 11: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/11.jpg)
ANALYSIS: Authenticity
Rationale PhonePass Test (PART B)
Proof of comprehension Sounds reorganization and repetition & Memory
Speech production Without discourse creation and interpretation
![Page 12: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/12.jpg)
ANALYSIS: Authenticity
Rationale PhonePass Test (PART C)
Vocabulary of professional employment level
low level of lexical knowledge
Extended production responses Limited production responses
Sociolinguistic knowledge N/A
![Page 13: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/13.jpg)
ANALYSIS: Authenticity
Rationale PhonePass Test (PART D)
Fluency (constructing, reading and repeating the sentences)
Inadequate constructing
Pragmatic knowledge N/A
Sentence mastery One in six chance of guessing word groups
![Page 14: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/14.jpg)
ANALYSIS: Authenticity
Rationale PhonePass Test (PART E)
Authenticity Extended production responsesNot scored
![Page 15: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/15.jpg)
ANALYSIS: Fairness
Rationale PhonePass Test (PART A)Minimal bias No biased content or language
Educational access (speaking & listening)
Advantages for test takers who have had more opportunities to learn and become familiar with the tasks on the test
Telecommunication access (land-line telephone, printer & internet connected computer)
The absence of supporting telecommunication in underdeveloped countries
Appropriate accommodations Uncertain physical (e.g. sight and hearing) impairment
Rescoring and reevaluation N/A
![Page 16: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/16.jpg)
CONCLUSIONS
• The performance on these tasks does not necessarily predict or reflect the speaking ability of the test taker to function in real-life domain of workplace.
• PhonePass SET 10 is considered relatively less authentic.
• The test tasks are relatively less specific to a particular domain and are not interesting and engaging.
![Page 17: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/17.jpg)
CRITIQUE
Strengths: The author used strong theoretical basis to support his
analysis. It provides a valuable evaluation as a reference, emphasizing
the importance of authenticity, to language assessment in workplace.
Weakness: This article does not offer further practical implication in
telecommunication in workplace. It would be better if the author provides more constructive
suggestions to the phone language test.
![Page 18: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/18.jpg)
IMPLICATIONS
• Since telecommunication lacks non-verbal compensation, such as body languages and facial gestures, the requirement of language is much higher than daily face-to-face communication.
• This analysis brings readers to have more deep provoking thoughts about telecommunication networks, for example, how can we communicate effectively without these compensation, and how can we minimize the misunderstandings in telecommunication.
• For employers, phone interview or language assessment via telephone should focus more on the authentic communicative competence than on linguistic performance.
• Communication effectiveness is a critically important criteria in assessing individual's language ability in workplace. Standards for communicative language capability in workplace is needed to be further explored.
![Page 19: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/19.jpg)
Questions AND/OR Comments?
![Page 20: Critical Review: Chun, C. W. (2006). An Analysis of a Language Test for Employment: The Authenticity of the PhonePass Test Tianfei (Faye) Jiang & Peipei.](https://reader036.fdocuments.in/reader036/viewer/2022081602/55170ba4550346f5558b5311/html5/thumbnails/20.jpg)
----- Tianfei Jiang & Peipei DongFebruary 11, 2014