RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES
description
Transcript of RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES
![Page 1: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/1.jpg)
RATERS:SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES
•ICAO WORKSHOP ON LANGUAGE PROFICIECY •Baku, Azerbaijan, December 7-9, 2005
•Evgenia Povaliy, Aviation English Instructor •Sergey Melnichenko, Deputy Director•Moscow ATC Training Center
![Page 2: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/2.jpg)
CASE STUDYCASE STUDY
Accidents where language was a Accidents where language was a causal or contributing factorcausal or contributing factor
Phraseologies only are not Phraseologies only are not enoughenough
Plain language is needed in non-Plain language is needed in non-routine and urgent situationsroutine and urgent situations
![Page 3: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/3.jpg)
English as an Instrument English as an Instrument of Communicationsof Communications
70 per cent of 28,000 safety reports 70 per cent of 28,000 safety reports were connected with communicationswere connected with communications
70 per cent of those who communicate 70 per cent of those who communicate are non-native English speakersare non-native English speakers
No other existing live or artificial No other existing live or artificial language is either better or worse than language is either better or worse than EnglishEnglish
English has become a “lingua franca”English has become a “lingua franca”
![Page 4: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/4.jpg)
Manual on the implementation Manual on the implementation of ICAO Language Proficiency of ICAO Language Proficiency
Requirements (Doc 9835)Requirements (Doc 9835)
Licensing authorities should require Licensing authorities should require specialized evaluation of those who do specialized evaluation of those who do not demonstrate Level 6 proficiencynot demonstrate Level 6 proficiency
Phraseologies-only testing is not Phraseologies-only testing is not appropriateappropriate
Demonstration of actual speaking and Demonstration of actual speaking and listening ability is requiredlistening ability is required
Until 5 March 2008 States may continue Until 5 March 2008 States may continue to use procedures they currently have to use procedures they currently have in place to assess English proficiency of in place to assess English proficiency of flight crews and controllersflight crews and controllers
![Page 5: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/5.jpg)
Requirements for Requirements for ratersraters
Higher education in LinguisticsHigher education in Linguistics Level 6 proficiency according to the Level 6 proficiency according to the
ICAO ScaleICAO Scale Highly developed moral standardsHighly developed moral standards Aviation background Aviation background Relevant training, preferably in an Relevant training, preferably in an
English-speaking countryEnglish-speaking country
![Page 6: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/6.jpg)
THE SCALETHE SCALE
Who has seen it?Who has seen it? Who is happy with it?Who is happy with it? Who has used it for Who has used it for
benchmarking evaluation?benchmarking evaluation? Who has been taught to use Who has been taught to use
the Scale?the Scale?
![Page 7: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/7.jpg)
TESTSTESTS
EntryEntry PlacementPlacement DiagnosisDiagnosis ProgressProgress ProficiencyProficiency
![Page 8: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/8.jpg)
ENTRY TESTENTRY TEST
Enables Airline Operators and Enables Airline Operators and ANS providers to avoid new ANS providers to avoid new personnel that is likely to personnel that is likely to have problems in 2008 have problems in 2008
because of poor knowledge of because of poor knowledge of English at presentEnglish at present
![Page 9: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/9.jpg)
PLACEMENT TESTPLACEMENT TEST
Enables Airline Operators and Enables Airline Operators and ANS providers to map out the ANS providers to map out the
level of existing personnellevel of existing personnel
![Page 10: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/10.jpg)
DIAGNOSIS TESTDIAGNOSIS TEST
Enables linguists to identify Enables linguists to identify the needs of individuals the needs of individuals (problem areas, course (problem areas, course
duration and its content)duration and its content)
![Page 11: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/11.jpg)
PROGRESS TESTPROGRESS TEST
Enables Airline Operators, Enables Airline Operators, ANS providers and linguists to ANS providers and linguists to
mark progress/no mark progress/no progress/regress progress/regress and to use and to use
other training path when other training path when neededneeded
![Page 12: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/12.jpg)
PROFICIENCY TESTPROFICIENCY TEST
Shows the actual level of Shows the actual level of English language proficiency English language proficiency
according to ICAO Scaleaccording to ICAO Scale
![Page 13: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/13.jpg)
Available TestsAvailable Tests
ChineseChinese KoreanKorean American (Oral Proficiency Interview)American (Oral Proficiency Interview) British (Mayflower College)British (Mayflower College) Russian (Moscow ATC Center)Russian (Moscow ATC Center) Eurocontrol (ELPAC – July 2007)Eurocontrol (ELPAC – July 2007)
![Page 14: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/14.jpg)
DrawbacksDrawbacks
Computer – no interactionComputer – no interaction Oral Proficiency – time and Oral Proficiency – time and
manpower-consumingmanpower-consuming THERE IS NO IDEAL TESTING TODAYTHERE IS NO IDEAL TESTING TODAY
BUT…BUT… ““Any test at all” is not better than Any test at all” is not better than
“no test” (Doc 9835)“no test” (Doc 9835)
![Page 15: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/15.jpg)
Steps to Make Steps to Make
Introduce ICAO qualification Introduce ICAO qualification requirements into national legislationrequirements into national legislation
Select (through independent linguistic Select (through independent linguistic institutions) and train ratersinstitutions) and train raters
Evaluate current level of pilots and Evaluate current level of pilots and controllerscontrollers
Ensure they are adequately taught Ensure they are adequately taught General and Aviation EnglishGeneral and Aviation English
Evaluate proficiency level of pilots and Evaluate proficiency level of pilots and controllers in 2008controllers in 2008
![Page 16: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/16.jpg)
LAST BUT NOT LEASTLAST BUT NOT LEAST
Code of ethicsCode of ethics
![Page 17: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/17.jpg)
Ethical PrinciplesEthical Principles (ILTA Code of Ethics)(ILTA Code of Ethics)
Raters shall have respect for humanity and dignity of each Raters shall have respect for humanity and dignity of each of their test takersof their test takers
Raters shall have all information in confidence and use Raters shall have all information in confidence and use professional judgment in sharing itprofessional judgment in sharing it
Raters should adhere to all relevant ethical principles Raters should adhere to all relevant ethical principles embodied in national and international guidelines when embodied in national and international guidelines when undertaking any trial, experiment or other research activityundertaking any trial, experiment or other research activity
Raters shall not allow the misuse of their professional Raters shall not allow the misuse of their professional knowledge or skillsknowledge or skills
Raters shall continue to develop their professional Raters shall continue to develop their professional knowledge, sharing it with colleaguesknowledge, sharing it with colleagues
Raters shall share the responsibility of upholding the Raters shall share the responsibility of upholding the integrity of language testing professionintegrity of language testing profession
Raters shall strive to improve the quality of language Raters shall strive to improve the quality of language testing, assessment and teaching servicestesting, assessment and teaching services
Raters shall be mindful of their obligations to the societyRaters shall be mindful of their obligations to the society Raters shall regularly consider the potential effects on all Raters shall regularly consider the potential effects on all
stakeholders, reserving the right to withhold their stakeholders, reserving the right to withhold their professional services on the grounds of conscience professional services on the grounds of conscience
![Page 18: RATERS: SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES](https://reader035.fdocuments.in/reader035/viewer/2022070503/5681563e550346895dc3e672/html5/thumbnails/18.jpg)
THANK YOU!THANK YOU!
Evgenia Povaliy, Aviation English InstructorEvgenia Povaliy, Aviation English Instructor [email protected]@atcm.ru Sergey Melnichenko, Deputy Director Sergey Melnichenko, Deputy Director [email protected] www.aviaenglish.ru Moscow ATC Training CenterMoscow ATC Training Center