© EUROCONTROL 2010
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010
Testing Air Traffic Controllers Testing Air Traffic Controllers for language proficiencyfor language proficiency
Adrian EnrightAdrian Enright
EUROCONTROLEUROCONTROL
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
1998 - 20101998 - 2010► ICAO Assembly 32ICAO Assembly 32ndnd Session Sept. 1998 Session Sept. 1998► PRICE study Group 2000 – 2006PRICE study Group 2000 – 2006► SARPs adopted by ICAO Council March 2003SARPs adopted by ICAO Council March 2003► ICAO Assembly resolution A36-11 (Oct 2006) ICAO Assembly resolution A36-11 (Oct 2006)
extends deadline to March 5extends deadline to March 5 thth 2011 2011► March 2010: how much further to go?March 2010: how much further to go?► Only 17/56 States indicate compliance …Only 17/56 States indicate compliance …
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
ICAO SupportICAO Support
► Action Plan for Implementation, regularly updatedAction Plan for Implementation, regularly updated► Checklist for regulators / service providersChecklist for regulators / service providers► LPRI workshops – 7th since Sept. 2006LPRI workshops – 7th since Sept. 2006► Language samples Language samples ► Doc 9835, Circular 318Doc 9835, Circular 318
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
Basics of test designBasics of test design
► Assess speaking and listening proficiencyAssess speaking and listening proficiency ICAO Rating scaleICAO Rating scale Holistic DescriptorsHolistic Descriptors
► Tests must demonstrate validity, reliability and Tests must demonstrate validity, reliability and practicalitypracticality
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
Language testing as seen by the air traffic controller …
START
FINISH
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
Holistic DescriptorsHolistic Descriptors
► A test should demonstrate:A test should demonstrate: visual and non-visual communicationvisual and non-visual communication communication in the context of the jobcommunication in the context of the job
► And the ability of the candidate to:And the ability of the candidate to: check, confirm and clarify informationcheck, confirm and clarify information handle an unexpected turn of eventshandle an unexpected turn of events use an accent or dialect intelligible to the aviation use an accent or dialect intelligible to the aviation
communitycommunity
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
Survey on testsSurvey on tests (Professor Charles Alderson of Lancaster University)(Professor Charles Alderson of Lancaster University)
► Survey conducted in 2008 and sent to numerous Survey conducted in 2008 and sent to numerous organisations using tests for pilots and controllersorganisations using tests for pilots and controllers
► Only 22 responsesOnly 22 responses► Additionally, all ICAO States contactedAdditionally, all ICAO States contacted
17/190 responded17/190 responded► Very little evidence attesting to the quality of testsVery little evidence attesting to the quality of tests► Unclear whether CAAs have the knowledge to Unclear whether CAAs have the knowledge to
judge the quality of testsjudge the quality of tests
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
Report concludesReport concludes► ““We can have very little confidence in the We can have very little confidence in the
meaningfulness, reliability and validity of the meaningfulness, reliability and validity of the aviation language tests currently available for aviation language tests currently available for licensure”.licensure”.
► ““The consequences of inadequate language tests The consequences of inadequate language tests being used in licensing pilots, air traffic controllers being used in licensing pilots, air traffic controllers and other aviation personnel are almost too and other aviation personnel are almost too frightening to contemplate”.frightening to contemplate”.
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
But, if I can say that my …But, if I can say that my …
► test specifications and test content match the needs of test test specifications and test content match the needs of test takerstakers
► test examiners (raters and interlocutors) are appropriately test examiners (raters and interlocutors) are appropriately trained and accreditedtrained and accredited
► tests have been adequately trialledtests have been adequately trialled► test takers are aware of how and on what they will be test takers are aware of how and on what they will be
tested and how their performance will be ratedtested and how their performance will be rated► the reliability of marking is monitored the reliability of marking is monitored
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
Then I’m on final for a safe landing …
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
But I must ask myself …But I must ask myself …► Am I confident that my test is fully compliant with Am I confident that my test is fully compliant with
ICAO LPRs?ICAO LPRs?► Are my controllers adequately prepared for the Are my controllers adequately prepared for the
test?test?► What actions am I taking to reassure my What actions am I taking to reassure my
controllers, especially the older ones, if they fail to controllers, especially the older ones, if they fail to get level 4?get level 4?
► Would I accept a controller with a level 4 in Test Would I accept a controller with a level 4 in Test “XYZ”?“XYZ”?
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
Beyond 2011?Beyond 2011?
► Programmes to maintain language proficiencyProgrammes to maintain language proficiency► Review and audit of testing systemsReview and audit of testing systems► At least 1 LPR Workshop per yearAt least 1 LPR Workshop per year
exchange experienceexchange experience education assistanceeducation assistance maintain standards of testingmaintain standards of testing
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
Food for thought …Food for thought …
► What is your organisation doing to prepare What is your organisation doing to prepare controllers for the language proficiency test?controllers for the language proficiency test?
► Why are we seeing a negative approach to Why are we seeing a negative approach to language testing from some of our controllers? language testing from some of our controllers? And what are we doing about it?And what are we doing about it?
► Whose responsibility is itWhose responsibility is it to find answers to to find answers to these and other questions?these and other questions?
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
Ensuring that student controllers have at least level 4 proficiency will lay the foundation for the future.
ICAO – Regional Workshop on Language Proficiency Requirements Implementation
Rome, March 3rd to 5th 2010 © EUROCONTROL 2010
Thank youThank you
Top Related