Bureau for International Language Coordination

Julie J. DubeauBILC Secretary

Istanbul, TurkeyMay 24, 2010

General information

LNA report

BAT Report

Outline of Presentation

What does BILC do?

Annual Conference in Spring June 2009 hosted by Italy - “Bridging the Gap:

Language Requirements vs. Language Reality”

May 2010 hosted by Turkey - “Mapping The Road: Success in Language Training”

Professional Seminar in Fall October 2009 hosted by Denmark –

"The 21st Century Classroom: Keeping up with the Times!“

October 2010 hosted by Bulgaria – “Aligning Training and Testing in Support of Interoperability”

Ratified by the nations and promulgated by NSA in February 2009

English and French versions can be downloaded from the BILC website www.bilc.forces.gc.ca

Steering Committee will be discussing a proposal for admin change during conference

Responsible for STANAG 6001

What else does BILC do?

Language Training Assessments Assistance to National Testing Programmes Language Testing Seminars:

LTS & ALTS

Special Projects:Benchmark Advisory Test (BAT)Language Needs Analysis (LNA)

General Information ACT is tasked by IMS to investigate the

feasibility of introducing a computer based learning tool for use by nations to familiarize with NATO terminology…

BILC is investigating requirement through JSSG

“ACT should develop ADL solutions that will complement nations’ efforts in language training, understanding of different local government structures and understanding familial, clan and tribal cultures”.

ACT has requested BILC’s assistance

Please contact the BILC Secretariat if you are interested in providing an ADL product

Complementary ADL SolutionsIMS Report Dated November 26 2009, Bratislava Follow-up Tasking on

Counter Insurgency (COIN)

November 2008 Chairmen’s Meeting of the NTG in Brussels

The aim of this study was to show whether language requirements appeared to be set at appropriate levels to enable military personnel to perform their duties adequately in the NATO OPS context, in this case ISAF.

LANGUAGE NEEDS ANALYSIS OFNATO CRISIS ESTABLISHMENT (CE) POSTS

To broadly identify whether the mandatory SLP requested for the post was:

At Level ~ A: meaning that from the tasks described in the JD, the profile appeared to be adequate;

High ~ H: meaning that the SLP requested for the post appeared to be higher than the functions defined in the job description; or

Low ~ L: meaning that the SLP requested for the post appeared to be lower than the functions defined in the job description.

LANGUAGE NEEDS ANALYSIS OFNATO CRISIS ESTABLISHMENT (CE) POSTS

Summary of Results

Ratings Total = 609 CE Post JDs

A - At Level 355 (58.2%)

H - High 203 (33.3%)

L - Low 41 (6.7%)

H/L - Mixed 10 (1.6%)

Job Descriptions requiring SLPs of Level 1: 30 JDs required SLPs of 1111 & all were High.

Job Descriptions requiring mixed SLPs of level 1 & 2:

4 required SLPs of 2211 & all were at level. 2 required SLPs of 2221 & both were at level.

Job Descriptions requiring SLPs of level 2: 61 JDs required SLPs of 2222. 15/61 (26.2%) were at level. 43/61 (70.4%) were High.

Results: Levels 1 & 2

Job Descriptions requiring mixed SLPs of Levels 2 & 3: 156 (25.6%) JDs required mixed SLPs of Levels 2 and 3. Of these 156, 66 (42.3%) were considered as having

SLPs that were at level

Further Breakdown: 1 required SLP 3221 (High/Low depending on skill) 9 required SLP 3222 (7/9 High) 1 required SLP 3223 (High) 4 required SLP 3232 (3High/1Low) 118 required SLP 3322 (47 at level (39.8%), 56 High

(47.4%) & 3 Low (2.5%)) 23 required SLP 3332 (17 at level, 6 Low note that these

6 also req Dari)

Results: Levels 2 & 3

Results: Level 3

Job Descriptions requiring SLPs of Level 3:

301 (49.4%) required SLP 3333 240/301 (79.9%) were considered at level 44/301 (14.6%) were considered High 13 were considered as Low SLPs for functions, (all

req expert language knowledge with 3s in Dari and Pashto)

Job Descriptions requiring mixed SLPs of level 3 & 4:

43 profiles of 4343, from which 22/43 were at level (51.1%), and 21(48.8%) were not scored at level (some H some L)

1 profile of 4344 (Low), 1 profile of 4443 (High)

Results: Levels 3 & 4

Results: Level 4 Job Descriptions requiring SLPs of level 4:

8 profiles of 4444 (6/8 were considered at level, with comments referring to the functions as requiring expert language ability, diplomatic or sophisticated language use, 2 were H)

Out of the 53 JDs requiring some skills at level 4, 10 (18.8%) also req languages other than Eng such as French, Dari, Pashto, etc.

Out of these 10, 6 (60%) were at level.

Recommendations NATO CE Post JD SLPs should be reviewed SLPs should be based on an analysis of the

tasks performed by incumbents in relation to STANAG 6001 Ed 3.

It is strongly recommended that the next analysis of requirements be done with the assistance of BILC language experts.

CE posts master sheet should include the SLPs requested for the posts.

IMS-Tasking on Language Proficiency and Education on NATO Standards & Terminology. 12 February 2010

Analysis of CE/PE posts Setting realistic requirements Review of linguistic requirement for Officers for operationally

deployed duties‘ has been initiated

Benchmarking

What’s next for interoperability?

Benchmark Advisory Test (BAT)- PURPOSE

To provide an external measure against which nations can compare their national STANAG test results

To promote relative parity of scale interpretation and application across national testing programs

To standardize what is tested and how it is tested

Benchmark Advisory Test (BAT)

Allocation to 11 Nations (200 ‘free’ tests) Tests administered by LTI, the ACTFL

Testing Office via the Internet and Telephone

January 2010 – End of Benchmarking Process

Positive Feedback from Nations

Benchmark Advisory Test (BAT) Results

Listening Speaking Reading Writing

Black (11) 64% (18) 72% (10) 60% (11) 55%

White (18) 61% (18) 56% (18) 94% (18) 39%

Red (18) 89% (18) 83% (18) 83% (18) 50%

Blue (20) 85% (19) 47% (20) 55% (20) 60%

Teal (8) 0% -- -- (8) 75% (8) 38%

Maroon (16) 69% (15) 47% (14) 64% (16) 44%

Pink (18) 67% (18) 50% (18) 78% (18) 67%

Purple (12) 8% -- -- (13) 54% (13) 62%

Orange (13) 77% (13) 46% (13) 54% (13) 62%

Yellow (17) 24% (18) 0% (18) 33% (18) 0%

GreenNation did not release national test scores, but indicated that, on average, they were higher than those attained on the BAT.

Test purpose Testing method Use of plus ratings Alignment of author purpose, text type, and

reader/listener task Inadequate tester/rater norming (productive skills) Inconsistencies in the interpretation of STANAG

6001 Cut-off score setting, etc.

FACTORS CONTRIBUTING TO NON-ALIGNMENT

BAT – Way Ahead

Project successful Demand for administrations will dictate future

development needs and modes

BILC SC to formulate recommendations

Nations who require BILC support (post-BAT) can request it

QUESTIONS?

Enjoy the Conference!