Peggy Garza Associate BILC Secretary

For Testing Programs

BILC Standardization Initiatives and

Conference Objectives

• Standardization Efforts

- Language Testing Seminars - Tailored Workshops - Toolbox for Language Testers- STANAG 6001 Testing Conferences

• Conference Topics and Goals

Outline

• Language Testing Seminar (LTS)

• Advanced Language Testing Seminar (ALTS)

• Language Standards and Assessment Seminar (LSAS)

BILC Testing Seminars

LANGUAGE TESTING SEMINAR(LTS)

Language Testing Seminar (LTS)

• Developed and conducted by BILC testing experts

• Background: A BILC initiative aimed at promoting a common interpretation of STANAG 6001 and consistency in language proficiency testing

• Specific objective: To give NATO/PfP language teaching professionals the opportunity to practice developing test items based on STANAG 6001

Language Testing Seminar (LTS)

• Held since Autumn 2000 at the Marshall Center, Garmisch, Germany

• Two-week graduate-level seminar

• Offered twice times a year (February and September)

• More than 350 graduates from NATO/PfP/MD nations

ADVANCED LANGUAGE TESTING SEMINAR

(ALTS)

Advanced Language Testing Seminar (ALTS)

• Need for continuing professional development for test developers identified

• Stakeholders’ meeting in September 2008

• Result: Advanced Language Testing Seminar (ALTS)

• Validated in 2009

• Target audience: LTS graduates who continue to work on STANAG 6001 testing- STANAG 6001 Edition 4 standardization

sessions- Focus on Level 3 for practical exercises- Opportunity to exchange best practices

• Modular approach to seminar

Advanced Language Testing Seminar (ALTS)

• Pre-seminar assignments- Approximately 40 hours

• Three-week seminar- Module 1: Receptive Skills: Listening and Reading (1 week)- Module 2: Productive Skills: Speaking and Writing (1 week)- Module 3: Test Analysis (3 days)- Module 4: Managerial Issues (2 days)

Advanced Language Testing Seminar (ALTS)

NEW

LANGUAGE STANDARDS AND ASSESSMENT SEMINAR

(LSAS)

Language Standards and Assessment Seminar (LSAS)

3-11 July 2013• For language teachers and program managers• Objectives– STANAG 6001 familiarization– Guidelines for aligning instruction to standards– Test and measurement principles– Classroom assessment and practical exercises

Testing Seminar Schedule in 2014

• Language Testing Seminar (LTS)‒ 10-21 February‒ 15-26 September

• Advanced Language Testing Seminar (ALTS)‒ 2- 20 June

• Language Standards and Assessment Seminar (LSAS)‒ 23 Apr – 1 May ‒ 3-11 December

• Recommended by ALTS participants

– Samples of listening and reading items at levels 1-3

– Samples of speaking and writing tests at levels 1-3, including plus levels

– Frequently Asked Questions (FAQs)

– Self-Assessment Outline for STANAG 6001 Testing Programs

STANAG 6001 Toolbox for Testers

BILC Website

www.natobilc.org

Tailored Workshops for National Testing Programs

• Ukraine‒ November 2011, 2012

• Moldova‒ May 2012

• Russia‒ June 2013

Tailored Workshops

• Zagreb, 2009– Focus on standardization, moderation, and validation

• Sarajevo, 2010– Dates: 31 August – 2 September

– Focus on testing speaking

• Stockholm, 2011– Dates: 6-8 September

– Focus on testing writing

• Copenhagen, 2012– Dates: 3-6 September, 2012

– Focus on testing listening

STANAG 6001 Testing Conferences

2013 STANAG 6001 Testing Conference

• Some of your questions:– Our stakeholders have questions. Is there advice for

how best to communicate with the stakeholders?– Our stakeholders are asking us to test more and more

people. What should we tell them?– We have developed items. What’s next?– How do we know if our tests are good?– We need to hire new testers. Is there a position

description for testers? How can we recruit for testers?

– How can we keep professionally current ?

2013 STANAG 6001 Testing Conference• Conference format– Presentations– Guest speakers from NATO and SHAPE– Panel discussions

o Resolving challenges

– Study Groups meetings and reportso Making a validity argumento Washbacko Action planso Renorming and standardizing testerso Piloting items and prompts

Specific Conference Goals• Review and clarify the entire test construction process• Identify action plans for test development and

administration • Provide ideas for selecting the appropriate testing

techniques• Discuss piloting and pretesting • Learn more about determining test validity• Identify practicality issues and discuss lessons learned• Discuss best practices in renorming and standardizing

testers• Learn about innovations in testing in the BILC community

– Computer-delivered testing– Proficiency and specialist testing– Testing at Level 4

Test Life Cycle Diagram

Communicating with the Stakeholders

• Identify the stakeholders• Coordinate with the stakeholders during the needs

analysis and planning stages– Stakeholders approve personnel and resources

• Close the loop with the stakeholders during the Test Life Cycle– Action plans– Technical reports– Updates– Washback

Test Construction

Needs Analysis

Planning

Design

Development

Administration

Monitoring

Test Specifications

Pre-testing / AnalysisStandard SettingEvaluation/review

Evaluation

Revision

Stage 2

Stage 3

Stage 4

Stage 1

Stage 5

Stage 6

Development Stage Part 1

Text selection

Item writing

Item review I (moderating)

Piloting on a small scale

Item review II (editing)

Assembly of trial test

• Retain

• Revise

• Reject More items needed?

Development StagePart 2

Pre-testing/trialling on a larger scale

Statistical Analyses

Standard Setting

Item banking

Assembly of final version of test

Item revision• Retain

• Revise

• Reject

2013 STANAG 6001 Testing Conference

• Expectations– Active participation and interaction– Frank and open discussions– Provide answers to the difficult questions– Create a forum for future networking and

cooperation• BILC appreciates your participation and we look

forward to a very productive conference!