Best Practices for Best Results - British Council · PDF fileTOEFL iBT ® writing tasks...

Copyright © 2012 by Educational Testing Service. All rights reserved. ETS, the ETS logo, TOEFL and TOEFL iBT are registered trademarks of Educational Testing Service (ETS). IELTS is a trademark of the University of Cambridge ESOL Examinations. 19030

Best Practices for Best Results: Using TOEFL iBT® Test Scores

for Admissions Decisions

Going Global

Sandy Bhangal, ETS GlobalDiane Schmitt, BALEAP


What is the TOEFL® Test?

• The TOEFL test measures the ability of nonnative speakers to use and understand English as it is spoken, written and heard in college and university settings

• Used primarily for admission into universities where English is the language of instruction– Accepted at both undergraduate and postgraduate

levels

• Also used for immigration, licensing, scholarships and placement

2


The TOEFL® Test Format

Section Description Testing Time Score ScaleReading 3-4 passages from academic texts;

12-14 questions per passage60-80 min. 0-30

Listening 4-6 lectures, some with classroom discussion, 6 questions each;

2-3 conversations, 5 questions each

60-90 min. 0-30

Break - 10 min. -Speaking 2 tasks to express an opinion;

4 tasks to speak based on reading and listening passages

20 min. 0-30

Writing 1 integrated task (write based on what is read and listened to);

1 independent task (support an opinion on a topic)

50 min. 0-30

Total - - 0-120

3

Note-taking is allowed in all sections.


• Scoring process ensures fairness and integrityof scores– Rigorous rater training– Daily calibration for Speaking and Writing raters– Multiple raters for Speaking and Writing

• Three to six raters for Speaking responses• Four ratings for Writing responses

– Scoring is separate from the testing site tomaintain score integrity• Raters see and hear worldwide responses,

not just from one country• Anonymous raters do not know the examinee

4

Scoring Sets the TOEFL® Test Apart


The TOEFL® program uses the e-rater®

automated scoring engine in a limited and responsible way for the Writing section

• Independent Writing task– one human rater + one automated rating

• Integrated Writing task– one human rater + one automated rating

Combines the judgment of humans for content and meaning and the consistency of automated scoring for linguistic features

5

What TOEFL® does


Measures the mechanical aspects of writing sufficiently, but content is measured only at a basic level

The kinds of test questions designed only for automated scoring are more coachable and promote rote learning, not practice of real communication skills

• Repeat a sentence out loud• Read a sentence out loud

At present, no automated scoring engine can evaluate content and meaning at the sophisticated level tested by the TOEFL iBT® writing tasks without also having the contributions of a human scorer

6

Automated Scoring


TOEFL iBT® Scores

• Four skill scores – Reading 0-30– Listening 0-30– Speaking 0-30 – Writing 0-30

• Total score: 0-120• Score reports sent within 10 calendar days

after test

TOEFL® test scores are valid for two years from the test administration date


Security-enhanced Score Reports

8


Performance Feedback on Examinee Report


TOEFL iBT® Reading Scores

10


TOEFL iBT® Speaking Scores

11


• Holistic score• Test takers are rated on:

– Topic development (especially for integrated tasks)

– Delivery– Language use

• Ratings are 0–4 from rubrics– Converted to a scaled score of 0–30

Speaking Scores


Speaking Benchmarks

• What is a benchmark?

• Listen to the four Benchmarks for SpeakingTask #2 – Independent Speaking


Independent Speaking Task


Benchmark Speaking Responses

Benchmarks – independent speaking

Annotations provide narrative description of the level

Each score represents a range of ability

• Level 4

• Level 3

• Level 2

• Level 1


Integrated Speaking Task

• Task #5 – Problem/Solution• A Listening/Speaking campus situation topic• Consists of one source, a listening passage,

a single question

The test taker hears a conversation between two people about a campus topic – less than 90 seconds

The test taker is asked to briefly summarize the problem and to express a personal opinion about the topic

• Note-taking allowed• Preparation time: 20 seconds• Response time: 60 seconds


Speaking Responses

• Listen for the following key points• Student #1 – stressed out because she has

too much homework – a paper to write, two exams to study for and math problems to complete

• Student #2 – recommends two possible solutions• Ask her professors for an extension on

paper and math work• Create a schedule to manage her time

and focus her energies

• Then test taker states which solution he/she prefers and explains why

Sample #1 Sample #2


TOEFL iBT® Writing Scores

20


TOEFL® Score Requirements

• ETS does not determine which students must take the TOEFL® test, nor does ETS establish the TOEFL score requirements institutions/agencies should use in making their decisions

• Each institution sets its own policy on whether a TOEFL score is required as well as the minimum TOEFL score they will accept

• ETS supports each university’s individual decision on who should be required to submit a TOEFL score

21


General Guidelines for Good Score Use

• Base evaluation of an applicant’s readiness to begin academic work on all available relevant information, not solely on TOEFL® scores– Look for other supporting evidence of

English-proficiency in the application

• Use score ranges rather than rigid cut scores on the TOEFL test to help make admissions decisions

22


TOEFL UKBA scores

23

B1 LEVEL B2 LEVEL

Reading 4 Reading 18

Listening 9 Listening 17

Speaking 16 Speaking 20

Writing 13 Writing 17

Totals 42 Totals 72


Setting TOEFL® Score Requirements

• There are many options for setting TOEFL®

score requirements– Require total score only– Require a total score but review applicant

section scores for the skills most important to your program

– Require a total score and a minimum section score for the most important skill(s)

– Require minimum section scores only (most restrictive)

• It is important to consider scores from all English-proficiency tests in the same manner

24


Using TOEFL® Section Scores

Reading Listening Writing Speaking TotalStudent 1 18 22 16 24 80Student 2 20 20 20 20 80Student 3 25 25 16 14 80

25

All three students have a total score of 80, but they have different profiles of English skills.

Which student is best for your program?


Review your score requirements regularly

• How much English is enough? – Different departments have different needs– What language skills are important for success in

this program?

• Level of ESL support available

• Competitive situation– Higher score requirements could reduce

applications– Lower scores could result in student failure, need

for ESL support or slowing down the rest of the class

26


• Conduct a Standard Setting Session using the free TOEFL® Standard Setting CD-ROM

• Select a panel of 10-12 faculty members, includingESL and admissions staff

• Follow procedures outlined on CD-ROM– Listen to TOEFL test items and responses at various

levels and determine minimum acceptable performance

– Analysis of panelists’ decisions determines panel’s selection of minimum acceptable TOEFL requirements for each skill

27

Selecting TOEFL® Score Requirement


Test and Score Data Summary for TOEFL iBT®

• Percentile data• Score means listed by total population,

native language, native country, gender and purpose for taking the test (i.e., Undergraduate, Graduate, or Professional License)

• Based on data from previous testing year• Annual publication available for download

from the Research section of the TOEFL®

website (www.ets.org/toefl) by the beginning of Q2

28

http://www.ets.org/toefl


TOEFL iBT® Score Comparison Tool

• Universities have asked for a way to compare TOEFL iBT® and IELTS® scores

• ETS collected more than 1,100 sets of TOEFL iBT and IELTS scores from individuals who had taken both tests

• An equipercentile approach was applied to each IELTS band score to develop the comparison table– Example: a TOEFL iBT score of 115 represents

the 90th percentile of the test takers in the sample, the same as an IELTS score of 8.5

• Related research report: “Linking TOEFL iBT Scores to IELTS Scores”

29


30

Comparing TOEFL iBT ® and IELTS® Scores

Research report at http://www.ets.org/toefl/comparescores


TOEFL iBT® Score Comparison Tool

• The comparison table was created as an initial step for universities that wanted to compare TOEFL iBT® and IELTS® scores

• ETS recommends the table be used for purposes of general understanding and not for establishing cut scores

• It is not appropriate to use the comparison table to project an applicant’s IELTS or TOEFL iBT score

• Follow up validity studies are recommended to validate the score comparisons

31


TOEFL® Information and Contacts

TOEFL® website for institutions:www.ets.org/toefl/institutions

TOEFL website for students: http://www.toeflgoanywhere.org

Sandy BhangalUK Client Relations [email protected] Schmitt [email protected]

ETS — Listening. Learning. Leading.®Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo, LISTENING. LEARNING. LEADING., GRE and POWERPREP are registered trademarks of Educational Testing Service (ETS). BETTER BY DESIGN and SCOREITNOW! are trademarks of ETS. All other trademarks are property of their respective owners.

http://www.ets.org/toefl/institutions

http://www.toeflgoanywhere.org/

mailto:[email protected]

mailto:[email protected]


Questions?

Best Practices for Best Results - British Council · PDF fileTOEFL iBT ® writing tasks...

Documents

Transcript of Best Practices for Best Results - British Council · PDF fileTOEFL iBT ® writing tasks...