lessons from the FCE/CAE revision project - НБУold.nbu.bg/PUBLIC/IMAGES/File/departments/foreign...

Cambridge Whiffs of Change:

lessons from the FCE/CAE revision project

Lets benchmark!

What did they change?What did they not change?What considerations informed these decisions?Is there anything we can use to improve our B2 test?

150 years of assessment

Cambridge Assessment was established as theUniversity of Cambridge Local ExaminationsSyndicate (UCLES) in 1858 to administer localexaminations for students who were not members ofthe University of Cambridge

http://bp2.blogger.com/_0oOkTuc1hkU/R_DHAOcQzTI/AAAAAAAAABg/HotpowQGY8w/s400/cambridge.jpghttp://www.cambridgeassessment.org.uk/ca/digitalAssets/114053_ca-logo3.jpg

The CPE 1913 Examination

Oral: Dictation hourReading aloud and Conversation hour

Written:(a) Translation from English into French or German 2 hours(b) Translation from French or German into English, and questions on English Grammar 2 hours(c) English Essay 2 hours(d) English Literature 3 hours(e) English Phonetics 1 hours

CPE 1953 paper

Oral: Dictation, Reading aloud and Conversation

Written:(a) English Literature 3 hoursor Science Textsor English Life and Institutionsor Survey of Industry and Commerce

(b) Use of English 3 hoursor Translation from and into English

(c) English Language (composition and a passage of English with language questions) 3 hours

Certificate of Proficiency in English, English Life and Institutions paper, question 12:

It is said that the English love thesea.What effect has this had, in youropinion, on their way of life?

CPE candidates in an examination hall in Rome, June 1956

1946 LCE Total time: 5 hrs + the Oral

Oral: Dictation, Reading aloud, Conversation

Written: (a) Translation from and into Englishor Prescribed Books 2 hours (b) Composition and Language 2 hours

1975 FCETotal time: 5 hrs 55 m

Paper 1: CompositionPaper 2: Reading ComprehensionPaper 3: Use of EnglishPaper 4: Listening ComprehensionPaper 5: Interview

The charm of history and its enigmatic lesson consist in the fact that, from age toage, nothing changes and yet everything is completely different.

Aldous Huxley

The FCE and CAE Review Project20042008

CAMBRIDGE ESOL : RESEARCH NOTES : ISSUE 30 / NOVEMBER 2007

http://www.cambridgeesol.org/rs_notes/rs_nts30.pdf

Studies in Language Testing (SiLT) forthcoming volume

http://www.cambridgeesol.org/rs_notes/rs_nts30.pdf

FCE Paper 1: ReadingTime: 1 h 15 min 35 items

Part Task type Tests ability to No. Q

Multiple matching

A text preceded by a list of headings or summarising sentences for each paragraph

read for gist and main idea

6 or 7

Multiple choice A text followed by four-option questions

understand detail, opinion, deduce meaning

7 or 8

Gapped text A text with missing sentences or paragraphs, to be found from a jumbled list

understand text structure (coherence and cohesion)

6 or 7

Multiple matching

Several sections of text candidates match statements or questions about the text to the relevant sections

scan for specific information

13-15

FCE Paper 1: ReadingTime: 1 h 15 min 35 items


Multiple choice A text followed by four-option questions.

understand detail, opinion, deduce meaning, ..

7 or 8

Gapped text A text with missing sentences or paragraphs, to be found from a jumbled list.


6 or 7

Multiple matching Several sections of text candidates match statements or questions about the text to the relevant sections.

scan for specific information, ..

13-15

FCE Paper 1: ReadingTime: 1 h 30 items


Multiple choice A text followed by four-option questions.

understand detail, opinion, attitude, tone, purpose and main ideas, deduce meaning

8

Gapped text A text with missing sentences, to be found from a jumbled list.


7

Multiple matching

Several sections of text candidates match statements or questions about the text to the relevant sections.

scan for specific information, detail, opinion and attitude

15

FCE 2008Paper 1: Reading

Changes Rationalereduce time from 1 h 15 min (35

questions) to 1 h (30 questions)reduce overall text length only slightly

from 2,000-2,300 words to 2,000 wordsremove current Part 1 (matching

headings or summary sentences to paragraphs of a text)

broaden scope of multiple-choice tasks on Part 2 (now Part 1) text

use Part 3 gapped text task only in sentence form (not paragraphs)

time effectiveness: shorter format but broader scope

inclusion of essential reading skills: infer writers tone, attitude and purpose

simplified format: no alternative tasks, item number fixed by part

.

But why did the matching headings task go?

How to Make Tests More Reliable

Exclude items which do not discriminate well between weaker and stronger students.

[] A small number of non-discriminating items may be kept at the beginning of a test to give candidates confidence and reduce the stress they feel.

Hughes, Arthur. Testing for Language Teachers. CUP, 2003.

FCE Paper 2: WritingTime: 1 h 30 min

Part Task type Tests ability to No. words

Compulsory task

Read some input of up to 250 words and use the information to write either a formal or informal letter.

5 content points to be covered

request/give information, explain, describe, suggest, complain, apologise, express enthusiasm, gratitude, etc., in an appropriate register

120-180

Choice of four

A situationally-based task to write one of the following:

articlediscursive compositionletter reportstory

Final question based on set reading texts.

write for a specific purpose and audience, with accurate language control

120-180



Compulsory task

Read some input of up to 250 words and use the information to write either a formal or informal letter, or


request/give information, explain, describe, suggest, complain, apologise, express enthusiasm etc. in an appropriate register

120-180

Choice of four


articlediscursive compositionletter reportstory..


write for a specific purpose and audience, in a specified genre

120-180



Compulsory task

Read some input of up to 160 words and use the information to write either a formal or informal letter, or an e-mail.


request/give information, explain, describe, suggest, complain, apologize, express enthusiasm etc. in an appropriate register

120-150

Choice of four


articleessayletter reportstoryreview


write for a specific purpose and audience, in a specified genre

120-180

FCE 2008Paper 2: Writing

Changes Rationalereduce time from 1 h 30 min to 1 h 20

minreduce input for Part 1reduce output for Part 1 from 180 to 150 broaden scope by

including production of emailsinclude review task in Part 2

reduce set texts from five to two, with questions related to specific book

reinstate the essay

writing more briefly is a relevant real life skill writing email messages is a relevant life skill

review is appropriate to the candidature in terms of both experience and lang. skills

On VRIP analysis,smaller input and output was found not to

affect adversely the validity and reliability of the exam

smaller input and output was found to affect positively the impact and practicality of the exam

But why didnt the set books go?

Backwash: advantages to be gained from studying set texts at FCE and CAE level

To provide a source of pleasure and stimulationTo encourage a lifelong interest in readingTo widen cultural and literary horizonsTo promote contact with other media: film, theatre, etc.To engage the reader on a subconscious level in terms of structure, vocab, collocation, which may influence the readers own writing skills

FCE Paper 3: Use of EnglishTime: 1 h 15 min 65 items

Part Task type Tests knowledge of No. Q

Multiple choice cloze

Choose 1 word from a set of 4 to complete a gap in a text.

collocations, phrasal verbs, idioms, phrases

15

Open cloze Think of one word to complete a gap in a text.

grammatical items 15

Word formation Form a new word from a given stem word to complete a text.

prefixes and suffixes 10

Key word transformations

Complete a paraphrase of a given sentence using 2 to 5 words incl. a given key word.

grammatical and lexical items

10

Error correction Read a text containing errors identify one extra word in some lines, mark others as correct.

proofread and edit a text for grammatical correctness

15

FCE Paper 3: Use of EnglishTime: 1 h 15 min 65 items





15


mostly grammatical items 15






10

FCE Paper 3: Use of EnglishTime: 45 min. 42 items





12


mostly grammatical items 12






8

FCE 2008Paper 3: Use of English

Changes Rationalereduced from 1 h 15 min

(65 questions) to 45 minutes (42 questions) - about 40% less

remove Part 4 (error correction)

time effectivenessthe grammar that the error

correction task tests is tested elsewhere in the paper

the skill of proofreading and editing can be tested through the writing paper

But why did they play down Use of English importance?

FCE Paper 4: ListeningTime: 40 min

Part Task type Tests ability to understand

No Q

Multiple choice

Listen to 8 unrelated dialogues of approx. 30 sec. each answer a three-option question on each.

function, location, roles and relationships, mood, attitude, intention, feeling or opinion

8

Sentence or note completion

Listen to a monologue or dialogue of about 3 minutes complete sentences or notes with info heard.

main points, opinion, detail or specific information, deduce meaning

10

Multiple matching

Listen to 5 topic-related monologues of 30 seconds each. Match each speaker to the correct option from a list of 6.

specific information, understand attitude and/or events from the text

5

Multiple choice orTrue/False

Listen to a 3-minute monologue or dialogue and answer three-option questions about it, or mark statements as true/false.


7



No Q

Multiple choice



8

Sentence or notecompletion

Listen to a monologue or dialogue of about 3 minutes complete sentences or notes with info heard.


10

Multiple matching

Listen to 5 topic-related monologues of 30 seconds each match each speaker to the correct option from a list of 6.


5

Multiple choice orTrue/False

Listen to a 3-minute monologue or dialogue answer three-option questions about it, or mark statements as true/false.


7



No Q

Multiple choice



8

Sentence completion

Listen to a monologue or dialogue of about 3 minutes complete sentences with info heard.


10

Multiple matching

Listen to 5 topic-related monologues of 30 seconds each match each speaker to the correct option from a list of 6.


5

Multiple choice

Listen to a 3-minute monologue or dialogue answer three-option questions about it.


7

FCE 2008Paper 4: ListeningChanges Rationale

only sentence completion in Part 2 (note-taking removed as an option)

only multiple-choice in Part 4 (true-false removed as an option)

sentence completion more suitable to the level than note-taking

standardised format will improve reliability by ensuring comparability between test versions

standardised format has a positive impact on test-takers and teachers

Ockhams razor: Entities are not to be multiplied without necessity.

Make everything as simple as possible, but not simpler.Albert Einstein

FCE Paper 5: SpeakingTime: ~14 min

Part Task type Tests ability to Length

Interview The interlocutor asks each candidate questions about their own life.

use language for social purposes

3 min.

Individual long turn (1 min) and response (20 sec)

The first candidate is given two pictures to talk about for 1 minute. The second candidate makes a short response, then the roles are reversed.

describe, compare and contrast, speculate, express opinion

1 min. + 20 sec. each

Two-way collaborative task

Candidates are given spoken instructions, along with visual and written prompts, and work on a decision-making task.

invite/give opinions, agree/disagree, ask for/make suggestions, evaluate options, take turns, negotiate a decision

3 min.

Three-way discussion

The interlocutor starts a discussion on the topic in Part 3

give & justify opinions, agree & disagree, take turns and develop a discussion

4 min.

FCE 2008Paper 5: Speaking

Changes Rationaleadd written replication of

task instructionswill help candidates recall

the taskswill improve objectivity and

impact

CAE Paper 1: ReadingTime: 1 h 15 min 45 items


Multiple matching

A text with several sections candidates match statements or questions about the text to the relevant sections.


12-18

Gapped text A text with missing paragraphs which have to be found from among a jumbled list.


6-7

Multiple choice A longer text followed by four-option questions.

find detail, recogniseinference, and interpret the text

5-7

Multiple matching



12-22

CAE Paper 1: ReadingTime: 1 h 15 min 34 items


Multiple choice 3 short themed texts, each followed by 2 four-option questions (as in CPE)

understand detail, opinion, tone, purpose, main idea, attitude, text organization (exemplification, comparison, reference)

6

Gapped text A text with missing paragraphs which have to be found from among a jumbled list.


6

Multiple choice A longer text followed by four-option questions.

find detail, recogniseinference, and interpret the text

7

Multiple matching



15

CAE 2008Paper 1: Reading

Changes Rationalereduce number of questions from

c. 45 (1 h 15 min) to 34 (1 h 15 min)minimal change in word length to

c. 3,000 wordsremove current Part 1 (multiple

matching)introduce themed texts (as in CPE

Paper 1)fix the no. of questions per part

avoid overlap in testing focus (former Part 1 = Part 4)

widen range of genres and registers: positive backwash effect

exam family resemblance: ensure a more straightforward progression from FCE through to CPE

CAE Paper 2: WritingTime: 2 h


Compulsory task

Read some input of up to 400 words and use the information to write one or two of the following: letter, report, proposal, article, info sheet, brochure, memo, leaflet, e-mail.

evaluate, express opinions, hypothesize, persuade; in an appropriate register and style

250

Choice of four

A situationally-based task to write one of the following: letter, report, proposal, article, info sheet, review, contribution to a longer piece, brochure, memo, leaflet, discursive composition, competition entry.


write for a specific purpose and audience in a specified genre; varying focuses incl. comparing, giving advice, justifying, persuading

250

CAE Paper 2: WritingTime: 1 hour 30 min


Compulsory task

Read some input of up to 150 words and use the information to write one of the following: letter, report, proposal, article.

evaluate, express opinions, hypothesize, persuade; in an appropriate register and style

180-220

Choice of four

A situationally-based task to write one of the following: letter, report, proposal, article, info sheet, review, contribution to a longer piece, essay.


write for a specific purpose and audience in a specified genre; varying focuses incl. comparing, giving advice, justifying, persuading

220-260

CAE Paper 3: English in UseTime: 1 h 30 min 74 items





15

Open Cloze Think of one word to complete a gap in a text.




Error correction Identify extra word OR correct spelling & punctuation.

proofreading and editing 15

Register transfer Use a text on one register to complete a text in another.

register 10

Discourse cloze Choose a sentence or phrase to complete a text from a list of 9.

text organisation 6

CAE Paper 3: Use of EnglishTime: 1 h 50 items





12

Open Cloze Think of one word to complete a gap in a text.




Gapped sentences

Three discrete sentences with a gap each write the gapped word that is common to all three sentences.

polysemy 5




8

CAE Paper 4: ListeningTime: 40 min


No Q

Sentence ornote completion

Listen to a monologue of approx. 2 min. twice complete sentences or take notes.

specific information 8-10

Sentence ornote completion

Listen to a monologue of approx. 2 min. ONCE complete sentences or take notes.

specific information 8-10

Multiple matching

Listen to 5 themed monologues of 30 sec. each do two multiple matching tasks , each requiring selection of 8.

gist, attitude, main points; interpret context; deal with a variety of accents and styles of delivery

10

Multiple choiceor sentence completion

Listen to a conversation b/n 2 or 3 speakers of approx. 4 min. complete sentences or answer four-option questions about it.

gist, specific information and attitude

6-12

CAE Paper 4: ListeningTime: 40 min


No Q

Multiple choice: three exchanges

Listen to 3 short dialogues answer 2 three-option questions on each.

feeling, attitude, opinion, purpose, function, agreement, course of action, gist, detail

6

Sentence completion

Listen to a monologue of approx. 3 min. twice complete sentences or take notes.

specific information 8

Multiple matching

Listen to 5 themed monologues of 30 sec. each do two multiple matching tasks , each requiring selection of 8.

gist, attitude, main points; interpret context; deal with a variety of accents and styles of delivery

10

Multiple choice Listen to a conversation b/n 2 or more speakers of approx. 4 min. answer four-option questions.

gist, specific information and attitude

6

Is there anything we can change in the NBU B2 test?

One must change one's tactics every ten years if one wishes to maintain one's superiority.

- Napoleon Bonaparte

Do we really want toattach greater importance to Use of English (30 p.) than Listening (15 p.) or Writing (15 p.)?test the skill of deducing meaning from context 7 times over (out of 18 shots) rather than broaden the scope of skills tested?test reading for gist (match headings), for one half of the reading test (12 points out of 24)?avoid testing coherence and cohesion at the receptive level while highlighting it at the productive level?compromise reliability by having 9 True/False questions in the Listening (out of 15)?have one single text in the C1Listening rather than broaden scope (dialogue/monologue, etc.) and give test-takers a fresh start with each of several (cf. CAE -10)

Wouldnt perhaps the NBUtest be better served by

......

Change does not necessarily assure progress, but progress implacably requires change.

NBU seminar

October 29th, 2008Presenter: Danya Doganova

Cambridge Whiffs of Change:Lets benchmark!The FCE and CAE Review ProjectFCE Paper 1: Reading Time: 1 h 15 min 35 items FCE Paper 1: Reading Time: 1 h 15 min 35 items FCE Paper 1: Reading Time: 1 h 30 items FCE 2008Paper 1: ReadingFCE Paper 2: Writing Time: 1 h 30 min FCE Paper 2: Writing Time: 1 h 30 min FCE Paper 2: Writing Time: 1 h 20 min FCE 2008Paper 2: WritingBackwash: advantages to be gained from studying set texts at FCE and CAE levelFCE Paper 3: Use of English Time: 1 h 15 min 65 items FCE Paper 3: Use of English Time: 1 h 15 min 65 items FCE Paper 3: Use of English Time: 45 min. 42 items FCE 2008Paper 3: Use of EnglishFCE Paper 4: Listening Time: 40 min FCE Paper 4: Listening Time: 40 min FCE Paper 4: Listening Time: 40 minFCE 2008Paper 4: ListeningFCE Paper 5: Speaking Time: ~14 minFCE 2008Paper 5: SpeakingCAE Paper 1: Reading Time: 1 h 15 min 45 items CAE Paper 1: Reading Time: 1 h 15 min 34 items CAE 2008Paper 1: ReadingCAE Paper 2: Writing Time: 2 hCAE Paper 2: Writing Time: 1 hour 30 minCAE Paper 3: English in Use Time: 1 h 30 min 74 items CAE Paper 3: Use of English Time: 1 h 50 items CAE Paper 4: Listening Time: 40 minCAE Paper 4: Listening Time: 40 minIs there anything we can change in the NBU B2 test?Do we really want toWouldnt perhaps the NBU test be better served by NBU seminar

lessons from the FCE/CAE revision project - НБУold.nbu.bg/PUBLIC/IMAGES/File/departments/foreign...

Documents

Transcript of lessons from the FCE/CAE revision project - НБУold.nbu.bg/PUBLIC/IMAGES/File/departments/foreign...