A Challenging Dimension to the

Assessment of Speaking: CEFR through ELF

Zeynep Mine Derince & Pelin Tekinalp Çakmak

EAQUALS International Conference

Budapest

25-26 April 2014

Outline

Integration of CEFR into the preparatory school curricula in

Turkey

Marmara University School of Foreign Languages

Before CEFR

After CEFR

Implementation of CEFR--In-service Training Sessions

ELF training workshop

Suggestions

Preparatory Schools in Turkey

One year intensive program

Prep Schools consist of students of various departments: medicine, engineering, social sciences, communication, tourism etc.

Prep students study in:

100 % English-medium departments

30 % English-medium departments

Proficiency levels of students vary at the beginning

Preparatory Schools in Turkey

Foundation and state prep schools

Number of preparatory schools: 150

Number of English instructors in each school: 50-150

Number of preparatory students in each school: 500-2500

Schools have testing offices, curriculum units, TDUs.

A survey for Prep Schools

30 schools were included.

Questions on missions, demographic info, practices.

11 out of 20 state schools have started using CEFR (only

half), however, 9 out of 10 foundation schools (all but one)

have employed CEFR.

Mission statements of

School of Foreign Languages

Which of the following sections are

included in the exams specified below?

0

5

10

15

20

25

30

35

Grammar Speaking Reading Listening Writing

Placement

Proficiency

Achievement

What does the graph display?

These schools claim that they have adopted a CEFR-based

curriculum, however ,

It is interesting to see in the results that grammar rates as

much as the reading and writing:

22 schools out of 30 ask grammar explicitly in the

proficiency, 27 in the achievement exam and 29 in the

placement exam.

Speaking is not tested at all in half of these schools.

Before CEFR

Curriculum

Elementary, Pre-intermediate and Intermediate levels

Grammar, Listening, Reading, Writing

Skills disintegrated

Assessment

1st achievement- Grammar and Listening

2nd achievement –Reading and Writing

3rd achievement- Grammar and Listening

4th achievement –Reading and Writing

Proficiency- Grammar-Listening-Reading- Writing-(Grammar made up 50% of the final score.)

After CEFR Curriculum

3 levels: A1-A2-B1

4 skills integrated instruction

Assessment

1st Progress Test- Reading, Writing, Listening & Speaking

2nd Progress Test- Reading, Writing, Listening & Speaking

3rd Progress Test- Reading, Writing, Listening & Speaking

4th Progress Test- Reading, Writing, Listening & Speaking

Proficiency- Reading, Writing, Listening & Speaking

All 4 Skills are assessed out of 25 points.

Before the 1st Progress Exam

Two analytical rubrics prepared by the curriculum team: A

rubric for basic users /A rubric for independent users

A short workshop on rubrics

Purpose of the study

1st progress exam: speaking skill scores are lower that the

other three skill scores.

Focused on two factors: Being familiar with the illustrative

scales of CEFR and English as a Lingua Franca.

The goal of this study is to find out if familiarization with the

CEFR and ELF enables raters to assign reliable scores in line

with other skills assessed in the test.

In-service Training

Materials

Relating Language Examinations to the Common European

Framework of Reference for Languages: Learning, Teaching,

Assessment (CEFR) –MANUAL

CEFTrain (www.CEFtrain.net)

Video samples of local learners

Workshops on ELF, native vs. non-native speakers

In-service Training- Session 1

Before the session

Section 3.6 in the CEFR (English pp. 3336) that describes

the salient features of the levels.

During the session

Sorting the text for the different levels in Table A1-The task is

to assign the descriptors to levels A1C2.

After the initial input on the CEFR,

Sorting the individual descriptors from a CEFR scale

More in-depth work and discussion of CEFR levels in

relation to the descriptors for speaking

In-service Training- Session 2

Before the session

The website CEFTrain (www.CEFtrain.net) -a hands-on

example.

During the session

The aim is to provide participants with sufficient awareness

of the CEFR levels to analyse and assess test tasks and

performances in relation to the CEFR levels.

In-service Training- Session 3

Before the session

Analytic rubrics for scoring

During the session

Rating video samples of local learners: non-native to non-native

interaction

In-service Training- Session 4:

English as a Lingua Franca

ELF has emerged as a way of referring to

communication of English between speakers with

different first languages, across lingua cultural

boundaries. ( House, 1999; Seidlhofer, 2002, 2005)

English is being shaped at least as much by its non-

native speakers as by its native speakers

For the majority of its users English is a Foreign

language, yet the vast majority of verbal exchanges in

English do not involve any native speakers.

In-service Training- Session 4:

English as a Lingua Franca

Global use of English is on the rise

Questioning the native speakers’ long accepted ownership of English (Widdowson, 1994)

Native speaker norms act as a powerful gate-keeping device which has little to do with intelligibility but a great deal with socio-economic factors.

Non-native speakers as language users in their own right (Cook,1999).

The importance of respecting and building on local values, beliefs and ways of doing things (Holliday, 1994).

Kachru’s circles

In-service Training- Session 4:

English as a Lingua Franca

Some language testing specialists have demonstrated how

irrelevant and potentially damaging it can be to insist on native

speaker norms when assessing the proficiency of English in the

Outer Circle( Lowenberg, 2000).

An argument that would have to be investigated for Expanding

Circle contexts too.

In-service Training- Session 4:

English as a Lingua Franca

ELF speech community should no longer be regarded as norm-dependent, but as norm-developing and norm-providing.

Speakers from different L1 backgrounds communicate through ELF and they co-construct ‘English’ in the process.

Degrees of cultural loadedness: As free as possible of prefabricated cultural baggage of inner circle countries and to create space for all cultures.

ELF aims at a formulation of minimal requirement for intelligibility and maximal flexibility for actual use. (Seidlhofer, 2002).

Progress Exam II

A transformative perspective

Transformation of language curriculum from native

based norms to ELF.

Open view of language models and standards,

challenging socially marked and restrictive models

Breaking the barriers imposed by inner-circle gate-

keepers in academia

Illustrative scales-Communicative Activities-

Overall Listening Comprehension

Understanding Interaction between Native Speakers

B2+ Can keep up with an animated conversation between native speakers.

B2 Can with some effort catch much of what is said around him/her, but may find it difficult to

participate effectively in discussion with several native speakers who do not modify their

language in any way.

B1 Can generally follow the main points of extended discussion around him/her, provided speech is

clearly articulated in standard dialect.

Illustrative scales-Overall Spoken Interaction

C2 Can understand any native speaker interlocutor, even

on abstract and complex topics of a specialist nature beyond

his/her own field, given an opportunity to adjust to a

non-standard accent or dialect.

B2 Can understand in detail what is said to him/her in the

standard spoken language even in a noisy environment.

Illustrative scales -Conversation

B2 Can sustain relationships with native speakers

without unintentionally amusing or irritating them

or requiring them to behave other than they would with a native

speaker.

Illustrative scales -Formal Discussion

C2 Can hold his/her own in formal discussion of complex issues,

putting an articulate and persuasive argument, at no

disadvantage to native speakers.

Structured overview of all CEFR scales

Native speaker: 16 times

Standard speech/dialect: 12 times

The term non-native: none

Conclusion

Language is an open, flexible, tool for communication

and identification with a community

The community of English language users worldwide is

growing so fast that it is transforming the way

communities of practice are seen. NNS teachers and

their perspective are contributing to this change by

challenging native speaker assumptions.

CEFR descriptor writers need to focus on this change

and include a perspective on ELF.

References

Cook, V. (1999). Going beyond the native speaker in language teaching. TESOL Quarterly, 33. 185-209.

Holliday, A. (1994). Appropriate Methodology and Social Context. Cambridge: Cambridge University Press.

House, J. (1999). Misunderstanding in intercultural communication: Interactions in English as a lingua franca and the myth of mutual intelligibility. In: Gnutzmann, C. (ed). Teaching and Learning English as a Global Language. Tubingen: Stauffenburg. 73-89.

Lowenberg, P.H. (2000). Non-native varieties and the sociopolitics of English proficiency assessment. In: Hall, J.K. and Eggington, E.G. (eds). The Sociopolitics of English Language Teaching. Clevedon: Multilingual Matters. 67-85.

Seidlhofer, B. (2002). ‘The shape of things to come? Some basic questions about English as a lingua franca’. In K. Knapp & C. Meierkord (Eds.), Lingua franca communication (pp. 269-302). Frankfurt/Main: Peter Lang.

Seidlhofer, B. (2005). ‘English as a lingua franca’. ELT Journal, 59/4: 339-441.

Widdowson, H.G.(1994). The ownership of English. TESOL Quarterly, 28. 377-389.

www.CEFtrain.net