CEFR in Finland – uses and adaptations – Possible

implications for JS?Sauli Takala

Tokyo, December 12, 2010

Overview• What is CEFR?• Work on curriculum development• Work on scale development and scale validation• Work on relating language examinations to the CEFR levels• Work on national assessments of FL proficiency in schools

Before CEFR• LOTS of seminars• LOTS of case reports (language study provision)• Narratives and anecdotes without a common frame

of reference: • ”All Dutch speak English very well.”• ”The further south you go in Europe, the less people

speak foreign languages.”• Case 1: ”In my country…”• Case 2: ”In my country…”• Ad infinitum

I SUPPOSE this is useful …

….but it´s so bOOORING!

(Wandering mind) ”I wonder if there isn´t a better way?”

50 years work on language education by the Council of Europe

1960s - A unit-credit scheme for languages (adults; migrant workers)1970s Development of a functional communicative approach (Threshold etc)1991 Coherence and transparency in language learning and teaching (start work on the CEFR and the European Language Portfolio, ELP)2001 publication of the CEFR and European Languages Portfolio2001 – 2008 Languages for Social Cohesion2008 + Towards a plurilingual, pluricultural, inclusive society; integration of all languages

CEFR & its Finnish translation

After CEFR •65% of students reach the target level, 15% one level and 5% two levels above it, 10% one level and 5% two levels below it.

Our target is B1 at the end of compulsory education in ”A”-language.

Our B2 looks like this. What is your B2 like?

What evidence do you have for such claims?

Our goal is A2 in ”B”-language at the end of compulsory education.

”The Blue Bible” (??)

Look! There are

the Portflio people.

How nice to meet them!

How do I know

that my B2 is your

B2?

What is the CEFR?

F. Kaftandjieva

• Title: Common European Framework of Reference for Languages: Learning, Teaching, Assessment

• Author: Council of Europe• Published:

– in 2001 – by Cambridge University Press– Available online: http://www.coe.int/T/E/Cultural%5FCo

%2Doperation/education/Languages/Language%5FPolicy/Common%5FFramework%5Fof%5FReference/

CEF construct of language proficiencyLanguage use, embracing language learning, comprises the actions performed by persons who as individuals and as social agents •develop a range of competences, both general and in particular communicative language competences. •They draw on the competences at their disposal in various contexts under various conditions and under various constraints •to engage in language activities involving language processes •to produce and/or receive texts in relation to themes in specific domains,• activating those strategies which seem most appropriate for •carrying out the tasks to be accomplished. The monitoring of these actions by the participants leads to the reinforcement or modification of their competences.

Frank Heyworth 2010 13

General competences Strategies Communicative competences

linguisticpragmatic

sociolinguistic

Language activities (rec/prod)

Domains & topics

TASK

Identifiable result

Situations

Why was there a need of a framework?

F. Kaftandjieva

Co-operation and co-ordination in educational efforts Mutual recognition of language qualifications

For what uses is the CEFR intended?

F. Kaftandjieva

Language learning programmes

Language certificationSelf-directed learning

What criteria must the CEFR meet?

F. Kaftandjieva

CEFR: Horizontal dimension

F. Kaftandjieva

Overall Language Proficiency

Communicative Language Competence

Linguistic

Pragmatic

Socio-linguistic

Communicative Language Activity

Reception

Interaction

Production

Mediation

Use of Strategies

Reception

Interaction

Production

F. Kaftandjieva

Language Proficiency: Common Reference Levels

F. Kaftandjieva

Breakthrough

Waystage

Threshold

Vantage

Effective Operational Proficiency

Mastery

Language Proficiency: Common Reference Levels

F. Kaftandjieva

Breakthrough

Waystage

Threshold

Vantage

Effective Operational Proficiency

Mastery

Basic user

Independent user

Proficient user

Broader Level Distinction

Language Proficiency: Common Reference Levels

F. Kaftandjieva

Breakthrough

Waystage

Threshold

Vantage

Effective Operational Proficiency

Mastery

Finer Level Distinction

A1.1A1.2

A2.1A2.2

B1.1B1.2

B2.1B2.2

C1.1C1.2

C2.1C2.2

Level A1 is the point at which the learner can interact in a simple way rather than relying purely on a rehearsed repertoire of phrases.Level A2 (Waystage), (a) majority of descriptors stating social functions (greet, short social exchanges…), and (b) descriptors on getting out and about (simple transactions ..)Level B1 (Threshold Level)- (a) maintain interaction and get across what you want to: eg. give or seek personal views and opinions in an informal discussion with friends…), (b) cope flexibly with problems in everyday lifeLevel B2 (Vantage) reflects three new emphases: (a) effective argument,(b) holding your own in social discourse, and (c) a new degree of language awareness (eg. correct mistakes if they have led to misunderstandings)Level C1 is characterised by access to a broad range of language: fluent, spontaneous communication.Level C2 represents the degree of precision and ease with the language of highly successful learners: convey finer shades of meaning precisely; a good command of idiomatic expressions etc. 100 hrs 200-300 hr s 450 hrs 1000s hrs

CEFR-adapted target levels defined in the

current curricula in the basic school (2003) and

the upper secondary school (2004)

CEFR – adaptation to school curricula• Needed to be adapted to the context, not just

adopted as such • Keep the well-established curriculum traditions:

balance between tradition and reform• Add as a new component the proficiency levels to

facilitate definition of progression • Indicate target levels for grades 6, 9 and 12• Need for more fine-grained levels at A1

– fast qualitative progress at lower levels– to sustain and support motivation

New Finnish FL Syllabuses

Level Listening Speaking Reading Writing

A1.1

A1.2A1.3A2.1A2.2B1.1B1.2B2.1B2.2C1.1

Breakthrough

Threshold

Vantage

Mastery

Language Proficiency Levels: in the Finnish Core Curriculum • A1: Limited communication in the most familiar situations

• A1.1 First stage of elementary proficiency • A1.2 Developing elementary proficiency• A1.3 Functional elementary proficiency• A2: Basic needs for immediate social interaction and brief narration

• A2.1: First stage of basic proficiency• A2.1 Developing basic proficiency• B1: Dealing with language use situations in everyday life

• B1.1: Functional basic proficiency• B1.2: Fluent basic proficiency• B2: Managing regular interaction with ’native´ speakers

• B2.1: First stage of independent proficiency• B2.2: Functional independent proficiency• C1: Managing in a variety of demanding language use situations• C1.1: First stage of fluent proficiency

B1.2

C1.2

New Finnish FL Syllabuses

Level Listening Speaking Reading Writing

A1.1 First stage of elementary proficiency

A1.2 Developing elementary proficiencyA1.3 Functional elementary proficiencyA2.1 First stage of basic proficiencyA2.2 Developing basic proficiencyB1.1 Functional basic proficiencyB1.2 Fluent basic proficiencyB2.1 First stage of independent proficiencyB2.2 Functional independent proficiencyC1.1 First stage of skilled proficiency

Taitotaso Kuullun ymmärtäminen Puhuminen Luetun ymmärtäminen Kirjoittaminen

A1.1

Kielitai-don alkeiden hallinta

*Ymmärtää erittäin rajallisen määrän tavallisia sanoja ja fraaseja (tervehdyksiä, nimiä, lukuja, käskyjä) arkisissa yhteyksissä. *Tarvitsee erittäin paljon apua: toistoa, osoittamista, käännöstä.

*Osaa vastata häntä koskeviin yksinkertaisiin kysymyksiin lyhyin lausein. Vuorovaikutus on puhekumppanin varassa ja puhuja turvautuu ehkä äidinkieleen tai eleisiin.* Puheessa voi olla paljon pitkiä taukoja, toistoja ja katkoksia. * Ääntäminen voi aiheuttaa suuria ymmärtämisongelmia. * Osaa hyvin suppean perussanaston ja joitakin opeteltuja vakioilmaisuja.* Puhujan hallitsemat harvat kaavamaiset ilmaisut voivat olla melko virheettömiä.

*Tuntee kirjainjärjestelmän, mutta ymmärtää tekstistä vain hyvin vähän. *Tunnistaa vähäisen määrän tuttuja sanoja ja lyhyitä fraaseja ja osaa yhdistää niitä kuviin. *Kyky lukea entuudestaan tuntemattomia sanoja on hyvin rajallinen.

*Osaa kirjoittaa kielen kirjaimet ja numerot, merkitä muistiin henkilökohtaiset perustietonsa ja kirjoittaa joitakin tuttuja sanoja ja fraaseja. *Osaa viestiä välittömiä tarpeita hyvin lyhyin ilmaisuin. *Vähänkin vieraampien sanojen kirjoittaminen virheellistä.

A1.2

Kehitty-vä al-keiskieli-taito 

*Ymmärtää rajallisen määrän sanoja, lyhyitä lauseita, kysymyksiä ja kehotuksia, jotka liittyvät henkilökohtaisiin asioihin tai välittömään tilanteeseen.*Joutuu ponnistelemaan suuresti ymmärtääkseen yksinkertaistakin puhetta ilman selviä tilannevihjeitä.*Tarvitsee hyvin paljon apua: puheen hidastamista, toistoa, näyttämistä ja käännöstä.

*Osaa viestiä suppeasti joitakin välittömiä tarpeita ja kysyä ja vastata henkilökohtaisia perustietoja käsittelevissä vuoropuheluissa. Tarvitsee usein puhekumppanin apua. *Puheessa on taukoja ja muita katkoksia. *Ääntäminen voi aiheuttaa usein ymmärtämisongelmia.*Osaa suppean sanavaraston, joitakin tilannesidonnaisia ilmaisuja ja peruskieliopin aineksia.*Puheessa esiintyy hyvin paljon kaikenlaisia virheitä.

*Pystyy lukemaan nimiä, kylttejä ja muita hyvin lyhyitä ja yksinkertaisia tekstejä, jotka liittyvät välittömiin tarpeisiin ja tunnistaa niistä yksittäisen tiedon, jos voi lukea tarvittaessa uudelleen.*Kyky lukea entuudestaan tuntemattomia sanoja on rajallinen.

*Osaa viestiä välittömiä tarpeita lyhyin lausein.*Osaa kopioida perustietoja ohjeista ja aikatauluista ja kirjoittaa muutamia lauseita ja fraaseja itsestään ja lähipiiristään (esim. vastauksia kysymyksiin tai muistilappuja). * Ulkoa opetellut fraasit voivat olla oikein kirjoitettuja, mutta itsenäisemmässä tekstissä vilisee kaikenlaisia virheitä.

A1.3

Toimiva alkeis-kieli-taito

*Ymmärtää henkilökohtaisia kysymyksiä ja jokapäiväisiä ohjeita, pyyntöjä ja kieltoja rutiinimaisissa keskusteluissa tilanneyhteyden tukemana.*Ymmärtää huomattavin ponnistuksin yksinkertaisia keskusteluja kiinnostavista asioista.*Ymmärtäminen edellyttää normaalia hitaampaa ja kuulijalle kohdennettua yleiskielistä puhetta.

*Osaa kertoa lyhyesti itsestään ja lähipiiristään. Selviytyy kaikkein yksinkertaisimmista vuoropuheluista ja palvelutilanteista. Tarvitsee joskus puhekumppanin apua. * Kaikkein tutuimmat jaksot sujuvat, muualla tauot ja katkokset ovat hyvin ilmeisiä.*Ääntäminen voi joskus tuottaa ymmärtämisongelmia.*Osaa rajallisen joukon lyhyitä, ulkoa opeteltuja ilmauksia, keskeisintä sanastoa ja perustason lauserakenteita.*Puheessa esiintyy paljon virheitä.  

*Pystyy lukemaan tuttuja ja joitakin tuntemattomia sanoja ja lyhyitä tekstikappaleita, jotka käsittelevät arkielämää, rutiinitapahtumia tai yksinkertaisia ohjeita.*Osaa paikantaa ja verrata yhtä tai useampaa yksityiskohtaa laajemmassa tekstissä. *Lukeminen ja ymmärtäminen on hyvin hidasta.

*Selviytyy yksinkertaisista helposti ennustettavista kirjoitustehtävistä tutuimmissa arkitilanteissa. Osaa kirjoittaa yksinkertaisia viestejä (postikortin, lyhyen esittelyn) todellisista tai kuvitteellisista henkilöistä ja hyvin tutuista aiheista. Osaa kopioida yksittäisiä tietoja ja kirjoittaa muistiin yksinkertaista sanelua.*Osaa muutamia kaikkein tavallisimpia sanoja ja ilmauksia, jotka liittyvät oman elämän yksityiskohtiin tai konkreettisiin tarpeisiin. Osaa kirjoittaa yksilauseisia virkkeitä tai rinnasteisia lauseita, joissa esiintyy perusaikamuotoja. *Tekstissä esiintyy hyvin paljon virheitä. 

A1.1 Speaking: First stage of elementary proficiency

• Can answer simple personal questions with short phrases. Needs considerable assistance and depends on gestures to express meaning. May also switch to first language at times.

• Makes often long pauses and repetitions. • Pronunciation difficulties may seriously impede

communication.• Has a very limited repertoire of basic vocabulary and a

few memorised sentence patterns. • Cannot produce extended speech, but shows

reasonable control of the very limited linguistic repertoire.

Categories incuded in the Finnish syllabus adaptation

Listening Reading1)Themes, texts, tasks 1) Themes, texts, tasks2) Conditions & 2) Conditions & constraints constraints

Speaking Writing- Themes/texts/tasks Themes/texts/tasks (narrative/interaction) Range of language- Fluency Accuracy-Pronunciatton-Range of language-Accuracy

Grade 6 Grade 9 Grade 12

A- English(starts in grade 3)

LC – A1.3S – A1.2RC – A1.3W – A1.2

LC – B1.1S – A2.2RC – B1.1W - A2.2

LC – B2.1S – B2.1RC – B2.1W – B2.1

B1- Swedish (starts in grade 7)

LC – A2.1S – A1.3RC – A2.1W – A1.3

LC - B1.2S – B1.1RC – B1.2W – B1.1

B2/3-languageGrade 8/10)

LC – A2.2S – A2.1 – A2.2RC – A2.2 – B1.1W – A2-1 – A2.2

Developing rating scales for assessing oral performance (computerized testing)

Story line with 6 different tasks

Task-specific criteria for:•Telephone conversation• Reading aloud• Summarizing a tex (mediation: L1-> L2)• Description (of pictures)• Pragmatic: situated responses• Monologue/speech

Task-specific rating scales are needed

Two main parts:a)Task performance and interaction (task-specific)b)Language-related part (basically indentical for all tasks)

See next slide for a visual summary

Levels Task perfor-mance and inter-ction

Range of language

Pronun-ciation

Fluency Accuracy

0 xx xx xx xx Xx

Below A1.1

xx xx xx xx xx

A1.1 xx xx xx xx Xx

A1.3 xx xx xx xx Xx

A2.1 xx xx xx xx Xxxx xx xx xx Xx

C1.1> xxx xx xx xx xx

Scale development and scale validation

CEFR Scale Validation/National Scale Validation

Interrater reliability: 0.87-0.99Correlation with CEFR: above 0.92Pattern matching (Trochim 1999): theory vs. empirical outcomes: L: .89, R: .91, W: .84Discriminant analysis: probablity of ”correct” membership - > .985Aiken agreement coefficient: L .90, R .90, W .80SAME level as CEFR: L 75%, R 65%, W 52%Conclusion: CEFR scales are valid enough to be used as a framework for FL teaching & assessmentSeveral scales have been developed/adapted and validated for different contexts in Finland

CEFR in the Examination and National Assessment context

Linking Matriculation Examinations (high stakes) to the CEFR:

• What level is obtained at the end of the Upper Secondary School (age 19)?

Distribution of Levels (%) in the Matric Exam (19yrs)

0%

20%

40%

60%

80%

100%

En-10

Ger-10

Rus-10 Fi-1

0

Ger- 3/

5

Rus 3/

5Fr 3

/5Sp3

/5Swe6

A1 A2 B1 B2 C1 C2

10: 10 years of study; 3/5: 3-5 years of study; 6 - 6 years of study

How to accommodate national grading and reporting systems and the CEFR (levels)?

• Matriculation exam grades from top to pass: roughly 5%, 15%, 20%, 24%, 20%, 11% <> CEFR 6 levels C2-A1

• One solution: by means of conversion tables/ charts, which show how national grades are related to the CEF levels.

0

20

40

60

80

100

120

140

160

180

200

220

240

260

280

300

<B1

Matr Exam GradeCEFR-level

Laudatur -5%

Eximia -15%

Magna - 20%

Cum laude -24%

Lubenter - 20%

Approbatur -11%

Improbatur

>C1

C1

B2

B1

Sum

sco

re (m

ax. 2

99)

Increased transparency and comparability: English (10 yrs) vs Swedish (6 years)

0

20

40

60

80

100

120

140

160

180

200

220

240

260

280

300

<B1

Matr Exam GradeCEFR-level

Laudatur -5%

Eximia -15%

Magna - 20%

Cum laude -24%

Lubenter - 20%

Approbatur -11%

Improbatur

>C1

C1

B2

B1

Sum

sco

re (m

ax. 2

99)

0

50

100

150

200

250

300

Matriculation GradeCEF Level

Laudatur

Eximia

Magna

Cum laude

Lubenter

Approbatur

Improbatur

>B2B2

B1

A2

A1

Sum

Sco

re (m

ax. 2

99)

1 2 3 4 5 60

10

20

30

40

50

60

Per

cen

t

CEF level

1=A12=A23=B14=B25=C16=C2

1 2 3 4 50

10

20

30

40

50

60

Per

cen

t

CEF level

1=A12=A23=B14=B25=C16=C2

FIGURE 1a. Distribution of CEF levels:) FIGURE 1b Distribution of CEF_levels: A-English B-Swedish

Linking National Assessment outcomes to the CEFR

What level is obtained in English at the end of the Comprehensive School after seven years of study (age 15-16)?

• cf: EU-project ”SurveyLang”

Level in English (%): grade 9 (15-16 years; 7 years of English, Tuokko, 2007)

A225

A234

A240

B141

B139

B132

B2>29

B2>23

B2>25

0%

20%

40%

60%

80%

100%

Receptive skills Speaking Writing

A1 A2 B1 B2>

Linking examinations results has only begun.

• Replication to verify tentative linkages.

•International co-operation to develop compeence in linking examinations/tests to the CEFR (cf. EALTA workshops Barcelona, 2007; Budapest, Turku; workshop in Siena, May 2011)

•International co-operation in mutual verifying of national efforts of linkage?

• International teams of judges?

• External validation by sharing tests?

A rough time/level (English) estimate based on CEFR

• In the Finnish context (L1 And L2 not related):

• Getting from A1.1 (age 9/10) to the average of B1 (age 15/16) takes about 300 lessons and perhaps 100 hours of homework -> 400 hours.

• Getting from the average of B1 to the average of B2 (at 18/19) takes about 250 lessons and probably some 200-250 hours of homework -> 450 – 500 lessons/hours

• A1 -> B2: 800 – 900 hours

I could never have predicted such develop-

ments!

0 500 1000 1500 2000 2500 3000

1

2

3

4

5

CE

FR L

evel

Time

Summing up

Finnish perspective on the CEFR- Implications?

• A valuable tool in all national language education

• A valuable tool in international contacts and co-operation

• Not prescriptive or dogmatic but descriptive; responsibility for thoughtful application lies with the user (eg. Japanese context)

• A reference tool – it is not a curriculum or a programme

• While comprehensive, does not cover everything

Finnish perspective on the CEFR – Implications?

• While the most useful tool around, needs to be elaborated through international co-operation

• Useful supplements: Manual for relating examins to the CEFR, Reference Supplement....

•CEFR and the Portfolio: excellent examples of transnational projects through voluntary co-operation, which serves enlightened national self-interests – no effort to force consensus or exercise power

The basis of the CEFR lies in the descriptions of:

The language user – what kind of person is s/he?How s/he uses languageThe texts to be processed, the tasks to be accomplishedThe competences needed to do this

Are these relevant and applicable in all contexts and for all languages?

The global levels, the general competence descriptors, many of the activitiecan probably be transferred directly?

New activities needed to be added or put at a different level – for example:

Greetings: more complicated in Japanese? Inter-cultural knowledge needs to be stressed?

DIFFERENT LANGUAGES

Spoken reception, production and interaction progresses in parallel with other languages

Development of reading and writing in L2 is probably slower and requires additional « can do » statements for recognising and producing the Latin alphabet?

Some references• Hildén, R. & Takala, S. (2007) Relating descriptors of the Finnish school scale to the CEF overall scales of communicative activities. (pdf available from sjtakala@hotmail.com; raili.hilden@helsinki.fi)

• Kaftandjieva, F. (2004) Standard setting. Section B in Reference Supplement to the Manual for relating language examinations to the CEFR. Council of Europe (available at: http://www.coe.int/T/DG4/Linguistic/Default-en-asp)• Kaftandjieva, F. & Takala, S. (2002) Council of Europe Scales of Language Proficiency: A validation study.In Common European Framework of Reference. Case studies, Council of Europe, 106-129. (pdf available from sjtakala@hotmail.com)

• Kaftandjieva, F. & Takala, S. (2003) Development and Validation of Scales of Language Proficiency. In: W. Vagle (ed.) Vurdering av språkferdighet, NTNU. Trondheim, 31-38 (pdf available from Takala: sjtakala@hotmail.com)• Takala, S. & Kaftandjieva, F. (2002) Relating the Finnish Matriculation Examination English Test Results to the CEF Scales. Helsinki Seminar, June 31- July 2, 2002 (available by request from Takala: sjtakala@hotmail.com)• Tuokko, E. (2007) What level do pupils reach in English at the end of the comprehensive school? U of Jyväskylä, Finland. (PhD thesis in Finnish, with English summary: tuokko@pp.inet.fi)