Susanne Reiterer_Talk 04.12
-
Upload
vanitystalks -
Category
Documents
-
view
117 -
download
0
description
Transcript of Susanne Reiterer_Talk 04.12
Brain + Multilingualism / Language Learning
Factors that matter
in Second Language Acquisition –
a Cognitive Neuroscience Approach to SLA
Ass. Prof. Susanne Reiterer, PhD
Dr.rer.nat. (PhD in Psychology) – Univ of Vienna, Austria
Mag.phil.(MA in Linguistics) - Univ of Vienna
University of Vienna & Tübingen (formerly)
Centre for Language Learning Learning and Teaching Research (FDZ ) – at
the Faculty for Philology and Cultural Studies, University Vienna
Language is for everyone ..
Factors for growth & development in L2
L1
age
..exposure
use..
soil
Plant type
dandelion (engl)
“pissenlit“ (fr)
Löwenzahn (ger)
wind
rain
sun
Individual variation !
Where do these abilities stem from? - theories
Mezzofanti
Kató Lomb
• God
• Brain structure &
function
• DNA/ Gene
(80% genes in linguistic functions
compared to g. IQ (K.
Stromswold
FOXP2, KE family;
JR family –semantics,
Briscoe2012 )
• “they work harder“
• Motivation
• exercise
• having much time
• everybody can do it
• musicality
• working memory / hearing
• personality (extraversion)
• empathy
Behaviorism Nativism
handedness
Individual Differences (Factors)
Brain structure
Attention
Extraversion (personality)
Openness
Conscientiousness
Emotional Intelligence / Empathy
Anxiety
Attitudes
Teaching method
Nature (genes, brain) aptitude? giftedness
Modifying factors Nurture (environment) practice
Second Language Learning Process
Language Aptitude
Motivation
Working Memory
DNA sex/gender
testosterone level
Exposure Time
Quality of training
Typology of language (linguistic similarity/distance)
Learning strategy
Purpose of language use
Age of Onset of L2
Age of Onset of L2
Age of acquisition versus proficiency level ?
Auditory input
Motor output
Bilingual (L2) Brain representation?
the big fight !
Factors for L1 and L2 attainment/proficiency
Brain (brain structure, lateralization, maturation)
DNA genetic equipment („developmental language disorders“…)
Sex (female linguistic advantage?)
Handedness (lateralization and language, left handers)
Age (Age of Onset) (as a consequence of brain maturation, deterioration) Bio
log
y
Intelligence/ Verb Intell (biol? social?)
Aptitude, language learning talent (biol? psychol? social?)
Domain general cognition / Executive Functions / Cognitive Control
Affective factors (Anxiety..)
Memory / Working Memory (biol?)
Learning strategies
Learner‘s Type (inborn? acquired?)
Personality (biol? cultural? psychol?)
Psych
olo
gy/
Co
gn
itiv
e
So
cia
l
Acquisition / Teaching Method
Quality of Input
Purpose of Language Use/ linguistic environment
Exposure Time
Language Attitudes
Polyglottism (L2, L3, L4… number of)
Bidialectalism (early experience of)
Amount / Quality of Training
Language typology Lin
gu
isti
c /
so
cio
lin
gu
isti
c
Language Learning Aptitude
• Definitions
• prediction of how well, relative to other individuals, an individual can
learn a foreign language in a given amount of time (speed and
accuracy are important) [John B Carroll, Charles Stansfield]
• As with many measures of aptitude, language learning aptitude is
thought to be relatively stable throughout an individual’s lifetime and
per definition there is a certain “given”, pre-existing element of
giftedness to it. [Niklas Abrahamsson]
what is this?
http://www.youtube.com/watch?v=FOiXtWcQ8GI 16 year old American boy, 20 languages
Language and its many faces (levels)
Grammar (syntax,
morphology) Phonetics/phonology
Pragmatics
Speech imitation aptitude: a large scale study Linguistic part Psychological part Brain Imaging part
Linguistic description (~10
tests)
Proficiency & Talent measures
• Language aptitude test (MLAT by Carroll 1960), • detailed phonetic language
recordings to determine
pronunciation fluency and aptitude
(own tests)
* 2 TOEFL (list comp, grammar)
• language learning questionnaires (history)
• psycholing. Questions (learn style, type of instruction, contacts in L2)
N=138
Psychological testing
(~15 Tests)
• Phonological working memory
• 2 Empathy tests
• 2 Personality questionnaires (Big
Five and Inhibition-Activation Sys)
* Verbal & non-verbal IQ (Raven)
• 2 Musicality (Gordon’s AMMA)
plus questions abt music hobbies
• Mental Flexibility (Simon task)
N~116
Brain Imaging
(3 types of measures taken)
• structure
MR anatomy – (mdeft)
VBM (voxel based morphometry – white / grey matter density)
• DTI (diffusion tensor imaging) (diff 12 dir)
• function fMRI (EPI, TR 2s; +Tr 12s)
N~70
How were the recordings done?
1. part: phonetic-
linguistic testing:
138 participants were recorded in (Uni Tübingen&Stuttgart) a soundproof room on 1
h speech production and 1 h speech perception.
Production tasks: English (L2): read texts, tell story, free speech, imitate sentences.
German (L1): imitate sentences (prosody), speak foreign-accented sentences in L1.
Hindi (L0): imitate sentences and words (no prior experience).
Perception tasks: English and German: disriminate sentences in diff prosody, Toefl test, listening comp,
detect foreign accented speech …
2 online native speaker ratings:
1. For English (L2): www.susannereiterer.eu/eval we need ~ 250 natives from all world (mainly US & GB) to make short online ratings (30 min) – work in progress!
2. For Hindi (L0): www.susannereiterer.eu/eval_hindi we had ~ 30 natives from all India (mainly Hindi speakers) to make long online ratings (2 h) – work completed!
Hindi score
for “pure“
imitation talent
without the
influence of
experience
English score
for “experience
plus ability“
- fluency (perform)
- pronunciation (Tal)
Sample in database:
138 Germans +
18 Native Indians
(to confuse the raters)
Sample in database:
138 Germans +
14 Native English
(to confuse the raters)
How to rate speech output? Imitation talent
native model (10) 7.2 / 6.4 / 4.6 / 2.2 / 0.6 / 0.1
native range 8-10
Rating
system:
10 =
highest score
0 =
lowest score
Highest score for German speakers for this sample: 7.2 138 German subjects+18 natives speakers=156 /
Sentence1: 8.8; Sentence3: 8.8 ! 30 (different) native raters (15f/ 15m).
Males were rated sig. better than females (by males and females)
(males mean: 4,9 points females 4,4; t-test: sig. 0.005
Ratings in internet were
performed for
3 Hindi sentences
and 1 word
Example here: sentence 2
(most difficult)
HINDI
Behavioral results: Hindi imitation scores
T – test f independent samples:
Males: 4.9 (SD±1.0)
Females: 4.4 (SD±0.9)
Sig: p= .005
normal distributions & linguistic theory
• Reconciling models/ theory
How to rate speech output? Pronunciation (ability)
Task 2: Reading of “The North Wind and the Sun”
The North Wind and the Sun
The North Wind and the Sun were disputing which was the stronger, when
a traveller came along wrapped in a warm cloak. They agreed that the
one who first succeeded in making the
traveller take his cloak off should be considered stronger than the other.
Then the North Wind blew as hard as he could, but the more he blew
the more closely did the traveler fold his cloak around him; and at last
the North Wind gave up the attempt.
Then the Sun shined out warmly, and immediately the traveller took off
his cloak. And so the North Wind was obliged to confess that the Sun
was the stronger of the two.
jbh10 / bfem10 / kgl5 / 6 / hbau2,2 / ahp 2
Rating system
10 = highest score
0 = lowest score
• 8 % (=11 subjects) of German speakers could
„cheat“ the native ears.
• They were all in the range between 8-10 points
• males rated significantly better than females by
American „raters“ (females).
• normally distributed
First results of native speaker rating English
Task 2: Reading of “The North Wind and the Sun”
How did we do the brain scans?
3. part
magnetic
resonance
imaging
MR earphones MR microphone
Button-press-device Drive-in starts!
First results - fMRI: high low ability (Hindi) groups
Preliminary results: fMRI: sentence imitation
Group versus group contrasts:
Nontalents versus Talents = Low Talent Group
First results - fMRI: sentence + word imitation (Hindi)
Results – structural MRI (VBM) – anatomy only (Hindi)
Voxel based morphometry measurement: gray matter volume
Reiterer SM, Hu X, Erb M, Rota G, Nardo D, Grodd W, Winkler S and Ackermann H. (2011) Individual differences in audio-
vocal speech imitation aptitude in late bilinguals: functional neuro-imaging and brain morphology. Frontiers in Psychology ,
2:271. (free access journal)
Hu X, Erb M, Ackermann H, Martin JA, Grodd W, Reiterer SM (2011) Voxel-Based Morphometry Studies of Personality: Issue
of Statistical Model Specification - Effect of Nuisance Covariates. NeuroImage, 54:1994-2005.
Dogil, G. and SM Reiterer (eds) (2009) Language Talent and Brain Activity. Mouton de Gruyter: Berlin, New York.
Talents
Non Talents
0
50
100
150
200
250
300
350
400
450
500
read_ger read_eng read_acc
Are
a u
pto
10
0 H
z
Talents
Non-Talents
Temporal Modulations (Hz)
Spectr
al M
odula
tions (
Cycle
s/K
Hz)
* p=0.002 * p=0.002 * p= <0.001
Results – acoustic spectrograms: sentence reading, faking accents
Read German (A) Read English (B) German with fake Engl Accent ( C)
r=0.7**
Reiterer S, Singh NC, Winkler S. (2012) Predicting speech imitation ability biometrically.(In: Empirical Approaches to Linguistic Theory
Studies in Meaning and Structure. Edited by Stolterfoht, Britta and Featherston, Sam. Mouton de Gruyter: Berlin, New York).
Results - fMRI: sentence reading and faking accents
Reading German with foreign accent
Reading tasks L1, L2, L1Acc
Increase BOLD activity in peak activation area
High ability group Low ability group
High ability group Low ability group
Low - High
L1 Germ
Low - High
L2 Engl
Low - High
L1 + ACC
N=9 (2 groups, high ability l low ability)
p<0.05, corrected at cluster level
L1 + Accent:
Increasing activation the lower
the scores
Differences low versus high ability group:
significant correlations with low ability
C
C
C
A
B
A B C C B A
Reiterer S, Singh NC, Winkler S. (2012) Predicting speech imitation ability biometrically.(In: Empirical Approaches to Linguistic Theory
Studies in Meaning and Structure. Edited by Stolterfoht, Britta and Featherston, Sam. Mouton de Gruyter: Berlin, New York).
Results – phonetic measurements: vowel duration measures
agreed high ability
low ability
Words: along, alike, agree, alive, asleep, …
r=0.7**
the shorter the vowel – the better the imitation score
High ability L0 (Hindi) speech imitators:
• better Work Memory (digit, nonword) 0.37***
• higher scores MLAT 0.34***
• English imitation 0.33***
• can “sing“ better 0.3**
• higher fun-seekingness (BAS_fun) 0.22*
• less conscientiousness -0.28*
preliminary behavioral results: correlations with language and
psychological measures (work in progress)
Individuals with high pronunciation aptitude in L2
(English) are also good in:
• higher scores MLAT 0.6***
• like “acting“ more 0.36***
• higher “openness to experience“ 0.33***
• more school years in L2 0.34***
• time spent abroad 0.33***
• speak more dialects in L1 0.3**
• higher musicality (Gordon test) 0.29**
• higher empathy 0.28*
• like singing more 0.28*
Hindi score English score
*** = .0001 / ** = .001 / *= .01
uncorrected for multiple comparisons
0.33***
• Subjects: n=113, age = 26 ± 6, from 19 to 40 years, onset =10
• Dependent Variable: Hindi Pronunciation Aptitude (imitation of Hindi sentences)
• Independent Variables: Verbal IQ, Non-Verbal IQ, Vocabulary Short-Term Memory, Non-Verbal Short-Term Memory (WAIS), Phonetic Coding Ability, Musicality, Neuroticism, Extraversion, Openness to Experience, Agreeableness, Conscientiousness, Empathy
• Results: Only Working Memory (digit span and nonword repetition) together “singing” (“like singing and can sing”) significantly predicted the Hindi Pronunciation Aptitude (Hindi score).
behavioural results (Hindi) – multiple regression
• Subjects: n=113, age = 26 ± 6, from 19 to 40 years, onset =10
• Dependent Variable: English Pronunciation Aptitude (imitation of Hindi sentences)
• Independent Variables: Verbal IQ, Non-Verbal IQ, Vocabulary Short-Term Memory, Non-Verbal Short-Term Memory (WAIS), Phonetic Coding Ability, Musicality, Neuroticism, Extraversion, Openness to Experience, Agreeableness, Conscientiousness, Empathy
Results: Only Phonetic Coding Ability (MLAT) and Empathy and Singing
together significantly predicted the English Pronunciation Aptitude. No significant interaction between phonetic coding ability and empathy.
=> No working memory influence in the late stage.
behavioural results (English) – multiple regression
• Xiaochen Hu, Hermann Ackermann, Jason Martin, Michael Erb, Susanne Winkler and Susanne M Reiterer. (2013)
Language Aptitude for Pronunciation in Advanced Second Language (L2) Learners: Behavioural Predictors and
Neural Substrates. Brain and Language: in press.
musicality (singing) and language learning?
sound learning / articulate sound imitation / flexibility in sound imitation
Original song (composition) lower score singer high score singer
Ongoing diploma-thesis Markus Christiner
(www.christiner.at), English Department, University of Vienna
(supervisors Prof. E. Keizer, Ass Prof. S. Reiterer)
“Happy Birthday“ (creativity)
Preliminary results:
high correlations between singing and pronouncing
(imitating) - and working memory,
lower with general musicality and
not with instrument playing , not with amount of singing
lessons
Acknowledgements
My sincere thanks to present and past collaboration partners, students and supervisors:
Ackermann, Hermann – University of Tübingen, Hertie Institute, Germany
Anus, Arev (project student) – University of Stuttgart, Germany
Baumotte, Henrike (PhD student) – University of Stuttgart, Germany
Bhattacharya, Joydeep - Goldsmith College, London; Academy of Sciences, Vienna
Christiner, Markus – (diploma student) – University of Vienna
Erb, Michael – University of Tübingen, CIN, Germany
Dogil, Grzegorz - University of Stuttgart, Germany
Grodd, Wolfgang – University of Tübingen, Germany
Hu, Xiaochen (PhD student) – University of Tübingen and Bonn Germany
Klose, Uwe – University of Tübingen, Germany
Kumar, Vinod – University of Tübingen, Germany
Nardo, Davide (project student) – Santa Lucia Foundation, Rome, Italy
Meier, Jens – University of Tübingen, Germany
Rappelsberger, Peter - Medical University of Vienna, Austria
Rota, Giuseppina (project student)- University of Pisa, Italy & Stuttgart, Germany
Singh, Nandini, Chatterjee – National Brain Research Centre (NBRC), Gurgaon, INDIA
Wildgruber, Dirk – University of Tübingen, Germany
Winkler, Susanne – University of Tübingen, Germany
book and papers
Reiterer, S. (2009). Brain and Language Talent: A Synopsis. In: Dogil, G. & Reiterer,
S. (eds.) Language Talent and Brain Activity. Trends in applied linguistics 1, Mouton
de Gruyter, Berlin-New York: 155-191.
• Hu X and Reiterer S (2009). Personality and pronunciation talent in second
language acquisition. In: Dogil, G. & Reiterer, S. (eds.) Language Talent and Brain
Activity. Trends in applied linguistics 1, Mouton de Gruyter, Berlin-New York: 97-129.
• Rota G and Reiterer S (2009). Cognitive aspects of pronunciation talent: how
empathy, mental flexibility, working memory and intelligence interact with phonetic
talent. In: Dogil, G.& Reiterer, S. (eds.) Language Talent and Brain Activity. Trends in
applied linguistics 1,
Mouton de Gruyter, Berlin-New York: 67-96.
• Nardo D and Reiterer S (2009). Musicality and phonetic language aptitude. In:
Dogil, G. & Reiterer, S. (eds.) Language Talent and Brain Activity. Trends in applied
linguistics 1, Mouton de Gruyter, Berlin-New York: 213-255.
• Xiaochen Hu, Hermann Ackermann, Jason Martin, Michael Erb, Susanne Winkler and Susanne M Reiterer.
(2013) Language Aptitude for Pronunciation in Advanced Second Language (L2) Learners: Behavioural
Predictors and Neural Substrates. Brain and Language: in press.
• Reiterer S, Singh NC, Winkler S. (2012) Predicting speech imitation ability biometrically.(In: Empirical
Approaches to Linguistic Theory Studies in Meaning and Structure. Edited by Stolterfoht, Britta and
Featherston, Sam. Mouton de Gruyter: Berlin, New York).
• Reiterer S, Hu X, Erb M, Rota G, Nardo D, Grodd W, Winkler S, Ackermann H. (2011)
Individual differences in audio-vocal speech imitation aptitude in late bilinguals: functional neuro-imaging and
brain morphology. Frontiers in Psychology (Language Sciences) 2:271, 1-12.
• Hu X, Erb M, Ackermann H, Martin JA, Grodd W, Reiterer SM (2011)
Voxel-Based Morphometry Studies of Personality: Issue of Statistical Model Specification - Effect of Nuisance
Covariates. NeuroImage, 54: 1994-2005.
Thank You!
Prelim. Behavioral Results1: Correlations with Lang measures
Females
Like
dancing! sig
Pronunciation Talents are also good in:
• proficiency measures (grammar, etc.)
• can speak more dialects in their L1!
• like to imitate dialects in L1
• like singing more! (and can sing better)
• have more contacts with natives
• time spent abroad (~ only MLAT)
• no of L2s spoken ( ~ only MLAT syntax)
Prelim. Behavioral Results2: Correlations with Psych tests
Byproduct:
sig. other Correlations with Sex:
• Females - higher empathy scores
• Females - higher neuroticism scores
• Females – higher agreeableness sc.
• Females – higher conscientiousness
• Females – higher inhibition scores !
Acc our tal measures, pron. talents
have:
• better Work Memory (digit, nonword)
• better Musicality (Rhythm)
• would be more open, fun-seeking
and less conscientious!
• have slightly more empathy
• more nonverbal IQ (acc. to the MLAT
only ! (beware of known correlations
between aptitude tests and IQ tests)
How to rate pronunciation (ability) – the
„foreign accent test“
(B) Mr. Richardson led his dog to a tree. (L2)
(A) Der Bauer erntete jetzt viel Getreide. (L1)
Task (3 conditions): Read sentences in A German/ B English/ C German with fake foreign accent
(C) Der Professor präsentiert das Resultat. (L1Acc)
(C) Der Bademeister rettete den Jungen.
(C) Der Beamte genießt seine Ferien.
High ability speakers:
MB
(A)
(B)
(C)
Low ability speakers:
AK
mid-range speakers:
SK
fMRI Paradigm What do the people do in the scanner? fMRI paradigm
Different tasks have to be completed inside the scanner. Speech production was recorded online during scanning.
A. Speech production (articulation, microphone recorded) (3 Tasks, 20 / 20 / 30 min)
1. WORD IMITATION (listen to and repeat WORDS in TAMIL (L0), English (Brit+American) (L2).
Ex: distributor, legislature, synthesizer, transparency / Tamil: pustagangal, patirigei, arasamgam.
2. SENTENCE IMITATION (listen to and repeat Sentences in English (Br+Am) (L2) and German (L1).
Ex: The teacher welcomed the student to her class./ Am: The engineer constructed the bridge with care.
Der Rettungsdienst brachte ihn ins Krankenhaus. Der Winter überraschte die Eichhörnchen.
3. SENTENCE READING (read sentences in (A) L1-German (B) L2-English and (C) German with fake English foreign
accent (LAcc)
Ex.: (A) Der Bauer erntete jetzt viel Getreide (B) Mr. Richardson led his dog to a tree (C) Der Professor präsentiert das
Resultat
First results - fMRI: sentence imitation (English score)
Main Effect Group: High > Low Aptitude Main Effect Language: English > German
Sentence Imitation
IFG STG, Ins
MR
SFG PC, SMA
BG
CB
STG HG IFG
Ins IFG Ins
SMA, ACC MCC
CB
BG
Comparison: High> Low Aptitude
A1: Sentence Imitation
STG
SFG
IFG
MR
Imitation
A2
Imitation
A3
Imitation
A4
First results - fMRI: sentence imitation (English score)
First results - fMRI: sentence + word imitation (Hindi)
• age range: 20-40 (mean: 25.9, ± 5.2)
balanced for:
• age of Onset English (AOL): around 10 years (in school) (late learners)
• L1: 138 German native speakers, studying in Tübingen or Stuttgart
• education: All completed „a-levels“, university students, young graduates/academics
• linguistic experience~ 50% language students (mostly English students)
(distributed over all „talent“ groups: high-middle-low)
• 90% right handed (in MR experiments – 100% right handed)
• No cases of neurological disorders
Who were the participants in our study?