Take-home message for today From Grammatical …...6/17/15! 1! From Grammatical Constraints to...

6/17/15

1

From Grammatical Constraints to Memory Mechanisms

Colin Phillips

Linguistics Department

Language Science Center Neuroscience & Cognitive Science Program

languagescience.umd.edu

Take-home message for today 1.  Evidence for specific memory architecture & access mechanism

Content-addressable Memory (CAM)

2.  Evidence that anaphor resolution exploits CAM

3.  Mapping linguistic constraints onto memory access

4.  Tension between CAM and specific structural restrictions

the key

to the cells were

S

Subj

PP

VP

V

the key

to the cells were

S

Subj

PP

VP

V

Two ways to search structures in memory

serial, structure-guided search parallel, cue-guided search In content-addressable memory

+plural +subject

1 0 0 0 0 0 0 1

structure-sensitive, avoids interference

slow, esp. for longer relations

susceptible to interference

fast, even for longer relations

McElree et al. 2003; Lewis et al. 2006; Wagers, Lau, & Phillips, 2009

Content-addressable memory in language processing

1.  Timing

2.  Facilitatory Interference

6/17/15

2

Bi-partite architecture of memory

Passive

Stringent limitations on the scope of information that can be concurrently processed

Broadbent 1958; Wickelgren et al., 1980; Garavan, 1998; Cowan, 2001; McElree, 2006; Verhaegen & Basak, 2007; Jonides et al., 2008

Active Passive

SAT: Possible Outcomes

Fig. 3 presents hypothetical SAT functions illustrat-ing how different SAT timecourse patterns can dis-criminate between alternative retrieval processes.Consider first the expected result that interpolating morematerial between the filler and gap position decreases theaccuracy of responding. Recall that this could be be-cause there is a lower probability that a representationof the filler is available when the verb is processed and/or

because there is a higher probability of misanalyzingmaterial up to and including the final verb. If additionalmaterial decreases only the overall accuracy of re-sponding, the corresponding functions will differ in as-ymptotic level alone. Panel A depicts two hypotheticalconditions that differ in this manner.

The pre-asymptotic portion of the SAT functionmeasures processing speed or dynamics, jointly specifiedby the intercept of the function (when accuracy departsfrom chance, d 0 ¼ 0) and the rate at which accuracygrows from intercept to asymptote. The intercept mea-sures the minimum time needed to form an interpreta-tion that would serve to discriminate acceptable fromunacceptable forms. The rate of the SAT function re-flects either the rate of continuous information accrual ifprocessing is continuous or the distribution of finishingtimes if processing is discrete or quantal (Dosher, 1976,1979, 1981, 1982, 1984; Meyer, Irwin, Osman, & Kou-nois, 1988). In either case, differences in intercept or rateimplicate underlying differences in the speed of pro-cessing. This situation is depicted in Panel B of Fig. 3,where the functions are associated with different inter-cepts and rates of rise to a common asymptote.

If access to the filler!s representation requires a searchprocess when the matrix verb is encountered, then theSAT intercept and/or rate of will systematically slow asmore material is interpolated between the filler and gap.Rate or intercept differences can arise from factors otherthan retrieval speed; for example, they might arise fromdifferences in the inherent complexity of computing

Fig. 2. Sample trial sequence illustrating the speed-accuracy tradeoff (SAT) variant of the acceptability judgment task.

Fig. 3. Hypothetical SAT functions illustrating two conditionsthat differ by SAT asymptote only (A) or by SAT intercept andrate (B).

B. McElree et al. / Journal of Memory and Language 48 (2003) 67–91 73

Asymptotic difference Reflects the strength of the representation or the likelihood of completing a parse/process.

Rate/intercept difference Reflects the speed of processing: how quickly information accumulates continuously, or the differences in an underlying discrete finishing time distribution.

Evidence from memory dynamics

ACCESSING SEMANTIC AND PHONOLOGICAL INFORMATION 179

position functions within each judgment. After identifyingthe best fit for each judgment, all 15 conditions (3 judg-ments ! 5 serial positions) were fit simultaneously inorder to isolate differences among the three judgments. Inall cases, the fits were performed on individual subjectdata. Fits of the average (over subjects) data were used tosummarize consistent patterns across subjects.

To quantify the impact of serial position on the retrievalfunctions for each judgment, sets of competitive fits were

performed that systematically varied the three parametersof Equation 1. These fits ranged from a null model, in whichthe 5 serial position functions were fit with a single ! (as-ymptote), " (rate), and # (intercept) parameter, to a fullysaturated 15-parameter model, in which each serial posi-tion was allotted a unique !, ", and # parameter. This analy-sis yielded a clear and consistent pattern across the threejudgments. In each case, the best fit to the data was a 5!-1"-2# model. With one exception (noted below), models

Figure 3. Average d $ accuracy (symbols) as a function of processing time for item(A), rhyme (B), and synonym (C) judgments. Smooth curves in each panel show thebest fits of Equation 1 with (the average) parameters listed in Table 1.

talk – yard – boat – store -‐ tales

Probe recognition – SAT response-signal task Wickelgren, et al., 1980, McElree & Dosher, 1989

FAST SLOOOOOWWWW

6/17/15

3

Distance affects asymptote, but not temporal dynamics Interpretation: whole sentence accessed in parallel – no serial search Concern: wh-constructions are not a good test of distance effects

“The key to the cells unsurprisingly were rusty …”

(Bock & Miller 1991; Pearlmutter et al. 1999; Deevy et al. 1998; Staub 2009; Wagers, Lau, & Phillips 2009; Eberhard et al. 2005)

Agreement Illusions

“The key to the cell unsurprisingly were rusty …”

It’s not simply ‘proximity concord’:

“The musicians who the reviewer praise so highly …”

“The musician who the reviewer praise so highly …”

And it is selective – plurals create illusions, singulars don’t

“The keys to the cell unsurprisingly was rusty …”

And it happens all the time …

Not only do we produce agreement errors – we generally fail to notice them

What causes agreement a8rac9on? 1.  Encoding: hallucina9ng plural subject

2.  Access: misretrieving irrelevant noun

Wagers, Lau, & Phillips 2009, J Mem Lang

Evidence: Gramma9cal Asymmetry Illusions of acceptability

The key to the cabinet were The key to the cabinets were

No illusions of unacceptability

The key to the cabinet was The key to the cabinets was

(Lewis, Vasishth, & van Dyke 2006, TICS; cf. McElree 2006)

Memory-‐access Model: ACT-‐R

Key features (i)  Parallel-‐access à 9me-‐constant retrieval (ii)  Content-‐addressable à suscep9ble to par9al-‐match interference (iii)  Limited-‐size buffers à ~ restricted focus of aXen9on; const. shun9ng

Some content cues

+plural +masculine +animate

+quan9fica9onal

subject main clause current clause

etc.

6/17/15

4

Martin & McElree 2008

Distance non-effects in VP-ellipsis

Foraker & McElree 2007

Pronouns & CAM

Foraker & McElree 2007

Pronouns & CAM

While she was taking classes full-time, Jessica was working two jobs to pay the bills.While she was taking classes full-time, Russell was working two jobs to pay the bills.

While she …Jessica …��Russell …

Gender Mismatch Effect

(Kazanina et al., 2007)

300

340

380

420

460

500

Good Bad

Match Mismatch

Pronoun interpretation is ‘blind’ to grammatically inappropriate nouns. (multiple constructions in English, ��also Russian, even Japanese)

Good co-reference

She was taking classes full-time while Jessica was working two jobs to pay the bills.She was taking classes full-time while Russell was working two jobs to pay the bills.

She …while Jessica …��while Russell …

Bad co-reference

6/17/15

5

Sturt 2003 Experiment 1

Accessible-‐mismatch/Inaccessible-‐mismatch

Jonathan was preXy worried at the City Hospital. He remembered that the surgeon had pricked herself with a used syringe needle. There should be an inves9ga9on soon.

Accessible-‐mismatch/Inaccessible-‐match

Jennifer was preXy worried at the City Hospital. She remembered that the surgeon had pricked herself with a used syringe needle. There should be an inves9ga9on soon.

Sturt 2003, Exp. 1 -‐ Early processing: first-‐pass at reflexive region

Reflexive Licensing (replica9on) The quiet librarian said that the studious schoolgirl reminded herself about the overdue book. The quiet janitor said that the studious schoolgirl reminded herself about the overdue book. The quiet librarian said that the studious schoolboy reminded herself about the overdue book. The quiet janitor said that the studious schoolboy reminded herself about the overdue book.

+gram +lure

+gram -‐lure

-‐gram +lure

-‐gram -‐lure

Parker & Phillips 2015 (like Sturt 2003)

The hippopotamus yawned. 1 2 3

4 5 6

7 8

First fixa9on 2

First pass 2 + 3 all fixa:ons before exi:ng region

Regression path 2 + 3 + 4 + 5 all fixa:ons before exi:ng region to right

Second pass 7 all fixa9ons aher first exi9ng to right

6/17/15

6

Sturt 2003, Exp. 1 -‐ Later processing: Second-‐pass at pre-‐final region

Sturt 2003, Exp. 1 -‐ Later processing: second pass RT at reflexive region

Ungramma9cal Interpreta9ons? Sturt 2003, Exp 2 Accessible-‐mismatch/Inaccessible-‐match Jonathan was preXy worried at the City Hospital. The surgeon [RC who treated Jonathan] had pricked herself with a used syringe needle. There should be an inves9ga9on soon. Accessible-‐mismatch/Inaccessible-‐mismatch Jennifer was preXy worried at the City Hospital. The surgeon [RC who treated Jennifer] had pricked herself with a used syringe needle. There should be an inves9ga9on soon.

Replicated effects of structurally accessible antecedent No effects of structurally inaccessible antecedent

6/17/15

7

Electrophysiology of Sentence Comprehension

•  Seman9c anomaly

N400

I drink my coffee with cream and sugar I drink my coffee with cream and socks

Kutas & Hillyard (1980)

N400

Morpho-‐Syntac9c viola9ons

Every Monday he mows the lawn. Every Monday he *mow the lawn.

The plane brought us to paradise. The plane brought *we to paradise. (Coulson et al., 1998)

(Slide from Kaan (2001)

The elementary-‐school teacher that Sophie liked very much scolded herself … The elementary-‐school teacher that Bernard liked very much scolded himself … The elementary-‐school teacher that Sophie liked very much scolded himself …

(Xiang, Dillon, & Phillips, 2009)

Reflexives

P600 response to viola9on No intrusion effect

(Xiang, Dillon, & Phillips, 2009)

6/17/15

8

a)  Jane thought that Bill owed himself another opportunity to solve the problem.

b)  John thought that Bill owed himself another opportunity to solve the problem.

Condi9ons differ only in the gender of the inaccessible antecedent of himself. Logic: mul9ple match => compe99on => slowdown => binding constraints did not immediately rule out binding-‐inaccessible posi9ons from the considera9on.

Badecker & Straub 2002, Exp 3

Badecker & Straub 2002, Experiment 4

Experiment 5

Experiment 6

No mul9ple-‐match effect

No mul9ple-‐match effect

Badecker & Straub 2002

Same Memory – Different Access

Subject-Verb Agreement The diva [that accompanied the harpist on stage] clearly was flawless … The diva [that accompanied the harpists on stage] clearly was flawless … The diva [that accompanied the harpist on stage] clearly were flawless … The diva [that accompanied the harpists on stage] clearly were flawless …

Subject-Reflexive Agreement The diva [that accompanied the harpist on stage] clearly presented herself … The diva [that accompanied the harpists on stage] clearly presented herself … The diva [that accompanied the harpist on stage] clearly presented themselves … The diva [that accompanied the harpists on stage] clearly presented themselves …

Dillon, Mishler, Sloggett, & Phillips, 2013

illusion

no illusion Both processes require access to same element -- the subject of the same clause.

6/17/15

9

Constraints à Cues? A pronoun is an instruction to find something in memory

E.g., himself •  Content = 3rd person singular masculine •  Location = subject of same clause

Constraints could be directly translated into retrieval cues. Or they could be used as post-retrieval checks. e.g., himself à retrieve [subject]

check [person, number, gender]

Stronger mismatches

Previous studies of reflexives limited to contexts involving a single feature mismatch, e.g., number or gender

Surprising modeling prediction (ACT-R) Multiple feature mismatches ➜ exponential increase in presence of illusion

Perfect Match

Size

of i

llusi

on

1 Mismatch

2 Mismatches

3 Mismatches

1-feature mismatch: gender mismatch (items adapted from Sturt 2003) Grammatical with lure The librarian said [that the schoolgirl reminded herself about the assignment]. Grammatical without lure The janitor said [that the schoolgirl reminded herself about the assignment]. Ungrammatical with lure *The librarian said [that the schoolboy reminded herself about the assignment]. Ungrammatical without lure *The janitor said [that the schoolboy reminded herself about the assignment].

2-feature mismatches: gender + number mismatch Ungrammatical with lure *The librarian said [that the schoolboys reminded herself about the assignment]. Ungrammatical without lure *The janitor said [that the schoolboys reminded herself about the assignment].

6/17/15

10

Experiment 1a

Rea

ding

Tim

e (R

efle

xive

)

1-FEATURE MISMATCH

2-FEATURE MISMATCH

No difference Illusion

GRAM UNGRAM UNGRAM

+LUR

E

+LUR

E

Experiment 2: Agreement aXrac9on baseline

Agreement a8rac9on, number manipula9on (like Dillon et al. 2013) The brief memo that the studious schoolgirl no9ced was posted on the no9ce board. The brief memo that the studious schoolgirls no9ced was posted on the no9ce board. The brief memo that the studious schoolgirls no9ced were posted on the no9ce board. The brief memo that the studious schoolgirl no9ced were posted on the no9ce board. Number + animacy manipula9on The brief memo that the studious schoolgirls no9ced reminded themselves to return the overdue books The brief memo that the studious schoolgirl no9ced reminded themselves to return the overdue books

Experiment 2: Agreement aXrac9on baseline

Number manipula9on Blue 1 = gramma9cal, no intrusion Blue 2 = gramma9cal, intrusion Red 1 = ungramma9cal, intrusion Red 2 = ungramma9cal, no intrusion Facilitatory intrusion observed for red 1 rela9ve to red 2 (=agreement aXrac9on) Number + Animacy Green 1 = ungramma9cal, intrusion Green 2 = ungramma9cal, no intrusion Facilitatory intrusion observed for green 1 rela9ve to green 2

a b c d e f

Total Time

0200

400

600

800

1000

Anaphor licensing not immune to illusions Undermines claim that access mechanisms are 9ed to specific types of gramma9cal dependencies.

Experiments 1a-c

Experiment 1a: gender + number Experiment 1b: gender + animacy Experiment 1c: number + animacy

6/17/15

11

Constraints à Cues? A pronoun is an instruction to find something in memory

E.g., himself •  Content = 3rd person singular masculine •  Location = subject of same clause

Constraints could be directly translated into retrieval cues. Or they could be used as post-retrieval checks. e.g., himself à retrieve [subject], check [person, number, gender] Structural constraints map differently onto retrieval cues. e.g., referential pronoun (‘him’) e.g., bound variable pronouns (c-command constraint) e.g., Mandarin long-distance ziji (intervention effect)

Principle B in Adults John thought that Bill owed him another chance to solve the problem. John thought that Beth owed him another chance to solve the problem. (Badecker & Straub, 2002)

Badecker & Straub 2002

Results are Mixed Immediate effects Nicol & Swinney 1989 Clihon et al. 1997 Lee & Williams 2006 Lewis, Chow, & Phillips 2012 Delayed effects Badecker & Straub 2002 Runner et al. 2003 Sturt et al. 2005 Kennison 2003

No ‘multiple match’ effects

-30

-20

-10

0

10

20

30

40

50

60

Exp. 1a

Exp. 1b

Exp. 1c Exp. 2

Exp. 3 Exp. 4

Exp. 5

B & S (2002)

Diff

eren

ce in

read

ing

times

(ms)

:

Inaccessible Match – Inaccessible Mismatch

John thinks {Bill/Beth} owes him another chance.

Chow, Lewis, & Phillips, 2014

c-‐command constraints •  Bound variable anaphora

–  No studenti [that listened to this physics professor] thinks that hei is a genius. –  *The student [that listened to no physics professori] thinks that hei is a genius.

•  Many other rela9ons : c-‐command is pervasive

–  Wh-‐dependencies –  Reflexive binding –  Nega9ve polarity –  Control

•  Possible effects on-‐line

–  c-‐command serves “ga9ng” func9on: only c-‐commanders accessed –  c-‐command as non-‐exclusive constraint, on par with others (number, case, etc.) –  c-‐command not directly implicated in retrieval at all

6/17/15

12

Core Ques9on

How do we encode and navigate structured mental representa9ons?

Structures are easy to draw. But what kind of mental object are they?

Linguis9c arguments: rela9onal no9ons like c-‐command are pervasive.

In some architectures it is trivial to capture rela9onal no9ons like c-‐command. In other architectures it is not trivial.

C-command is not content “E c-commands H” is not an inherent property of node E, that exists independent of H’s presence

Kush, Lidz, & Phillips (2015)

Kush, Lidz, & Phillips (2015) Kush, Lidz, & Phillips (2015)

Structural configura9on maXers for binding, not for coreference. Is binding blocked without c-‐command? Not yet clear.

6/17/15

13

Binding vs. Coreference No cyclist thought the spectators loved him. The spectators [that no cyclist acknowledged]

thought the people loved him. The cyclist thought the spectators loved him.

The spectators [that the cyclist acknowledged]

thought the people loved him.

(Kush, Lidz, & Phillips, 2015)

Binding vs. Coreference 2 x 2 Design: GenderMatch x Referentiality The troop leaders . . . Match-Quant [that no girl scout had respect for] Mismatch-Quant [that no boy scout had respect for] Match-Ref [that the girl scout had respect for] Mismatch-Ref [that the boy scout had respect for]

had scolded her after the incident … (Kush, Lidz, & Phillips, 2015)

Binding vs. Coreference n = 30

* n.s. *

n.s.

C-command gates access to antecedents in retrieval But what about Principle C effects in retrieval? (Kush, Lidz, & Phillips, 2015)

Strong Crossover

Mary knew which man said that he liked cheese. Mary knew which man it was that he liked ___.

+masc. + sing.

+masc. + sing.

(Kush, Lidz, & Phillips, in prep)

6/17/15

14

Strong Crossover and Retrieval

Jane asked … NoCrossover Sentences which janitor which lunchlady

_____ had said that he had spoken with Donna ... Jane asked … Crossover Sentences

which janitor which lunchlady

…it seemed that he had spoken with _____ ...


Strong Crossover: SPR Times n=30


Strong Crossover: Results

NoCrossover Conditions Crossover Conditions

* n.s.

Antecedent retrieval accesses feature-matching fillers – only if they are grammatically appropriate. Retrieval “ignores” feature-matching wh-phrases if the pronoun c-commands the gap-site.


Strong Crossover and Retrieval

Jane asked … NoCrossover Sentences which janitor which lunchlady

_____ had said that he had spoken with Donna ... Jane asked … Crossover Sentences

which janitor which lunchlady

…it seemed that he had spoken with _____ ...

wh-gap before pronoun

wh-gap after pronoun


6/17/15

15

Japanese (a)soko (Hoji 1991, 1995, Ueyama, 1999)��

Dono-kaisya-mo soko-no kogaisya-o suisensita.every-company-mo soko-gen subsidiary-acc recommended‘Every companyi recommended itsi subsidiary.’

*Dono-kaisya-mo asoko-no kogaisya-o suisensita.every-company-mo asoko-gen subsidiary-acc recommended‘*Every companyi recommended itsi subsidiary.’

Reconstruction effects ��

Soko-no kogaisya-o dono-kaisya-mo suisensita.soko-gen subsidiary-acc every-company-mo recommended‘Every companyi recommended itsi subsidiary.’

Asoko-no kogaisya-o dono-kaisya-mo suisensita.Asoko-gen subsidiary-acc every-company-mo recommended‘*Every companyi recommended itsi subsidiary.’

Acceptability Rating Study II. Acceptability rating study (n = 48)

Context: A president and his secretary are discussing the current business situations of automobile companies. The president says,…

(A) Simbunkiji-niyoruto, [sokoi-no itiban yakunitatanai yakuin]-o Newspapers-by soko-gen most useless executive-acc [dono jidoosyagaisyai]-mo kubinisita toiu hookoku-ga dehajimeta-rasii. every automobile company-mo fired that report-nom appeaedr-seem ‘The newspaper says that there appears a report that to its most useless

employee, every automobile company fired.’ (B) *Simbunkiji-niyoruto, [sokoi-no itiban yakunitatanai yakuin]-ga Newspapers-by soko-gen most useless executive-nom [dono jidoosyagaisyai]-mo uttaeta toiu hookoku-ga dehajimeta-rasii. every automobile company-mo sued that report-nom appeaedr-seem ‘The newspaper says that there appears a report that its most useless

employee sued every automobile company.’

ACC-NOM order

NOM-ACC order

Aoshima, Yoshida, & Phillips 2009, Syntax

soko-no syain dono kacyoo dono kaisya

INCONGRUOUS CONGRUOUS its employee

every manager every company

Scrambled ACC-NOM Order

Canonical NOM-ACC Order

Aoshima, Yoshida, & Phillips 2009, Syntax

Relational Access •  Real-time sensitivity to relational constraints

–  Retrieval can target items based on c-command (bound vbls) –  Robustness of Principle C effects not due to forwards search

•  What makes this possible?

–  Serial access mechanism? Easily captures c-command sensitivity. … but how to invoke only when QP antecedent is available?

–  Relational properties as content (via costly encoding function)?

–  Even cases that look like (backwards) retrieval involve forwards updating e.g., QP & whP induce tracking of c-command path (cf. Kush 2013 PhD)

•  Why doesn’t everything use this?

–  Varying usefulness/reliability of deploying structure-sensitive mechanisms –  Feasibility of identifying elements likely to act as c-commanding licensors

6/17/15

16

Intervention John found a picture of himself in the cabinet. *John found Mary’s picture of himself in the cabinet. Zhangsan zhidao Lisi piping-le ziji. Z. knows L. criticized self *Zhangsan zhidao wo piping-le ziji. Z. knows I criticized self Nobody said that Bill would ever vote for Clinton. *Nobody said that everybody would ever vote for Clinton. What do you know that Sally likes __? *What do you know who likes __?

Mandarin Long-Distance Reflexives

Lisi nongshang-le ziji Lisi harm-PERF self “Lisi harmed herself”

Zhangsan shuo Lisi nongshang-le ziji Zhangsan says Lisi harm-PERF self “Zhangsan says that Lisi harmed him / herself”

Antecedent for ziji must be (i) a subject (ii) animate (iii) c-commanding

LD antecedent: Coach Zhang say [that report [..] underestimate ziji] Local antecedent: Auto-biography say [Coach Zhang [..] underestimate ziji] No antecedent: Auto-biography say [that report [..] underestimate ziji]

+ control conditions without ziji.

Speed-Accuracy Tradeoff (MR-SAT) ziji conditions

[common scaling of d’ based on asymptote, for visualization of dynamics]

control conditions (identical words, except for ziji)

(Dillon et al. 2014)

local

long-distance

Timing advantage for local antecedent à serial access to antecedents

Take-home message for today From Grammatical …...6/17/15! 1! From Grammatical Constraints to...

Documents

Transcript of Take-home message for today From Grammatical …...6/17/15! 1! From Grammatical Constraints to...