INFORMATION TO USERS - Library and Archives Canada · of the possible parses by taking into account...

INFORMATION TO USERS

This manuscript has been reproduced frorn the microfilm master. UMI films the

text directly from the original or copy submitted. Thus, some thesis and

dissertation copies are in typewriter face, while others may be from any type cf

cornputer printer.

The quality of this reproduction is dependent upon the quality of the copy

submitted. Broken or indistinct print, colored or poor quality illustrations and

photographs, print bleedthrough, substandard margins, and improper alignment

can adversely affect reproduction.

In the unlikely event that the author did not send UMI a complete manuscript and

there are missing pages, these will be noted. Also, if unauthorized copyright

material had to be removed, a note will indicate the deletion.

Oversize materials (e-g., rnaps, drawings, charts) are reproduœd by sedioning

the original, beginning at the upper left-hand corner and wntinuing from left to

right in equai sections with small overlaps. Each original is aiso photographed in

one exposure and is inciuded in reduced form at the back of the book.

Photographs inciuded in the original manuscript have been reproduced

xerographically in this copy. Higher quality 6" x 9" black and white photographic

pnnts are available for any photographs or illustrations appearing in this copy for

an additional charge. Contact UMI directly to order.

Bell & Howell Information and Leaming 300 North Zeeb Road, Ann A W r , MI 48106-1346 USA

800-521 -0600

NOTE TO USERS

The original manuscript received by UMI contains pages with indistinct andlor slanted print. Pages were microfilmed as

received. r

This reproduction is the best copy available

Designed Intelligence: A Language Teacher Model

1. Staatsexamen, Weingarten University, 1983

II. Staatsexamen, Weingarten University, 1985

M a s e r of Arts, Simon Fraser University, 1993

THE REQUIREMENTS FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

in the Department

Linguistics

O Gertrud Doris M- He* 1998

S M O N FRASER UNIVERSITY

July 1998

All rights reserved. This work may not be

reproduced in whole o r in part, by photocopy

or other means, without permission of the author.

National Library j*I of Canada Bibliothèque nationale du Canada

Acquisitions and Acquisitions et Bibliographie Services services bibliographiques

395 Wellington Street 395, rue Wellington Ottawa ON K I A ON4 OttawaON K1AON4 Canada Canada

Your Iiie Voire relerence

Our nle Notre rëfërence

The author has granted a non- L'auteur a accordé une licence non exclusive licence dowing the exclusive permettant à la National Library of Canada to Bibliothèque nationale du Canada de reproduce, loan, distribute or seIl reproduire, prêter, distribuer ou copies of this thesis in microfom, vendre des copies de cette thèse sous paper or electronic formats. la forme de microfiche/film, de

reproduction sur papier ou sur format électronique.

The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts fkom it Ni la thèse ni des extraits substantiels may be printed or otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation.

Abstract

This thesis presents the design of an Intelligent Language Tutoring

System that models a language teacher by evaluating and responding to

student performance o n foreign language exercises. The design is

implemented in Head-driven Phrase Structure Grammar and illustrated with

German.

A review bf the literature provides an ovenriew of Intelligent

Language Tutors, examining existing systems with respect to error coverage,

user scope, and computational generality. Design criteria are reevaluated,

emphasizing a pedagogically-infomed, student-centered approach.

The design presented facilitates this view, d s o solving several extant

problems in Intelligent Language Tutoring Systems: errors need not be

anticipated. Additionally, the techniques successfully address ambiguous

readings, contingent and ambiguous errors. Finally, the system is modular

making it adaptable to different languages, specinc student needs, andlor to

reflect the pedagogy of a particular language instructor.

The design of the Intelligent Language Tutoring System consists of - five components: The Domain Knowledge, the Licensing Module, the Analysis

Module, the Student Model, and the Filtering Module. The Domain

Knowledge provides the linguistic analysis, the remaining components add

pedagogical value.

The Domain Knowledge consists of a parser with a grammar which

parses sentences and phrases t o produce sets of phrase descriptors. A phrase

descriptor is a mode1 of a particular grammatical phenornenon. It records

whether o r not the grammatical phenornenon is present in the input and Kso,

whether it is correctly or incorrectly formed. The Licensing Module selects one

of the possible parses by taking into account the likelihood o f the error. The

..- Designed Intelligence: A hnguage Teacher Mode1 111

Analysis Module detides on the source of the error. It takes a phrase

descriptor as input and generates sets of possible responses to the leamer's

input that the instruction system can use when interacting with the student.

The Student Model records mastery of grammatical structures as well as

structures with which the leamer has problems. The level of the learner

according to the current state of the Student Model, determines the particular

feedback displayed. Finally, the Filtering Module decides o n the order of the

instructional feedback by considering the importance of an error in a given

exercise and the dependency between syntacticdly higher and lower

constituents.

- -

Designed Intelligence: A Language Teacher Model iv

Acknowledgments

Joy and despair visited me in completing this work, but my constant

companions were my supervisor, my f d y ~ and my friends. You always had

another nice word to say to keep me going.

Paul, you devoted much of your time and knowledge to my success;

such insights as can be found in this work are due to your outstanding efforts

and guidance as my supervisor. You were a wonderfiil person t o work with.

Many, many th&.

Thanks, too, to my cornmittee members, especially Fred Popowich, for

the thoughtfid commentary.

To ali my fkiends and fellow graduate students, all of whom 1 have not

seen much lately, thanks for your support and patience. And to the monitors in

the Language Leaming Centre, thanks for being so considerate during the

time when my foms was on this work.

To my f d y in Germany, my heartfelt thanks for a l l your care and

thoughts for me.

And Chris and Bodhi, during the little time I had to spare you gave

me aIi the support, love, and energy needed to complete this work. Our life

suffered, but k d y my mind can be with you again. LYTD.

Designed Intelligence: ALanguage Teacher Mode1 v

Table of Contents

. . Approvd ........................................................................................................... 11

... Abstract ........................................................................................................... iii

Acknowledgements ........................................................................................ v

List of Figures ...fi ........................................................................................ ix

CHAPTER 1 Intelligent Language Tutoring Systems . 1

1.1 Introduction ........................................................................ 1

........................... 1.1.1 Natural Language Processing 4

.................................................. 1.1.1.1 Syntactic Parsers 5

1.1.2 ParskgIll-Formedhputin

........... Intelligent Language Tutoring Systems 7

............... 1.1.2.1 Anticipating Ill-Formed Student Input 8

.................................... 1.2 Augmented Transition Networks 12

1.3 Intelligent Language Tutorhg Systems Using

1.4 Evaluation of Intelligent Language Tutoring Systems .. 28

........................................................... 1.5 The German Tutor 30

... ......................... 1.5.1 The Domain Knowledge .. 32

1.5 -2 The Licensing Module ............................ .. ..... 37

1.5.3 The halys is Module ......................... .... ..... 37

.......................................... 1.5.4 The Seiident Mode1 40

....................................... 1.5.5 The Filtering Module 41

Designed Intelligence: A Language Teacher Mode1 vi

CHAPTER 2

CHAPTER 3

............................... Analyzing Student Input 45

Introduction ...................................................................... 45

Errors in Feature Matchîng ............................................. 50

General Technique for Errors in

Feature Matching ............................................. 50

............................................ Ambiguous Errors 55

Two Likely Sources of an Error ......................... 56

......................... An Unlikely Source of an Emor 57

Analyzing Noun Phrases ................................. 60

Feature Percolation ................... .... ............. 62

........... Phrase Descriptors and Feature Values 67

............................................ Errors in Linear Precedence 69

2.3.1 General Technique for Verb-Second Position .. 70

................... 2.3.2 Verb-Second Position .,. ................ 72

.... 2.3 -3 General Technique for Verb-Final Position 75

.......................................... 2.3.4 Verb-Fin al. Position 77

........................................................................ Summary 80

Disambiguating Multiple Sentence Readings .................................................................. 83

Introduction .................................................................... 83

Licensing Module ........................................................ 88

3.2.1 Subject-Head and Head-Subject Licensing .... 90

3.2.2 Head-Subject-Complement and

................................. Head-Subject Licensing 93

3.2.3 Licensing vs . Fewest Error Parse .................... 96

......................................................... Ad Hoc Techniques 99

. ............................... 3.3.1 Finite Verbs vs hibitives 99

Designed Intelligence: A Language Teacher Mode1

3.3.2 Summary ....................................................... 105

.......... CHAPTER 4 Pedagogically-Informed Feedback 107

Introduction

...................................................... 4.2 The Analysis Module 113

......................................................... 4.3 The Student Mode1 119

................................ 413.1 Implicit Student Models 121

................................ 4.3.2 Explicit Student Models 122

4.3.3 Feedback Suited t o Leamer Expertise ......... 124

..................................................... 4.4 The Filtering Module 127

.............................. 4.4.1 The Error Priority Queue 130

.......................................... 4.4.2 Contingent Errors 132

4.5 Summary ....................................................................... 138

CHAPTER 5 Conclusion ....................................................... 140

5.1 Summary ........................................................................ 140

.................................... 5.2 Further Research .... . . 144

App endix

References

Designed Intelligence: A Language Teacher Mudel viii

List of Figures

Figure 1.1 : An Augmented Transition Network ............................................ 13

Figure 1.2 : The German Tutor .............................. .. ...................-................ 31

Figure 1.3 : Partial Lexical Entry for geht ................. .... ............................ 33

Figure 1.4 : M a r h g Number Features for Singular Nouns ....................... .. 35

Figure 1.5 : M a r h g Number Features for Plural Nouns ............................. 35

Figure 1.6 : Recordhg Number Features forgeht ....................................... 36

Figure 1.7 : DATR Code Listing for a Finite Verb in a Main Clause ............ 39

Figure 2.1 : Partial Feature Structure for geht .................... .. .................... 46

Figure 2.2 : Lexical Entry for g e b t .................... .. ........................................ 51

Figure 2.3 : Lexical Entry for er ................................................................. 51

Figure 2.4 : Marking Person Features for er ........................ .. .................. 52

Figure 2.5 : M a r b g Person Features for gehst .................... ....... ........ 53

Figure 2.6 : Marking Person Features for du .................... .. ...................... 53 ......... Figure 2.7 : Case, Number, and Gender Features ........................ .. 60

............................... Figure 2.8 : Lexical Entry for der ..... .......................... 62 Figure 2.9 : Lexical Entry for Gotter .......................................................... 64

.......................... Figure 2.10 : Lexical Entry for der Gotter ........................ .. 65

Figure 2.11 : Lexical Entry for zürnen ....................... .. .............................. 66 Figure 2.12 : Phrase Descriptors for Der Gotter ziinzen ................................. 66

Figure 2.13 : Partial Structure of Finite Verb Position .................................. 71

Figure 2.14 : Feature Values for Verb-Second Position ................................. 71

............... Figure 2.15 : Percolation Pattern for Er geht ............................. .. 72

Figure 2.16 : Unary Assigngos Rule ............................................................. 73 ............................. Figure 2.17 : Phrase Descriptor for Verb-Second Position 73

............... ................... Figure 2.18 : Percolation Pattern for Sie ihm hi&! .. 75

-

Demgned Intelligence: A Language Teacher Mode1 ix

Figure 2.19 : Partial Structure of Nonfinite Verb Position ............................ 76

Figure 2.20 : Feature Values for Verb-Final Position ................................... 76

Figure 2.21 : Percolation Pattern for Er kann gehen .................................... 77

.............................................................. Figure 2.22 : Unary Assigngos Rule 78

Figure 2.23 : Phrase Descriptor for Verb-Final Position ................................ 78

Figure 2.24 : Percolation Pattern for Sie Rann Mfen KZaus ......................... 79

Figure 3.1 : Two Analyses for Er fragt das Maddzen ..................................... 84

Figure 3.2 : Two Analyses for Er sieht die Frau ............................................. 91

........................................................................ Figure 3.3 : Subject-Head Rule 92

Figure 3.4 : Two Analyses for *Heute sieht sie er ......................................... 94

............................................................. Figure 3.5 : Lexical En* for g e k n 101

............................................................... Figure 3.6 : Lexical Entry for kann 103 .

Figure 3.7 : Phrase Descriptors forgehen as a Finite Verb ......................... 104

Figure 4.1 : Pedagogic Modules ..................................................................... 108

Figure 4.2 : Granularity Hierarchy for Constraints in Feature Matching . . 115

Figure 4.3 : DATR Code Listing for a Finite Verb in a Subordkate Clause 116

Figure 4.4 : Granularity Hierarchy for Constraints in Linear Precedence . 118

................................... ................................ Figure 4.5 : Contingent Errors .... 135

Remgned Intelligence: A hgriage Teacher Mode1 s

CHAPTER I Intelligent Language Tutoring Systems

1.1 Introduction

Cornputer-Assisted Language Learning (CALL) is the result of the

convergence of several fields of research addressing the use of computers in

language processing. Whde the influence of computational linguistics and

machine translation may be indirect, modern CALL s y stems draw heavily

upon the kdings of these two areas. The handling of naturd language by the

computer contributes greatly to the fluency of interaction between the human

user and the machine.

We k d a direct influence on CALL in experiments in Programmed

Instruction (PI). In the late sixties, CALL systems were primarily developed

on large-scale, maidkame syçtems in universities where computer sessions

were intended to replace classroom instruction. Programmed Instruction .. -. - .

Designed Intelligence: A Language Teacher Mode1 1

proposed the view that the best way of learning a task is to split it into small

units, where the successfd completion of one building-block ieads to the next.

The generally sound pedagogical principle of dividing a large Iearning

tasir into conceptudy smaller units was, however, distorted by an over-

emphasis on rote mernorization manifested in repetitive drills, multiple choice

answers, and uninformative feedback. For example, in a typical drill exercise,

the student was a&ed to type in the answer which was checked as each

character was entered. If any error resulted, the computer assumed control

and typed out the word "wrong" Barker & Yeates 19851.

CALL questioned the effectiveness of such systems. A program with a

simple letter t o letter match is incapable of dserentiating types of errors: not

only is it, therefore, incapable of providing any valuable, evaluative feedback,

but, in ignoring the source of the error when selecting another problem, it

relies upon an inflexible, program-centered, rather than student-centered,

definition of clBiculS. m e , in considering the early systems, one should

make allowances for the limitations of the then current technology, the fad

remains that, although PI has been refhed over the years, it has never

achieved a high degree of popularity P r i c e 19911. The original h e a r

programs (representing fixed sequences of instruction) improved, becoming

more sophisticated branching programs; most, however, remained based on

multiple choice answers. Inasmuch as the new is oRen an offshoot of the old,

as among teaching approaches which emulate some procedures of the previous

approach while rejecting others, CALL is hiçtorically related to PI, adopting

however a concern for individualized learnuig, self-pacing, and immediate

feedback.

Due to an increased emphasis of carrent teaching approaches on the

student as an active participant in the learning process, increasingly more


attention is beîng paid to the interactive aspects of CALL systems Burns,

Parlett, and RecEeld 19911. As a consequence, CALL research has shifted its

focus from drill-and-practice to tutoring systems. I n tutoring programs, the

computer is still the "judge-of-the-right-ansmer? But as opposed to drill-and-

practice applications, the path to the right answer involves a fair amount of

student choice, control, and in particular, student-compter interaction

m s c h a u e r 19961.

Marshall Cl9881 identifies the significant interactive palities of

Computer-Assisted Language Learning as one advantage of irnplementing the

computer into the language classro om. l'rue interaction, however, requires

intelligent behaviour on the part of the computer. Without intelligence, the

system is merely another method of presenting information, one not especially

preferable t o a static medium like print. Instead of multiple choice questions,

relatively uninformative answer keys, and gross mainstreaming of students

characteristic of workbooks, modern C U L is aixning a t interactive computer

systems possessing a high degree of &cial intelligence and capable of

processing Naturd Language input molland et al. 19931. For a program to be

intelligent, however, i t must emulate the way a Ianguage teacher evaluates a

student response.

A language teacher fkst examines the sentence and locates the error.

Once the error has been identified, s/he decides on the reason for the error. In

some instances, the source of an error might not be easily determined due t o

error ambiguity. For this reason, a teacherys evaluation of the error will take

into account fùrther factors. These might include the student's previous

performance history &or the Gequency and difnculty of a particular

grammatical construction. The final result of this process is error-contingent

feedbackl suited to learner expertise.


The goal of this dissertation is to design a mode1 of a language

instructor by evaluating and responding to student performance o n foreign

language exercises. The steps taken by a language instructor in the correction

process are achieved by the computational analysis described and

implemented, by way of example, in the Geman Tutor, an Intelligent

Language Tutoring System (ILTS) for German.

- The &st step ... in the evaluation process of students' input is the

sentence analysis. In ILTSs, this is handled by the parser and the grammar.

The parser analyzes a sentence according to the knowledge of the language

encoded in the grammar. In section 1.1.1, I will discuss the importance of

Natural Language Processing (NLP) to ILTSs. In section 1.1.2,I will introduce

existing techniques for parsing ill-forrned student input, a challenging task

not unique to ILTSs.

1.1.1 Natural Language Processing

The strength of NLP is that it dows for a sophisticated error analysis

where student tasks . can go beyond multiple-choice questions andor f2.l-in-

the-blanks. Simple drills are based on str ing matching algorithms, that is, the

student response is compared letter for letter against an answer key. However,

one obviously cannot enter the arbitrarily many sentences required for

meaningfd practice into memory for purposes of cornparison. NLP provides

the analytical complexiG underpinning an ILTS.

The pedagogicd goal behind an ILTS is t o provide error-contingent

feedback suited to learner expertise. For example, if a student chooses an

incorrect article in Gerrnan the error might be due t o incorrect infiection for

1. "F'eedback tailored to the nature of the student's error is called error-contingent feedback." Alessi & Trollip 119851, p. 116.


gender, number, or case. In such an instance the program must be capable of

distinguishing between the three error types. For cognitive leaniing to occur,

instructional feedback must address the different sources of an error

mumehart & Norman 1975, Venezky & Osin 19911.

The error analysis performed by the computer system f o m s the basis

for error-contingent feedback. Garrett Cl9871 describes four kinds of feedback:

1. "presents only the correct answer,

2. pinpoints the location of errors on the basis of the cornputer's letter-by ietter cornparison of the student's input with the machine-stored correct version (pattern markup),

3. based on analysis of the anticipated wrong answers, error messages associated with possible errors are stored in the computer and are presented if the studentrs response matches those possible enors (error-anticipation technique),

4. most sophisticated, uses an intelligent, Natural Language Processing (NLP) approach such as the "parsing" technique in which the computer does a linguistic analysis of the student's response,

Studies [Nagata 1991, 1995, 1996, van der Linden 19931 addressing

the question of what End of feedback a computer program should give have

shown that not only do students appreciate the more sophisticated feedback

made possible by NLP, but also perform better on the language skills being

taught. The fact that students learn better provides the rationale for

emploping parsers in Computer-Assisted Language Learning.

1.1.1.1 Syntactic Parsers

Natural Lanmage parsers show great promise in the syntactic

analysis of students' input. They are best employed for ILTçs in introductoryl

intermediate language courses where the focuç is primady on form rather

than on content. At the advanced level where a stronger emphasis is placed on

content, syntactic parsers become less usefiil because they are less reliable.

2. Garrett [1987], as cited in Nagata 119911, p. 330,


Inevitably, grammatical constructions become more elaborate and ambip~ous

at the higher levels which causes difficulties for parsers in general.3

A rider of studies have proven the usefuhess of syntactic parsers

in language learning. For example, a study conducted by Juozulynas Cl9941 at

Miami University showed that only 20% of errors in the essays of second-year

students of German are of a semantic nature.4 Juozulynas collected 349

students compositions. In d, 360 pages (313 essays) were included in the

study. The error distribution in his study was:

syntax: 28.6%

morphology: 24.4%

punctuation: 12.3%

spelling: 14.7%

semantics: 20%

A study by Rogers Cl9841 who collected 26 Gerrnan essays with an

average length of 569 words revealed similar results. Her distribution of

errors is:

syntax: 35%

morphology: 34.5%

lexical: 15.6%

orthography: 9.5%

complete transfer of EngIish expression: 5.4%

Adjusting Rogers' error classification to match ~ u o ~ y n a s ~ , 30% of

errors are of semantic origin. The higher percentage of semantic errors in

Rogers' study might be due to the fact that "the Miami University student

samples were Gom secondyear students, while the students in Rogers' study

3. There are a number of ILTSs which focus on communicative language leaming and thus emphasize fluency over accuracy. See Holland et al. [1995al and Swartz & Yazdani Cl.9921. 4. Juozulynas classified semantic errors as errors of meaning, such as wrong word choice, "made-upn words, and errors in pronoun reference. 5. For an even cornparison, Juozu lpas took Rogers' classification and assigned her categones lexical errors, complete transfer of EngZish expression, and some types of syntactic and morphological enors to the semantic category.


were advanced, with at least f o u years of learning Geman in a formal

environment, in many cases supplemented by visits to ~ e r m a n ~ . " ~

Given the outcornes of these studies, syntactic parsers can treat a

large percentage of student errors and thus are a powerfd tool in second

language learning.7 The following section, section 1.1.2, will discuss the

techniques employed by existing ILTSs in handling dl-fonned input.

. .

1.1.2 Parsing Ill-Formed Input in Intelligent Language Tutoring Systems

Although parsing ill-fomed input has been a challenge for all NLP

applications, ILTSs diEer fundamentally fkom other NLP applications in how

and why they handle ill-formed input. In most applications, the goal is to

successfidly parse and analyze a sentence, despite any errors. In ETSs,

however, the focus lies on tracing student's language knowledge rather than

o n the linguistic analysis of well-formed input. Such systems are inherently

prescriptive; for error-contingent feedback, they need to analyze where the

student deviated from the expert knowledge. In this respect, LTSs have a

more difficult task. Fortunatelx however, they usually do not deal with the

relatively large input domain found in other NLP applications.

In ILTSs, errors occur, not because the studentys howledge is a strict

subset of the expert knowledge, but because the learner possesses knowledge

potentially different in quantity and quality. For other NLP applications,

Weischedel & Sondheimer 119831 analyze errors as either absolute o r relative.

Absolute errors refer to errors in the language output. Examples are tense

and number mistakes, o r word order problems. Relative errors address

6. Juozulynas f19941, p. 16. 7. See also Sanders C1991J.

Deçigned Intelligence: A Langaage Teacher Mode1 7

grammatical structures which are correct but are beyond the scope of the

grammar.

Although this definition of absolute errors addresses the concems of

an ILTS, there is nonetheless a Meren t goal behind the two conceptions:

ILTSs aim at providing the student with error-contingent feedback. Thus

ILTSs need to analyze student errors, and not simply withstand them. The

error -analysis performed by an ILTS provides the source of an error and thus

enables the student to leam grammatid constructions of the target

language. Simply withstanding errors fails the purpose of a paner-based

KTS.

Relative errors refer to language the user possesses that is beyond the

system's knowledge, which is rarely the case with an Intelligent Language

Tutor. ILTSs focus on a language subset as described in students' grammar

books. It is a diagnostic tool for language learners to improve their second

language grammar skills. The system responds t o a student's work on the

basis of any errors the student may have made and the model of the student

which the system has constructed to that point.

1.1.2.1 Anticipating Ill-Formed S tudent Input

ILTSs augment grammars that parse grammatical input in one of

three ways. Each method overcomes some obstacles; however, ail have

concomitant disadvantages which will be discussed in the context o f specific

implernentations in section 1.2. The following is a brief description of the

three general approaches.

In the h t technique, ILTSs may augment d e s such that if a

particular rule does not succeed, specific error routines, that is meta-iules8,

force application of the ni le by systernatically relaxhg its constraints. Meta-


rules are commody employed for agreement errors. An example of a meta-

rule as d e h e d by Chen and Kurtz Cl9891 is given in (cl:

(la) sentence(s(Np, Vp, Num)) --> noun-phrase(Np, Num),

verb-phrasewp, Num).

Cb) verb-phrase(vp(vN), Num) --> intrans-verbW, Num).

(c) intrans-verbw, Num, Relax-flag) -->

it-verbCV, Num); . .

Relax-flag == true, it-verbW, WrongNum),

WrongNum == Nurn,

error-fiaggingCX, subject-verb-agreement,

TV, Num, ~ r o n ~ ~ u m l ) . ~

The g r m a r rules given in (la) and Cb) specZy that a sentence

consists of a subject and an intransitive verb which agree in number. If the

noun and the verb do not agree in number, the parse would simply fail. The

meta-rule given in (cl, however, relaxes the eonstraint on number and will

allow the pane t o proceed. The predicate errorfluggiing will add an error

message and the student can be informed that a rnistake in number has

occurred.

In a second approach, ILTSs may augment the grammar with d e s

which are capable of parsing ill-formed input (buggy rules)1° and which apply

if the grammatical iules fail. Buggy rules are distinct fiom meta-rules in that

they do not force application of the same d e , but rather, provide a distinct

rule for every ill-formed construction. For example, in German the paçt

participle always occurs sentence-finally in a main-clause, given in example

(2):

8. The term meta-nile used in this dissertation does not refer to schema for other context-free d e s as used i n grammar formalisms such as GPSG- 9. The meta-de given has been adapted from Chen & Kurtz 11989:58-91 who provide a d e for a transitive verb. 10. Buggy d e s are sometimes called mal-rules.


( 2 ) Der Mann hat das Buch &. The man has read the book.

Due to native language interference'', English leamers of German

will tend to place the past participle in the position which reçembles English

word order, that is, between the auxiliary and the noun phrase. A buggy d e

would anticipate precisely this enor by describing a structure where the past

participle incorrectly occurs in the position derived fkom the English n o m ,

succe~sfüLly parsai the ungrammatical sentence and providing error-

contingent feedback.

Finallx with feature grammar formalisms, lLTSs may also alter the

unification algorithm itself such that in the event of contlicting feature values

the parse does not fail, but instead applies a different set of procedures.

Parsers designed for language instruction typically contain

components which anticipate o r search for errors in the event that the

grammatical rules are not successfd, buggy rules being a common instance.

Searching for errors, however, presents a number of problems.

First, due to the vast error scope and unpredictability of some errors,

ill-formed input can only be partially anticipated [Yazdani & Uren 19881. The

fewer errors anticipated, the srnaller the error coverage. Sentences containing

errors which have not been anticipated by the designer cannot be processed by

the system, inevitably resulting in generic rather than error-contingent

feedback, and failùlg to achieve one of the primary goals of an ILTS.

11. Native language interference, or interlingual transfer, refers to the use of elements fiom one Ianguage while speakinglwriting another [Richards 19711. In leaming a second language, the Iearner maps onto previously existing cognitive systems, which usually is the native language. The school of Contrastiue Analysis [Lado 19571 t i e d to e~pla in errors through reference to the native language of the learner. However, this view was rejected by later work and studies in Errorhalysis [Corder 1967, Selinker 19721. See also Larsen-Freeman & Long C19911.


Second, anticipating errors lacks generalit~ Most systems base the

search for errors on the native language of the student. Any system which

anticipates errors requires that the same error in the target language is

presented in each different source language. The fewer source languages

considered the more user-limited the system. For example, a system which

anticipates errors made by English learners of German, will tend not t o

handie errors speciiically - . made by French leamers of German.

For the reasons of error coverage, generality, and user scope, the

methods employed in handling incorrect student - input in ILTSs should be

those that provide the most general treatment of errors, and that make

minimal use of emor anticipation.

The methods available t o detect errors are determined by the

programming language and the grammar formalism. KTSs have been

implemented in both procedural and declarative programrning languages-

Programming language to some extent determines the choice of grammar

formalism, which in turn determines what detection methods c a n be applied,

and how errors are perceived in the system Matthews 1993,1994, Sanders &

Sanders 19891.

The following @scussion wiU focus on LTSs which have been

implemented in either Augmented Transition Networks Woods 1970 1 or

declarative representations.12 The systems will be discussed +th respect to

the method used to detect errors and their e r r o r scope.13

12- A m , in their original design, were conceived as parsing rather than grammar formalisms. However, they have developed into a grammar-l3~e concept, especially in computational linguistics [Shieber 19861. 13. The systems discussed al1 place a pedagogical focus on form rather than content.


1.2 Augmented Transition Networks

Transition Networks (TNs) consist of a set of states, movement

between which is controlled by rules according t o the next elernent in the

input. States are referred to as nodes; transitions between states are referred

t o a s arcs. TNs are ipherently procedural.

Augmented Transition Networks (ATNs) additiondy employ

registers which hold grammatical information. ATNs also allow actions to be

associated with each arc, for instance, the setting of a register, o r the caUing

of another network.

The Augmented Transition Network Uustrated in Figure 1.1 parses

sentences by following the arcs, and accepting, from the input string,

constituents that are on the arc labels. The ATN can either accept a word, a

proper noun for example, o r it can c d another entire subnetwork, VP. Parsing

proceeds und a node is reached in which parsing can stop, indicated by the

double circle. In addition, the arcs set and test register NUM to control

number agreement between the subject and the verb.

The ATN given in Figure 1.1 parses sentences such as Mary sleeps by

going through the following steps:

Start in state 1 of S network. Accept NP (Mary, NUM = ssingular). Set register NUM of NP = singular. Set register NUM of S = singular.

Proceed to state 2 of S network, Cal1 VP network. Start in state 1 of VP network, Accept V (sleeps, NUM = singular).

Set register NUM of VP = singular.


NP VP set NUM of S test NUM of S =NUMofNP =NUMofVP

S 7 t n n

Figure 1.1: An Augmented Transition Network [Covington 19947

Proceed to state 2 of VP network and exit.

Test that NUM of S = NUM of VP

(the test succeeds since both have the value singular).

Proceed to state 3 of S network and exit.

While t h e Augmented Transition Network will successfully parse

Mary sleeps, note, however, that the sentence Mary sleep will fail since the

register on number agreement is violated. Violations of this End, however, are

common among second language students. An ILTS needs to parse such

sentences, and analyze the errors t o provide error-contingent feedback.

Meta- and buggy rules are the most common procedures employed for

handling students' errors in ILTSs. In ATN systems, agreement constraints

are commonly relaxed with meta-des, while buggy-des are used for errors

in word order. Buggy d e s might also make up a complete second g r m a r ,

the student's native language. In this case, meta-rules are used for analyzing

overgeneralization errors14.

The pioneer work in ETSs was developed by Weischedel et al. [1978].

The system is a prototype German tutor implemented as an Augmented

Designed Intelligence: A Language Tacher Mode1 13

Transition Network with a semantic and syntactic component. The authors

address four classes of errors:

2. agreement,

3. fkequently occurring errors due to native language interference, and

- Spelling edors are not treated winithin the system; instead a spelling

qrrection algorithm is applied before the sentence is analyzed by the parser.

The remaining three error classes will be discussed in the following sections.

Agreement

For agreement errors (subject-verb agreement, d e c t e d noun phrase

endings, word order among adverbial elements) the system uses predicates on

the constituents. The predicates check whether the forms are correct. If a

predicate evaluates to fdse, an error message is added. An example of a

predicate might be "CASE-FAILED?". If the predicate evaluates to tme, a

meta-nile is instantiated which loosens the constraint on case agreement.

mThile relaxing constrâints wïü lead to a successfid parse, the strategy

does not necessarily lead to the desired pedagogical outcorne. It is a technique

for permitting ill-formed structures rather than for analyzing 3.l-formedness.

For error-contingent feedback, a system needs to diagnose the source of an

14. Overgeneralization errors are the effects of particular leaming strategies on items within the target language, that is, overgeneralization of target language rules. For example, a shident might choose for the verb schlufen the regular past tense s u f i -te *schlufte as opposed to the correct irregular form schlief. According t o Richards [19711 such learning strategies appear to be universally employed when a leamer is exposed to second language data; this is confirmed by the observation that many errors in second language communication occur regardless of the background language of the speaker. Error Analysis [Corder 1967, Seünker 19721 states that equal weight can be given to interlingual and overgeneralizatio n errors.


error rather than merely relaxhg the constraint. Consider example (3a), cited

by Schwind [1990a, 19951:

(3a) *Der Gom zümea.

(b) Die Gotter Wnen.

me gods are angry.

The subject of the sentence is Der Gotter and thus requires

nominative case. Der Gotter, however, is a e c t e d for genitive plural and the

feedbàck would be me determiner is incorrectly inflected for case. However,

the mistalre is ambiguous and thus depends o n the context.15 In example (3a),

it is very unlikely that the student constructed a subject assigning genitive

case which is the least cornmon a d thus hardest of the four cases in German

[Schwind 19951. The desired feedback should be The determiner is incorrectly

inflected for number since Gotter is plural and der is singular. This analysis is

not possible by simply relaxing the c o n s t r k t on case. It is the context in

which the particular error occurs which needs to be considered t o achieve the

desired feedback.

Errors like the one given in example (3a) cause because the

analysis performed by the parser is correct from a computational point of view

but the feedback is pedagogically wsound. The parser and grammar will not

dag der G ~ e r as an error in number because the determiner and the n o u n

make up a correct noun phrase in the genitive plural. However, when the verb

combines with the noun phrase a case error is recorded because the verb

requires nominative case.

15. Articles in Geman are ambiguous. Depending on where the e r o r occurs der, f o r example, could be a mistake in case, gender, o r number.

Designed Intelligence: A Laquage Teacher Model 15

Frequently O c c h g E m r s due to Native Language Interference

For fkequently occurring errors in word order due to native language

interference, Weischedel et al. Cl9781 add each incorrect form as a buggy rule.

An example of a native language interference error concerns the position of

the past participle in Geman, illustrated in example (4a).

- . --

(b) ~r && das Kind -. He has seen the child.

ATNs do not allow for a very general treatment of errors, particularly

errors in word order. Each individual error has to be anticipated and a buggy

rule needs ta be implemented. Overgeneralization errors in word order, for

example, although not implemented in Weischedel's system, would need to be

treated in a similar way as language transfer errors in ATNs, that is, by ad

hoc d e s . 16

Similar failings, due to the need to anticipate errors can be found in

Liou [1991]. That ATN system consists of an expert model, containing correct

grammatical constructions and a bug model which anticipates the ill-formed

structures. Liou's program covers seven types of errors: det-noun phrases,

conjunctionç, verb morphology, subject verb disagreement, capitalization,

although ... but, and no ngttez.. combinations. The error scope is fairly small

and an extension would involve increasing the bug model by painstakingly

encoding a buggy d e for each likely error

16. Larsen-Freeman & Long L1991J list examples for errors due to factual misconception which they label simplification and induced errors In addition there are mistakes as opposed to errors, a distinction made by Corder 119671. Mistakes are random performance slips. In such cases, the student is aware of the lexemes and gramrnar rules of the target language but due to a lack of Sping skills, concentration, and/or fatigue mistakes occur. Al1 of these error types would require ad hoc rules.


Another example of the same heuristics is found in ALICE-chan, a

multimedia foreign language leaming environment for ~ a ~ a n e s e ' ~ developed

by Levin & Evans [1995]. The system covers grammatical topics typically

taught in a fitçt-year Japanese course and includes about 100 d e s . The

limitations are again that speaal rules need to be designed to parse errorful

structures. Thus their treatment is n o t very general.

The h a 1 error class Weischedel addresses is ambiguous readings.

Accordhg to Weischedel et al. [1978] the intended meaning in example (5) is

The man is giuing the girl a hat rather than The girl is giving the man a hat.

(5) Dem Miidchen gibt der Mann einen Hut.

neuter, dative rnasc, nom musc, accusathe

The heuristic Weischedel's system uses t o determine the

interpretation intended by the student is ta select the parse that gives the

fewest errors. Ambiguous readings are rarely addressed by ILTSs although

they are a common phenornenon with highly idected languages such as

~erman. l8

Weischedel's method is a computationally effective way to resolve

ambiguous readings. However, from a pedagogical perspective "a more

sophisticated routine w o d d take into account the likelihood of various kinds

of e r ~ o r s " ~ ~ . The likelihood of an error is determined by the leamer level and

the frequency of a grammatical construction. Consider example (6):

17. The system also includes a Spanish module. 18. Levin and Evans [1995] resolve ambiguity through interaction with the user Covington & Weinrich [1991] use the same heuristics as Weischedel et al. [1978]. However, they express their doubts about the usefùlness of the rnethod employed. 19. Covington & Weinrich [1991], p. 153.


(6 1 Sie liebt er.

fem-, nom or fern., acc masc, nom

I f s her he loves.

In example (6) the intended meaning could either be It's her he loves

o r Ske loues him. According t o Weischedel's approach, the former

interpretation would win since the sentence contains no errors, while the

latter has an error associated with the direct object. However, a beginner

student of German most likely intended to write she loves hin, not knowing o r e

overlooking the fact that lieben assigns accusative case to its object. The

interpretation It's her he loues is c e r t d y grammatical if the object receives

stress. But in writing, even native speakers would probably underline Sie

indicating the emphasis of the direct object. An even more puzzling example is

given in (7):

(7) * Sie dankt e r.

fern., nom or fem., acc musc, nom

It's her he thanks or She thanks him.

Choosing the parse with the fewest errors, would not be sufncient for

example (7). The verb danken assigns dative case. The two possible readings

are either It's her he thanks o r She thanks him but in both interpretations

there is one mistake: either sie, nominative is used uistead of ihr, dative o r er,

nominative instead of ihm, dative, respectively In such an instance, relying on

the parse with the fewest errors makes the choice of parse arbitraq even

though the student most likely intended Sie dankt ihm. A better approach to

treating ambiguous readings will be provided in Chapter 3.

Augmented Transition Networks are limited in their application to

ILTSs. In procedural programs, the data and its implementation are

interwoven which makes a system less general and makes it more diffcult t o

provide a large coverage of emors, because errors need to be anticipated and

encoded similar t o a pattern-matching rne~hanisrn.~~


?

Furthermore, alI the ATN systems desaibed above are tied to a

specSc native language of the student, in Weischedel's system, for example,

English. For students with different native languages, the systems wodd

require even more buggy ni les since the error anticipation is derived from the

native language of the student. Examples (8) - (10) illustrate the problem:

(8) German: Ich & es fesehert.

(9) French: Je l'd ou. . -

(10) ~ n & s h : 1 have - it. In German, the direct object appears in between the auxil iq and the

past participle, the auxïiiary always being in second and the past participle in

final position in main clauses. A French leamer of German is likely t o commit

errots concerning the position of the auxiliary, while the EngIish leamer of

German will have problems with the position of the past participle. For an

ATN to deal with the error caused by either native language, buggy d e s of

each er ror would have to be implemented.

A further shortcoming of procedural systems is that they are not

modular. Such systems cannot be easily altered and applied to another

Ianguage. The whole system has t o be rewritten, when preferably one could

replace only the language-dependent grammar withi-n a language-

independent shell.

In distinction tg Augrnenkd Transition Networks, declarative

systems contain feature grammar formalisms which are based on unification.

In addition to meta- and buggy rules, ill-formed input can be processed by

20. Johnson's evaluation of generd ATN systems can be well applied to ILTSs: 'Zike any procedrval model, an ATN gives extensive possibilities for optimization and tight control over details of parsing strategy. The pnce to be paid in return is a danger that as the system increases in size, its logic becornes obscure, modification progressively more risky, and debugging more and more problematic." Johnson [1983J, p.72. See also Loritz 11992, 19951 who M e r discusses the implemenbtion o f a s in ETSs.

Designed Intelligence: A Langaage Teacher Model 19

altering the unification algorithm itself. Section 1.3 will discuss ILTSs

employing declarative formalisms.

1.3 Intelligent Language Tutoring Systems Using Declarative Formafisms

. * *

The bulk of ILTSs are implemented as declarative representations.

The main types of grammar formalisms used in ILTSs are Government

Binding [Chomsky 1981, 19861 and Logic Grammars (Colmsrauer's

Metamorphosis Grammar Cl9781 and De=te Clause Grammar Pereira &

Warren 19801). In addition to the main types, KPSG as part of the GPSG

family [Gazdar et al. 1985, Gazdar 19821 has been implemented in an ILTS by

Hagen [1994].

There are three common methods of analyising ill-formed input used

in declarative-based ILTSs. In the f k s t approach, in addition t o the target

grammar, the student's native language is explicitly encoded to cover errors

due to native language interference. Depending on the error scope of the

system, meta-des may be added for overgeneralization errors. The second

approach does not rely on the native language of the student, instead relying

on meta-des t o relax the constraints of the target language. In the third

approach, the iinification algorithm is altered in such a way that the system,

despite clashing features, performs a parse and keeps track of the conf l ict ing

features.

An example of the fkst approach is Schuster's [1986] system for

Spanish Ieamers of English. The system focuses o n the acquisition of English

verb-particle and verb-prepositional phrase constructions. The program

Designed Intelligence: A Language Teacher Mode1 - .-

contains two grammars, Spanish and English. An obvious shortcoming of the

system is that it can only deal with native language interference errors.

Overgeneralization errors, another significant source of students' errors, are

ignored and cannot be recognized by the system.

Catt & Hirst [1990] overcome the shortcoming in Schuster's system

by implementing meta-des for overgeneralization errors. Their system

&ripsi consists of t h e e grammars: English as the target language, French

and Chinese as the native languages of the leamers. Whenever the input

cannot be recognized by English rules alone, the system applies the rules of

French or ~hineçe.~' Although the system represents an extension of

Schuster's system, nonetheless, an important error class passes unrecognized:

errors in word order can only be detected if due t o native language transfer. A

further, rather apparent, limitation of the authors' approach lies in its lack of

generality. First, the same error in the target language has to be encoded in

each source grammar, French and Chinese. Second, identical grammatical

structures are encoded in all three grammars leading to redundancies.

Wang & Garigliano [1992,1995] designed a Chinese Tutor for English

native speakers. In their design, they attempted to avoid the kind of

redundancies found in Catt and HirçPs system. The Chinese Tutor models

only the fragments of the grammatical rules in the native language which are

different from the corre&onding d e s in the target language. Additionally,

the authors address only those transfer errors j u s s e d by empirical data they

collected. They found that 78% of the errors made by English leamers of

Chinese are due to transfer. The remaining 22% are ignored in their system.

21. It is not quite clear from the description in Catt & Hirst 119901 whether a sentence would actually run through three grammars or whether the students possib1y identie their native language when uçing the program and required switches are set.


The percentage of transfer errors is natwally cIosely dependent on the two

languages ~ o r n ~ a r e d . ~ ~

The deficiencies of the systems employing native grammars are

either poor coverage of errors, as in the case of neglecting enors due other

than to native language interference, and lack of generality The systems also

overtly rely on a particular native language of the user and are thus JEmited

in their scope. This-=limitation becomes more problematic due to the increase

in multi-culturalism and on-line Distance Education over the Internet. Any

overt presumption of the native language of the student is severely

restrictive.

A variation of error handling is found in other dedarative systems.

For example, Kurtz, Chen, and Huang Cl9903 developed a DCG system XTRA-

TE for Chinese learners of English. The authors only use correct grammar

rules with meta-des that relax constraints at multiple levels. Level 1

contains all grammatical constrâints, Level 2 relaxes syntax, Level 3

semantics, and Level4 both syntax and semantics. In addition, for translation

exercises the system performs a literal word-by-word translation. The

authors' system is more general in the sense that they do not anticipate

errors on the basis of the native language of the student. However, the system

cannot handle errors in word order. The only constraints which can be relaxed

are agreement emors, both syntactic and semantic.

A similar pitfall can be found in a system for leaming English

developed by Covington & Weinrich [1991]. The program is written in Gulp,

an extended form of Prolog which includes a notation for feature structures

22. Weinreich 119531 asserted that the greater the m e r e n c e between the two languages, i.e. the more numerous the exclusive forms and patterns in each, the greater is the learning problem and the potential area of interference.


[Covlngton 19941. The system focuses o n agreement, insertion, and omission

errors. Insertion and omission errors are due to a violation of the

subcategorization List of the verb. For example, a student might use an

inherently transitive verb without an object. The authors relax constraints by

implementing a lenient d e for each rule which parses well-formed input.

However, errors in word order pass unrecognized.

- Meta-rulespby dennition, cannot address errors in word order. They

c m on ly relax constraint violations of grammatical features. Ali systems

which handle word order errors anticipate these by implementing buggy

rules. However, in ail cases the result is very user-speciiîc since the systems

rely on a p a r t i d a r native language of the student. The ones which

implement more than one native language lack generality The same error in

the target language is presented in each source language.

The third approach in error detection within declarative systems

makes use of the feature grammar formalism i t s e ~ ~ ~ In declarative

implementations, the grammar makes heavy use of the operation on sets of

featues called unification. Unification-based grammars place an important

restriction on unification, namely that t w o categories A and B fail t o unifv if

they contain mutually inconsistent information [Gazdar & P d u m 1985,

Knight 19891. However, this inconsistent information constitutes exactly the

errors made by second language Ieamers. For example, if the two categories A

and B do not agree in gender a parse will fail. A system designed to

accommodate the errors of second language leamers' requires either a

moditication o f the unification algorithm itself or feature specScations

capable of overcoming the gender restriction.

23. Covïngton & Weinrich 119911 use features t o describe the particular error but their error detection mechanism is based on buggy d e s rather than on the feature grammar formalism-

Designed Intelligence: A 1I;anguage Teacher Mode1

Hagen [1994] developed an ILTS, GrammarMaste< for French as a

second laquage. The program is driven by an object-oriented, &cation-

baçed parser mitten in Hypeflalk. The systern addresses three particular

grammatical constructions: conjunctions, reflexive binding, and dislocated,

missing, and s u p e ~ u o u s constituents.

The goal of the system is to show an implementation of some "thorny

problems of complqx grammatical c~nstruct ions-~~~ Hagen's systern borrows *

analyses fkom a number of grammar formalisms: Head-Driven Phrase

Structure Grammar (HPSG), Generalized Phrase Structure Grammar

(GPSG), and Government Binding (GB). Missing conçtituents are handled

with the SUBCAT feature of HPSG, reflexive binding makes use of the foot

feature principle (GPSG), and superfluous constituents are controlled by

thematic assigrunent (GB).

Hagen relaxes the unification algorithm to block parsing faïiure such

that if the parser udetects contradictory features like [geiMJ and [gefl, it

preserves the features o n the major part of speech (in the case of noun

phrases, the noun) and inserts the contradictory feature o n a minor part of

speech like an article into an error stack dong with a corrective message."25

Hagen's implementation of the prinaples fhm GPSG, HYSG, and GB

d o w for a general treatment of the errors considered. T h e errors do not need

t o be anticipated. If any of the grammar principles does not hold, the

constituent is flagged and an error message is attached. However, Hagen

addresses a very small range of errors; the system also lacks a uniform

syntactical fkamework and seems linguistically rather than pedagogically

r n o t i ~ a t e d . ~ ~ In addition, altering unification implies inserting procedural

24, Hagen C19941, p. 5- 25. Hagen 119941, p. 16.

techniques in an inherently dedarative algorithm. Thus the andysis becomes

closely tied to its implementation, that is, the grammar only works if

unincation is changed in a certain way

Schwind [1990a, 19951 developed a system for German based o n

Colmeraueis Metamorphosis Grammar [Colmerauer 19781. Her system

covers a broad range of errors: agreement, syntactic and semantic errors. Her

method of analyzing-agreement errors is based solely o n the feature gramrnar

formalism. The class of agreement errors covers errors in gender, number,

person, and case of noun phrases. The system does not anticipate these errors.

~nçiead, each lexeme is specified for every possible grammatical occurrence27

In addition, the unification-based algorithm builds up two sets of feature

structures: "... iiriifv (a,b,r,e) holds whenever r is the result of the unification

and e is the set wnsisting of di the pairs of values for which a and b could not

be uniIied, together with all the symbols contained in the symmetrical

difference between a and b."28 Thus Schwind's defkition of unification m e r s

fkom the usual one 19841 in the sense that the parse does n o t fail

but instead records the elements which do n o t un3y

Schwind's analysis also allows for discriminating among errors that

are phonetidy identical, but differ in their source. These errors occur within

26. Grammatical constructions which are linguistically and computationally complex do not necessady present difEculties for the language learner. For example, the errors second language leamers make with conjunctions are Emited to subjectherb number agreement and case assignment of the subject or verb complement. While conjunctions present a challenge from a computational point of view since i t is of utmost importance for the linguist that the parser displays the correct analysis, for the language leamer i t is the feedback that outweighs the linguistic and computational analysis [Farghali 19891. 27. In German, there are three genders, four cases, and two numbers. While this results in 16 potentially distinct lexical forms, there are only six f o m s for definite articles, which are phonetically distinct. For example, the definite article der can be Cgenitive, plural], [genitive, feminine, singular], [nominative, singular, mascl, [dative, feminine, singularl. Schwind [19951 uses only one lexical entry for der, see also lMcFetridge & Heift 19951. 28. Schwind [19951, p. 305.

Desrigned Intelligence: A Language Teachex Mode1 25

noun phrases. The problem with noun phrases in German derives f?om the

fact that there are three features (gender, nurnber, and case) any of which can

be wrong.29 Consider examples ( l l a ) - (13a), as cited by Schwind [1990a,

19951:

(lia) *Der Kind spielt.

(b) Pas Kind spielt.

nominatiue

The ,&ild is p Zay ing.

(=a) *Er gibt der Milch,

Cb) Er gibt dem Milch.

dative

Ne is giuirrg the child milk.

(13a) * Sie kennt der E n d .

Cb) Sie kemt-.

accusative

She knows the ~ h i l d ? ~

In examples (lla) - (13a), the error occurs with der Kind. According to

Schwind, the desired feedback in example (lla) and (12a) is ZJie de teminer is

incorrectly inflected for gen&r. while in example (13a) it is The deteminer is

incorreêtly inftected for case. In the three examples, although the surface error

der is identical, the sources of the errors are distinct. This is due to the

distinct case requirements of the three examples and the ambiguity of the

determiner del: In example (lia), the source of the error is gender since der is

a possible article in the nominutive (der, die, das). However, &ce der is not a

possible accmative article (den, die, das) as required in example (13a), the

source of the error lies in case. Schwind Cl9953 applies case filtering to achieve

the desired feedback. Her system fmt checks whether the article is a possible

determiner for the required case. If it is, the system responds with a gender

29. These three features do not cover the declension of adjectives. Adjectives also i d e c t according to whether or not they are preceded by a definite o r indefinite article. 30. Schwind C19951, p- 312.

Designed Intelligence: A Language Teacher iMode1 26

error; if not, the source of the error is case. The shortcoming of her analysis is

that the system assumes that students in general are more likely to know

grammatical case than gender. This is apparent in example (12a) which

Schwind Cl9951 analyzes as an error ingender as opposed to in case. The error

is ambiguous due t o the grammatical ambiguity of der. If the student

assumed that Kind is feminine, the error is due to wronggender. However, if

the student knows the correct gender but did not assign the correct case, the

error-source is wrdiig case. Ambiguous errors will be further discussed in

Chapter 2.

Schwind uses the same analysis for semantic errors. The system

recognizes the violation of semantic restrictions on verbs and their

compIements. For this, she provides a semantic network where semantic

constraints are expressed as feat~res.~ ' For example, while cornputer is

[inanimate], woman is marked as [animate, human]. The analysis is again

very general, that is, the errors do not have to be anticipated.

One sigdicant deficiency of Schwind's system is its handling of high-

level syntax errors. In syntax, Schwind makes a distinction between low- and

high-level. Low-level syntax errors are errors of insertion or omission while

high-level errors refer to errors in word order. Schwind uses one buggy rule

for each constituent of insertion and omission. High-level syntax errors,

however, must be anticipated and according to Schwind Cl9951 their

treatment is not general.

Feature grammars are more promiçing than ATNs for Intelligent

Tutoring Systems. The data and implementation are kept separate which

allows for easier expansion. Feature grammars also accommodate more

31- see also Weischedel et al. [1983]. The authors present a very similar analysis of semantic networks. However, i t is not as complete as the one by Schwind C1990b1.

Designed Intelligence: A Tnnguage Teacher Mode1 27

methods of error detection. In addition to meta- and buggy-rules, they allow

for alteration of the unification algorithm itself Altering the unification

algorithm has its pitfalls as rnentioned; however, a more general treatment of

errors than with meta- and buggy rules alone can be achieved.

Feature relaxation presents an alternative t o existing methods and is

the method adopted in t b i s dissertation. Unlike existing ILTSs, the analysis

descrïbed does not 'keek or anticipate errors, but instead emulates the way a

language instructor evaluates a student's response. The computational

analysis reflects the pedagogicd bias of the system.

1.4 Evaluation of Intelligent Language Tutoring Systems

The foregoing discussion illus trates t w o developmental stages in

ILTSs that primarily focus on form rather than o n content. The early ILTSs,

as the one by Weischedel e t al. [1978], concentrated solely on tackling the

computational problems of parsing iIl-formed input as opposed t o embedding

pedagogic considerations into such systems. For example, while Weischedel's

system considers ambiguous readings, they are addressed fkom a

computational rather than a pedagogic point of view. This is evident in the

algorithm used to handle such errors. Selecting the parse wi th the fewest

errors is a computationally effective method, but it does n o t address the likely

cause of error. A pedagogic system would consider the performance level of the

learner andior the language leamhg task in order t o address ambiguous

readings.

Desigaed Intelligence: A Language Teacher Mode1 28

The second phase of ILTSs, while still extending the parsing

capabilities of iu-formed input, also attempts t o emphasize pedagogic

considerations. For example, Schwind [19951 and Holland [1994] seek a wider

error coverage in a computationally more generd way, and also address

language learning pedagogy in c o n s i d e ~ g ambiguous errors. However, their

pedagogic focus is Zimited. While the alg0ntb.m addresses ambiguous errors to

some extent, contingent errors cannot be handled adequately - - #

The discussion of existing lLTSs shows that neither phase has been

completed. From a computational point of view, some errors such as errors in

word order, for example, s t i l l need to be anticipated and thus are not

addressed in a general way. The algorithms are based on a particular native

language of the student and the systems are thus limited in error coverage

and user scope. From a pedagogical perspective, ambiguous and contingent

errors are important to language instruction.

The analysis presented in this dissertation addresses both

developmental phases in ILTSs. From a computational point of view, a wider

class of errors is addressed in a general wax thus n o t s a d c i n g error

coverage and user scope for errors in word order. From a pedagogical point of

view, the andysis considers the learner by emulating the ways a language

instnictor evaluates and responds to a student's performance in language

learning tasks. As a resdt, ambiguous and contingent errors, as well as

multiple sentence readirigs, are treated with a strong pedagogic focus. In

addition, instructional feedback is suited to learners' expertise.

The following section will outline the Gennan Tutor, which austrates

the analysis desrribed in this dissertation by way of example.

- -


1.5 The G e r m a Tutor

The NLP component of the G e m a n Tutor parses students' answers to

introductory German exertises and returns structures which specdy student

model updates and possible feedback of diEerent levels of granularity - - -

coaelated with çhidents' expertise. The concept of granularity has been

previously applied to an Intelligent Tutoring System for LISP programming

[see Greer & McCalla 19891. For ILTSs, granulari@ is particularly important

in &amhg responses to learners' errors. Inexperienced students require

detailed instruction while experienced students benefit best fkom higher level

reminders and explmations LaReau & Vockell19891.

The design of an ILTS implemented in the G e m n Tùtor consists of

five components: the Domain Knowledge, the Licensing Module, the Analysis

Module, the Student Model, and the Filtering Module. The five components,

given in Figure 1.2 correspond t o the steps a language instiuctor applies in

evaluating and responding to students' errors.

The Domain Knowledge represents the knowledge of the language. It

consists of a parser with a grammar which parses sentences and phrases t o

produce sets of phrase descriptors NcFetridge & He* 19951. A phrase

descriptor is a model of a particular grammatical phenornenon such as case

assignment or number agreement. A phrase descriptor records whether o r not

the grammatical phenomenon is present in the input and correctly or

incorrectly formed.

Due to structural ambiguity found in language, a parser produces

more than one parse for many sentences. The Licensing Module selects a


Student Input

knowledge of the language encoded in the grammar

. - J + sends phrase descriptors of dl parses

Licensin~ Module

seIects the desired parse

i sends al1 phrase descriptors of the selected parse

retums student model updates and produces instructional

feedback of increasing abstraction

4 sends a student model update and instmctional feedback

l Student Model I l keeps learner model updates

and decides on the student level I 1 sends instructional feedback suited to student expertise

fil ter in^ Module decides on the order of instructional feedback

Error-contingent Feedback Suited to Learner Expertise

Figure 1.2: The German Tutor


parse by taking into account factors a language instructor considers in

evaluating a student's response. In determining the most likely sentence

reading, a language instructor considers the level of instruction, the

eequency of the grammatical construction, and thus the likelihood of the

error.

The Analysis Module incorporates a language instructor's knowledge

in piqointing the precise source of an error The Analysiç Module takes a

phrase descriptor as input and generates sets of possible responses to the

leamer's input that the instruction system can use when interacting with the

student. The level of the learner, either expert, intermediate, o r novice

according t o the m e n t state of the Student Model, determines the particular

feedback displayed. The Student Model records mastery of grammatical

stnictures as well as structures w i t h which the learner has problems.

The Filtering Module determines the order of the instructional

feedback displayed t o the learner. The system displays one message a t a time

so as n o t to overwhelm the student with multiple error messages. In addition,

the Filtering Module takes into account contingent errors, a class of multiple

errors t o be discussed in Chapter 4.

1.5.1 The Domain Knowledge

The parser analyzes students' input according t o the knowledge of the

language encoded in the grammar. The parser and the gramrnar thus provide

the linguistic analysis of students' input. The grammar fo r the German n t o r

is written in ALE (The Attributed Logic Engine), an extension of the

cornputer language Prolog. ALE is an integrated phrase structure parsing

and definite clause p r o g r d g system in which grammatical information is

expressed as typed feature structures. Typed feature structures combine type


inheritance and appropriateness specifications for features and their values

[Carpenter & Penn 19941.

The grammar formalism used is derived &om Head-driven Phrase

Structure Grammar Vollard & Sag 1987, 1994, Nerbonne et al. 19941. This

theory is one of a family which share several properties. Linguistic

information is presented as feature/value matrices. Theories in this family

are -to yarying dk-&ees lexicalist, that iç, a considerable amount of

grammatical information is located in the lexicon rather than in the grarnmar

rules. For example, Figure 1.3 illustrates a minimal lexical en- forgeht. The

subcategorization list of the verb, notated with the feature subi, specifies that

geht takes a subject which is minimdy specifîed as a singular noun. Rules of

gra-ar specify how words and phrases are combined into larger units

according to the suhcategorization List. In addition, there are principles which

govern how information such as the head features is inherited.

phon < geht z

vat

head tt

subj cat [head n]

content kndex [num

Reln geht [Geler 1

Figure 1.3: Partial Lexical Entry forgeht

Designed Intelligence: A Isnguage Teacher Mode1 33

The philosophy of the analysis described differs fkom other systems

designed to parse ill-formed input in ILTSs in that it does not seek or

anticipate errors, but instead records whether or not grammatical constraints

are met, the goal being to analyze students' language input rather than the

more modest aim of recognizing ïü-formed constmctions. The analysis rnodels

the steps taken by a language instructor in the correction process of students'

input. - - f-

The grammar itself is written as a set of constraints. Any constraint

such as the one in Figure 1.3 that the subject of the verb geht must be a

singular noun may block parsing ifnot successfully met. The terminology used

in Figure 1.3 will be described in more detail in Chapter 2.

In HPSG, the information flow is modeled by structure-sharing and

not by transformation o r movement. Two o r more distinct attributes (or paths)

within a feature structure are said to be structure-shared if their values are

specitied by one and the same feature structure Pollard & Sag 19871.

Structure-sharing in feature-value matrices is indicated by multiple

occurrences o f a coinderring box labeling the single value. For example, in the

lexical entry for the verb geht, given in Figure 1.3, the number feature of the

verb and its subject share one common value, x. Ifthis condition is not met,

that is, either the subject o r the verb is not singular, unification fails. In ILTSs,

however, sentences contghing such inconsistent information need to parse

successfully so that the error can be discovered, analyzed, and reported t o the

student.

The constraint on number agreement c m be relaxed, however, by

changing its structure so that, rather than checking that the noun is singular,

the system records whether or not the subject of geht is in the singular. To

achieve this, the noun is no longer marked as [num d, but instead the path


numl sg terminates with the values ermr or çorrect- For example, for a

singular n o u n phrase, the value of the path num I sg is correct, while it is error

for a plural noun phrase. The two partial lexical entries are given in Figure

1.4 and Figure 1.5, respectively

-

2- synsem local content index I

Figure 1.4: Markhg Number Featmes for Singular Nouns

.-

synsem

C

local content

Figure 1.5: Marking Number Features for Plural Nouns

The verb geht records the value of ~g hom its subject (Figure 1.6). If

t h e value of the path n v I sg is correct, the subject is in the singular. In case

of a plural noun, geht records the value error for number agreement.

The goal of the parser and the grammar is the generation of phrase

descriptors. A phrase descriptor is implemented as a fiame structure that

models a grammatical phenornenon. Each member of the fiame consists of a

name followed by a value. For example, subject-verb agreement in number is

modeled by the Game [number, value] where value represents an as yet


phon < geht >

local

cat * - i

head v

subj

des criptor

synsem 1 cat [head n]

local

Figure 1.6: Recording Number Features forgehf

uninstantiated value for number. If the grammatical phenornenon is present

in the student's input, the value is either correct or error depending o n

whether the grammatical conskaint has been met or not, respectively. If the

grammatical constraint is missing, the feature value is absent. Consider

examples (14a) and (b):

6) Er geht.

He is leaving.

The phrase descriptor for subject-verb agreement in number in

example (14a) is [numberyerrorl, while that for the sentence in (b) is

[number,correct]. For either sentence, (14a) o r (b), the information will be

recorded in the Student Model. A system presented with (14a), however, will

also instnrct the learner on the nature of subject-verb agreement in number.

Designed Intelligence: A Language Teacher Model 36

In addition to the grammat ica l features defined in HPSG the

grammar uses a type descriptor representing the description of the phrase

that the parser builds up. This type is set-valued and is initially

underspecified in each lexical entry- During parsing, the values of the features

of descri~tor are specified. For example, one of the members of descriptor,

vp num in Figure 1.6, records the number agreement of subject-verb in a

main-clause. Its value is inherited from the feature speded in the verb - -

geht. Ultimately, descnotor records whether the sentence is grammatical and

what errors were made.

1.5.2 The Licensing Module

The Licensing Module determines the most likely reading of a

sentence. Due to language ambiguity. parsers generally produce more than

one syntactic structure. Feature relaxation leads to an even greater number o f

parses. Selecting the fkst parse o r the parse with the fewest errors is

inappropriate for the design of an ILTS described. Neither solution takes into

account the likelihood o f an enor, possibly resdting in misleadhg feedback.

The design of the Licensing Module reflects a language instructor's decision in

selecting the most likeIy reading of a sentence by considering the source

language of the learner. the level of instruction, and the fiequency of a

grammatical construction. This is discussed in detail in Chapter 3.

1.5.3 The Analysis Module

The third component of the system is an Analysis Module which

encodes knowledge of the language instructor to determine the exact source of

an error. The Analysis Module takes phrase descriptors as input and

generates possible responses that the instruction system can use when


interacting with the leamer. A response is a pair that contains a message the

system will use to inform the leamer if a phrase descriptor indicates there

has been an error and a Student Model update. The Student Model update

contains the name of a grammar constra.int in the Student Model dong with

an instruction to increment or decrement the corresponding cumulative total.

The Analysis Module generates sets of instructional feedback of

increasing abstraction. As an example consider the ungrmatical sentence

in (15a). An inexperienced student should be informed that Madchen is a

neuter noun, that the verb danken is a dative verb and that the determiner

das is incorrect. A student who has mastered case assignment (as indicated

by the Student Model) may be informed only that the case of the object is

incorrect.

(15a) *Der Mann dankt m. (b) Der Mann dankt

TILE man t h a h the girl.

The Analysis Module is implemented in DATR Evans and Gazdar

19903, a language designed for pattern-matching and representing multiple

i n h e r i t a n ~ e . ~ ~ The Sussex version of DATR is implemented in Prolog.

Nodes in DATR are represented by the name of the node followed by

paths and their values. A partial representation of the node which

corresponds to the phrase descriptor that records the position of a finite verb

in a main clause is given in Figure 1.7.

The paths in a node definition represent descriptions of grammatical

constraints monitored by phrase descriptors. The matching algorithm of

32. The andysis makes minimal, although essential, use of the multiple inheritance capabilities of DATR.

Demgned Intelligence: A Language Teacher Mode1 38

<main,clause position-mainclause finite errer> ==

Cposmainclfin' '3' ' m e ' 'The verb in the main clause is not in the correct position.'

'posmainclfin' '2"true1 'The finite verb in the main clause is not in the correct position.'

'posmainclfin' 'S 'true' 'The finite verb in the main clause is not in the correct

position. I t has to be the second element of the sentence.')

<main-clause positionmaincIatuse finite correct> == ('posmaincEnl ' 1' 'false' "

'posmainclfin' '1' 'fdse' "

'posmainclfin' '1' 'fdse' ")

<main-clause position,mainclause finite absent> = (").

Figure 1.E DATR Code Listing for a Finite Verb in a Main Clause

DATR selects the longest path which matches left to right. Each path in a

node is associated with atoms on the right of '=='. There are three learner

levels considered: expert, intermediate, and novice each of which has four

a t o m ~ . ~ ~ The first atom in each set refers to a gmmmar node, followed by an

increment, a Boolean value, and a message?

For example, the node gosrnainclfin, is responsible for monitoring

finite verb position in a main clause. If there has been an error identified by

the parser, the parse will generate the phrase descriptor [mainainclause

[positi~n~mahclause [fùiite errer]]]. This will match the path anain_clause

position-mainclause h i t e errorx This clause will concatenate the name of a

d o t in the Student Model pusnainclfin and its i n c r e ~ n e n t ~ ~ with a message.

The message provides the student feedback, and the remahhg information is

used to adjust the Student Model.

33. A phrase descriptor tha t contains the value correct indicates that the grammatical constra.int has been met in which case no feedback message is specified. A phrase descriptor that contains the value absent is ignored by the system. Thus the Est to the right of '==' is empw- 34. The Analysis Module will be explained explicitly in Chapter 4. 35. The Boolean values indicate whether it is an increment or decrement: true for increment, false for decrement.

Designed Inteiligezcs: A Language Teacher Mode1 39

If the phrase descriptor is [mainclause [position-mainclause [n i te

correct]]] and thus there has been no error, no message is associated with the

name of the slot in the student model and its demement. The information,

however, st i l l contributes a Student Model update in order to record the

success.

Finallx if the grammatical phenornenon was not present in the

student input, the phrase descriptor [main-clause [position-maindause [finite

absent]]] is generated. However, the phrase descriptor is uninstantiated and

thus no information is associated.

The &al result of the Analysis Module is a student model update and

a list of possible learner responses fiom a coarse t o fine grain size. The

information is further processed by the Student Model.

1.5.4 The Student Model

The Student Model keeps track of the learner history and provides

l e m e r model updates. There are three learner levels considered in the

system: novice, intermediate leamer, and expert. The Student Model passes

instructional feedback suited t o learner expertise to the Filtering Module.

The Student Model consists of 79 grammar constraints which are

equally distributed among main, subordinate and coordinate clauses.35 The

distinction between the three clause types is necessary for two reasons. First,

if not kept distinct the system would overwrite the descriptors instantiated by

one clause type once it parses the next. Second, in some instances, there are

35. Each clause contains 25 m m a r constraints per clause. In addition, there are two grammar constraints monitoring noun phrases practiced in isolation, one for verb-initial position, and one for the entire sentence. A complete Est of the grammar constraints is provided in the Appendix.


difXerent grammatical rules which apply to main and coordinate as opposed to

subordinate clauses. For example, in a subordinate clause in German the

finite verb appears in final position while in a main clause it is always the

second constituent. To capture this distinction and to provide error-contingent

feedback suited to learner expertise the system needs to i d e n m in what kind

of clause the constraints were met o r violated.

- Each of the grammar constraintç represents a grammatical

phenornenon; the phrase descriptors report which constraints have been met

o r not. Each student starts out at the intermediate level. Once a student uses

the system, the Student Model adjusts the counter of each grammar node

accordingly. Setting the initial level to intermediate is arbitrary but

reasonable, and the model will quickly adapt for various profiles.

On the basis of the dynamic leamer model, the system decides on the

granularity of the instructional feedback. The central idea is that an expert

requires less detailed feedback than a beginner leamer Fischer & Mandl

19881. For each gramma. node, the Student Model checks which level the

student has achieved and sends the appropriate instructional feedback to the

F'iltering Module.

1.5.5 The Filtering Module

The 6nal component of the design is the Filtering Module which

decides on the order of the inçtmctional feedback displayed t o the learner. In a

language teaching situation a student might make more than one error mithin

an exercise. In a typical student-teacher interaction, however, the language

instructor does n o t overwhelm the student with multiple error reports.

Instead, a language instructor reports one error at a time by considering the

salience of an error. This pedagogy has been transferred t o the analyçis of this


dissertation. The grammar constraints produced by the phrase descriptors are

hierarchically organized. An E r o r Priority Queue determines the order in

which the instructional feedback is displayed by considering the importance of

an error in a given exertise and the dependency between syntactically higher

and lower constituents. Once the student makes the required correction, the

whole evaluation process repeats.

- C~ntingent~errors , a class of multiple errors which presents a

particular problem for existing systems, are taken i n t o consideration. For

instance, example (16a) illustrates that denken subcategorizes for the

preposition an, while von is a dative preposition requiring the dative pronoun

dir. Ignoring the dependency of errors, the feedback to the student would be

Titis is the wrong preposition and This is the wrong case of the pronoun. The

feedback is correct with regards to von, however, the pronoun dich is not

incorrect if the student correctly changes the preposition von to an. Depending

o n the order of the feedback, the student might end up changing von to an and

dich to dir and wind up utterly confused because the case of the pronoun had

been %agged as an error in the original sentence.

U6a) * Ich denke - von dich.

dat prep- acc pronoun

6) Ich denke - an - dich.

acc prep. acc pronoun

I am thinking of you.

The error in the pronoun is correctly ffagged by the system fkom a

purely logical point of view. However, f?om a pedagogical perspective reporting

the error in the pronoun is redundant and even misleading- In such instances,

only the error in the preposition is reported while the error in the pronoun is

recorded in the Student Model. Once recorded in the Student Model the

information can be used for assessing and remediating the student.


ILTSs aim at modelling the human tutor. Thek goal is to provide

error-contingent feedback and to allow for an individualization of the leaming

process. From a computational point of view, the challenging task lies in

pro cessigg iIl-formefi input.

The existing systems discussed all depend o n buggy and meta-rules,

and/or an alteration of the dcat ion-based algorithm itself. Even in the most

general systems, errors in word order need to be anticipated and thus cannot

be treated in a very general wap. The search for particular errors is also to a

large extent based on the native language of the student. Any system which

anticipates errors requires that the same error in the target language be

presented in each dinerent source language. The fewer source languages

considered the more user-limited the system. In addition, in a number of

systems the data and its implementation are interwoven complicating their

design, maintenance, and ultimately restricting their error coverage.

The strengths of the analysis of this dissertation are manifold: first,

the design contains a pedagogic component and thus it emdates a language

instructor by evaluating and responding to students' language input. The

result of the entire process is error-contingent feedback suited to learner

expertise. Second, the grammar is sufncient-y general that it treats

grammatical and ungrammatical input identically for the phenornena it is

designed to handle. This generality has the advantage of reducing the number

of rules required by the grammar. Third, the decoupling of the parsing system

from the analysis of whether or not the input is grammatical has the practical

advantage that development of each can proceed independen* Fourth, the


analysis described can successfùlly handle ambiguous and contingent errors,

a capability beyond that found in existing systems.

The modular design of the entire system also allows for easier

adaptation to other languages. Although the grammar itself is language-

dependent, the system does no t anticipate specinc errors and thus is not built

entirely fiom the perspective of a particular native language. The Licensing

Module, the Analysis Module, the Student Model, and the Filtering Module /

are langjuage-dependent only to a trivial degree. While the phrase descriptors

and thus the instructional feedback will Vary hom one language to another

the overd architecture of these modules can be applied to languages other

than Gennan.

Designed InteUigence: A Language Teacher Mode1 44

C H A P T E R 2 Analyzing Student Input

2.1 Introduction

The f i s t step in evaluating students' input is sentence analysis. In

ILTSs, this is handled by the parser and the grammar. The parser analyzes a

sentence according to linguistic information encoded in the grarnmar.

In Head-driven Phrase Structure Grammar (HPSG), linguistic

information is formally r-epresented as feature structures. Feature structures

spe- 'values for various attributes as partial descriptions of a hguistic

sign. For example, the feature structure aven in Figure 2.1 provides partial

information about the verb geht.

According t o Pollard & Sag [1994], each linguistic object is assumed t o

have at least the attributes Phonology (phon) and Syntax-Semantics

(synsem), and in the case of phrases, Daughters (dtrs). The phon value

Designed Intelligence: A ILanguage Teacher Mode1 45

1 phon < geht >

1 L

Figure 2.1: Partial Feature

[cat [head n]

content 1 index l

Structure forgeht

'1

specines the phonetic f o m of the sign; the svnsem value provides syntactic

and semantic information; and the dtrs value describes the constituents of

the phrase.

HPSG adopts a lexicalist approach in which syntactic information is

described within each lexical en- Subcategonzation, a specification by the

head for its argunent(s), is represented by the features (specïfier), subi - .

(subject), and comr>s (complements). For example, the feature stmcture given

in Figure 2.1 illustrates that the verb geht subcategorizes for a subject. The

subject is minirilally specified as a noun and its person and number features

are stnicture-shared with the agreement features of the verb. Structure-

sharing is indicated by multiple occurrences of a coindexing box labelhg the

single value. The comns Est, which refers to the number and kind of

complements the verb takes, is emp@ since geht is an intransitive verb.

Designed Intelligence: A Langnage Teacher Mode1 46

Because syntactic information is expressed within each lexical entry, HPSG

requires only a few generalized syntactic d e s to spec* how words and

phrases combine into larger units.

In unification-based grammars, grammatical phenornena are

modeled by feature structure matrices. The analysis in this dissertation has

adapted the syntax/semantics component of HPSG to the error analysis

student input twugh the concept of phrase descriptors. A phrase

descriptor is a mode1 of a grammatical phenomenon that records whether o r

not the grammatical phenomenon is present in the input and, if so, whether it

is correctly or incorrectly formed. In either case, detailed information is

collected, which can then be passed o n ta other program modules for further

analysis, and dtimately used to generate appropriate feedback.

A phrase descriptor is implemented as a fiame structure. Each

member of the frame consists of a feature followed by a value. For example,

subject-verb agreement in person is represented by the & m e [per, value]

where value represents an as yet uninstantiated vdue for person. If the

grammatical phenomenon is present in the studentys input, the value is either

correct or error depending o n whether the grammatical constraint has been

met or not, respectively. If the grammatical constraint is misskg, the feature

value is absent. To examine this concept, consider examples (la) and (b):

(la) *Er schlafst.

(b) Du schlafst.

You are sleeping-

The phrase descriptor for subject-verb agreement in person in (la)

would be [per, errorl, while that for the phrase in (b) wodd be [per, correctl. In

either instance, the error o r success will be recorded in the Student Model.

Presented with (la), however, the system will additionally inform the learner

of the error.


A grammar conçists of a lexïcon and syntactic rules. For an ILTS,

there are thus two places for recording whether grammatical constraints have

been met: within the leicon or within the grammar rules. The difference lies

in the suitabdi@ of each location in application to a particular type of error.

For instance, under the design I am describing, errors are grouped into two

main classes: errors in feature matching and errors in linear precedence.'

Errors in feature matching are treated within the lexical entries; errors in

h e & precedence a& addressed in the grRmmar rules.

Most of the errors made by introductory students of ~ e r m a n ~ can be

analyzed by feature rnatching. Examples of such errors are gender, number

and/or case mismatch between two or more constituents. These are found in

noun phrase concord, o r in the agreement between a preposition and its NP

argument, for example. In addition t o violation of agreement of syntactic

featues, a violation of agreement of semantic features also f d s i n t o this

class.

Errors in word order are identified withjn the grammar rules. One

conçtra.int o n word order in German is verb position. For instance, there are

three possible positions for finite verbs: initial, second, and last. Examples (2)

and (3) illustrate that the f i t e verb occurs in initial position in both

questions and imperatives. It appears in h d position in embedded clauses

(example (4)) and in all-other instances it is the second constituent of the

sentence (examples (5),(6),(7),(8)).

(2) Sitzt Klaus im Garten?

Is Klaus sitting in the garden?

1. This classification is language-independent. 2. Studies [Nemark 1976, Rogers 1984, J u o d y n a s 19941 have identified agreement phenornena as a major source of errors for students of German.


(3) G& in den Garten!

Go into the garden!

(4) Ich weiss, dass Klaus im Garten m. I know that &?am is sitting in the gariien-

(5) Wem g& du das Buch?

To whom are you giuing the book?

(6) Hlaus && im Garten. . -

Kluug is sitting in the garden.

(7) Heute sitzt Klaus im Garten.

irbday maus is sitting in the garden-

(8) Dass maus irn Garten sitzt, weiss ich.

That =arts is sitting in the garden, 1 know.

For nonfinite verbs, there are t w o possible positions: the nodinite

verb occurs in &al position in main clauses (example (9)) or second t o last in

embedded clauses (example (IO)), the last element being the h i t e verb

(example (10)).

(9) Khus hat im Garten -. dSaus mas sitting in the garden-

(10) Ich weiss, dass Klaus im Garfxn m- hat.

1 know t h t maus was sitting in the garden.

The following discussion will focus o n the design underlyhg the

generation of phrase descriptors for errors in feature matching and errors in

linear precedence. Unlike other ILTSs, the analysis presented does n o t

anticipate errors, a problem discussed in Chapter 1. Instead the parser -. analyzes a sentence by comparing input against the knowledge of the

language spedied in the grammar. A phrase descriptor records a grammatical

phenornenon present in the input by describing its contextual occurrence.


2.2 Errors in Feature Matching

Errors in feature matching are handled within the lexicon by

adjus ting how grammatical phenornena are represented in lexical entries. The

following section will explain the approach for recording student performance

on eriors in feature-matchhg. The algorithm will be illustrated with subject-

verb agreement in person.

2.2.1 General Technique for Errors in Feature Matching

To illustrate the approach of errors in feature matching, the

discussion in this section will focus on the two sentences given in example

(lla) and (b):

(lla) *Er g&&.

(b) D u m .

You are leauing.

The sentence in (lla) contains a mistake in person agreement: er is a

3rd person, singular, nominative pronoun while gehst is a 2nd person,

singular verb.

A grammar is written as a set of constraints. In example (lla), the

constraints between the subject er and the verb gehst require agreement in

number, person, and case. Any of the three constraints could block parsing if

not successfully met. Figure 2.z3 illustrates that the subject of the verb gehst

must be a 2nd person, singular, nominative, noun. Er, however, given in

3. The lexical enties illustrated in this chapter contain only partial feature specifications for breviw


cat .

head v

local

- phon < geh

Reln gehst 'ont"t[Geher 1

synsem

-.

f

1

-.

cat [head ~ m s e nom.]

content [index

d

2

d

-

1

Fignre 22: Lexical Entry for gehsf

Figure 2.3, is intlected for a person, singular, nominative. Thus *Er gehst

local cat [head ,[case nom]] 7 p&{

Figure 2.3: Lexical Entry for er

A constraint can be relaxed, however, by changing its structure so

that, rather than enforcing agreement between two constituents. it instead


records whether or not agreement is present. To achieve this, the subject er,

given in Figure 2.4, is no longer marked as [per 3rd1, [num &, [case nom].

Instead the feature gt% for example, specifies all possible person features,

that is, 1st. 2nd. and 3rd. For er, the value for and 2nd is error, while for

3rd it is correct. Conceptually, the feature structure States that it is incorrect

to use er as a 1st and 2nd person pronoun, but correct for 3rd person.

cat ihead n]

Figure 2.4: Marking Person Features for er

The verb gehst, given in Figure 2.5, no longer subcategorizes for a

subject marked [per 2ndl. Instead, the verb gehst will inherit the value of 2nd

fiom its subject during parsing. For the subject er, the value for 2nd is error

indicating that the constraint on person agreement has been violated, but,

importantly, allowing the parse to succeed.

For the correct sentence D u gehst, given in (b), the value for 2nd

would be correct because du is a second person, nominative pronoun, as

iIlustrated in Figure 2.6. During parsing, gehst will inherit the value correct

indicating that the constraint on person agreement has been met.

In addition t o the HPSG features, the grammar uses a feature

descrintor representing the description of the phrase that the parser builds ~~~~~ -


phon < gehst >

head v

subj synsem local

content index per 2nd 2 [ [ q R e h gehst

lGeher

lescriptor

Figure 2.5: Marking Person Features for gehst

synsem local

cat [head n]

content index I

Figure 2.6: Markïng Person Features for du

Designed Intelligence: A h g u a g e Teacher Mode1 53

up. During parsing, the values of the features of descri~tor becorne ~ ~ e c i n e d . ~

For example, the phrase descriptor v p q e r records the constraint on person

agreement. For the sentence *Er gehst, v p g e r will inherit its value f?om the

feature 2nd, given in the lexical entry ofgehst in Figure 2.5. The nnal result is

the phrase descriptor [mainclause [vpger [Znd errorl]], indicating that the

required grammatical constraint on person agreement has not been met.

- The responpibility of the head of a phrase is t o collect the phrase

descriptors of its ~orn~lernents .~ In the implementation in this dissertation,

the phrase descriptors are percolated up the syntactic tree via the Deschptor

Prtnciple. This principle states that the descriptor features of the mother node

are the descriptor features of the head daughter.

The advantages of the method described for analyzing constraints in

featue matching are:

First, the technique is very general in that it can be applied to any

agreement phenomenon. The phrase descriptors record whether a

grammatical phenomenon is present in the input and, if so, whether it is

correctly o r incorrectly formed.

Second, errors are not treated differently from well-formed input.

Thus, errors in feature matching need not be anticipated. Neither the parsing

nor the grammar formalism needs to be altered and no additional d e s are

required.

4. takes three features: main cocoordinate c h , and sub clause. These record the constraints within the three clause types. Each of these in turn, contains a number of features. 5. This knowledge must be specified in the lexical entries as opposed to the grammar d e s . For errors in word order, for exampie, a head inherits phrase descriptors from its complement which in many caseç are not instantiated until afker the parse. To obtain the instantiated values, the lexical entries define which phrase descriptors a head d l inherit fkom its complement.


Third, the method used tc idenfi@ errors in feature matching can also

handle ambiguous errors, and therefore represents an improvement over the

existing techniques. The method closely modeIs the reasoning a language

instructor employs in disarnbiguating enors. Ambiguous errors will be

discussed in the following section.

2.2.2 Ambiguous Errors

Ambiguous errors are distinct f?om ambiguous readings, discussed in

Chapter 1. Ambiguous readings are due to stnictural ambiguity in the

language and manifest themselves as multiple parses. For these, the task is to

select the appropriate parse among many parses. Ambiguous errors, however,

do not necessarily result in multiple parses. They are ambiguous with regard

to the source of the error. The task here is to iden* the most likely error so

that appropriate feedback c m be generated.

Ambiguous errors are those whose source must be identified f?om a

range of possibilities. An intelligent system must emulate the techniques a

language instructor employs in disambiguating errors. The rationale a

language teacher uses is based on the Lkelihood of an er ror as determined by

the student level, previous performance history, and the frequency and

dinicuky of a particular grammatical construction [see Rogers 19841.

The following discussion will provide examples illustrating

ambiguous errors. The example given in section 2.2.2.1 will show an instance

in German where a parser might flag o n l y one of t w o possible sources of an

error. In such cases, however, a language instmctor will typically address both

reasons for the error so as to not misguide the student. Section 2.2.2.2 will

give an example where a parser might flag a n unlikely error resulting in

feedback which is pedagogically unwarranted. In section 2.2.3, 1 will show

Designed Intelligence: A tanguage Teacher Mode1 55

how the method for errors in feature rnatching can emulate a language

instructor in analJrzing both kinds of ambiguous errors. The method employed

provides a more general analysis than existing systems.

2.2.2.1 Two Likely Sources of an Error

Ambiguous errors have more than one possible reason for the error.

There are instances, however, where the feedback will be misleadhg if the j-

syçtem reports only one of the potential errors. For example, in sentence (IZa),

given by Holland [1994], the student made a mistake with the determiner die

of the prepositional phrase. The source of the error is indeed ambiguous: it is

either an error in gender, but far more Iikely, an error in genekr and case.

(12a) * Wm stehen auf Berg.

pl / fem / mm, acc singular / masc / nom, acc, dat

(b) Wir stehen a u f h Berg.

We are standing on the rno~ntain.~

Die and Berg agree in the nominative and accusative case. Berg,

however, is singular masculine, while die occurs only with singular, ferninine

nouns. According to Holland [1994] it is likely that the student made a

mistake in gender as weU as case, because the determiner die is not a dative

determiner; yet dative is required by a state verb (stehen) with auf. In CALL

systems described by HoIland [1994], however, only an error in gender is

reported. The reason is that the noun Berg carnes the case feature of dative

which is projected up the tree, regardless of the features of the determiner.

Because dative matches the requirements of auf with stehen, no error is

flagged beyond the determiner-noun level.


Reporting only the emr in gender, however, is pedagogicaLIy unwise.

It would UeIy cause the student to change die to de< which is still incorrect

since, although the determiner der is now the correct gender for the noun

Berg, the determiner is not infiected for the required dative case?

Example (12a) illustrates a . instance representing two underlying

emo&. m e a pmer will typically flag only one source of the error, the

teacheis feedback,,however, would be The determiner "die" iç incorrectly

inflecteed for gender and case, addressing both sources of the error. Such

feedback is not misleading and supplies the student wi th the appropriate

information to correct the error. An intelligent system should possess t h i s

pedag0gi~a.U~ important awareness.

2.2.2.2 An Unlikely Source of an Error

Unlike ambiguous errors where both sources of the error should be

reported, there are errors which may be classified as trivially ambiguous:

given fmo possible sources of an error, one is potentially so misleading o r so

unlikely that a language teacher would easily dismiss it.

(13a) * Der Gotter zürnen.

(b) Die G6tter ziifnen.

The gods are angryry8

Example (13a) containç an ambiguous error with the noun phrase.

Either the student made a mistake in number agreement between the

determiner and the noun since der is singuiar and Gotter is plural, or s/he did

not aççign the correct case to the noun phrase. Der G6tter could be a genitive

plural noun phrase; however, as a subject of the sentence, the expected case is

nominative. But as Schwind Cl9951 states "F]t is vëry unlikely that a student

should to cons*& a genitive plural, which is a 'difEcult' case, when the

nominative is required." Thus a language instructor's feedback would be The

deteminer "der" is incorrectly inflected for number. However, the parçer will

not flag an error in number since der Gotter is correctly i d e c t e d for number

in the genitive. ARer the verb and the noun phrase have combined, the parser

will detect a case error and that is the error that wïü be reported.

Holland [1994] and Schwind [1995] propose a solution whereby lower

level errors are ad&essed f i r ~ t . ~ These are errors inside a noun phrase: errors 3

in gender and number between determiner and noun, for example. Schwind's

[1995] argument is that once these errors are reported t o the student, the

higher level errors, such as case concord, become more salient and can be

dedt with.

7. Rogers [1984:71 provides examples of similar ambiguity caused by adjectives. In these instances, however, it is difficult for even a human

indefmite articles and kmguage instructor to

pinpoint the exact source of an error. In the absence of additional data, theteacher wodd typically address both sources of the error. Consider examples 1-2.

(la) * Ich *de Ihnen & Urlaub empfehlen. (b) Ich -de Ihnen einen Urlaub empfehlen,

I wodd recommend a holiday for you. (2a) * offentliche Interesse (b) offentTic)ies Interesse BUT &s o f f a Interesse

public Utterest In example (la), the error in the indefinite article is due either to wrong case or wronggender; ein would be appropriate for either nominativdmasculine, neuter or accusative/neuter. Either the student incorrectly assumed that Urlaub is a neuter noun and thus chose the correct case ùiAection or the student knows that Udaub is a masculine noun but did not apply the required accusative case. In example (2a), the emor is due either to wrong gender or wrong adjective inflection. It is likely that the student incorrectly assumed that Interesse is a feminine noun since most of the nouns sufiked with -e are feminine in German. But it is also possible that the student is aware that Interesse is a neuter nom, but did not apply the correct adjective infiection for an urzpreceded adjective. The adjective chosen in example (2a) shows the correct inflection if preceded by a deh i t e article. This particular kind of amtiiguity, given in example (2211, only occurs with w r e c u adjectives since in the absence of an article the student's gender assignment to the noun is not apparent. It is therefore ambiguous whether the student applied the wronggender o r chose the wrong adjective inflectwn. In instances of modified NPs nreceded by an article, it is the article which provides the clues as to whether the student applied the wrong gender or case to the NP or whether s/he is not aware of the correct adjective paradigm.


Addressing lower level errors first is usefd in the sense that it is a

very general d e which can be easily implemented in an ILTS. However, this

approach makes an assumption about the student's knowledge of German

grammar - that with all ambiguous errors the student is more Iikely t o have

learned case and adjective inflections than gender or number. The supposition

is unwarranted, as well as pedagogically unsound. For example, Juozulynaç

Cl9941 who analyzed an approximately 400 page corpus of German essays by

American college $tudents in second-year German courses found that

students made more case errors than errorç in number. For this reason, a

system which i d e n s e s al1 ambiguous emors as lower level errors is in many

instances likely to misrepresent the actual source of the error.

Addressing lower level enors first would also not solve ambiguou

errors where the feedback should address both sources of errors, as required

for example (12a) described previously

A M e r shortcoming of the technique used by Holland Cl9943 and

Schwind Cl9951 is that addressing lower level errors nrst is not desirable for

contingent errors. Contingent errors require a focus on higher level errors.

They will be explained in detail in Chapter 4.

The analysis described for errors in feature matching can favourably

respond to ambiguous errors due to a number of techniques which mill be

discussed in the following sections.

8, She cites m e r examples which require basically the same analysis as given for examples (12a). See Schwind 119953, p- 314. 9. Schwind [1995] refers to the technique as Case Filtering.


2.2.3 Analyzing Noun Phrases

The analysis presented here makes no distinction between lower and

higher level information. Instead, case, number, and gender agreement of the

determiner are analyzed a t the syntactic level of the verb phrase: the case

feature of a noun and the three features of its determiner, case, gender, and

number are all percolated t o the syntactic level of the verb phrase. To examine

the concept, considei the tree structure given in Figure 2.7.

NP- case

v number + gender

Det- N number, gender

Figure 2.1: Case, Number, and Gender Features

Figure 2.7 illustrates that in Gerrcan, the verb assigns grammatical

case to its çubject, that is, the noun and the determiner. The noun determines

the gender and number idiection of its determiner. As a result, there are a

number of possible errors with noun phrase constituents at distinct syntactic

levels :


1. the noun can show an error in case, and

2. the determiner can be incorrectly inflected

for case, nunber, or gender.

The three possible errors of the determiner, however, are intertwined.

Consider again the example Qted earlier by Schwind [1995].

(14a) * Der Gotter ziïmen. - - 1-

(b) Die Gotter ziirnen.

The gods are angry.

The problem with sentences like (14a) is that if the number and

gender agreement of the determiner are analyzed ùidependently of its case,

the feedback might not be accurate For instance, the gender and number

agreement of der Gotter is correct at the noun phrase level because both der

and &tter are possible constituents in the genitive plural. Only after the verb

and the subject have coinbined will an error in case be ffagged -- because the

subject requires nominative case. Yet a language instructor's feedback would

be l%e deteminer "der" is incorrectly inflected for number. The parser,

however, cannot ff ag an error in =ber because number agreement is correct

at the syntactic level of the noun phrase.

ln the technique described thus far for errors in feature matcbing, the

verb analyzes the determiner for case, number, and gender. This method,

however, requi-es that the gender and number of the noun be apparent. In

addition t o percolating all features of the determiner to the syntactic level of

the verb phrase, the phrase descriptor that records the agreement of the

determiner marks exadly which particular determiner was involved in a

given noun phrase. To achieve this, determiners use a fber-grained feature

value distinction: deremor, for example, as opposed to simply grror. This h e r

distinction d o w s the designer to fully reconstruct the error context. The - - - - - - - -

Designed Intelligence: A ILangaage Teacher Model 61

following section will dismss the analysis of errors in feature rnatcbing as

applied to the sentence *Der Gotter ziinen.

2.2.3.1 Feature Percohtion

To account for ambiguous errors, the analysis of errors in feature

matching requires that a determiner be specified for a l I possible case, number,

and gender featurer For example, the lexical entry of the determiner der is

given in Figure 2.8. The features nom, dat, and g m

~hon < der >

represent case

synsem Local head

[masc corred neut dererror

nom l fem dererror plu dererror

neut dererror fem dererror

lPlu dererror ]

dat

neut dererror fem correct

Figure 2.8: Lexical Entry for der

agreement between the determiner and the verb. The three features masc,


neut. fan correlate to the three genders of a noun in the singular; the feature

& represents the agreement between a determiner and a plural noun. The

determiner specifies its value for each possible occurrence of case, number,

and gender. For instance, the value for nominative/ plural is dererror because

the nominative, plural determiner is die and not der.'' For the genitive plural,

it is correct.

- For the se)ence *Der Gotter zürnen, the n o u n Gotter determines the

gender and number of the determiner. Gotter is a plural noun. For this reason,

the nom wi l l inherit the value for the feature pLia for aU. grammatical cases

fkom the determiner, given in Figure 2.9. For clarity, the values of the

determiner have been instantiated for the phrase &r Gotter in Figure 2.10.

In addition to the three agreement constraints of a determiner, case

agreement between the noun and the verb also needs t o be analyzed. The

feature noun of Gotter, given in Figure 2.9, specifies a value for each possible

grammatical case of the n o m . For example, for nominative and genitive the

value is correct since the noun is correctly inflected for these two grammatical

cases. For dative, however, it is error- Gotter is not a dative plural noun.

For its subject, the verb zürnen, given in Figure 2.11 collects values

for the nominative case. The verb will inher i t the values for the gender and

number features mas& neut> fem, and &, respedively, from the noun. In addition, zZirnen will also-inherit the value for the feature noun representing

the case of the noun itself. Finallx the leAcal entry of ziirnen further s p e s e s

that its phrase descriptors will inherit ail values fkom the agreement featwes

of its subject.

10. Determiners can take a number of values: b e r r m , dererror. b e r r o t and m. For example, errors specified for the debite article der aU take the

value dererror, illustrated in the lexical entry of der in Figure 2.8. The error types correspond to the phonetically distinct definite articles in German.

- - - - - - - -. . . -


?bon < Gotter >

head ;--

n

case

I

synsem

lplu O nom noun correct

acc noun correct 1

noun correct 1

local head case

nom [plu O]

dat rPlu O]

Figure 2.9: Lexical Entry for Gotter

m e r parsing the sentence Der Gotter ziirnen, the sentence coIlected

the five phrase descriptors, given in Figure 2.12.

Note, however, that the values for the features rnasc, neut, and fem

carry the value absent since these constraints were not present in the

student's input: Gotter is a plural noun. As a result, the features masc, neut,

and fem did not get instantiated during parsing and are thus ignored by the

system. But the two phrase descriptors [mainaindause [vp-subj [plu deremorIl]

and [rnainainc1ause Evp_subj [noun -11 recorded the number, gender, and


phon < der Gotter >

head

n

case plu O

[no, ,,O,)

gen noun correct

Local head case ! nom [plu dererror]

acc lplu dererror 1 dat [plu dererror]

gen [*lu correct]

Figure 2.10: Lexical Entry for der G6tfer

case agreement of the'determiner and the case concord of the noun,

respectively As a result, the agreement featues of the determiner and the

noun were percolated t o the verb phrase.

While feature percolation plays a fundamental role in the treatment

of ambiguous enors, there are two additional techniques which assist in

obtaining the desired feedback. They will be discussed in the following section.


phon < zürnen >

head u

local rat aead

n

-

case nom

-

Figure 2.11: Lexical Entry forziirnen

1 phon < der Gotter zürnen > cat [head i

descriptor

-

main-clause

masc [~i' neut fern plu I- noun n

masc absent1 11 neut absent fem absent plu dererror noun correct 1

Figure 2.12: Phrase Descriptors forDer Gotter zümen


2.2.3.2 Phrase Descriptors and Feature Values

In analyzing students' input, the grammar itself does not encode the

precise source of an error. Lexical entries enumerate the possible constraint

violations, and then a phrase descriptor notes the constraint and provides the

grammatical context in which the construction occurred. As a result, the

grammar and pedagogy are kept separate. The designer can interpret the

source of the error on the basis of the error context provided by the phrase

de~c&~to r . In the fchowing example, two different sentences have generated

an error with a feminine noun but they result in distinct feedback.

(15) [main-clause lvp-subj Wern dererrorlll

(16) [main-clause [vp-dirobj [fem dererrorlJJ

In (151, the student made a mistake with a femiriine noun as the

subject. Phrase descriptor (16), however, shows an error with a feminine noun

as the direct object. In both sentences, however, the determiner der has been

used. This knowledge is apparent because determiners use a finer-grained

feature value distinction, dererroz dieerror, et al., as opposed t o simply error.

This h e r distinction allows the designer to fully reconstruct the error context

and thus encode the desired feedback. The phrase desrriptor given in (15) will

yield the feedback 2 h r e is a rnistake in gender with the deteminer "der" of the

subject because der is a possible determiner in the nominative for a masculine

but n o t a feminine noun. In contrast, in (16), the student made an error in

case; der is correct for a feminine n o m in the dative, but not the accusative as

required by a direct object. F o r such phrase descriptor, the system will

generate the feedback mere is a mistake in case with the determiner "derJ' of

the direct object.

The analysis for errors in feature matching also solves the ambiguous

error in the example *Der Gotter ziirnen provided earlier. Given the phrase

descriptor [mainclause [vp-subj [plu dererrorll], the designer can condude


that the student used der for a plural subject noun in a main clause. In such

occurrences, the feedback will dways be The determiner "der" is incorrectly

znflected for nurnber, which is the appropriate response:

F o r ambiguous errors where the student made two mistakes, both

errors should be reported. For the sentence *Wir stehen auf die Berg provided

earlier, the parser generates the phrase descnptor [main-clause (ppdat

[masc dieerrorll]. The fact that the student used the dennite article die for a

r n a s c ~ e noun in a dative prepositional phrase is apparent. In such cases,

the feedback will always be D i e determiner "die" is incorrectly infiected for

gender and case which is the desired feedback as discussed in section 2.2.2.1.

Because the approach percolates al1 features of the determiner and

the noun to the syntactic level of the verb phrase, and separates pedagogy

fkom the grammar, it is possible to encode precisely the feedback a teacher

would provide for an error in a particular context. The algorithm described

provides the information required as part of the conceptual design of phrase

descriptors. In a modular system, it becomes a simple matter to alter feedback

according to the pedagogy of the designer. The grammar does not need to

make any assumptions o n which grammatical phenornenon a student is more

likely t o Irnow, a shortcoming of HoI.Iandys Cl9941 and Schwind's [1995]

systems, nor will the sfxdent ohtain any misleadhg feedback. In addition,

such analysis does not impose any restrictions o n other classes of errors, such

as contingent errors.

In the following section, section 2.3, I will discuss the approach for

errors in word order. 1 will introduce two algorithms for creating phrase

descriptors that record verb-second and verb-ha1 position, which is of

particular interest for German. The approach chosen, however, can be applied

to other word order constraints found in other languages."


2.3 Errors in Linear Precedence

Existing systems anticipate errors in word order on the basis of the

native language of the student. If not anticipated, buggy rules cannot ident*

them. For example, for the sentence given in (17a), EngIiçh learners of

Germ-an wi l l tend to place the past participle in the position which resembles

English word order, that is, after the auxiliary. A buggy rule would anticipate

precisely this error by describing a stnicture where the past participle

inconectly occurs in the position derived from the English nom, successfUy

parsing the ungrammatical sentence. However, such a grammar could not

analyze the sentence given in ( c ) where the verb is in a different, but also

incorrect position.

(17a) *Ich habe & ihrn das Buch-

(b) Ich habe ihm das Buch gejg&m.

I have giuen him the book.

(c) *Ich habe ihm das Buch

By contrast, the following approach does not anticipate errors in word

order. Instead the system records where the verb occurs and thus is capable of

analyzing either enor, given in (17a) and (c).

The following sections will focus on the analyses of verb-second and

verb-ka1 position. In main clauses in German, unlike many other European

languages, the finite verb is always the second constituent of a sentence, an

damiiliar concept for most learners. Thus, this grammatical constraint

II. A word order constraint of French, for example, is the position of adjectives. Unlike English and German, the adjective follows the noun in French. The same approach as illustrated for recording verb position in Geman can be applied.

- -

Designed Intelligence: A hnguage Teacher Mode1 69

presents dif5nilties. Section 2.3.1 will describe the general technique for

analyzing verb-second position.

2.3.1 General Technique for Verb-Second Position

In German, the &te verb can be followed o r preceded by the subject,

o r can even be preceded by a subordinate clause. In example (18), the subject

precedes the verb a e in example (191, due to the t h e adverbial the subject

follows the verb. In example (20) the verb is preceded by a subordinate clause.

In ail instances, the verb is the second constituent of the sentence.

(18) Er peht. He is leaoing.

(19) Morgen & er. Tomorrom he is leavirtg.

(20) Dass er geht, & ich.

%t he is Zeauing, I know.

Examples (18) - (21) reveal why the position of a finite verb in German

cannot be analyzed by simply relying on a particular grammar rule to assign

the value. At the t ime of each rule application it cannot be determined

whether the verb is indeed the second constituent of the sentence. For

example, for both the sentences given in (18) and (211, the subject er will

combine with the verb geht. Yet after the sentence has been parsed, the f i t e

verb in (18) is in the correct position, while in (21), it is in the wrong position

due to the time adverbial preceding the verb phrase.

Examples (18) - (20) have a common structure although they are

formed by different d e s . The syntactic tree, given in Figure 2.13 illustrates

that the finite verb in German always occurs as the second constituent in a

sentence regardless of t he phrasal type. Under this analysis, the grammar

Designed IntelLigence: A Language Teacher Mode1 70

Verb Y

Figure 2.13: Partial Structure of Finite Verb Position

spec&es a feature representing the position of a verb. All words except finite

verbs ignore the value of the position feature. The grammar takes the second

constituent of a sentence and assigns the value correct t o the position feature

of its first constituent and the value erro to the position feature of all other

constituents. The method ensures tha t the position value correct is always put

in second position while for a l l other positions, the value is error, given in

Rgure 2.14.

correct error

Figure 2.14:Feature Values for Verb-Second Position

The phrase descriptor that records verb-second position will inherit

its value fkom the position feature of the verb. The final result is a phrase

descriptor indicating whether the constraint on verb-second position has been

met. Section 2.3.2 will illustrate the concept in detail.

Designed Intelligence: A Language Teacher Modei 71

2.3.2 Verb-Second Position

In analyzing a sentence, ail grarnmar rules build up a list for verb-

second position in the same way: the fkst value of the Est is assigned to the

l e b o s t constituent and the rest to the rightmost constituent. Each member

of the list represents a constituent in a sentence. Figure 2.15 (a) illustrates

the syntactic tree for verb-second position for the correct sentence Er geht.

< error, correct, ... >

NP VF' I

Er I

geht error correct

Figure 2.15: Percollation Pattern for Er geht

During parsing, the list < X, Y> is constructed where Y represents tbe

position value of the verb geht. Mer the sentence has been parsed, the

Assignqos rule instantiates the values of the list, given in Figure 2-16? the

first item of the list receives the value error, while the value for the second

constituent is correct, illustrated in Figure 2.15 (b). As a result, for the

sentence er geht, geht records the value correct for its position feature.

The grammar dso specifies that the position feature of the verb

geht is structure-shared with its appropriate phrase descriptor, given in the

partial lexical entry of the verb geht in Figure 2.17.

12. The number of terms of a list represents the longest possible sentence anticipated by the system. Each term corresponds to a constituent in a sentence. For brevity, there are only six terms given in Figure 2.16


cat

[ ~ y m ~ r n 10.. [cat [ h e d [il]]]]

Figure 2.16: Unary Assigngos Rule

phon c geht >

local

cat [head v]

descriptor rnaixl-c1auçe position-mainclause [finite m]] [

Figure 2.17: Phrase Descriptor for Verb-Second Position


The h a 1 result for the sentence Er geht is the phrase descriptor

[mainâU1clause [position-mainclause [finite correct111 indicating that the verb

geht is in the correct position.

Note, however, that this analysis will also parse sentences such as the

one given in example (22a).

(Ba) *Sie ihm hm.

(II) Sie hil& ihm.

In example (22a), the f i t e verb hilft is in the wrong position. It needs

to be the second constituent of the sentence. The analysis desrribed will parse

the sentence by the rule that combines a noun phrase with a following verb,

the Complement-Head d e , and the rule which combines a subject with a

following verb, the Subject-Head d e . The phrase descriptor which is

generated, however, will be [main-clause [position-mainclause [-nite errorll1.

To illustrate the analysis of a sentence with a verb in the wrong

position, consider the syntactic tree structures given in Figure 2.18. .

For the sentence, *Sie ihm hilff, the grammar rules wiU build up the

list c X, Y, Z > where Z represents the position value of the verb hi@. The

Assigngos d e will instantiate the values of the list as < error, correct, error,

... >, given in Figure 2.18 (b). As a result, the verb hi@ will record the value

error for its position feature. The position value is structure-shared with the

phrase descriptor that records the position value of a fhite verb. The final

result with respect to analyzing verb-second position of the sentence *Sie ihm

hile is the phrase descriptor [main-clause [position-mainclause [fuite errorll J

indicating that the constraint on verb-second position has not been met.


< error, correct, error, ... > S

I < correct, error > Sie

error

Y Z correct error

Figure 2.18: Percolation Pattern forSie ihm hilft

mThile the parser and grammar build up a Est for verb-second

position, verb-final position is analyzed in a diff'erent way. In the following

section, 1 will illustrate the approach for recording verb-final position.

2.3.3 General Technique for Verb-Final Position

In German, infinitives and past participIes always occur in &al

position in main clauses. Examples are given in (23) and (24), respectively

(23) Er kann pehm. He can l e a k

(24) Er hat sie getroffen.

He has met her:

Figure 2.19 shows the general structure of the position of a nodiaite

verb in German.


Figure 2.19: Partial Structure of Nonfinite Verb Position

The nonfinite verb is in the wrong position if it occurs as the lefcmod

constituent in a phrase. If the verb is the rightmost constituent, its position is

correct only if the verb is the final constituent of the sentence. It cannot be

determined, however, whether the verb is indeed in h a 1 position until the

entire sentence has been parsed. For this reason, the grammar rules spe@ a

position enor for the leftmost constituent of a phrase. The position value for

the rightmost constituent, however, is assigned by a final rule, one which

applies only &r the sentence has been parsed. Al1 words except nonfinite

verbs ignore the value of the position feafxre. Figure 2.20 illustrates the

position values for a nonfinite verb.

error correct

Figure 2.20: Feature Values for Verb-Final Position

In addition, the phrase descriptor that records the position value will

inhent its value fkom the position feature of the nonhi te verb. The h a 1

Designed Intelligence: A ILanguage Teacher Mode1 76

result is a phrase descnptor indicating whether the constraint o n verb-fkid

position has been met. The following section will provide examples of the

algorithm employed.

2.3.4 Verb-Final Position

To follow the analysis f o r verb-fmal posit ion for the correct sentence - Er kann gehen, consider the tree structure, given in Figure 2.21.

error Er A v VP

l kalm

I gehen

error Z

Figure 2.21: Percolation Pattern forEr kann gehen

Figure 2.21 reveals that the position value of the nonfinite verb gehen

is n o t assigned in the grammar d e s . Instead, the as yet uninstantiated

position value is percolated to the sentence level. After parsing the sentence,

the Assigngos ru le will assign the position value correct to the infinitive

gehen. The unary Assigngos rule is partially given in Figure 2.22.

The grarnmar dso specifies that the position feature of the nordbite

verb is stmcture-shared wi th the phrase descriptor that records verb-fkal

position, given in the lexical e n t q ofgehen in Figure 2.23.


-

synsem

phon c gehen >

synsem local

-

local

-

1 cat [head m] r

Figure 2.22: Unmy Assign-pos Eeule

cat [head v]

Figure 2.23: Phrase Descriptor for Verb-Final Position

The final result is the phrase descriptor [main-clause

[position-mainclause [idmitive correctlu indicating that the S t i v e is in

t h e correct position.


Note, that the grammar will also allow sentences to be parsed with a

nonfinite verb in the wrong position. Consider the example given in (25a).

(25a) *Sie kann M e n Klaus.

(b) Sie kann Klaus Belfea.

She can help Klaus.

In example (25a), the nonfinite verb is in the wrong position. The

infinitive, helfen, needs to be in sentence-final position. The grammar

descnbed will par& the sentence by the n i le that combines a verb with a

folloming noun o r verb phrase, the Head-Complement rule, and the rule which

combines a subject with a following verb, the Subject-Head nile. The phrase

descriptor which is generated, however, will be [mainclause

To illustrate the analysis of a sentence with a nodinite verb in the

wrong position, consider the tree structure, given in Figure 2.24.

I kann

error

I heKen I

Klaus error Z

Figure 2 . a Percolation Pattern for Sie kann helfen maus


Figure 2.24 shows that for the sentence *Sie kann helfen KZaus, the

position value of the infinitive helfen is error. The position value of helfen is

structure-shared wit the phrase descriptor that records the position value of

an infinitive. The final result is the phrase descriptor [mai-clause

[position-mainclause [infinitive error111 indicating that the infinitive is in the

wrong position.

- The two algorithms for errors in linear precedence provide a general

analysis for parsing students' ill-formed input. The potition of a finite o r

nonfinite verb need no t be anticipated. The technique for verb-final position

has also been applied to identifying verb positions in subordinate clauses in

German. PVhile the fkite verb in subordinate clauses appears clause-fùially-,

the nonfinite verb occurs second-to-last,

The main advantage of the techniques for errors in feature matching

and errors in linear precedence is that the grammar treats grammatical and

ungrammatical input identically. The grammar's main task is t o assign phrase

descriptors. As a result, the treatment of errors is very general: none of the

errors covered by these methods needs t o be anticipated, neither the parsing

nor the grammar formalism needs t o be altered and no additional rules are

required. For this reason, the system does no t s&er &om poor error coverage,

limîted user scope, o r lack of generality, problems associated with other

approaches discussed in Chapter 1.

While an argument might be made that there is an element of error

anticipation in fully s p e c m g each possible grammatical constraint in a - --

Designed Intelligence: A Language Teacher ~ ~ d e l

lexical en- of a sign, there is an important difference. The systems discussed

in Chapter 1 search for a specific student error. F o r example, they search for

the infinitive to the right of the auxiliary based o n English word order. For

verb positions which have not been anticipated, however, the sentence cannot

be analyzed. In contrast, the analysis here records whether the verb is in the

correct position and if not, will dways report an error no matter where the

verb occurs in a sentence. Thus, the main referent under this approach is the

Germa language as the target language rather than specific errors baçed on

the native language of the student.

The approach applied t o errors in feature matching can also

successfully address ambiguous errors, not generally handled by existing

ILTSs with the exception of Holland [1994] and Schwind [1995J Unlike these

systems, however, the approach described does not need to make any

assumptions about which grammatical phenornenon a student is more likely

to know, nor will it produce misleading feedback. It successfully emulates a

human tutor in evaluating and responding t o a student's performance by

implementing the following h o techniques:

First, the parser percolates al1 agreement features of the determiner

and the noun to the syntactic level of the verb phrase. There is no distinction

between lower and higher level errors required since the information a t both

syntactic levels is presemed.

Second, the analysis is modular in that it separates the grammar

from pedagogy: the language grammar is applied to parsing while the

language pedagogy is implemented in the feedback. As a result, the phrase

descriptors provide the error context and the designer decides on the desired

feedback by defining which error(s) shodd be addressed.


While the approach presented in this dissertation provides a general

treatment of errors in feature matching and linear precedence, an

unconstrained grammar increases ambiguity Ambiguous readings will be

discussed in the follo wing chapter.


C H A P T E R 3

Multiple Sentence Readings

Introduction

A standard problem in parsing is that most sentences can be assigned

more than one syntactic structure. For example, van Noord [1991 states that

the Alvey Tools Grammar with 780 rules averages about 100 readings per

sentence on sentences ranging in length between 13 and 30 words. The

maximlfl~~ was 2,736 readings for a 29 word sentence.

Although typiczh language exercises do not contain 29 word

sentences, an uncons trained grammar generally produces more parses than

systems designed for o n l y well-formed input. Consider a system for German

which relaxes the constraint on grammatical case. Without case marking, the

parser has no way of knowing if a constituent is a subject or a verb

complement. As a result, more than one sentence analysis w d l be produced

and the errors flagged in each sentence reading can vary For instance, for the


sentence given in (1) the parser can assign at least two legitimate syntactic

structures. They are illustrated in the two tree diagrams, given in Figure 3.1.

(1) Er fragt das Mâdchen.

S Subject-Head

Det N

das ~ i i d c h e n

He is asking the girl.

Sentence Structure (a)

*S Head-Subject

Complement-Head \ NP

Specifier-Head

Det N

das Miidchen

me girl is asking him. Sentence Structure (b)

Figure 3.1: Two Analyses forEr f i a . das Miidchen - -

In sentence structure (a), er is the subject of the sentence, das

Müdchen is the direct object. The parser and grammar will not flag an error.

The sentence is correct. In the second sentence analysis, however, the subject

and the direct object are reversed: das Müdchen is the subject and er is the

direct object. But in this sentence reading, an error is flagged for the direct

object since er must be ihn in the accusative case.

The two distinct sentence analyses arise £rom two, conjoint causes:


First, due to variable word order in German, even a grammar

designed to parse only well-formed input must s p e c . d e s capable of

parsing SV0 and OVS constructions. For instance, sentence structure (b) in

Figure 3.1 is ungrammatical because the object er should be ihn as a direct

object in the accusative case. However, the Complement-Head and Head-

Subject rules are nonetheless required for the correct sentence Ihn fragt das

Madchen-

Second, the feature case which disambiguates a subject and a verb

complement in a well-formed system, has been relaxed. Since the rules for the

alternative sentence analysis are in place, the grammar and the paner wiIl

produce the t w o syntactic structures given in Figure 3.1.

NLP applications frequently incorporate techniques for s e l e d g a

preferred parse. One method is to select the fkst parse produced by the

system. For an ILTS, however, this criterion is insufEcient because the b t

parse chosen might address an unlikely error, as in the case of sentence

reading (b) of example (1). As a result, the feedback might be confusing,

misleading, o r even incorrect. A pedagogic system should approximate the

students' intentions-

Weischedel et al [1978] implemented a technique whereby the

selected parse is based on the number of errors identined. m e r all parses

have been generated, the one with the fewest number of errors is chosen.

Such a technique, unfortunately, is not suitable for a pedagogic system for a

number of reasons which will be discussed in detail in section 3.2.

The prefemed method for disambiguating multiple sentence readings

more closely follows the approach of a Ianguage teacher who considers the

likelihood of an error. The likelihood of an error takes into account the level of


instruction and the frequency andlor difEculty of a grammatical construction.

The paramount concern is avoiding misleading or incorrect feedback.

In analyzing student input, there are two Ends of ambiguous

sentence readings which may result in misleading and/or incorrect feedback.

For the first kind, the parser produces sentence readings which are

unlikely considerigg the level of instruction and the diEculty of a

grammatical construction. For example, German exhibits a fairly fkee word

order with respect to subject and object(s) position. However, it is doubtful

that a beginner student of German would astign OVS word order over SV0 in

a main clause: object topicalization is a rare construction, even at a more

advanced level.

The second kind contains sentence readings which are potentially so

misleading that a language instmctor would easily dismiss them. For

instance, for the sentence given in (2a), although the parser might interpret

the verb gehen as an infinitive, a language instructor would clearly iden-

the sentence as containhg an error in number agreement between the subject

and the finite verb. The ambiguity arises because the 3rd plural finite form of

a verb in German is phonologically identical to its infinitive.

(2a) *Er gehen.

(b> Er geht.

He is leaoing

The two Ends of ambiguous readings are resolved by implementing

two techniques. One uses licensing conditions t o mark sentences for the rules

which constructed them and selects the appropriate sentence reading on the

basis of the Iikelihood of a grammatical construction. The other blocks parses

f?om being generated through ad hoc techniques. Both methods assist in

Designed Intelligence: A Eanguage Teacher Model 86

obtaiMg the sentence structure a student most likely intended and thus

result in the desired feedback.

The task of the licensing conditions is t o distinguish multiple

sentence readings with respect to word order. During parsing, three of the

syntactic d e s , the Subject-head, the Head-Subject-Complement, and the

Head-Subject d e each assign a distinct licensing descriptor. Any of the three

licensing conditions c m license a sentence. After parsing, the Licençing

Module prioritizes multiple sentence readings so that, in the event of a choice,

the parses are selected in a particular order. The chosen parse is passed o n to

the next module of the system for firther processing.

The order in which parses are selected c m , however, be customized.

The licensing conditions, for example, for an introductory course in German

specify that if a parse exists that contains SV0 word order, i t will be selected.

Failing to find any SV0 structure, the system will çearch for a VSO. Finallx if

no parse contains either SV0 or VSO, parses with VS word order will be

licensed.' The flexlbility of customizing the ranking of the licensing

conditions is highly desirable in a pedagogic system for the following reason.

No pedagogic system should dismiss grammatical conshctions

arbitrarily. It ideally will reflect the level of instruction and the focus of a

particular exercise. F o r example, object topicalization is not a grammatical

construction explicitly taught at the introductory level where the focus is on

1. Examples of inverted verb-subject constructions with a transitive and intransitive verb are given in (1) and (Z), respectively- Example (1) illustrates a verb-Nbjeet-cornpiement (VSO) consb-ùction, the matrix clause in example (2) shows verb-subject (VS) word order.

(1) Morgen sieh t er sie,

lbmorrow he'll see her.

(2) Dass er geht, weiss ich.

That he is leauing, I k rtow.

Designed Intelligence: A Langnage Teacher Mode1 87

the grammar of more commonly used rules of German. However, if an

instructor nonetheless wanted to implement topicalization, the system should

be fleeble enough to d o w it. The ranking of the licensing conditions can be

adjusted for topicalization exercises while the default setting reflects the

more general goal of a particular level of instruction.

In contrast to the Licensing conditions, the task of the ad hoc

techniques is to block parsing of sentence readings out of hand. These are

sentence readhgs which are so misleading that a language instructor would

not even consider them. For example, ad hoc techniques disambiguate lefical

entries which are phonologically identical, illustrated earlier with the verb

gehen in example (2a). While in a well-formed grammar, phonologically

identical forms are generally mutually exclusive due to their fùlly spedied

features, in an ILTS, these very same constraints might have been relaxed.

The ad hoc techniques block such sentence readings.

In sections 3.2 and 3.3 1 wi l l illustrate how the licensing conditions

and the ad hoc techniques successfidly disambiguate multiple sentence

readings.

Licensing Module

The task of the Licensing Module is to select the sentence reading a

shident most Likely intended, even given errorful input. No computational

method can always guarantee correct selection since the system has no direct

knowledge of what the student meant to express.2 For a pedagogical system,

2. While in some cases intentions can be clarified by a query to the student, less common sentence readings do not likely warrant the disruption.


however, the technique c a n dosely mode1 a language instructor by taking i n t o

account the likelihood of an error.

The following analysis implements three licensing conditions, given

in (3) - (5), t o disambiguate multiple sentence readings.

. . (4) [main-clause flicense-head-subjcomp truell .--

. (5) [ma&clause Clicense-head-subj truel13

The three Iicensing descriptors are assigned in their corresponding

grammar d e s : Subject-Head, Head-Subject-Complement and Head-Subject

rule, reflecting SVO, VSO, and VS word order, respectively ARer parsing, any

given sentence will contain one of the three licençing desc~ptors. Note,

however, that these three Iicensing conditions are mutually exclusive because

each d e combines the sub~ect with the rest of the sentence.

Any one of the three licensing descriptors can license a sentence The

conditions are prioritized in the system so that, in the event of a choice, the

Licensing Module selects the parses in the order presented in (3) - (5) The

chosen parse will be passed on to the next module in the system for further

processing.

3. The concept of licensing conditions can be applied to other grammatical phenornena. For example, the parser also poduces multiple sentence readings for sentences containing

.prepositional phrases. For instance, (lbl and (Ic), below, d lu t ra te h o sentence analyses of the clause given in exampIe (la):

(la) .,-, dass er in dem Garten Liest.

(b) ..-, dass [[er [[in dem Garten] liest]]].

(c) *..-, dass [[er in] [[dem Gartenl liestll].

.-., that he is reaùing in the garden. The sentence reading given in (b) is correct: er is the subject of the phrase, [in dem Gartenl is the prepositional phrase. In sentence structure (c), however, the subject is [dem Gartenl and [er in] is treated as the prepositional phrase. Two licensing descriptors can be implemented to disambiguate the multiple sentence readings with regards to the prepositional phrase.


The following sections will provide examples of the Licensing

conditions and discuss their order. Section 3.2.1 will provide examples for the

Subject-Head and Head-Subject licensing conditions. Section 3.2.2 wiU give

examples for disambiguating Head-Subject and Head-Subject-Complement

constructions.

3.2.1 . - Subject-Head and Head-Subject Licensing . -

The licensing conditions reflect the factors a language instructor

takes in to account in evaluating a student's sentence. To examine this

concept, consider example (6):

(6) Er sieht die Frau,

He sees the wornan.

A sentence which is unambiguous in many other NLP applications

can nonetheless result in multiple sentence readings in a systern designed to

parse fi-formed input. For instance, there are two syntactic structures for the

sentence fkom example (6). They are illustrated in the tree diagrams given in

figure 3.2.

For the sentence Er sieht die Frau, er could be taken to be either the

subject or the direct object of the sentence. Assuming that the choice between

the two parses were arbi-ary, sentence structure (a) where er is the subject of

the sentence contains n o errors while sentence reading (b) would yield the

feedback mis is not the correct case for the direct object.

The introductory student probably did not intend to topicalize the

object, that is, apply OVS over SVO word order. But given the feedback

provided for (b), the learner would most likely attempt to correct die Frau,

creating an incorrect sentence - one which was correct initially4 . -- . . . - - - - -


Det

die

He sees the woman. Sentence Structure (a)

NP

N V

1 1 Det N sieht I

die Frau

me woman sees him. Sentence Structure (b)

Figure 3.2: Two Analyses for Er sieizt die Frau

The parse selected by the Licensing Module emulates the decisions

made by a language instructor in interpreting a student's response. For an

introductory ievel, this mode1 strongly prefers the SV0 word order, given in

sentence reading (a), for the following reason: a beginner student is not being

taught about object topicalization and thus is iinlikely t o attempt the

construction. Even if the student had intended the OVS word order, in

considering the level of instruction and the f?equency and difficulty of the

grammatical constmctioh, a language instructor would assume that, more

likely, the SVO was meant.

The two licensing conditions, [mainclause Clicense-subj-head truell

and [mainclause [license-head-subj true]], disambiguate the two sentence

4. The feedback would probably be even more confusing if e r as the direct object were underlined, highlighted, or specificaUy mentioned in the feedback.


readings. The conditions are stored in their corresponding g r m a r d e s ,

Subject-Head and Head-Subject rule, respectively For illustration, the

Subject-Head rule with its associated licensing descriptor is partially given in

Figure 3.3.

.O cal r descriptor mainclause I

Figure 3.3: Subject-Head Rule

During parsing, the Subject-Head d e assigns the phrase descriptor

[main-clause [license-subj-head truell to sentence reading (a). For the

alternative sentence reading, the Head-Subject d e generates the phrase

descriptor [mainclause mcense-head-subject true]]. For an introductory

course, the order of the licensing conditions determines that sentence reading

(a), that is, the parse containing SV0 word order wiU be selected. Thus, the

same grarnmar c a n be used for different purposes.

The ranking of the licensing descriptors can be customized according

t o the goals of an exercise and the level of instruction. For example, a


particular exercise might aim at practicing object topicalization. For such an

instance, the priority ranking of the licensing descriptors can be changed such

that if a choice is given, the parse containing the head-subject licensing

descriptor wilI be selected.

While the two licensing conditions in the Subject-Head and Head-

Subject rules would be sufficient for disambiguating the two sentence

readings of example (61, the Licensing descriptor assigned in the Head-

~ubjec t -~om~lement d e is required for verb-subjed inverted constructions

in German. Examples are yesho questions, rnatrix clauses following a

subordhate clause, or main-clauses containing an adjunct as the first

eiement. The following section will pravide examples illustrating the need for

the Head-Subject-Complement licensing condition and show how it differs

Çom the Head-Subject Licensing condition.

3.2.2 Head-Subject-Complement and Head-Subject Licensing

In German, the hi te verb is always the second constituent of the

sentence. For this reason, there are a number of verb-subject inverted

constructions which also cause ambiguits In the example given in (7a), a time

adverbial precedes the verb, while the subject follows the verb.

(7a) *Heute sieht sie a

6) Heute sieht sie h.

Sentence (7a) results in two syntactic structures, given in Figure 3.4:

For sentence structure (a) in Figure 3.4 the Head-Complement rule

fïrst constructs the phrase [sieht siel. The Head-Subject rule then combines

the verb with its subject, [[sieht sie] er]]. The rule assigns the licensing

descriptor [mainâinclause mcense-head-subject truc]]. In this sentence reading

no error is flagged: sie is correctly infiected for a direct object and er for a - - -- --


sieht sie Head Complement Subject

lbday, he21 see her.

Sentence Smcture (a)

Adv S

sieht sie er

Head Subject Cornplentent

M a y , she'll see him.

Sentence Structure (II)

Figure 3.k Two Analyses for *Heu& sieht sie er

subject. For sentence structure (b) in Figure 3.4, however, the Head-Subject-

Complement rule assigns the licensing descriptor [main-clause

[licens e-head-subject-comp true]] and the feedback is The direct object is

incorrectly inflected for case since er, as a direct object should be ihn. The

correct sentence reading is that in (b). Although the parser does not flag an

error for sentence reading (a), the sentence is in fact ungrammatical. In

German, an object cannot be topicalized in inverted verb-subject

constructions. In this analysis, the ungrammatical construction is ruled out by

the ordering of the licensing conditions, given in (8) to (10).

(8) [main-clause Clicense-subj,head me13

(9) (main-clause Clicense-head-subjcomp truell

(1.0) [main-clause ficense-head-subj meIl

-- --


The order of the three licensing descriptors specifies that the parse

containing the Head-Subject-Complement descriptor, given in (9, will always

be selected over the Head-Subject licensing descriptor, given in (10); thus

eliminating the ill-formed topicalized sentence reading.

The ranking of the three licensing descriptors determines that if a

parse e i s t s that contains the SV0 word order it will be licensed. Failing t o

find any SV0 structure, the system will search for a VSO. As illustrated in the

example Heute sieht sie er, the VSO word order is also subsumed by VS order,

and must therefore be considered first to yield the desired feedback. Findy,

the Head-Subject licensing condition, given in (IO), wiU only be selected if n o

parse contains any of the other two conditions. This applies t o sentences with

intransitive verbs in inverted sentence constructions. An example is given in

(Il):

(11) Dass er geht, weiss ich.

%t he is leaving, 1 know.

The Head-Subject d e is the only rule whicb can apply to the matrix

clause combining the verb with its subject. For this reason, the descriptor

[mainclause Bcense-head-subj t rue] is required to License such sentences!

The licensing conditions mode1 a language instructor in

disambiguating multiple sentence readings in word order. The analysis

presents an alternative t o the technique first implemented by Weischedel et

5. Evea in an exercise where students practice topicalization, the Head-Subject-Complement licensing condition will always be considered over the Head-Subject condition because the former is subsumed by the latter. To allow for topicalization, the Subject-Head licensing condition would be considered last. 6. Unlike main clauses, embedded clauses do not require licensing descriptors. Under this analysis, errors in word order are reported first and parses are never arnbiguouç with respect t o verb position. Once the verb is in dause-final position, there is no ambiguity with regard t o the subject or object. IR subordinate clauses, al1 constituents occur to the left of the verb and the subject-verb d e requires a saturated complement Iist of the verb.


al. Cl9781 which selects the parse with the fewest number of enors. The two

are contrasted in the following section.

3.2.3 Licensing vs. Fewest Error Parse

The Zicensing approach is more suitable to feature underspecification

than a technique that is based on the fewest number of errors for a number of

reasons:

Unlike Weischedel's [1978] method, the licensing technique excludes

S e q u e n t (emphatic, poetic, etc.) sentence readings, if a choice of a more

fiequent usage is available. For example, for the sentence given in example

(12), ifsie is the subject of the sentence, then er is incorrectly infiected because

er should be ihn as a direct object in the accusative case. If sie is topicalized

and er is the subject, the sentence is correct.

(12) Sie liebt er.

She loves him or It is h r he loues.

However, a language instructor would not assume that a beginner

student intended the more poetic second reading. For this reason, the

preferred feedback would be This is not the correct case for the direct object

which corresponds to the grammatical constmction the student most likely

intended In Weischedel's Cl9781 technique, no error would be reported since,

an error-free sentence reading exists, albeit an improbable one.

To give equal weight t o syntactic structures that are grammatically

possible, but rare, would cloud feedback. Querying the student to

disambiguate sentences is one solution, but unwarranted if one alternative is

S e q u e n t . Feedback messages would double o r triple in length in order to

accommodate a tiny percentage of constmctions. A human tutor makes

assumptions about context and student sophistication. These assumptions

Designed InteLligence: A Language Teacher Model 96

have been transferred as much as possible to the approach presented here. If

the student indeed intended to topicalize the direct object in the example sie

liebt er, the feedback message will not be confiising since the student is aware

of the construction. However, accepting the sentence as correct when the

student did not airn a t topicalization would leave the student with a

misconcep tion.

- In addition, the ordering of the licensing conditions can be

customized. Exercises which aim at topicalization can be implemented and

the order of the licensing conditions changed.

The licensing technique also handes multiple parses where different

sentence readings contain an equal number of errors. An example is given in

(13)-

(13) *Sie hiM er.

She helps hirn or It is h r he helps.

For the sentence given in (131, the paner produces two sentence

analyses relevant t o this discussion. In the first reading, sie is the subject and

er is incorrectly infiected for dative. If the object is topicalized, er is the subject

and sie is incorrectly i dec t ed for the indirect object. Taking into account the

leamer level, the licensing conditions wiu select the first reading. Relying on

the parse with the fewest errors does not provide a rationale for choosing one

over the other since both-readings contain one error, either with er o r sie.

Finally, a system should present a consistent sentence analyçis. The

licensing technique consistently chooses the same parse through each

iteration of the correction process. As will be discussed in Chapter 4, a system

should display one feedback message a t a tirne. Once the student responds t o

the correction, the sentence is processed for further errors. A heuristic based

on the number of errors could easily shiR sentence readings in mid-stream,

Designed Intelligence: A Language Teacher -Mode1 97

leading to confusing errors messages. This applies to instances where more

than one sentence reading contain the same number of errors. To examine this

concept, consider example ( 14).

(14) *Der Groflvater sucht der Mann.

The grandfather is Zooking for the man.

The sentence given in (14) contains an error with the direct object.

There are two sentence readings: either der Grobvater o r der Mann could be

the direct object. However, in both instances the determiner has to be idected

for accusative, where der should be den. Thus, both sentence readings contain

an error in case. Assuming that the parser fiags one error at a tirne, once the

student makes the required correction, the sentence w i l l be sent for further

error checking. When the parser analyzes the sentence for the first tirne, it

might interpret der Mann as the direct object and an error in case will be

reported. If the student incorrectly changes der Mann t o d e n Mann, perhaps

assuming that the verb assigns dative as opposed t o accusative case, the

sentence is again wrong. In the next iteration process, however, the parser

might shiR interpretations and choose the alternative reading where der

Mann is the subject. This is possible because both sentence readings still

contain one error. The result is that the feedback will be confusing for the

student. The licensing technique described wiU always select the sentence

reading with SV0 word order, if a choice is given. Thus, the parser will

consistently choose the same sentence for analysis until the sentence is

correct.

The default setting of the priority ranking of the licensing technique

is aimed at the beginner level of German and assumes, like a language

instructor that a student would not attempt to use rare constructions a t that

level. For an intermediate course or even particdm exercises, the system can


be set to d o w for topicalization. The priority ranking of the three phrase

descnptors can be reordered to prefer OVS over SV0 word order.

I W e the licensing conditions sift possible sentence readings

according to the likelihood of an error, the ad hoc techniques block sentence

structures that arise £rom feature relaxation.

3.3 - Ad HocTechniques

If our goal is to mode1 a language instructor in evaluating students'

responses, then it is imperative that the analysis reject implausible analyses.

Whïle the licensing techniques select the most mely sentence reading, ad hoc

techniques will block undesirable sentence readings from being parsed.

Section 3.3.1 will provide an example of an ad hoc technique which has been

applied to phonologically identical finite verbs and ~ f i v e s .

3.3.1 F'ïnite Verbs vs. Tnfinitives --

Ad hoc techniques are warranted because of sentence readings arising

fiom ambiguity of lexical entries. A problem for any relaxed grarnrnar is

leecal entries that are phonologically identical, but have distinct grammatical

functions. For instance, examples (15) and (16) illustrate that the 1st person,

plural ïdect ion of a finite verb in German is identical to its infinitive. Either

form could be selected in any given parse. It does not, however, foUow that the

choice must be arbitrary. The choice of sentence readings should reflect the

student's likely intention, and avoid inaccurate o r misleadhg feedback.

(15) Heute g.eben wir.

Today we are leaoing.

Designed hteUigence: A Language Teacher Mode1 99

(16) E r kann a. Nè can go.

The sentences given in (15) and (16) wiU each produce a t Ieast 2

sentence readings. For each sentence, there will be one parse withgehen as an

infinitive and one parse with gehen as a finite verb. Ifgehen is interpreted as

an infinitive, then errors in case, gender, number, and person would be

overlooked. The reason is that infinitives do not impose agreement constraints

on the& subjects and thus do not contain phrase descriptors that record these

grammatical phenomena.7 In addition, there are instances, where the

feedback might be misleading. For example, idbitives need to be in sentence-

final position in German. For the sentence Morgen gehen wir, interpreting

gehen as an infinitiue would raise an error for verb position, although the

sentence with gehen as the finite verb is correct. The multiple sentence

readings are possible because the distinctive feature of the finite vs. the

nonfinite verb has been relaxed.

There are two ways of overcoming the problem with phonologically

identical formç: either o n l y one lexical entry is used, o r each f o r m has its own

lexical entry in which case they need to be made mutudy exclusive through

implementation of an additional feature. The analysis presented here uses one

lexical entV The advantage is that no additional features are required.*

To illustrate the analysis, consider the partial lexical entry for gehen

which functions as both a f i t e verb and an infinitive, given in Figure 3.5.'

7. The agreement features are marked with the modal which appears with the infinitive. 8. In some instances, giving each form its own lexical entry can be fairly elaborate. For example, finite verbs are the head of the sentence. Thus, the only dependent relationship with respect to the head exists between the subject and the verb. An analysis which considers two distinct lencal entries for a finite verb and an infinitive either requires that each possible subject is specified for the distinctive feature of the two verb forms or that the rules which combine a subject with a finite verb block a nonfinite verb. 9. For the purpose of this discussion, the case features have been collapsed. In addition, specifications for subordinate and coordinate clauses have been omitted.


phon < gehen >

local

content index !

position-maincl

Figure 3.5: Lexical Entry for gehen

Figure 3.5 illustrates that the verb gehen will generate three phrase

descriptors that record agreement constraints: vp-subj, vp-num, and vpqer .

The descriptors record number, gender, and case agreement of the subject of

gehen and subject-verb agreement in number and person, respective13 In


addition, the phrase descriptor position-maincl will collect a position value for

gehen as a nnite verb as well as for an infinitive. Finallx to record the

agreement between an aUX1liar-y o r a modal and its verb complement, gehen

spe&es two features: and nonfinite. They are required in instances where

a student might incorrectly choose an a d i a r y with an infinitive o r a modal

with a past participle.

- To obtain, accurate feedback, the analysis requires that the

appropriate phrase descriptors for either form are generated and that they be

mapped onto the correct verb form. To achieve this, the lexical entry of the

verb gehen specifies the phrase descriptors required for both forms. The

phrase descriptors for an infinitive, however, are only reported if they are

inherited by an auxiliary or a modal. The phrase descriptors for gehen as an

infinitive d be discussed in the following section.

Phrase Descriptors for gehen as an Infinitive

To capture the constraints of an infinitive, two grammatical

phenornena have to be recorded: verb-bal position and agreement between

an auxiliary o r a modal and the n o f i t e verb form.1° An auxiliary requïres a

past participle while a modal subcategorizes for an infinitive. An example of

an error with the modal kann and a past particïple is given in (17a).

(l7a) *Er kann eeg.angen.

(b) Erkann&

He can leave.

To record verb-bal position, the lexical entry of gehen, given in

Figure 3.5, includes the phrase descriptor [main-clause [position-maincl

10. In the aetual implementation, there are two phrase descnptors for recording verb-final position: one for infinitives and one for past participles. The two phrase descriptors allow for precise feedback of the position of the two verb inflections.


[infinitivenoreport XJ]]? An auxïliary o r a modal will inherit the value of the

phrase descriptor. The lexical entry of the modal kann is given in Figure 3.6.

phon 4 kann > head

comps pos 1 mainel [infinitive 011

L

descriptor

descriptor

mainclause

main-clause

Figure 3.6: Lexical Entry for kann

Figure 3.6 illustrates that kann will inherit the position value of the

infùlitive fiom the descript o r [mainclause [position~rnainclause

rinfinitivenoreport a]]. Note, however, that the descriptor of the modal is

[main_clause [position-mainclause rinfinitive Xf J J While the phrase

descriptor of the modal will contribute to a student mode1 update, the phrase

descriptor [main-clause [position~mainc~ause [infinitivenoreport X]]] is

otherwise ignored by the system.

11. X stands for an as yet uninstantiated value.


To record the agreement between an auxiliary o r a modal and its verb

cornpiement, gehen specities two features: & and nonfinite. For the idbi t ive

gehen, the modal kann, given in Figure 3.6, spedes a phrase descriptor that

records the agreement. It will inherit its value from the feature bse of the

nonfinite verb. The rernaining phrase descriptors given in the l e ica l entry of

gehen are n o t inherited by an auxilïary o r a modal and are thus ignored for

gehen as an infinitive. This applies to the phrase descriptors that record

subject-verb agreement in case, number, gender, and person and the position

of a f i t e verb.

Phrase Descriptors for gehen as a Finite Verb

To examine the constrâints of a finite verb, the grammar requires

three phrase descriptors for recording the agreement phenornena between the

subject and the verb: vpçubj, vpger, and vp-num. The verb gehen specifies

that it will inherit the values hom its subject. In addition to agreement, verb-

second position of the n n i t e verb also needs to be analyzed. The phrase

descriptors illustrated in Figure 3.7* record the constraints of a finite verb.

position-maincl [finite ( 1

Figure 3.7: Phrase Descriptors for gehen as a Finite Verb

12. For the purpose of this discussion, the case, number, and gender features have been collapsed.


Note, however, that the phrase descfiptor that records the position value of an

infinitive is ignored for gehen'as a finite verb. In addition, none of the phrase

descnptors of the finite verb are inherited by the modal and thus are not

recorded for gehen as an infinitive.

The ad hoc technique allows for disambiguation of multiple sentence

readings arising fkom phonologically identical lexical entries. The analysis

uses -one lexical entry and the phrase descriptors for each grammatical

phenomenon are successfully mapped ont0 the correct verb fom. As a result,

potentially misleading parses are blocked.

The technique can be applied t o further instances of phonologically

identical foms also found in other languages. For example, English presents

the same grammatical phenomenon of infinitives vs. finite forms.

Multiple parses present a challenge for any ILTS because the parser

might choose an unlikely sentence reading. As a result, the learner might

receive misleading feedback o r an error might be overlooked.

The Licensing conditions described in this chapter mark sentences for

the d e s which conçtructed them and select the appropriate interpretation on

the basiç of the likelihood of a given grammatical construction. The ranking of

the licensing conditions can be customized to reflect the focus o f a particular

exercise and/or the level of instruction,

In contrast to the licensing conditions, the ad hoc techniques block

sentence interpretations that are potentially so misleading that a language

teacher would easily dismiss them. The technique described c m be applied to

further instances of phonologically identical forms.


Both methods described in this chapter disambiguate multiple

sentence readings by emulating the language instructor in providing

intelligent feedback to students' input.


CHAPTER 4 Pedagogically- Informed Feedback

4.1 Introduction

Analyzing student input in an ILTS requires linguistic and pedagogic

knowledge. The task of the grammar and the parser is to generate phrase

descriptors which provide the linguistic analysis of students' input. The

phrase descriptors, however, need to be processed M e r to reflect principles

of language learning. The Analysis Module, the Student Model, and the

Filtering Module, respectively, constitute the pedagogic components carrying

out the analysis described.

The Licensing Module, discussed in Chapter 3, chooses the desired

parse. AU phrase descriptors associated with the selected parse are then sent

t o the Analysis Module, and the information obtained is subsequently passed

to the Student Model and Filtering Module. Figure 4.1 illustrates the relation

between the Analysis Module, the Student Model, and the Filtering ~odule. '


AU Phrase Descriptors ikom the Selected Parse

Analvsis Module

returns student model updates and produces instructional feedbac i of increasing abstraction

Student Model

keeps learner model updates and decides o n the student level 1

Filterine Module

decides o n the order of instructional feedback

Error-contingent Feedback Suited to Learner Expertise

Figure 4.1: Pedagogic Moddes

The Analysis Module takes all incoming phrase descriptors and

generates possible instructional feedback at different levels of granularity

comelated with student expertise. In ILTSs, granularity is particularly

important t o fkaming responses t o learners' errors. Inexperienced students

1. Of these three modules, only the Analysis Module is language-dependent but to a trivial extent: the feedback messages in the current implementation happen to be in English.

Designed intelligence: A Language Teacher Mode1 108

require detailed instruction while experienced students benefit best from

higher level reminders and explanations naReau % Vockell 19891.

For instance, in example (la) the student made an error with the

determiner einen of the prepositional phrase. Von is a dative pïeposition and

UrLaub is a masculine noun. The correct article is einern.

(la) *Sie traurnt von eineq Urlaub. -

fi) Sie t&umt von ~inem Urlaub.

She is dreaming of a uacation.

In a typical student-teacher interaction, feedback depends on the

students' previous performance history. For instance, for the error in example

(la), a leamer who generdy has mastered the concept of dative assigning

prepositions might merely receive a hint indicating that a mistake occurred

within the prepositional phrase. For the learner who generally knows the

grammatical r u l e but s t i l l needs practice in its application, the feedback is

less general: in addition t o location, the teacher might point o u t the m e of the

eaor (case). For the novice learner, the feedback would be as speciiic as

possible. In addition to the location and type of the error, information as to the

exact source of the error, that is, the precise grammatical rule that has been

violated (dative case), would be provided.

Note, however, that even the most specific report aimed at the novice

leamer refkains fkom revealing the correct answer. For the error in example

(la), the beginner student is s t i l l required t o decide on the correct Sec t ion of

a masculine indefinite article in the dative case. The pedagogical principle

underlying this design is guided discovery learning. According t o Elsom-Cook

[1988], guided discovery takes the student dong a continuum fiom heavily

structured, tutor-directed learning t o where the tutor plays less and less of a

role. Applied to feedback, the pedagogy scales messages on a continuum fkom

least-to-most specific guiding the student towards the correct answer.


Burton and Brown LI9821 state that hinting a t the source of an error

supports the development of students' self-regdation, a prerequisite for

guided discovery learning. A system can lead the learner towards the correct

answer rather than simply supplying it, thus assisting the students in finding

the source of an error themselves. The ability to discover the source of an

enor, however, strongly correlates with the expertise of the student: the more

exp- the leamer the less explicit feedback is necessary Fischer & Mandl

19881.

G r a n u l e captures this pedagogicdy intuitive concept: the

Analysis Module generates three categories of instructional feedback

corresponding to three learning levels: expert, intermediate, and novice. The

responses are s d e d fkom least-to-most specific, respectively Provided with

three categories of feedback by the Analysis Module, the Student Model

selects the error response suited to the student's expertise according to a

record of the student's previous performance history o n a particular

grammatical phenornenon.

The Student Model tracks 79 ~~ar constraints, corresponding t o

the grammatical concepts being monitored in an introductory course for

~ e r m a n . ~ For each of the grammar constraints, the Student Model keeps a

counter which, at any given instance in the evaluation process, falls in the

range of one of the three learner levels. If a grammatical constraint has been

met, the counter is decremented. If the constraint has not been met, the

counter is incremented and a feedback message suited to the performance

level of the leamer is selected.

2. The grammar constraints have been chosen according to Wie geht's, the course book commonly used in an introductory course for German fsee Sevin, Sevin, & Bean 199U.


A . example of a grarnmar constraint is pp-dat which records the

student's performance on dative assigning prepositions. Each student is

initially assessed as an intermediate, which has been chosen as a reasonable

default l e ~ e l . ~ A leamer who violates the constraint on dative prepositions

will, a t first, obtain the feedback message for the intermediate. If the student

commits the same error in subsequent exercises, s h e will soon be assessed a

novice. At this point, the Student Model will select the more detailed feedback

message suited to ihe beginner. However, each time the student applies the

grammatical constraint correctly, the Student Model records the success- &?ter

demonstrating ~roficiency, the student will again be assessed as intermediate,

or, even expert. Maintaining a large number of grammatical constraints

allows for a very detailed portrait of an individual student's language

cornpetence over a wide-range of grammatical phenomena.

The Filtering Module decides on the order in which instructional

feedback is displayed. Feedback is provided for one error at a time to avoid

overloading the student with extensive error reports. According to van der

Linden [1993], displaying more than one feedback message at a time makes

the correction process too complex for the student.

The ordering of the instructional feedback is decided by an Error

Priori@ Queue which ranks the 79 grammar constraints maintained by the

Student Model. The ordering of the g r m a r constraints and ultimately

feedback is, however, adjustable. Depending o n the pedagogy of a particular

language instructor, they can be reordered t o reflect the focus of a par t idar

exercise. In addition, errors which are not relevant to a particular exercise

3. The intention of the system is primarily to augment but not to replace classroom instruction. The system is therefore a practice tool for the student who previously has been exposed to a particular grammatical concept of Gennan- It is thus safe to assume that the learner will not be a complete beginner nor an expert on the grammatical phenomena practiced.


need not be reported at dl, although the error itself' still is recorded in the

Student Model.

The ordering of the grammar constraints also accommodates

contingent errors, a special class of multiple errors, and thus avoids

misleading or redundant feedback. F o r instance, in example (2a) the student

made a mistake with the prepositional phrase. The verb denken

subcategorizes for the preposition an which requires the accusative pronoun

dich, while von is a dative preposition requiring the dative pronoun dir. If we

igaore the dependency of the errors with the preposition and the pronoun, the

feedback t o the student would be "von" is the wrongpreposition and This is the

wrong case of the pronoun "dich". However, the pronoun dich is not incorrect if

the student changes the preposition von to an because an requires the

accusative pronoun dich. Depending on the order of the feedback, the student

might end up chmghg von to an and dich t o dir and wind up utterly

confused. S h e might ultimately fd to fmd the correct case of the pronoun

because it had been flagged as an error in the original sentence.

(2a) * Ich denke y~a. dkh-

dat prep. acc pronom

(b) Ich denke dich.


I am thinking of you.

The error in the pronoun is correctly flagged by the system f?om a

purely logical point of view. However, fiom a pedagogical perspective reporting

the eaor in the pronoun dich is redundant and possibly misleading. In such

instances, only the error in the preposition von is reported although the error

in the pronoun is recorded in the Student Model. Thus, the system, while

silently recording dl errors, does not necessarily comment on all of them.


The £bal result of the three modules is error-contingent feedback

suited to students' expertise. The following sections will discuss the Analysis

Module, the Student Model, and the Filtering Module in detail.

4.2 The Analysis Module

The function of the Analysis Module is to take all incoming phrase

descriptors as input and retum student model updates and potential error

feedback at dinerent levels of granulari@ for each phrase descriptor.

Granularity has been previously applied to an Intelligent Tutoring System for

LISP programming [McCalla & Greer 1994, Greer & McCalla 19891. In their

system SCENT, Greer and McCalla implemented granularity to "recognize

the strategies novice students employ when they solve simple r e m i v e LISP

progranunhg problemç."4 in t h i s analysis, however, granuiarity is used in

&aming responses t o learners' errors: inexperienced students obtain detailed

instruction while experienced students receive higher level reminders and

explanations. For example, consider the ungrammatical sentence in (3a).

(3a) *Der Mann dankt dem Eiwu. -.

(b) Der Mann dankt der Baia,

The man thanks the woman.

In example (3a), the student has provided the wrong determiner for

the indirect object. For the error dem Frau, the system generates feedback of

increasing abstraction that the instruction system can use when interacting

with the student. The level of the learner, either expert, intermediate, o r

novice according t o the current state of the Student Model, determines the

4. Greer & McCaIla C19891, p. 478-


particular feedback displayed. The responses, given in (4a) - (c) correspond to

the three leamer levels for the error in example (3a), respectively:

(4a) There is a mistake with the indirect object,

(b) There is a rnistake in gender with the indirect object.

(c) This is not the correct article for the indirect object. The noun is feminine.

For the expert, the feedback is most general, providing a hint t o where - Z

in the sentence the error occurred (indirect object). For the intermediate

leamer, the feedback is more detailed, providuig additional information on the

type of error (gender). For the beginner, the feedback is the most pretise. It not

only pinpoints the location and type of the error but also refers to the exact

source of the error (feminine noun).

Figure 4.2 displays the partial Granulârity Hierarchy for conçtraints

in feature matchingg5 The Granularity Hierarchy is a representation of the

instructions given to the student conelated with the grammatical constraintç

monitored by a phrase descriptor. Each term in a phrase descriptor

corresponds t o a level in the Granularity Hierarchy For example, for the

indirect object of the sentence *Der Mann dankt d e n R a y given in (3a) on p.

113, the grammar and the parser will generate the phrase descriptor

[main-clause [vp-indirobj [fem errer]]]. The top node of the Hierarchy

specitîes in which kind of clause the error occurred. The phrase descriptor

indicates that a mistake- was made in a min-clause. The next level in the

Granulari@ Hierarchy Lists possible errors in each clause type. As indicated

by the phrase descriptor, the mistahe refers t o the indirect object. A n even

her-grained constraint speafication is found in the next lower level of the

Granularity Hierarchy For instance, an indirect object can be incorrectly

5. The hierarch~ here is simplified for the purpose of illustration. The lowest nodes in the hierarchy split into further nodes as illustrated with gender.

Designed Intelligence: A Language Teacher Mudel 114

idected for either case, number, o r gender. The phrase descriptor specifies

that an error in gender occurred, çpecifically with a femirzine noun which

corresponds to the lowest level in the Granularity Hierarchy

Main Clause Coordinate Clause Subordinate Clause

Decreasing . S pecificity

Semantic Constraints

Case Gender Number

Preposition Semantic Constraints

Verb Inflection ModaVlnfhitive

Figure 4.2: Granularity Hierarchy for Constraints in Feature Matching

Each node in the Granularity Hierarchy corresponds to a level of

speciûcity of a feedback message. Granularity works dong one dimension,

namely, abstraction: travelling downward through the Hierarchy, nodes lower

in the Hierarchy will have assotiated with them messages of increasing

specificity. Generally, the more experienced the student the coarser-grained

the message and the higher the node.

The Analysis Module is implemented in DATR [Evans and Gazdar

19901, a language designed for pattern-matching and representing multiple

inheritance. Nodes in DATR are represented by the name of the node followed


by paths and their values. For example, the node given in Figure 4.3

corresponds to the phrase descriptor that records the position of a &te verb

in a subordkate dause.

('possubclfin' '3' 'true' T h e verb in the subordinate clause is not in the correct position.'

'possubclfm' '2' 'true' T h e h i t e verb in the subordinate clause is not in the correct - .

' posit-ion.'

'possubclh' '1' 'true' T h e finite verb in the subordinate clause is not in the correct

position, I t has to be the last element of the sentence.')

<sub-clause position-subclause fmite correct> = ('possubclfin' '1' 'false' "

'possubclfin' '1' 'false' "

'possubclfin' '1' 'false' ")

esub-clause positionçubcIause finite absent> -- (").

Figure 43: DA= Code Listing for a Finite Verb in a Subordinate Clause

The paths in a node defbition represent descriptions of grammatical

conçtraints monitored by phrase descriptors. The matching algorithm of

DATR selects the longest path that matches leR to right. Each path in a node

is associated with atoms on the right of '=='. For example, if there has been an

error in word order in a subordinate clause, the parse will contain the phrase

descriptor [sub-clause [position~subclause [ f i t e error]]] . This will match the

path csub-clause position-subclause fmite er ron which s p e d e s four atoms

for each of three groups. Each group represents a learning level. The three

learning levels considered are: expert, intermediate, and novice6, given in

Figure 4.3.

6. IFiner distinctions can be made by looking at the actual error count either for the purposes of evaluation and remediation; however, three levels are sufficient ta distinguish among the feedback messages encoded.


The f k s t atom in each group specifies the grammar constraint as

described by the incoming phrase descriptor. For example, the grammar

constraint possubclfi represents finite verb position in a subordinate clause.

The second atom specifies a value that is decremented or incremented

depending on whether a grammatical constraint has been met o r not,

respectively

- - The Boole& values, as the third atom, indicate whether it is an

increment o r decrement: true for increment, false for decrement.

Finally, the fourth atom specifies a feedback message. For example,

for the path csub-clause position-subclause finite erron provided earlier, the

specificity of each response corresponds to the grammar constraints in the

Granularity Hierarchy which fiames responses to constraints on verb

position, given in Figure 4.4.

The three instructional responses associated with each node

correspond to the three leamer levels. The feedback for the beginner student

reflects the lowest node in the Granularity Hierarchy, while for the

intermediate and expert student the message will refer t o nodes higher in the

hierarchy. In a typical

more profitient in the

becomes less specinc.

student-hacher interaction, as the student becomes

use of a grammatical construction, error feedback

If there has been no error in the student's input, the phrase descriptor

is [sub-clause [position~subclause [finite correct]]] and no message is

associated with the name of the grammar constraint. However, the first three

atoms, the grammar constraint, the decrement, and the Boolean value are

7. A phrase descriptor that contains the value &=.t is ignored by the system. Thus, the list t o the right of '==' is empty.


1 Verb Position 1 Decreasing Specificity Main Clause

Subordinate Clause Coordinate Clause

A

- - - - - - - - - - - -

Finite

Infinitive

Past Participl e

Figure 4.4: GranulariSr Hierarchy for Constraints in Linear Precedence

still spe&ed, given in Figure 4.3 on p. 116. They contribute to a Student

Model update to record the success.

Finallx if the phrase descriptor is [sub-clause [position-subclauçe

[finite absent]]], then the grmat ica l phenomenon was not present in the

student input. As a result, the phrase descriptor is ignored by the system,

indicated by an empty list in Figure 4.3 o n p. 116.

The &al output of the Analysis Module is a student mode1 update

and a list of possible instructional responses h m a coarse to fine grain size

for each incoming phrase descriptor. The more expert the student with a

particular grammatical construction, the more general the feedback.

Designed IntelIigence: A Language Teacher Model 118

The modular design of the system also allows the encoding of

instructional feedback suited to the pedagogy of a particular teacher. F o r

instance, an instructor who preferred to avoid the term finite verb in beginner

messages could substitute the simpler verbS8

The list of alternative instructional responses fkom the Analysis

Module has to be processed further before being displayed t o the student. The

Student Model, which picks a response suited to the level of the student, will

be dismssed in the following section.

4.3 The Student Mode1

Individualization of the learning process is one of the features of a

student-teacher interaction that distingtishes it &om gross mainçtreaming of

students characteristic of workbooks. Students l e m at their o w n pace and

often, work for their own purposes. Leamers also vary with respect to prior

language experience, aptitude, andior learning styles and strategies [Odord

19951. According to the Individual DSerences Theory as described by Oxford

C19951, if learners learn diaerently then they Likely benefit fiom

individualized instruction. An ILTS can adapt itself to different learner

n e e d ~ . ~ The requiremerit, however, is that the system incorporates and

8. There is an argument to be made for softerllng technical terms in the intermediate and novice categories. Rather than implementing a pseudo-vocabulary, however, a system can underline or highlight the error and provide hot links to information and examples on the linguistic terminolog' The advantage of using preuse linguistic terminology for al1 learner levels might result in students not only learning the grammatical concepts but the proper terminology a t the same time. 9. In its strict definition, ILTSs which do not implement a student mode1 are ICALL systems rather than ILTSs. However, in this thesis al1 systems which make use of Natural Language Processing are referred to as ILTSs.


updates knowledge about the learner. Student modelling provides the key to

individualized knowledge-based instruction McCalla & Greer 19921.

Student modelling has not been a strong focus of paner-based ILTSs,

likely because the challenging taçk of representing the domain knowledge in

ILTSs is s t i l l largely incomplete molland & Kaplan 1995b1.'~ If the grammar

is not accurate and complete, even a precise student model cannot

compensate. For inStance, HoIIand Cl9941 states that a system which doeç not

flag ambiguous and contingent errors accurately will obscure a student

model. A further possible reason is that many significant dif3erences in

language learning swles (situational, aural, visual, etc.) are precluded fkom

consideration in text-based ILTSs.

ILTSs which do have a student model primarily concentrate on

subject matter performance. M o d e b g students' surface errors assists in

individualizing the language learning process and 3 s sufncient to model the

student to the level of detail necessary for the teaching decisions we are able

t o make."ll The technique aids in teaching the required skills and

remediating the student.12

10. According to Holland and Kaplan C1995bJ there are two trends with student modelling in systems for language learning. The North American focus lies on the NLP module, the domain knowledge. Due to the c o m p l e x i ~ of the NLP component, these systems implement student models only to the extent. of analyzîng surface errors. In contrast, Eumpean systems concentrate on more sophisticated student models by hypothesizing causes of enors (see Chanier et al., 119921). However, they "opt for narrow NLP bandwidthn. Holland & Kaplan 11995b1, p. 364, Il. Elsom-Cook [19933, p. 238. 12. The rnodelling techniques employed in language learning are distinct fiom those applied to procedural tasks. In procedural tasks, the modelhg techniques presume that the student can learn the skill in terms of a number of formal steps. Yet although languages are rule- govemed systems and we can represent linguistic ability in terms of formal step-by-step niles, we do not produce language by consciously following them [Bailin 19901. As a result, the modelling techniques in language learning primarily diagnose the sources of errors rather than model strategies which the student used in solving a particular problem.


McCaIla Cl9921 makes a distinction between implicit and explicit

student m o d e h g which is particularly useful in classi&ng the student

models in ILTSs.

An implicit student model is static, in the sense that the Student

Mode1 is reflected in the design decisions inherent to the system and derived

fkom a designeis point of view. For instance, in an ILTS the native language

of the leamer can be encoded as a bug model and ultimately used to diagnose

errors.

In contrast, an explicit student model is dynamic. It is a

representation of the leamer which is used to drive instructional decisions.

For W S s , for instance, the student model can assist in guiding the student

through remedid exercises o r i t can adjust instructional feedback suited to

the level of the learner. In either case, the decisions are based on the previous

performance history of the learner. The following discussion will provide

examples of KTSs which have implemented implicit and explicit student

models.

4.3.1 Implicit Student Models

Implicit student modelling has been applied to ILTSs to diagnose

errors. For example, in-Catt & Hirst's Cl9901 system Scripsi the native

language of the student represents the learner model. It is used to model the

leamer's interlanguage. With regard to student modelling, the pitfall of such

an implementation is that it is a static conception. The system3s view of the

learner cannot change across interactions with the system. It has no impact

o n instructional decisions and provides only a gross individualization of the

learning process when ideallx a student rnodel is dynamic Bol t et al. 19941.

Designedl Intelligence: A Language Teacher Mode1 l.21

In a more hdividualized example, Bull Cl9941 developed a system

that teaches clitic pronoun placement in European Portuguese. The student

model is based on the system's and the student's belief measures, language

learning strategies, and language awareness.

The system's belief rneasure is comprised of the proportion of

incorrect/correct uses of the rule; the students provide the data for the

studiea.tk belief measuse, being required to state t h e 5 confidence in their

answer when entering sentences. Learners also iden* their preferred

leaming strategies when using the program. Accordhg to Bull [1994],

language awareness is achieved by dowing the student access to all

information held in the system. None of the information, however, is used to

drive the instructional process. In addition, a number of studies have s h o m

that students tend to not take advantage of the option tu access additional

information. For example, Cobb & Stevens Cl9963 found that in their reading

program learners' use of self-accessible help was villtually non-existent, in

spite of their previously having tried it in a practice session, and also having

doubled the success rate as compared to either a no help or dictionary help

option in the practice session.

4.3.2 Explicit Student Models

In developing & explicit student model one typically starts by

making sorne initial assumptions based on pretests or stereoSpical

postdations about the leamer. For example, initially every student could be

asçessed as an intermediate. During the instructional process, the student

model adjusts to student's behaviour moving to a novice o r expert profile, as

appropriate. This technique is used in explicit student models t o make

instructiond decisions.


Explicit student modelling has been used in a number of ILTSs,

primarily in the form of tracking. Tracking can be as simple as calculating

percentages of correct answers o r more sophisticatedly, i d e n e g particular

errors which occurred in the student's input. The information is then used to

alter the instructional process, either in the form of further language tasks or

feedback,

- - Explicit student modelling is found in the system The Fawlty A h d e

Tutor [Kurup, Greer & McCalla 19923 which teaches correct article use in

English. The system presents the student with scenarios whereby the student

must select the correct article form and the appropriate rule. The tutor keeps

an error count and selects the scenarios on the basis of the performance of the

student; thus the path through the program is individualized by altering the

instructional process according to prior performance of the student.

Bail.in [1988,1990] in his system Verbcon /Diagnosis also employs the

tracking method. Diagnosis provides practice in using English verb forms in

written texts. All verbs are presented in their infinitid form challenging the

student to provide the appropriate verb form. The system tracks the most -

fkequent error occurrence and the context in which the error occurred. The

information is used to provide informative feedback based o n contrasting

correct and ungrammatical uses of tenses. In addition, Diagnosis suggests

exercises to help with the remediation process. .

The student mode1 under the analysis of this dissertation is based on

students' prior performance and it ultimately has two main functions. The

first is to select instructional feedback suited to learner expertise and second

t o use the student's performance for assessrnent and remediation.13 The

following section will discuss the technique employed.


4.3.3 Feedback Suited to Learner Expertise

To select instructional feedback suited t o the level of the learner, the

Student Model keeps track of 79 grammar constraints. These grammar

constraints correspond t o the grammatical concepts to be monitored in an

introductory course for Geman. The grammar conskaints are split among

the three clause types, main, subordhate, and coordinate, each containing 25

nodes. In addition, there are two gramrnar constraints monitoring noun

phases practiced in isolation, one for verb-initial position in main-clauses,

and one for the entire sentence.

The Student Mode1 assumes three types of leamers: the novice, the

intermediate, and the expert. Each student level is represented by a range of

vdues14:

novice: 20 < X I 30

intermediate: 10 I X < 20

expert: O I X < 10

hitially, the leamer is asseçsed with the value 15 for each grammar

constraint, representing the mean score of the intermediate learner. The

values are used by the Student Model to decide on the specincity of the

instructional feedback being displayed. The intermediate learner has been

chosen as a reasonable default. While the messages might be in i t idy too

overspecified for the exp.ert and too underspecified for the novice, they will

quickly adjust t o the actual leamer level.

individualize the mode1 fkom the outset.

13. The system does not yet contain any pre-defined

Pre-testing could be used to

exercises, but in an extended version error counts held in the ~ G d e n t Model be available to the l e k e r . They form the basiç for branching decisions and remediation recommendations. 14. The ranges chosen for each student level roughly correspond to the average size of a grammar exercise unit. Since the default decrement for the error count is 1, suc ces^ completion of 10-15 exercises will record a change in student level,

Designed Intelligence: A Lanpage Teacher Mode1 124

For each learner level, the Student Mode1 receives four atoms from

the Analyçis Module, described previously: the name of a gramrnar

constraint, an inmernent/decrement, a Boolean value and in case of the latter

being true, a feedback message.

The counter of the grammar constraint determines which of the three

leamer responses is selected. If the number is 20 or greater, the system

displays the message suited for the beginner. If it is less than 10, the system

treats the learner as expert, and any number in between points to an

intermediate. The counter is bounded a t O and 30-

Once the Student Model has selected the feedback message to be

diçplayed to the learner, the counter is incremented by the incoming weighted

constant. For the expert, the increment is generally 3, for the intermediate 2,

and for the novice 1. The consequence of this sequence is that in case of errors,

the student wiU switch quickly from expert to intermediate, and somewhat

less quickly &om intermediate to novice. As a result, if a student is not an

expert with a given grammatical construction aRer all, the feedback wiIl

quickly become more informative fkom the expert to the intermediate level.

The transition 60m the intermediate t o the begimer level is slower since the

feedback for the former already points at a specific error type. For instance, for

the grammar constraint possubclfin, given on page 116 the feedback for the

intermediate level specifies The finite verb in the subordinate clause is not in

the correct psi t ion. l5

15. Ultimately, i t might be desirable to incorporate knowledge of the student's past performance- For example, at the end of each practice session the current error count for a particular grammar constra.int could be averaged with a historical count representing the last N sessions. By considering a histox-ical error count in deterrnining student level, momentary lapses in performance would be balanced by previous performance.


In the case of a correct response, as indicated by the Boolean value

false received fkom the Analysis module, the constant of the grarnmar

constraint is çubtracted from the counter. The decrement for al1 graRîmar

constraints is 1. Thus, the transition f%om novice to intermediate to expert is

gradual. The result of assigning a small decrement at all learner levels is that

any student has to apply a correct construction many times before the

feedback becomes sparse.

The Student Model presented has a number of advantages. I t takes

into account students' past performance, and by adjusting the value to be

incremented o r decremented, it is adaptable to a particular grammatical

constraint in an exercise or the pedagogy of a particular instructor. For

example, a language instructor might rate some errors more saLient than

others in a given exercise. In such an instance, the increment/decrement of

some grammar conçtraints can be tuned to change their sensitivity.

Its main strength, however, lies in the fact that a single e r r o f i

sentence will not drasticdy change the overall assessment of the student.

The phrase descriptors collect errors that indicate precisely the grammatical

context of the mistake. This enables the Analysis Module to create grammar

constraints which reflect verg specSc grammatical concepts that are

maintained in the Student Model. The consequence is that a student can be at

a diaerent level for any given grammar constraint reflecting the performance

of each particular g r m a r skill. This is desirable in a language teaching

environment because as a student progresses through a language course a

single measure is not sunicient t o capture the lmowledge of the learner and t o

distinguish among leamers. The Student Model described allows the student

to be geared toward enor-contingent and individualized remediation.


The Student Model can also be used for assessrnent and remediatioc.

A branching program can be implemented where students' tasks are

determined by their prior performance. The grammar constraints can be

weighted for a partinilar set of exercises, so as to be especidy sensitive to

salient errors. In an authorhg program, these settings could be adjusted by

the language instructor.

- Finallx the data obtained can be used for further research in

improving the overall performance of the system, and might prove usefüi in

providing objective measures of student performance.

The final step in analyzing students' input is handled by the Filtering

Module. The task here is to accommodate multiple errors. Multiple errors

have been largely overlooked in ILTSs. From a pedagogical perspective,

however, instructional feedback messages need to be prioritized by the system

and displayed one at a time to the student to avoid multiple error reports and

redundant and/or misleading feedback in the case of contingent enors. The

following discussion will focus on this task

4.4 The Filtering Module

While it is desirable to construct a system capable of detecting and

accurately explaining all errors, it does not follow that the system should

display each and every error detected. In the absence of an error filtering

mechanism, the sheer amount of feedback would overwhelm the student. For

example, in evaluating her own system Schwind [1990a] reports that

"[s]ometimes, however, the explanations were too long, especially when

students accumulated errors."16 In a language learning exercise, a student

Desigaed Intelligence: A Language Teacher Model 127

might make more than one error. However, a language instructor typically

skips krelevant errors, and discusses the remaining ones one a t a t h e .

Example (5a) illustrates an example of multiple errors.

(5a) * Heute meine &dem haben mit & Ball.

(b) Heute haben meine Kinder mit dem Ball &- Today my children were playing with the ball.

In example (5a) the student made the following five errors:

1. word order: the finite verb haben needs to be in second position

2. word order: the nonfinite verb gespielt needs to be in final position 3. spelling error with the verb spielen 4. wrong plural inflection for the subject =rider 5. wrong case for the dative determiner dem

From a pedagogical and also motivational point of view, a system

should no t ovemhelm a student with instructional feedback referring to more

than one error at a time. Schwindys [1990a] solution to this problem is that

rnuItiple errors should be avoided from the outset. She suggests that sentence

construction exercises should focus on s p e s c grammatical phenornena such

as prepositions o r verb cases [see also Kenning & Kenning 19901.

While Schwind's approach is probably inherent in many ILTSS~~,

lUniting the teaching domain is only a partial solution. Even a basic sentence

in German, as illustrated in example (5a), requires a number of rules and

knowledge about the case system, prepositions, word order, etc.l8

Little research has been done in Computer-Assisted Language

Learning regarding the volume of feedback for different kinds of learners at

16. Schwind (1990a1, p. 577. 17. ILTSs concentrate on sublanguages to also achieve a higher degree of accuracy. [Levin & Evans 19953. 18. Holland [1994], in her system BRIDGE, displays only one error at a time, and permits instructors to divide errors into primary, which are automatically displayed, and secondary, which are displayed only at the student's request.


different stages in their language development. However, van der Linden

[1993] found that "feedback: in order to be consulted, has to be concise and

precise. Long feedback (exceeding three k e s ) is not read and for that reason

not use^."^^ She M e r states that displaying more than one feedback

response at a time makes the correction process t oo complex for the student

[van der Linden 19931.

- Van der Lirgien's [1993] study makes three final recommendations:

r. feedback needs to be accurate in order to be of any use to the student,

2. displayhg more than one error message at a time is not veIy usefid because a t some point they probably will not be read, and

3. explanations for a particular error should also be kept short.

With regard to feedback display, van der Linden's Cl9931

recommendations require a system submodule to sift al1 incoming errors. The

errors have to be reported one at a time and the emor explanations should be

brief This provides the student with enough information to correct the error,

but not an ovemhelming amount, and yet records detailed information within

the student mode1 for

The analyçis

ranks student errors

assessrnent and remediation.

described implements an Error Priority Queue which

so as to display a single feedback message in case of

multiple constraint violations. The ranking of student errors in the Error

Priority Queue is, however, flexible: the grammar constrâints can be reordered

t o reflect the desired emphasis of a particular exercise. In addition, a language

instructor might choose not t o report some eaors. In such an instance, some

grammar constraints will display no feedback message at all, although the

19. Van der Linden C19931, p. 65.

Designed Intelligence: A Language Teacher Mode1 l29

error will s t i l l be recorded in the Student Model. The following section wiU

discuss the Error PrioriQ Queue.

4.4.1 The Error Priority Queue

The Student Model maintains grammar constraints and selects

instructional feedback suited to leamers' expertise. In case of multiple errors,

the Error Priority, Queue determines the order in which instructional

feedback messages are displayed t o the leamer. It ranks instructional

feedback with respect to

1. the importance of an error within a given sentence, and

2. the dependency of errors of syntactically lower and higher constituents.

The Error Priority Queue for the grammar constraints of a main

clause is partially given in (61.~' The names of the grammar constraints

generated and rnaintained by the Analysis Module and Student Model,

respectively are given in parentheses.

(6) Error Prior i ty Queue

1. Word Order in a Main Clause

1. position of a finite verb in a main-clause (posmainclfin) 2. position of a nonfinite verb in a main-clause (posmainclnonfm) 3. position of a finite verb in initial position (posmainclinitial)

direct Obiects in a Main Clause

1. case of the noun of the indirect object (indirobjnounmaincl) 2. case, nuniber, and gender of the determiner of the indirect object

(in dirobjmaincl)

1. case agreement of conjoined nouns of indirect objects (indirobjconjnounmaincl)

20. A complete Es t of the grammar constraints implemented in the system is provided in the Appendix,


2. case, number, and gender of conjoined determiners of indirect objects (indirobjconjmaincl)

N- Preriositional Phrases in a Main Clause

1. choice of preposition (prepmaincl)

2. case of the noun of a prepositional phrase in the accusative (ppaccnounmaincl)

3. case, number, and gender o f the determiner of a prepositional phrase in the accusative (pp-accmaincl)

4. case of the noun of a prepositional phrase in the dative (pp-datnounmaincl)

5. case, number, and gender of the determiner of a prepositional phrase in the dative (pp-datmaincl)

Each grammar corstraint, given in the Error Prioriw Queue in (6)

correlates to a node in the Granularity Hierarchy, provided earlier. The

grammm constraints are grouped according to grammatical phenornena. For

example, the group preuositional phrases in a main clause contains all

constraints monitored w i t h prepositional phrases and it corresponds to the

node prep. phrase in the Granularity Hierarchy in Figure 4.2 on p. 115. Each

member of a group in the Error Priority Queue refers to a node lower in the

Granularity Hïerarchy: the grammar constzaint choice of preposition, for

example, correlates with the node preposition in the Granularity Hierarchy.

The groups in the Error Priority Queue are sorted according to the

importance of an error within a sentence and the members within the group

are sorted according t o the dependency of errors of syntactically Iower and

higher constituents. If the student made multiple enors, the system ranks

instructional feedback messages according t o the order specined and displays

them one a t a time.

The Enor Priority Queue shown in (6) reflects the default setting for

the importance of an error in a given exercise. For example, grammar

constraints 1. t o 3. of the group Word Order in a Main Clause refer t o errors in

linear precedence. In the default setting, they are reported first since word


order is one of the fundamental concepts of a language and thus likely to have

high priority in most exercises.

The ordering of the groups of grammar constraints can, however, be

altered to reflect the pedagogy of a particular language instnictor. For

example, an instructor might want t o centre exercises around dative case

assignment. In such an instance, the grammar constraints can be reordered so

that . érrors of indirect objects are reported first. In addition, a language

instructor might choose t o suppress some errors, so as not to distract the

student from the main task. These would be errors which are not relevant to a

particular exercise. Suppressing certain errors, does not affect their

contribution to the Student Model, on the rationale that behind-the-scenes

information should be as detailed as possible.

The E n o r Priority Queue also takes into account contingent errors, a

special class of multiple errors. Contingent errors are due to a dependency

between syntactically higher and lower constituents and can result in

redundant or even misleading feedback. Contingent errors require that the

syntacticdy higher constituent be reported e s t . The following section will

discuss the process in detail.

4.4.2 Contingent Errors

The E r r o r Priority Queue also rads g r m m constraints according

to the position of the error constituents in the syntactic tree. For example,

grammar constraints, 1. - 5. of group on p. 131, refer t o prepositions and

their noun complements. Here it is especially important to report the error on

the preposition before an error with its noun complement to avoid redundant

and misleading feedback due t o the contingency between these errors. To


illustrate the problem with contingent errors, consider example (7a) as noted

by Schwind [1990a].

(?a) * Ich warte fii~ dk

acc prep. dat pronoun

(b) Ich warte &-

acc prep. acc pronom

I am waifing for you.

In example'(7a), the student made two errors: Für is an acnisative

preposition which requires the accusative pronoun dich. In addition, warten

subcategorizes for auf and not for fùr. The lurking cause here is likely

language interference. The English verb to wait subcategorizes for for, thus

the misuse of für. For the two errors flagged in example (7a) the student would

receive the feedback This is the wrong preposition and This is the wrong case

of the pronoun. However, the case of the pronoun depends solely o n the

preposition. In example (Ta) the pronoun is in the incorrect case for either

preposition, für and ouf both of which take the accusative. However, feedback

on contingent errors can even mislead the student. This applies to instances

where two prepositions require different cases, as given in example (8a):

(8a) * Sie denkt a ibn.

dat prep. acc pronoun

Co) Sie denkt rn ihn.


She is thinking of him.

Example (8a) illustrates that denken subcategorizes for the accusative

preposition an, while von is a dative preposition requiring the dative pronoun

ihm. As in example (7a), the feedback to the student would be mis is the

wrong preposition and mis is the wrong case of the pronoun. However, the

pronom ihn is not incorrect if the student changes the preposition von t o an

because an requires the accusative pronoun ihn. Depending on the order of


Figure 4.5: Contingent Errors

An example of a system which reports the redundant emor of a

syntactically lower constituent i s desaibed by Schwind C1990aJ who reports

how students evaluated such ETS:

".-., fiequently a wrong preposition for a verb was chosen and then the wrong case for that preposition. Our system then explained both errors but some students felt tha t they did not need to h o w the comect case of a preposition which they were not going to use anyway. Clearly i t would be possible not to check the agreement of the case of a noun phrase and the preposition when the preposition is already wrong, but this means that the student is left with a misconception about a grammar rule which has been detected by the s y ~ t e r n . ~ ~ ~

As Schwind states, the student should n o t be left with misconception

in the case of contingent errors. However, as human tutors, we do not

inundate students with error explanations, particularly when an error is not

immediately pertinent. A human tutor might make a mental note to monitor

22. Schwind [l990al, p. 577.


the lower-level agreement construct, but would focus on the more important

error.

Only a few scholars molland 1994, Schwind 1990a, 19951 have

addressed the problems of contingent errors. Holland [1994] suggests several

improvements to her own system:

",.- omitting error classification entirely and showing students only the locations of errors in a sentence, which are then stored in sturtents' performance records. However, this means losing the relevant information about enor classification in the clear cases. "... surpressing the outermost flags in a nested series and making it obligatory to address the innermost flags. The rationale is that having students first correct lower ievel errors; typically the less serious gender disagreements gives higher level grammatical problems a chance to present themselves. This strategy makes sense for a flag series in which the outermost flag is perfectly redundant, however, this strategy runs the risk of misrepresenting student error for certain other flag configurations."23

The suggestions made by Holland Cl9941 are unsatisfactory.

The h t solution is a step backward in the development of ILTSs. For

example, Nagata [1991, 1995, 19961 studied the efficacy of simply u n d e r k g

errors as opposed to providing the student with detailed feedback about the

nature of the error and she found that the latter is indeed more effective and

more appreciated by students.

The strategy of addressing syntactically lower level (innermost) errors

&st, given in 2., is strongly motivated by the need t o account for ambiguous

errors, Houand Cl994 and Schwind [1990a, 19951, as discussed in Chapter 2.

However, as Holland Cl9941 states 'the strategy runs the risk of

misrepresenting student enors for certain other flag which

23. Holland [1994], p. 249. For the purpose of discussion, the suggestions made by Holland Cl9943 have been reordered. Also, Holland [1994] list a third suggestion which, however, is very general and provides no recommendation for a c t d implementation.

- - - - -


are precisely those contingent errors illustrated in examples (7a) - For

these, i t is the syntactically higher level (outermost) errors, such as the choice

of a wrong preposition which need to be reported first. The problem with both

Holland's Cl9941 and Schwind's [1990a, 19951 approaches is that they attempt

to handle ambiguous and contingent errors with basically the same technique.

The assumption is that one can process either higher level errors o r lower

level errors first. Yet as with ambiguous errors, any system which cannot deal

witL contingent errors effectively runs the risk of producing misleading

feedback, and M e r misrepresents the student's knowledge of the

grammatical structures involved. As a result, the Student Model will not be

accurate.

The analysis described can successfully address ambiguous and

contingent errors because a separate technique for each class of errors is

employed. Ambiguous errors are handled by feature percolation and fher-

grained feature values speafied in phrase descriptors, as discussed in Chapter

2. In contrast, contingent errors are resolved by the Filtering Module. The

Fïitering Module s a s incoming errors in such a way that the çyatactically

lower level error is not reported to the student. This is achieved by the Enor

Priority Queue which specifies that errors of syntactically higher constituents

are reported first. For example, if the student chooses an incorrect preposition,

the error will be reported before an error with its noun complement. The error

with the noun complement WU, however, be recorded in the Student Model.

Once the error with the preposition has been corrected successfiill~ the error,

if still present, with the noun complement will be addressed. The analysis

24. Holland 119941, p. 249. 25. The same shortcoming is found in Schwind's [1995] system. As discussed in Chapter 2, Schwind [1995] also addresses lower-level errors first to handle ambiguous errors.


ensures that the system does not misrepresent the studenfs howledge, but

at the same time, provides the student with relevant and accurate feedback.

The grammar constraints of the E r o r Priori@ Queue, can always be

reordered to reflect the pedagogy of a particular language instructor. However,

the ranking of the grammar constraints referring to contingent errors, as with

grammatical constraints 1. - 5., which correspond to the choice of a particular

prepqsition and its noun complement, constitute a fked subgroup of grammar

constraints. While the whole group c a n be assigned priority over other

grammar constraints, the order of individual grammar constraints within the

group is fked to ensure that contingent errors are addressed in a

pedagogically sound wax

The Analysis Module, the Student Model, and the Filtering Module

are essential elements of the approach presented in this dissertation. They

incorporate pedagogical and psychological aspects of language learning

essential to an ILTS that models a language instructor in evaluating and

responding to student input.

The strength of .the Analysis Module lies in its ability t o generate

instructional feedback of different granulari* ~ranularity is implemented

unidimensionally ammging gritlllIIlar constraints and instructional feedback

fkom a coarse t o fine grain size

The Student Model classifies students according t o three performance

levels: the novice, the intermediate, and the expert. 79 g r m a r constraints

are maintained by the model, reflecting the grammatical constraints to be

Desïgned Intelligence: A Language Teacher Mode1 138

met in an introductory course for Germa. A student can be at different levels

for any given grammar constraint reflecting performance on each particular

grammar skill. As a consequence, a single enor will not drastically change

the assessment of the student. The information stored can be used for

tadoring instructional feedback suited to the level of the leamer and also for

assessment and remediation.

- The Filtering Module is responsible for handling multiple errors. The

approach to reporting multiple errors is pedagogically motivated, also taking

into account contingent errors. The system displays one response at a time

and the ranking of feedback responses is established by an Eaor Priori@

Queue. The Error Priori@ Queue is, however, flexible and can be easily

adjusted to reflect the focus of a particular grammatical constraint in a given

exercise. The &al result of the analysis is a single error message tailored to

the level of the leamer and pedagogical considerations.


CHAPTER 5 Conclusion

The design of an Intelligent Language Tutoring System presented in

this dissertation emulates a language instrudor by evaluating and

responding to student performance in foreign language exercises. The

techniques employed generate error-contingent feedback suited t o student

expertise,

The approach presented solves several extant problems in ILTSs.

First, errors in feature matching and linear precedence are treated in a very

general way: the main referent under this design is the target language rather

than spedic errors based on the native language of the student. As a result,

the system does not s d e r fIom limited error coverage, a narrow

preconception of the user, o r lack of generalia all problems associated with

anticipating errors based on the source language of the student. Second, by


emphasizing a pedagogicdy-informed, student-centered approach, the

techniques successfully address ambiguous readings, contingent and

ambiguous errors. Third, the andysis is modular making it adaptable to

specific student needs andior to reflect the pedagogy of a particula. language

instructor. Furthemore, although Gerrnan is used by way of example

throughout this dissertation, the rnodular design facilitates the task of

developing ILTSs for other languages. Language-dependencies, that is, the

domain knowledge o f the system are codîned t o the gramnar, which exists

within an othermise predominantly language-independent shell.

The approach presented consists of five components in total: The

Domain Knowledge, the Licensing Module, the Analysis Module, the Student

Model, and the Filtering Module. The five components closely correspond to

the steps a language instructor takes in evaluating a student's response to

foreign laquage exercis es.

The Domain Knowledge provides the linguistic analysis. It consists of

a Natural Language parser which analyzes student input according t u the

lmowledge o f a language encoded in the grammar. There are two main classes

of errors considered: errors in feature matching and errors in Iinear

precedence. Errors in feature matching are addressed in the lexical en- for

a linguistic sign; errors in word order are handled ui the grammar rules.

The main task of the Domain Knowledge is to assign phrase

descriptors. A phrase descriptor is a mode1 of a particular grammatical

phenomenon such as case assignment o r number agreement. A phrase

descriptor records whether or not the grammatical phenomenon is present in

the input and correctly or incorrectly formed. Ultimately, a phrase descriptor

speafies the contextual occurrence of a grammatical constraint, without,

however, deciding on the exact source of an error. - -


The remnining components add pedagogical value to the overall

design. Due to structural ambiguity found in language, a parser produces

more than one parse for many sentences. The Licensing Module chooses fkom

among parses by taking into account factors a language inshc tor would

consider in evaluating a student's response. In deterrnining the most likely

sentence reading, a language instructor considers the level of instruction, the

frequency of the grammatical construction, and thus the likelihood of the

error.

The Analysis Module incorporates a language instructor's knowledge

in pinpointing the precise source of an error. The Analysis Module takes

phrase descriptors as ioput and generates sets of instructional feedback of

increasing abstraction correlated with students' expertise. The Analysis

Module implements the pedagogical principle of guided discovery learnïng.

The more expert the learner, the Iess explicit the feedback required to lead

the student toward the correct answer. The modular design of the analysis,

however, makes it possible to customize feedback for a particdar instmctor o r

course.

The level of the learner, either expert, intermediate, o r novice

according to the m e n t state of the Student Model, determines the particular

feedback displayed. The Student Model keeps track of the learner history and

provides learner model updates. It records mastery of grammatical structures

as well as structures with which the leamer has problems.

The Student Model maintains a record of each grammatical

constraint to be monitored in an introductory course for German. As a resdt,

a student c m be at a meren t level for any given grammar constraint

reflecting performance of each particular grammar SU. The Student Model -- - - - -- -- - - -


allows for individualization of the leaming process by correlating feedback

with student expertise.

The Filtering Module decides on the order in which instructional

feedback is dispiayed. Feedback is provided for one error at a time t o avoiding

overloading the student with extensive error reports. The ordering of the

instructional feedback is decided by an Error Priori@ Queue which ranks

gramkir constraink freely with respect to the importance of an error in a

given exercise and more restrictively t o handle dependencies between

syntactically higher and lower consti~ents. The ordering of the grammar

constraints and ultimately feedback is, however, adjustable. Depending on

the pedagogy of a particular language instructor, the grammar constraints

can be reordered t o reflect the focus of a particular exercise. In addition,

errors which are not relevant t o a particular exercise need not be reported at

all, although the error itself stiu is recorded in the Student Model. Thus the

system, while silently recording all errors, does not necessarily comment on

aU of them.

The design of an ILTS outlined in this dissertation solves many

computational and pedagogical problems found in earlier ILTSs. Significant

progress has been made toward the goal of an intelligent, student-centered

tutor, but, inevitably, further work remains. For instance, while HPSG is well-

suited for the a l g o r i t h m described, there are, however, errors which pose

problems for KPSG as a head-driven grammar formalism. These concern

errors in omission and insertion. In addition, a number of pedagogical

research questions suggest themselves. The system and underlying design

will undoubtedly mature over time, benefiting from the experience gathered

through practical use. The following section will discuss further research.


Further Research

Errors in omission and insertion cause problems for any ILTS

because these errors are highly unpredictable. Although missing and exfxa

constituents Gan, t o some extent, be traced to language interference, these

errorS can certainly occur due to causes other than native language

intederence. For instance, a learner might even be aware of the grammar

d e s of the target language but due to a lack of Sping skills, coxentration,

o r fatigue fails to demonstrate that understanding. For this reason, any

system which anticipates these Ends of errors on the basis of the native

language of the student sufFers fiom a narrow preconception of the user,

limited enor coverage, o r lack of generality

The approach taken in this dissertation is consistent with the overall

HPSG philosophy in that it maintains a s m d set of d e s . For this reason, the

grammar strictly eschews buggy rules that trg, for example, t o p m e a phrase

missing a word. Yet HPSG does not immediatdy offer any tools for handluig

errors in omission and insertion

briefly discuss missing and extra

of example.

in a general way. In the following 1

adjuncts, complements, and heads, by

Adjuncts are optional in that they contribute to the semantic content

of the whole phrase. Homever, missing or extra adjuncts do n o t cause a

violation of the subcategorization list of a head. In contrast, a tighter

dependency holds between complements and their heads. Heads

subcategorize for their complernents and thus a head determines the number

and kinds of complements it requires to be saturated. For errors in omission,


for example, the three distinct types of constituents present varying degrees

of di f f icu l ty for an ILTS.'

From a computational point of view, the least problem is posed by

adjuncts. A missing adjunct d l not cause a parse t o fail. However, from a

pedagogical perspective adjuncts still need to be addressed. For instance, in

example (la) the student has omitted an article within the adjunct, the

prepasitional phrase. The sentence will parse; however, without additional

techniques the error will not be reported- In a chart-based parsing technique

such as the one described in this dissertation, a solution might be found in

examining the chart.

(la) *Er Lest ein Buch in Garten.

(b) Er l ied ein Buch -- He is reuding a book in the garaen-

In contrast, missing complements cause a violation of the

subcategorization List of a head. For instance, in example (2a) the direct object

is missing. Unlike English, the verb sehen is always transitive in German.

(2a) *Ich sehe.

(b) Ich sehe den Stem.

1 see t h star.

For missing complements a technique might make use of the

subcategorization principle of HPSG [see Hagen 19941. If, a t the end of a

parse, the verb hzs not-found its complement(s) as specined in its lexical

entm then the verb remainç unsaturated and unsaturated words signal an

error response. Tracing verb complements presents some challenge but, at

1, There are a few errors of omission and insertion which are covered by the technique used for errors in feature matching. For example, an error with a missing auxiliary does not present a problem for the analysis in this dissertation. The sentence will parse with a nonfinite verb. A phrase descriptor indicating a missing finite verb, however, will be generated.


least, a record in the chart is present.

The problem becomes intractable if the head of the sentence, the verb

itselfis missing. In a head-driven approach, there is no record in the chart if

the head is missing and thus these errors require at least some extension to

the theory of HPsG?

Problems simiiar to those found with miçsing constituents are

apparent with errors in insertion. Consider example (3a):

(3a) *Er ist & Englander.

(b) Er ist Englander.

He is an Englishrnan.

Example (3a) illustrates the classic example of English speakers

incorrectly inçerting the indefinite article before nationalities in German.

However, as in the case with rnissing constituents, errors in insertion are

highly unpredictable. A sfxdent might even repeat a word in a sentence due

to, for example, a lack of concentration.

For extra constituents, HPSG also cannot offer an immediate tool for

a general solution because HPSG maintains a small set of general schemata

that do not refer to specific categories of their constituents. Thus, an answer

outside the grammar formalism has t o be found. For example, scanning the

chart for singleton words and an algorithm which drops the word in question

would be considered,

From a pedagogical perspective, a research question of further

interest concerns student rnodelling. For example, due to the implementation

2. The fact that the phrase descriptors are also head-driven is a secondary problem. If the grammar is capable of recognizing that the verb is missing and instructional feedback cm be provided to the leamer, then as soon a s the sentence contains a verb, the phrase descriptors of the verb complements can be collected.

Designed Intelligence: A Laquage Teacher Mode1 146

of phrase descriptors in the overd modular design, the system is capable of

referring not only t o the source but also, in the case of native language

interference, to the cause of e r r ~ r s . ~ A study with actual language learnerç

might reveal whether feedback which addresses both the source and the

cause of an error would enhance the language learning process for different

learners at different stages in their language development.

- A b-roader study might address the role of an KTS in a given language

learning environment. For example, parser-based ILTSs inherently focus o n

grammaticd form rather than content. However, advances in multimedia

development have enabled ILTSs to d o w for a more communicative focus

without, however, neglecting form. Given this potential, empirical research

could reveal whether ETSs can form the basis of a language learning activity

rather than seroing as a second- best, that is, a mere mechanical adjunct to

dassroom grammar instruction.

3. For errors in linear precedence, for instance, a her-grained error distinction similar to that applied to ambiguous enors can be implemenbd. For example, the position value of a verb could reAect the error due to native language interference.

- - -- - -


Appendix Grammar Constraints

position of a finite verb in a main clause (posrnainclfin)

position of a nonfinite verb in a main clause (posmainclnonfin)

position of a finite verb in initial position (posinitid)

inflection of the verb in a main clause (vformmaincl)

inflection of a nonfinite verb in a main clause (compverbmainclj

choice of auxiliary vs. modal in a main clause (auxmainci)

number agreement in a main clause (vp-nurnmaincl)

person agreement in a main clause (vp-permaincl)

case of the noun of a noun phrase (npnoun)

case, number, and gender of the determiner of a noun phrase (np)

case of the noun as the subject in a main clause (subjnouamaincl)

case, number, and gender of the determiner as a subject in a main clause (subjrnaincl)

case agreement of conjoined nouns as subjects in a main clause (subjconjnounmaincl)

case, number, and gender of conjoined determiners a s a subject in a main clause (subjconjmaincl)

- - case of the noun as a direct object in a main clause (dirobjnounmaincl)

case, number, and gender of the determiner as a direct objectin a main clause (dirobjmaincl)

case agreement of conjoined nouns as direct objects in a main clause (dirobjconjnounmaincl)

case, number, and gender of conjoined determiners a s a direct object in a main clause (dirobjconjrnaincl)

case of the noun as an indirect object in a main clause (indirobjnounmaincl)

case, number, and gender of the determiner as an indirect object in a main clause (indirobj maincl)


case agreement of conjoined nouns as indirect objects in a main clause (indirobjconjnounmaincl)

case, number, and gender of conjoined detenniners as an indirect object in a main clause (indirobjconjmaincl)

choice of preposition in a main clause (preprnaincl)

case of the noun in an accusative prepositional phrase in a main clause (pp-accnounmaincl)

case, number, and gender of the determiner in an accusative prepositional phrase in a main clause (pp-a&maincl)

case of the noun in a dative prepositional phrase in a main clause (pp-datnounmaincl)

case, number, and gender of the determiner in a dative prepositional phrase in a main clause (p pdatmain cl)

semantic constraints in a subordinate clause (semmaincl)

position of a finite verb in a subordinate clause (possubclfin)

position of a nonfinite verb in a subordinate clause (possubclnonfïn)

S e c t i o n of the verb in a subordinate clause (vformsubcl)

inflection of a nodinite verb in a subordinate clause (compverbsubcl)

choice of auxiliary vs. modal in a subordinate clause (auxsubcl)

number agreement in a subordinate clause (vp-numsubcI)

person agreement in a subordinate clause (vpgersubcl)

case of the noun as the subject in a subordinate clause (subjnounsubc2)

case, number, and gender of the determiner as a subject in a subordinate clause

(subjsubcl j

case agreement of conjoined nouns as subjects in a subordinate clause (subjconjnounsubcl)

case, number, and gender of conjoined determiners as a subject in a subordinate clause (subjconjsubcl)

case of the noun as a direct object in a subordinate clause (dirobjnounsubcl)

case, number, and gender of the determiner as a direct object in a subordinate clause (dirobjsubcl)


42. case agreement of conjoined noms as direct objects in a subordinate clause (dirobjconjnounsubcl)

case, number, and gender of conjoined determiners as a direct object in a subordinate clause (dirobjconjsubcl)

case of the noun as an indirect object in a subordinate clause (indirobjnounsubcl)

case, number, and gender of the determiner as an indirect object in a subordinate clause Cindirobjsubcl)

case agreement of conjoined nouns as indirect objects in a subordinate cIause (indirobjconjnoun%bcl)

case, number, and gender of conjoined determiners as an indirect object i n a subordinate clause (indirobjconjsubcl)

choice of preposition in a subordinate clause (prepsubcl)

case of the noun in an accusative prepositional phrase in a subordinate clause (ppaccnounsubcl)

case, number, and gender of the determiner in an accusative prepositional phrase in a subordinate clause (ppaccsubcl)

case of the noun in a dative prepositional p h a s e ui a subordinate clause (pp-datnounsubcl)

case, number, and gender of the determiner in a dative prepositional phrase in a subordinate clause (pp-datsubcl)

semantic constraints in a subordinate clause (semsubcl)

position of a finite verb in a coordinate clause (poscoordclfin)

position of a nonfinite verb in a coordinate clause (poscoordclnonfin)

inflection of the verb in a coordinate clause (vformcoordcl)

inflection of a nonfinite verb in a coordinate clause (compverbcoordcl)

choice of auxiliary vs. modal in a coordinate clause (auxcoordcl)

number agreement in a coordinate clause (vp-numcoordcl)

person agreement in a coordinate clause (vp-percoordcl)

case of the noun a s the subject in a coordinate clause (subjnouncoordcl)

case, number, and gender of the determiner as a subject in a coordinate clause (subjcoordcl)


case agreement of conjoined nouns a s subjects in a coordinate clause !subjconjnounsubcl)

case, number, and gender of conjoined determiners as a subject in a coordinate clause

(subjconjsubcl)

case of the noun as a direct object in a coordinate clause (dirobjnouncoordcl)

case, number, and gender of the determiner as a direct object in a coordinate clause

(dirobjcoordcl)

case agreement of .conjoined n o m s as direct objects in a coordinate clause

(dirobjconjnounsubcl)

case, number, and gender of conjoined determiners a s a direct object in a coordinate

clause (dirobjconjsubcl)

case of the noun as an indirect object in a coordinate clause (indirobjnouncoordcl)

case, number, and gender o f the determiner as an indirect object in a coordinate clause

(indirobjcoordcl)

case agreement of conjoined nouns a s indirect objects in a coordinate clause

(indirobjconjnounsubcl)

case, number, and gender o f conjoined determiners as an indirect object in a coordinate

clause (indirobjconjsubcl)

choice of preposition in a coordinate clause (prepcoordcl)

case of the noun in an accusative prepositional phrase in a coordinate clause

(pp-accnouncoordcl)

case, number, and gender o f the determiner in an accusative prepositional phrase in a

coordinate clause (pp-acccoordcl)

case of the noun in a dative prepositional phrase in a coordinate clause (pp-datnoun)

case, number, and gender of the detenniner in a dative prepositional phrase in a

coordinate clause (pp-datcoordcl)

semantic constraints in a coordinate clause (semcoordcl)

an error in any clause type (sentence)


Ahmad, K et al. (1985). Computers, Language Learning and Language Teaching. Cambridge University Press.

Alessi, S. M., and Trolip S., R. (1985). Cornputer-Based Instruction: Methods and Development. Englewood CliEs, New Jersey: Prentice-Hall, Inc.

Badin, A. (1995). "AI and Language Learniag." Intelligent Language Trrtors: Theory Shaping Technology. Holland, M. V., Kaplan, J.D., Sama, M.R., eds. Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc.: 327-343.

Bailin, A. (1990). "Skills in Context and Student Modeling." CALICO Journal, 8: 7-22.

Bailin, A-, and Thomson, P. (1988). "The Use of Natural Language Processing in Computer-As sisted Language Instruction." Cornputers and the Hurnanities, 22: 99-110.

Barker, P., and Yeates, H. (1985). Introducing Ccrnputer-Assisted Learning. London: Prentice-Hall International.

Bolc, L., ed. (1978). Natural Language Communication with Computers. Springer Verlag.

Brown, D. H. (1994). Principles of L a n s a g e Learning and Teaching. Englewood CWs, New Jersey: Prentice Hall, Inc.

Bull, S. (1994). " ~ t u d e k Modelling for Second Language Acquisition." Computers and Education, 23 (1-2): 13-20.

Burns, H. et al, eds. (1991). Intelligent î k tor ing Systems: Euolutions i n Design. Hillsdale, New Jersey: Lawrence Erlbaum Assoc., Inc.

Burton, R. K., and Brown, J. S. (1982). Investigation of Computer Coaching for Znformal Learning Activities." Intelligent TWoring Systems. Sleeman, D. and Brown, J. S. eds. Academic Press.: 79-98.

Designed kitelligence: A Langaage Teacher Mode1 152

Carpenter, B., and P ~ M , Gerald. (1994). The Attribute Logic Engine: User's Guide, Version 2.0. Computational Linguistics Program, Carnegie Mellon University, Pittsburgh.

Catt, M., and Hirst, G. (1990). "An Intelligent CALI System for Grammatical Error Diagnosis." Computer Assisted Language Learning, 3: 3-27.

Chanier, T., Pengelly, M., Twidale, M., and Self, J. (1992). "Conceptual Modeling . in Error Analysis in Computer-Assisted Language Learning Systgms." Intelligent ntoring Systems for Foreign Language Leaming. The Bridge to International Communication. Swartz, M. L. and Yazdani, M., eds. Nato AS1 Series F: Computer and Systems Sciences. Vol: 80. Springer Verlag.

Chen, L., and Kurtz, B. L. (1989). 'XTRA-TE: Using Natural Language Processing Software to Develop an ITS for Language Learning." Proceedings of the 4th International Conference on Artifxial Intelligence and Education: 54-70,

Chomsky, N. (1986). Barriers. Cambridge, MA: MIT Press.

Chomsky, N. (1981). Lectures on Government and Binding. Dordrecht: Foris.

Cobb, T., and Stevens, V. (1996). "A Principled Consideration of Computers and Reading in a Second Language." The Power of C . . Pennington, M. C . ed. Houston: Athelstan Publications: 115-137.

Colmerauer, A. (1978). Metamorphosis Grammars. Natural Language Communication with Computers. Bolc, L., ed. Springer Verlag.

Covington, M. A. (1994). Natural Languczge Processing for Prolog Programmers. Englewood Cliffs, New Jersey: Prentice Hall Inc.

Covington, M.A., and Weinrich, KB. (1991). "Unification-Based Diagnosis of Language Leamers' Syntax Errors." Literary and Linguistic Computing, 6 (3): 149-154.

Corder, S. (1967). "The Significance of Leamer's Errors." International Review of Applied Linguistics, 5: 161-170.


Costa, E., ed. (1992). New Directions for Intelligent Trctoring Systems NATO AS1 Series F: Computer and Systems Sciences. Springer Verlag. Vol: 91.

Elsom-Cook, M. (lW3 ). "Student Modelling in Intelligent Tutoring Systems." The A r t i f ~ i a l Intelligence Reuiew, 7 (3-4): 227- 240.

Elsom-Cook, M. (1988). "Guided Discovery Tutoring and Bounded User Modelling." Artificial Intelligence and Human Learning." Self, J., ed. Bristol: J, W. Arrowsmith Ltd.: 165-178.

-

Evans, R., and Gazdar, G. (1990). m e DATR Papers, Volume 1. Brighton, School of Cognitive and Computing Sciences. The University of Sussex.

Fischer, P. M., and Mandl, H. (1988). "Improvement of the Acquisition of Knowledge by Informing Feedback." Learning Issues for Intelligent Tutoring System. Mandl, H. and Lesgold, A., eds. Springer Verlag: 187-241

Farghali, A. (198 9). "A Model for Intelligent C omputer Assisted Language Instruction (MICALI)." Cornputers and the Humanities, 23: 235-250.

Fotos, S., ed. (1996). Multimedia Language Teaching. Tokyo, Logos International,

Frasson, C., Gauthier, G-, and McCda, G.I., eds. (1992). ~ n t e l l i ~ e n t Tutoring Systems. Lecture Notes in Computer Science. Springer Verlag.

Garrett, N. (1995). 'WALL and Second Language Acquisition." Intelligent Language %tors: Theory Shaping Technology. Holland, M . V., Kaplan, J.D ., Sama, M.R., eds. Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc. 344-358.

Garrett, N. (1987). "A Psycholinguistic Perspective on Grammar and CALL." Modern Media in Foreign Language Education: Theory and Implementation. Ed. Smith, W . F. L ~ ~ c o ~ ~ w o o ~ ~ K: National Textbook. 169-196.

Gazdar, G. (1982). Phrase Structure Grammar. The Nature of Syntactic Representution. Jacobson, P . and Pdum, G, eds. Dordrecht: Reidel: 13 1-186.


Gazdar, G., and PuUum, G. (1985). "Computationally Relevant Properties of Natural Languages and their Grammars." New Generation Computing. 3: 273-306.

Gazdar, G., Klein, E., Pullum, G-, and Sag, 1. (1985). Generulked Phrase Structure Grammar. Cambridge, MA: Harvard University Press.

Greer, J. E., and McCalla, G.I., eds. (1994). Student Modelling: The Key to Individualizec? Knowledge-Based Instruction. Nato AS1 Series F: Computer and Systems Sciences. Vol: 125. Springer Verlag.

Greer, J. E., and McCalla, G.I. (1989). "A Computational Framework for Granularity and its Application to Educational Diagnosis ." Proceedings of the International Conference on Artificial Intelligence. Detroit: 477-482.

Hagen, L. K. (1994). "Unification-Based Parsing Applications for Zntelligent Foreign Language Tutoring Systems." CALICO Journal, 12 (2): 5-3 1.

Holland, M. V., Kaplan, J.D., and Sama, M.R., eds. (1995a). Intelligent Language Tutors: Theory Shaping Technology. Mahwah, New Jersey: Lawrence Erlbaum Associates, hc.

Hol la~~d , M. V., and Kaplan, J.D. (1995b). Naturd Language Processing Techniques in Cornputer-Assisted Language Learning: Status and Instructional Issues. Instructional Science, 23: 35 1-380.

HoIland, M. V. (1994). uLessons Leamed in Designing Intelligent CALL: Managing Communication across Disciplines ." Computer Assisted Lang-uage Learning, 7 (3): 227-256.

Houand, M., Maisano, R., Alderks, C., and Martin, J. (1993). "Parsers in Tutors: What Are They Good For?" CALICO Journal, 11 (1): 28-46.

Holt, P., D u ~ s , S., Jones, M., Greer, J. (1994). "The State of Student Modelling." Student Modelling: The Key to Indiuidualized Knowledge-Based Instruction. Greer, J . E., McCalla, G, eds. NATO AS1 Series. Series F: Computer and Systems Sciences. Springer Verlag. Vol: 125: 39-35.


Holt, P., and Wood, P. (1990). "Intelligent Tutoring Systems: A Review for Beginners." Canadian Journal of Educational Communications, 19 (2): 107- 23.

Johnson, R. (1983). Parsing with Transition Networks. Parsing Natural Language. King, M. (ed). Academic Press Inc., London: 59-73.

Juozulynas, V. (1994). "Errors in the Compositions of Second-Year German Students: An Empirical Study for Paner-Based ICALI." CALICO Journal, 12 (1): 5.17.

Karttunen, L. (1984). "Features and Values." Proceedings of the International Conference on Cornputational Linguktics.

Kenning, M.-M., and Kenning, M. J. (1990). Cornputers and Language Leaming. Ellis Horwood Limited, England.

King, M., ed. (1983). Parsing Natural Language. London: Academic Press Inc.

Kunip, M., Greer, J. E., and McCda, G. (1992). The Fawlty Article Tutor." Intelligent Tutoring Systems. Frasson, C., Gauthier, G-, McCaIla G.1, eds. Lecture Notes in Computer Science. Springer Verlag: 84-91.

Kurtz, B. L., Chen, L., and Huang, X (1990). English-Chinese Language Learning System Using Adaptive Correction and Multipass Error Diagnosis." J o u m l ofArtificial Intelligence in Education, 1 (4): 51-64.

Labrie, G., and Singh, L.P.S. C19911. Tarsing, Emor Diagnostics, and Instruction in a French Tutor." CALICO Journal, 9: 9-25.

Lado, R. (1957). Linguistics across Cultures. Ann Arbor: University of Michigan Press.

LaReau, P., and Vockell, E. (1989). The Computer in the Foreign Language Curriculum. Santa Cruz, CA: Mitchell Publishing.

Desigxled Intelligence: A Language Tacher Mode1 156

Larson-Freeman, D., and Long, M. H. (1991). An Introduction to Second Langzmge Acquisition Research. London: Longman Group UK Limited.

Last, R. W. (1989). Mificial Intelligence Techniques in Language Learning. Ellis Horwood Limited, England.

Levin, L. S., and Evans, D. A. (1995). "ALICE-chan: A Case Study in ICALL Theory and Practice." Intelligent Language Tutors: Theory Shaping Technology. Holland, M. V., Kaplan, J.D., Sama, M.R., eds. Mahwah, New Jersey: Lawrence Erlbaum Assotiates, Inc.: 77-99.

Liou, H.-C. (1991). "Development of an English Grammar Checker: A Progress Report." CALICO Journal, 9: 27-56.

Littlewood, W. (1984). Foreign and Second Language Learning. Cambridge University Press.

Loritz, D. (1992). "Generalized Transition Network Parsing for Language Study: T h e GPARS System for English, Russian, Japanese, and Chinese." CAL,ICO Journal, 10 (1): 5-22.

Loritz, D. (1995). "GPARS: A Suite of Grammar Assessrnent Systems." Intelligent Language Tutors: Theory Shuping Technology. Holland, M. V., Kaplan, J.D., Sama, M.R., eds. Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc.: 77-99.

Mandl, H., and Lesgold, A-, eds. (1988). Learning Issues for Intelligent Tutoring Systems. Springer Verlag.

Marshall, D. V. (1988). CALICBT - The Great Debate. ChartmeIl-Bratt, Brornley.

Matthews, C. (1994). "Intelligent Computer Assisted Language Learning as Cognitive Science: The Choice of Syntactic Frameworks for Language Tutoring." Journal of Artificial Intelligence in Education, 5 (4); 533-556.

Matthews, C. (1993). "Grammar Frameworks in Intelligent CALL." CALICO Journal, I l (1): 5-27.

- -

Designed Intelligence: A Language ~Acher Mode1

McCalla, G. (1992). The Centrality of Student Modelling to Intelligent Tutoring." New Directions for Intelligent Trctoring Systems Costa, E., ed. NATO AS1 Senes F: Computer and Systems Sciences. Springer Verlag. Vol: 91: 107-131.

McCalla, G., and Greer, J. (1994). "Granularity-Based Reasoning and Belief Revision in Student Models." Student Modelling: The Key to Individualized Kiiowledge-Based Instruction. Greer, J. E., McCalla, G, eds. NATO AS1 . Series F. Computer and Systems Sciences. Springer Verlag. Vol: 125: 39-62.

MCC&, G., and Greer, J. (1992). "Special Issue on Student Modelling: Editors' Introduction." Journal of Artificial Intelligence in Education, 3 (4): 377-380.

McFetridge, P., and He*, T. (1995). "Grammar and Analysis for Implementing Granula* in Cornputer-Assisted Language Instiuction." Educational Multimedia and Hypennedia, 1995. Graz, Austria: Association for the Advancement incomputing in Education, 1995. pp. 460-465.

Nagata, N. (1996). "Computer vs. Workbook Instruction in Second Language Acquisition." CALICO Journal, 14 (1): 53-75.

Nagata, N. (1995). "An Effective Application of Natural Language Processing in Second Language Instruction." CALICO Journal, 13 (1): 47-67.

Nagata, N. (1991). "Intelligent Computer Feedback for Second Language -

Instxuction." The Modern Language Journal, 77: 330-8.

Nerbonne, J., Netter K., and Pollard C., eds. (1994). G e n a n in Head-Driven Phrase Structure Grammar. CSLI Lecture Notes. Stanford, Center for the Study of Language and Information.

Newmark, P. (1976). "The Importance o f Accuracy." CILT reports and Papers 13, Gennan in the United Kingdom, Problems and Prospects.

Oxford, R. L. (1995). "Linking Theories of Learning with Intelligent Cornputer-Assisted Language Learning (CALL)." Intelligent Langrcage Tutors: Theory Shuping TechnoZogy. Holland, M. V., Kaplan, J.D., Sama, M.R., eds. Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc. 359-369.


Pennington, M. C., ed. (1996). The Power of CALL. Houston: Athelçtan Publications.

Pereira, F.C.N., and Warren, D. H. D. (1980). "Definite Clause Grammars for Language Analysis - a Survey of the Fomalism and a Cornparison with Augmented Transition Networks." Artificial Intelligence, 13: 231-278.

Pollard, C., and Sag, 1. (1994). Head-Driuen Phrase Structure Grammr. Chicago UGversity Press.

-

Pollard, C., and Sag, 1. (1987). .hformtwn-Based Syntax and Semantics: Fundarnentals. CSLI Lecture Notes. Stanford, Center for the Study of Language and Information.

Price, R. V. (1991). Cornputer-Aided Instruction: A Guide for Authors. Pa&c Grove, CA: BrooWCole Publishing Co.

Richards, J. (1971). "Error Analysis and Second Language Strategies." Languuge Sciences, 17: 12-22.

Rogers, M. (1984). UOn Mayor Types of Written Error in Advanced Students of German-" International Review of Applied Linguistics in Languuge Teaching, 22 (1): 1-39.

Rumelhart, D., and Norman, DA. (1975). Explorations in Cognition. San Francisco: Freeman.

Sanders, R. (1991). YError Analysis in Purely Syntactic Parsing of Free Input: The Example of German." CALICO Jourrurly 9 (1): 72- 89.

Sanders, A. F., and Sanders R. H. (1989). &Syntactic Parsing: A Survey." Cornputers and the Humanities, 23: 13-30.

Schuster, E. (1986). The Role of Native Grammars in Correcting Enors in Second Language Learning." Cornputafional Intelligence, 2 (2): 93-98.

Schwind, C. B. (1995). "Error Analysis and Explanation in Knowledge Based Language Tutoring." Cornputer Assisted Languuge Learning, 8 (4): 295-325.


Schwind, C. B. (1990a). "An Intelligent Language Tutoring System." international Journal of Man-Mcchine Studies, 33: 557-579.

Schwind, C. (1990b). Teature Grammars for Semantic Analysis ." Computational Intelligence, 6: 172-178.

Self, J., ed. (1988). Artificiul Intelligence and Human Learning- London, New York: Chapman and Hall Ltd.

SelinJcer, -L. (1972). "Interlanguage." International Eeview of Applied Lingilistics, 10: 2091231.

Sevin, D., Sevin, I., and Bean K.T. (1991). Wie gehfs. Holt, Rinehart and Winston, Inc. The Dryden Press. Saunders College Publishing.

Shieber, S. M. (1986). An Introduction to Unîf~aion-Based Approaches to Grammar. CSLI Lecture Notes Number 4. Stanford, Center for the Study of Language and Information.

Sleeman, D., and Brown, J. S. eds. (1982). Intelligent Btoring Systems. Academic Press,

Smith, W. F., ed. (1987). Modern Media in Foreign Language Education: m o r y and Impkmentation. Chicago: National Textbook Co.

Swartz, M. L., and Yazdani, M., eds. (1992). Intelligent Tutoring Systems for Foreign Language Leanzing. The Bridge tu Intenational Communication. Nato AS1 Series F: Computer and Systems Sciences. Vol: 80. Springer Verlag.

Van der Linden, E. (1993). "Do es Feedback Enhance Cornputer-Assisted Language Learning." Computers & Education, 21 (1-2): 61-65.

Van Noord, G. (1997). "An Efficient Implementation of the Head-Corner Parser." Computational Linguistics, 23 (3): 425-456.

Venetzky, R., and Osin, L. (1991). The Intelligent Design of Computer-Assisted Instruction. New York: Longman Publishing Group.

Wang, Y., and Garigliano, R. (1995). "Empirical Studies and Intelligent Language Tutoring." Instnrctional Science, 23: 47-64.


Wang, Y., and Garigliano, R. (1992). "An Intelligent Language Tutoring System for Handling Errors Caused by Tramfer." Intelligent Tutoring Systems. Frasson, C., Gauthier, G., McCalla G.1, eds. Lecture Notes in Cornputer Science. Springer Verlag: 395-404.

Warschauer, M. (1996). "Computer-Assisted Language Learning: An Introduction." In Fotos, S. (ed). Multimedia Langxage Teaching. Tokyo: Logos International: 3-20.

Weinreich;-U. - - . (1953). L a m a g e in Contact. Linguistics Circle of New York.

~eischedel , R.M., and Sondheimer, N.J. (1983). "Meta-Rules as a Basis for Processing Ill-Formed Input." American Journal of Computational Linguistics, Special Issue on I l l -Fomd Input, 9 (3-4): 161-177.

Weischedel, R.M., Voge, W.M., and James, M. (1978). "An Artincial Intelligence Approach to Language Instruction." Artificial Intelligence, 10: 225-240.

Wenger, E. (1987). Artifziul Intelligence and Tutoring Systems: Computational and Cognitive Approaches to the Communication of Rnowledge. Los Altos, CA: Morgan K a h a n n Publishers, Inc.

Woods, W. (1970). "Cascaded ATN Grammars." American Journal of Computational Linguistics, 6 (1): 1-12.

Yazdani, M., and Uren, J. (1988). "Generalising Laquage-Tutoring Systems: A French/Spanis h Case Study, Using LINGER." Instructional Science, 17: 179-188.

Designeci Intelligence: A Language Teacher Mode1 161

IMAGE EVALUATION TEST TARGET (QA-3)

APPLlE D IMAGE. lnc - = 1653 East Main Street - --2 Rochester. NY 14609 USA -- -- - - Phone: 71 6/482-O3OO -- -- - - Fa: 71 6/288-5989

0 1993. Appiied Image. Inc.. All Rights Reserved

INFORMATION TO USERS - Library and Archives Canada · of the possible parses by taking into account...

Documents

Transcript of INFORMATION TO USERS - Library and Archives Canada · of the possible parses by taking into account...