UNIVERSITY-LEVEL CAl IN FRENCH

by

JOHN BARSON

Department of Freru:h and ItalianStanford University

DAVID LEVINE

Intermetms, Inc.Cambridge, Mass.

ROBERT SMITH

Department of Computer ScienceRutgers University

New Brunswick, NJ.

MARYSE SCHOLL and

PIERRE SCHOLL

IMAG Laboratoire d'Informatiqne et deMathematiqnes Appliquees de Grenoble,

France

THIS ARTICLE describes an experiment to use computer-assisted instruc­tion (CAl) for teaching Introduction to French at Stanford University. Thecomputer program and associated curriculum were used for the first quar­ter (approximately ten weeks) of a three quarter sequence of French Iduring 1972 and 1973. Students involved in the experiment took lessons oncomputer terminals located at the Institute for Mathematical Studies in theSocial Sciences (IMSSS) at Stanford and also attended standard classes fordrill and conversation practice taught by one of the authors, John Barson.

This article describes the main features of the program, its deficiencies,and the role the French program played in the development of CAl atStanford during the 1970s. The French program incorporated the follow­ing ideas and techniques of instruction.

1. An underlying pedagogy: A method for foreign language instruction calledthe Rationalist Direct Method was the basis of the instruction. This methodand its implications for CAl are discussed in section 1.

2. Division of the curriculum into strands: The French curriculum was dividedinto strands, each incorporating instruction and exercises on a facet of thematerial; this is an approach that was pioneered at the Institute for othercurricula such as reading (Atkinson, Fletcher, Lindsay, Campbell, & Barr,1973). Items of the curriculum are coded in a pattern language that allowsthe same pattern to be used for many vocabulary items. The use of thiscurriculum design and its relation to the Rationalist Direct Method is dis­cussed in section 2.

3. Random~access audio: The course used random-access computer-generatedaudio for the generation of spoken French. The audio system is described insection 4.3.

4. Error diagnosis: An effort was made to provide careful analysis of studenterrors, diagnosing various orthographic problems that occur frequently.This is discussed in section 4.4.

The research reported in this article was partially supported by National Science Founda­tion Grant NSF GJ443X. We would like to thank Gary Morgenthaler for his programming ofparts of the project. Barbara Anderson and Dianne Kanerva assisted in the preparation of thisarticle.

685

686 BARSON, SMITH, D. LEVINE, SCHOLL, & SCHOLL

The French program succeeded to some degree in implementing thepedagogical philosophy of the Rationalist Direct Method. In doing so, itpressed some of the techniques to their limits. As discussed in section 5, theaudio system in use at the time, the algorithms for error diagnosis, and thestrand curriculum representation all presented problems. As we discuss insection 6, these difficulties suggested some of the concerns that becameprimary in more recent work in the later 1970s at the Institute.

I. PEDAGOGICAL PHILOSOPHY

We wanted the computer to do as much of the teaching as possible. Thecomputer did all the instruction in grammar and vocabulary and ,providedexposure to spoken French. Computer-generated random-access audioprovided exposure to spoken French during the CAl lessons. Students metthree times weekly in a normal classroom group where French was spokenaloud. The classroom teacher did not introduce new vocabulary or gram­mar, but instead used what the students had learned from the computer asa basis for conversations and other oral exercises. In addition, studentswrote short compositions in French and the (human) teacher analyzedthese assignments.

1.1 Teaching Method

We followed the Rationalist Direct Method approach to language in­struction (de Sauze, 1961; Pucciani & Hamel, 1967). The main tenets ofthis method are:

1. The student should hear the language before speaking, reading, and writingit.

2. The curriculum should be constructed using vocabulary and syntactic struc­tures that describe everyday situations.

3. A constant appeal should be made to the student's innate sense of language,his 1 capacity for grammatical inference based on given examples, and hiscapacity for logical organization.

4. Extensive practice should be given in listening and imitation to develop basicautomatic responses.

5. Creativity should be encouraged, both in writing and speaking, even at thebeginning.

The implementation of points I, 2 and 4 on the computer posed nounusual difficulty given the audio system. Point 5, of course, was done inthe classroom, as was the oral practice for point 4. Point 3 was handled byproviding grammatical explanation only at the request of the student andgenerally structuring the presentation to provide a challenge to the stu-

1 We shall use the masculine English pronoun to refer to students in general, regardless ofsex.

FRENCH 687

dent's inductive capabilities. How these objectives were achieved is de­scribed in detail in the foIlowing sections.

2. STRUCTURE OF THE CURRICULUM

2.1 Strands

To structure the curriculum in accordance with the principles of theRationalist Direct Method, we had to balance two conflicting tendencies.On the one hand, language is composed of a multitude of rules which aretraditionaIly taught one at a time; on the other hand, the student must seethe language as an integrated whole and each fragment of instructionshould contribute to the development of this whole.

A particular exercise, while remaining a part of the overall unifiedprogression of the,curriculum, may emphasize one particular grammaticalconcept. We have chosen to organize the grammatical concepts into severalbroad categories. Each category is essentiaIly independent of the others,although any given exercise involves the student's general grasp of thelanguage and thus necessarily involves. the other categories.

Each category is applicable throughout most of the course. Every exer­cise belongs, by definition, to a particular category. Each category forms astrand of exercises running the length of the curriculum. In a sense, eachstrand represents a suhcurriculum, being an ordered progression of mate­rial illustrating the particular conceptual category associated with thatstrand. The strand does not, however, stand alone; its exercises are validinstruction only as part of the ensemble comprising all the strands. Thestudent's study must take him through all the strands to preserve the unityof the language. The presentation strategy allows for a certain freedom inthe student's relative progress through different strands, but the overallsynchronization is maintained.2

We selected the following underlying conceptual categories for theFrench language:

1. Gender and number.

2. Replacement and use of pronouns.

3. Verbal reflex.

4. Temporal awareness.

5. Modification and qualification.

6. Sentence structure.

7. Pronunciation.

8. Idiomatic expression.

We shall illustrate the exercises in these strands by considering varia­tions on the sentence

2 The synchronization was at times extended by specification of order relationships be­tween the strands.

688 BARSON, SMITH, D. LEVINE, SCHOLL, & SCHOLL

"Jacques lit Ie journal"(Jacques reads the newspaper).

We shall follow the practice of quoting French text with English transla­tions following in parentheses. Boldface type will indicate emphasis.

Strand 1 focuses on the article, with exercises using such phrases as

"Ie journal" (the newspaper)"un livre" (a book).

Strand 2 deals with the use of pronouns:

"II lit Ie journal." "II Ie lit."(He reads the newspaper.) (He reads it.)

Strand 3 discusses the verbal reflex (conjugation):

"Je lis Ie journal.':(I read the newspaper.)"II lit Ie journal."(He reads the newspaper.)

Strand 4 focuses on tense and time indication:

"J'ai Iu Ie journal hier,"(I read the newspaper yesterday.)

Strand 5 deals with modification and qualification (adverbs and adjec­tives):

"Jacques lit Ie bon journal."(... the good newspaper.)

Strand 6 discusses sentence structure (negation and interrogation):

"Jacques ne lit pas Ie journal."(Jacques did not read the newspaper.)"Qui lit Ie journal?"(Who reads the newspaper?)

In actual conversation, many forms are combined and the curriculumcould in principle reflect this richness. This would, however, present thestudent with too much to learn simultaneously. The strand concept allowsus to focus the instruction on one set of closely related features with neither

FRENCH 689

the richness of conversation nor the problems associated with simulta­neously presenting too many stimuli.

We preserve the essential unity of the curriculum by the simple instruc­tional strategy of changing strands frequently. If an individual student ishaving difficulty with a particular concept, the presentation algorithmprovides a somewhat higher percentage of exercises from the associatedstrand. Section 3 describes the presentation algorithm in more detail.

2.2 Units and Patterns

A strand is subdivided into units. Each unit concerns a specific rule orgrammatical point. The strand is a linear ordering of these units.

For most of the curriculum, the unit is an exercise-block. This is a patternor template from which an actual exercise is generated by substitutingactual vocabulary.

For instance, a simple exercise in sentence formation might contain thefollowing exercises. (We indicate the audio messages generated by Audio:.Student responses, generally what we intend for students to type to thesystem, are preceded by Student:. Comments intended to facilitate this ex­position are enclosed in square brackets, and English translations are en­closed in parentheses.)

Audio: Paris est une ville.(Paris is a city.)

Audio: Un lion ... un animal.

[Desired response follows]

Student: Un lion est un animal.(A lion is an animal.)

Audio: Un lion est un animal.

[Repeating the response aloud]

The above exercise may be generated from the following pattern, where Aand B are variables ranging over certain classes of words and phrases.

Audio: Paris est un ville.

Audio: A ... B

Student: A est B

Audio: A est B

The variables A and B in the pattern are chosen dynamically from a list ofpairs associated with the item, such as:

A Bun lion un animalun chien un animalM. Dupont un homme

See section 4.1 for a more detailed description of the pattern language.

692 BARSON, SMITH, D. LEVINE, SCHOLL, & SCHOLL

Between blocks: The major motion of the student through the cur­riculum takes him from one block to the next on each strand. As a block isusually associated with- a single grammatical point, it also forms a naturalunit for evaluation. When the student reaches the end of a block, hisperformance is checked according to the following algorithm:

PRESENT BLOCK satisfaction criteria:

P +- percentage correct in block;IF P > 80 percent THEN

PROGRESS to next block ELSEIF P < 30 percent THEN

REPEAT current block ELSEBEGINC +-correct-in-sequence percentage;IF C > 50 percent THEN

PROGRESS to next block ELSEREPEAT items missed;END;

The P (percentage correct in block) value is computed simply over all itemsin the block. The C (correct-in-sequence percentage) value is a measure ofthe number of correct responses in a sequence at the end of the block. It isthe quotient of the number of items answered correctly at the end dividedby the total number correct. The intuitive justification for the use of thismeasure is that the student may have had some initial difficulty with theblock that was fully overcome by the end.

The threshold percentages (80, 50 and 30) in the above algorithm aredefaults that are under control of the curriculum author.

Within the block: Each pattern block is written with a set number ofitems that will be presented to all students. In most cases, several additionalitems will be available to give extra practice to those students who arehaving difficulty (as evidenced by repeated wrong answers). In addition,the curriculum author specifies a tolerance level. The algorithm on the fol­lowing page illustrates the action of presenting a block to the student.By supplying the tolerance level and the additional items, the curriculumauthor controls the remediation available within the block. The mechanismof comparing the number of errors to the supplied tolerance level providesa simple yet effective mechanism to control the presentation of additionalexerCIses.

During the repetition of a block or portions of a block due to failure tomeet the performance criteria, the strategy is automatically set to rejectextra or repeated items.

PRESENT BLOCK:

FRENCH 693

COMMENT present the regular items, counting errors;ERRORCOUNT+- 0;FOR EACH regular item DO

BEGINPRESENT item (after pattern variable substitution);IF incorrect THEN

ERRORCOUNT +-ERRORCOUNT + 1;END;

COMMENT decide how many additional items to present, and doit;

ADDITEMS +-tolerance level - ERRORCOUNT;FOR CNT +-1 UPTO ADDITEMS DO

BEGINPRESENT an extra item if there are any left else one of

the regular items incorrectly answered;END;

Within an item: Under control of the curriculum author there are twostrategies for handling an incorrect student response within an item. Theusual strategy allows the student to try again, having first given him apreliminary diagnostic in the form of his answer with misspelled or Ollt­

of-order words underlined appropriately.' If the student's second trial isalso wrong. the program gives him the correct response and continues. Forthe purpose of tallying performance, the response is not counted wronguntil the second error, but the correct-in-sequence count is reset on the firstmistake in an item.

The second strategy allows the student only one attempt to solve theproblem. An error causes the correct answer to he printed without anyerror diagnosis of the student's response. This strategy is automaticallyapplied to all extra and repeated items.

3.2 Student Initiative

As described above, the control over presentation of material resides intwo places, the algorithms contained in the program and indicatorssupplied by the curriculum author. The French program provides animportant third dimension, control by the student himself. The presenta­tion of explanatory material is entirely under the control of the studenthimself. Explanations of grammar, vocabulary, and other information are

4 See section 4.4 for a detailed discussion.

694 BARSON, SMITH, D. LEVINE, SCHOLL, & SCHOLL

always available, but are presented only upon request. The student mayinterrupt an answer to make a request for information or he may takeadvantage of one of the many times the program pauses for this purpose.In the following, we discuss a number of features of the program that allowthe student to ask for help and information.

Grammar and vocabulary: The Rationalist Direct Method requires thatnew material be experienced before it is explained: the student must begiven an opportunity to induce relevant principles whenever he can. At thesame time, there will clearly be occasions when the student needs anexplanation of what is happening or access to the meaning of an unknownword. The necessary information is provided by the curriculum author andmade available to the student upon demand. While it is hoped the studentwill first try to use his inductive language skills, the alternative of asking forassistance is always available to the student.

Grammar help: Each block (pattern) illustrates a particular generalpoint. A short message stating the applicable rule is coded as part of theblock and is available on request. The curriculum author can controlwhether or not this message is available to the student. The student isencouraged to try to do a problem without asking for the rule given in thismessage.

Vocabulary help: An on-line French-English dictionary is available onrequest. When the student indicates that he wants the dictionary, theprogram gives a short statement of the English meaning of the requestedword, as well as pertinent French grammatical information such as gender,irregularities, etc., of French words from the curriculum. In addition to allthe stem forms employed in the curriculum, the dictionary contains irregu­lar forms of words and some inflected forms that must be used prior topresentation of the relevant grammar. Access to the dictionary is forbiddenonly when testing comprehension, as during dictation.

Messages: To save time, the student receives only brief instructions atthe terminal. For example, he might only be told "Completez" ("complete")or "repondez oui" ("answer yes"). A longer message, giving more details onthe nature of the required work, is available on request. (The longermessage is usually in English and the shorter one in French). It is ourintention that the longer explanation would only be needed once or twicein a given pattern.

Often the correct answer itself constitutes an effective way of illustratinga given structural point. By typing "?" when a response is requested, thestudent can receive the correct answer directly.5 Student use of this optionseemed to be motivated by both misunderstanding of the substantive

5 Available only in 1973. In 1972, the student was told the correct answer after he made twoerrors.

FRENCH 695

grammatical point involved and by confusion concerning the program'soperation.

To insure against excessive use of the "?" (which prints out the correctanswer) option, its use causes the current item to be counted as wrong.Used sparingly at the beginning of a pattern, the "?" would not impede thestudent's progress. Excessive use, or too many errors, leads to repetition ofthe items on which the student did not choose to respond'

Starting over: If the student desires, he can ask the computer to restartat the beginning of the current item. This is the only minor alteration thatthe student can make in the actual item presentation strategy. It did provehelpful when the student was confused about the focal point of a new item.After being told the answer, he would realize that he had been misinter­preting what had preceded. Since a substantial portion of the presentationis done orally, the ability to go back and listen to it again proved popular.

It is not clear how to account for the number of restarts in the tally of thestudent's performance. A single incorrect response .breaks the correct-in­sequence count in any case. By restarting after a single wrong response, thestudent could in effect gain three tries at answering correctly. No attemptwas made to discourage the use of the restart option.

Audio control-repeat: The student has the option of having the latestaudio message repeated as many times as desired. Selective repetition maywell be valuable when dealing with a foreign language and words whichmay be unfamiliar to the student. In the present experiment, the audio wasnot of the highest fidelity. The students often had to ask for repeats of theaudio messages, particularly those involving new words or unfamiliar struc­tures. A curriculum option, under control of the curriculum author, directsthe program to pause after nearly every audio message, waiting for a replayrequest from the student.

Audio control-typeout: Except during a dictation, the student can havea given audio message typed out simultaneously. While the lack of fidelitydoubtless contributed to the use of this option, many students appeared tohave difficulty visualizing unfamiliar vocabulary and often asked for atypeout. The program automatically repeats the audio after printing it inorder to reinforce aural comprehension.

Unfortunately, once the audio message has been displayed on the ter­minal, most exercises become trivial. Usually the words required for ananswer will have appeared in the audio and now only need to be copied.Many of the students indicated their awareness of this difficulty, but stillfound the typeout option necessary to clear up confusion about what theyhad heard.

6 Unlike the other requests, the "?" for answer was only available at a response point, not atthe other stopping points; also, it cannot be inhibited by the curriculum coder.

696 BARSON, SMITH, D. LEVINE, SCHOLL, & SCHOLL

3.3 Miscellaneous Requests

A number of other features of the program increase the ease of studentuse, and are necessary for an operational CAl system, but will not bediscussed here.

4. SYSTEM IMPLEMENTATION

We have already given the basic structure of the strand curriculum andthe algorithms involved in strand selection. In this section we discuss anumber of aspects of the implementation of the system including thepattern description capabilities of the curriculum authoring language, theaudio system, and the algorithms used to analyze student responses.

4.1 Pattern Description

A pattern block contains the specification of all the information necessaryfor the presentation and' analysis of a group of related items. A blockshould illustrate only one grammatical point. All its constituent items willhave the same general formulation, differing from one another in vocabu­lary. The block description contains the following details:

1. Preamble - Identification (strand, time-slice, and block number)- Number of main items.- Remediation strategy

How many tries at response (lor 2)- Tolerance level (Number of wrong responses allowed

before going to extra items.)Performance evaluation values .(Optional in case the stan­dard values of 80%, 30%, and 50% are not appropriate.)

2. Information Text of the grammar explanation.Text giving more details on the nature of the work re­quired.

3. Formulation - A series of commands to be executed sequentially to pre­sent the i.tem: Type-out, Play and Audio Message, Accept aresponse from the student. With each command must begiven the corresponding text. (A similar exact text is givento specify the correct response.) If the same text is to bepresented for each instance of the pattern, the text is writ­ten directly. If different text is required for successive in­stances, then the pattern contains variables. The entire listof substitution instances for the variables are given with thepattern.

These patterns were encoded using the INCa author language, designedand implemented for this project by Pierre Scholl.

FRENCH 697

4.2 Types of Patterns

The use of the pattern facility includes a variety of constructions andstudent/computer interaction. Often, the curriculum author tries to con­struct a "dialogue," as in the following examples.

Computer: Est-ce que vallS etudiez Ie franl;ais?("Do you study French?")

Computer: Qui, ...

Student: J'etudie Ie fran<;:ais.

Or

Computer: Dites que ju suis un professeur intelligent.("Say that I am an intelligent teacher.")

Student: Vousetes un professeur intelligent.

The availahility of the audio system increases the flexibility of the pre­sentation. Often; the instructions and stimulus are given orally. In the twoexamples given above, the lines defined as Computer are actually spokenby the audio system, except for the short prompt "Oui, ..." Viewed merelyas a succession of written words, many exercises require only simple trans­formations in the symbols, with the bulk simply copied from stimulus toresponse. The oral presentation, on the other hand, encourages the stu­dent to begin thinking in French, to absorb the stimulus as a meaningfulthought, and then produce the reply as a whole utterance.

As noted above, the student can request that any audio message beprinted as well. With a better quality audio system, printing the audiomessages only on the student's request would be quite effective. In thepresent system it represents a slight reduction in effectiveness of presenta­tion.

4.3 Audio System

The audio system in use for this project was designed by IMSSS for aprogram in elementary reading as described by Atkinson et al. (1973).Briefly, the technique uses analog to digital conversion hardware to recordvoice in digitized form and store it on a disk as digital information. In thisform the sound information can be easily manipulated by an editing pro­gram. Playback is by means of a digital to analog converter which recreatesa standard audio signal for the student's headphones. (Further discussionof this technique, its advantages and disadvantages for CAl, can be foundin Sanders, Benbassat, & Smith, 1976.)

A difficulty with the audio system somewhat specific to language instruc­tion results from the practice of assembling sentences from individualwords. Often words recorded by the same person on successive days will be

698 BARSON, SMITH, D. LEVINE, SCHOLL, & SCHOLL

noticeably different when played together in a continuous phrase. Moreserious troubles are encountered regarding voice inflection and pitch. Bymaintaining an even tone while recording, one creates sentences that arenot misleading in their inflection, but are bland aud uninteresting. Sepa­rate recordings were made for words appearing at the end of interrogativesentences to capture the necessary rising tone. In those instances in whichthe overall phrase inflection can be very important for correct comprehen­sion, an entire phrase must be recorded as a single unit.

A related problem specific to French is the liason between successivewords. If a word ending with a silent consonant precedes one beginningwith a vowel, the consonant is sounded as part of the second word: ("estELIDE un"). In each such case, the word pair has been recorded togetherand treated as a single sound unit for reproduction.

4.4 Response Error Analysis

The French program contains facilities for analyzing and correctingstudent responses under certain assumptions about the kind of errorsstudents make. This includes phonetic and spelling errors related to thetask of learning French. We describe these facilities here. The student'sanswer is first checked against the expected· one for an exact match disre­garding multiple spaces and miscellaneous punctuation. If that check fails,a word-by-word match is performed. This analysis gives two types ofdiagnosis, one relating to individual words and the other to their position inthe sentence.

Individual word match: The matching of an individual word may beeither exact, correct except for accents or misspelled, or the match may failentirely. The misspelling program works on a phonetic basis. It takes thetwo words, one the expected word, the other from the student's answer,and processes both as follows. First all accents are deleted; agreement ofthe two at this stage will be reported as a match with accent errors. Other­wise, the program deletes certain final letters usually associated with inflec­tional endings, for example, plural endings ("s", "x"), some verb endings("ent", "er"), and mute consonants ("d", "t", "p", "z"). Many letters arereplaced by phonetic equivalents, shown on the following page.

After the replacements, double letters are reduced to single ones, non­initial vowels are eliminated, and the resulting strings again compared forequivalence. If they agree, the program assumes that the student has madea spelling error, otherwise the match fails.

The misspelling algorithm is far from perfect; for example, it may fail torecognize actual misspellings (or incorrect forms) or it may confuse twowords which are actually different. In spite of these shortcomings, it doesplaya siguificant role in the response diagnosis when it functious properly.Unless a correspondence can be establisbed between the words of the ex­pected and actual response, no word order analysis will be possible.

- .

FRENCH

LETTER REPLACEMENT

699

Letter

All vowelsHK, Q, S, X, zJNWGiPhPtTi + vowelY + vowel

Its replacement

ASuppressedBecome CGMVNiFTC except if initialL + vowel

In 1973, a new misspelling algorithm was developed following a schemeused by the PLATO project (Tenczar & Golden, 1972). The new algorithmuses several different criteria, thereby avoiding reliance on any singlefeature. Word length, initial letter, letter constituency, letter pairs, andconsonant-vowel pairing are all taken into consideration. This schemeappears to function at least as well as the pure phonetic transliterationtechnique. Its behavior is not as predictable, however, and no major testshave yet taken place to determine its exact characteristics. As the newalgorithm was only used at the very end of the experiment, no extensivestudent reaction or other data are available.

Word order: After the recognition and misspelling routines have as­sociated the words of the student response with those in which the wordsappear, an elaborate algorithm compares the expected sentence with thestudent's, disregarding unrecognized words in the latter. This helps iden­tify several kinds of errors of word order including a single word out ofposition, two words in inverted order, and three words in permuted order.(Examples: Expected ~ ABCDE; one out of order = ACDEB; two invert­ed = ACBDE; three permuted = ADCBE.)

Diagnostic messages: Results of the analysis are presented to the stu­dent by retyping his response and then underlining erroneous words withvarious mnemonic codes. In evaluating the success of these techniques, wediscovered that students were not very aware of the meaning of the nota­tions even though explanation was available on-line.

4.5 Dictionary

The dictionary was prepared from word usage lists provided automati­cally by the lesson preprocessor and, thus, includes substantially the entirevocabulary of the course. Each definition includes a part of speech abbrevi­ation and a short statement of the word's meaning, usually running less

700 BARSON, SMITH, D. LEVINE, SCHOLL, & SCHOLL

than a line of printing. For irregular words, there is an extra entry for theirregular forms directing the student to the main (stem) entry. Similartreatment was given certain inflected forms which the student may en­counter before he knows enough grammar to derive the stem.

The dictionary resides on a separate disk file specially indexed forrandom access. No misspelling detection is included in the accessmechanism; the student is simply told "word not found" in the absence ofan exact match.

4.6 Operating Environment

The French program was written in SAIL (Reiser, 1976) on a DECKAIO processor operating under TENEX at IMSSS, and was used forinstruction at Stanford during 1972-1973.

5. SUGGESTIONS FOR IMPROVEMENT

This section details some weaknesses of the program and describes someof the suggestions for the further work that helped form the outlook offuture research at the Institute.

5.1 Response Analysis

A considerable amount of development is still possible in responseanalysis and the control of progression through the curriculum. Theseareas represent the main contribution of the French program.

The response analysis algorithm does not properly do the limited taskswhich it attempts. Several expansions of its functions would be of interest.As a start, the misspelling detector needs more adjustment. It does notreliably recognize many obvious errors. As a result, the students tend not topay attention to its diagnosis. They do not distinguish between the diagnos­tic underlining meaning "misspelled word" and that for "unrecognizedword". Part of the difficulty stems from the program's reliance on themisspelling algorithms to recognize words which have been improperlyinflected-a particularly severe form of misspelling. The usefulness of thespelling corrector is severely limited by its inability to deal with any contex­tual, structural, or semantic problems. For example, if the student incorrect­ly types "de Ie journal," the spelling corrector will accept the student'sinput even though the rules of French orthography require that "de Ie" becontracted to "du."

Incorporating full grammatical knowledge into a response can be anextremely complex task. (See Levine, 1981, for details of a computer-basedanalytic grading for German grammar instruction.) As a compromise, inthe context of the Erench program, the following might prove effective:

1. Recognize and check for certain common very irregular forms, like ("to be"):etre, suis, es, est, sommes, etes, and sont.

FRENCH 701

2. R{':cognize special processing situations in which contractions (apostrophes)mayoccur.

3. Build into the recognizer some knowledge of known morphological patternssuch as -aux plurals, verb endings, and so forth.

Word order and missing word checks are at the same disadvantage asthe misspeller in that no use is made of any information about grammaticalstructure. The problem is not too severe here, as mistakes will usually showup as misplaced words which the program will diagnose properly. Occa­sionallya particular student response demonstrates the pitfalls of a simplesyntactic approach to checking word order, for example:

Expected: "Georges est un homme."(George is a man.)

Student: "Un Georges est un homme."(A George is a man.)

The computer matches the first three words of the expected response withthe first three words of the student response and complains about thepermutation of word order, then it goes on to complain that the second"un" is extra! If the analyzer had any grammatical information, it might beable to associate articles with their nouns and would thus have correctlynoticed the extra article preceding "Georges".

In any case, one must be careful with diagnostic information. If it isincorrect or misleading, the student will be annoyed and tend to disregardall such messages, even when they are helpful. In particular, the programcurrently is injudicious about telling the student that a word is missing fromthe answer, producing that message when the difficulty is merely an inabil­ity to recognize a word of the student response.

5.2 Evaluation

When has the student mastered a particular concept sufficiently toadvance on that strand? The present program sometimes moves on too fastand other times dallies forever on a trivial point. Some sensitivity to thetime required- for a response might help out-as the student learns aconcept he should respond more rapidly. (Unfortunately, a poor typist mayobscure that data by his general clumsiness.) Another clue that the pro­gram currently disregards is the nature of an error. Does the mistake haveanything to do with the current grammatical concept? Although the re­sponse analyzer goes to great length to diagnose and localize errors, ofseveral different types, none of the specific information is available to theevaluation routines which simply count the number of errors. 7 A spellingmistake or error in an unrelated part of the sentence will register the sameas a failure to understand the central point.

Perhaps the student could supply some explicit information to theprogram about how he feels he is doing. For instance, the student might

7 Nor does the curriculum coding indicate to the program what words are important.

702 BARSON, SMITH, D. LEVINE, SCHOLL, & SCHOLL

type a "+" to indicate he is confident about his work or a "-" if he doesn'twant to go forward quite yet. These indications could be given at any time,rather than in response to a direct request for such information. Theywould be weighed along with the other performance indicators.

5.3 Remedial Help

The program currently incorporates various strategies to help the stu­dent who has made an error, for example, extra items and selective repeti­tion of missed items. There is, however, no second-level strategy to handledifficulties encountered on the initial remediation. If a problem is missedon its remedial presentation, it is simply brought back for a third andfourth time.

Another second-level strategy problem concerns the decision to advanceat the end of an exercise-block. Typically, a student scoring less than 80%correct will repeat those items he answered incorrectly. Often there willonly be a few such items. Sometimes, the student will fail to answer cor­rectly on the second presentation. Perhaps he understands the generalrule, but not this specific instance which accounts for a few of the mistakes;or perhaps he has trouble typing a certain letter combination. The pro­gram at present uses the original 80% - 30% criteria to evaluate the reme­dial pass through the block, but the percentages are no longer appropriatefor the small number of items considered. In particular, a student missingboth of two repeated items will then be sent back to repeat the entireexercise-block.

Perhaps explicit second-level remedial items should be coded, or maybeentire remedial exercise-blocks could be activated. The need for somespecial second-level strategy is clear, but more consideration must be givento implementation.

5.4 Review

Once an exercise block has been mastered, it is never again presented asthe program now stands. Some sort of review would be a desirable feature.One proposal is to have the curriculum writers prepare review exercises tosit on a separate strand. A more ambitious plan would have the programitself responsible for automatic review. A strategy could be set up so that allprevious material receives some coverage, with extra emphasis given toexercise-blocks that caused trouble the first time through. Further defini­tion is needed for the automatic strategy; How far back should the reviewpointers be set? Should a block be reviewed more than once? At whatintervals should review items be presented? If review takes place followingcompletion of a main exercise on a given strand, some special handling isneeded for strands which do not appear in every time period.

FRENCH 703

5.5 Multi-block Concepts

In constructing the curriculum. a conscious effort was made to limiteach exercise-block to instruction relative to a single grammatical concept.Sometimes, however, the difficulty is not in avoiding multi-concept blocksbut in coding concepts which require more than one block for properpresentation. With the present architecture, the program handles suchsituations quite poorly. The blocks will be presented sequentially just as ifthey were independent. Performance criteria (for moving on) areevaluated for each block independently. If the curriculum is such that thestudent must see material from both blocks before any evaluation is appro­priate, the lesson designer has no way of controlling the presentationproperly. He must set the progression criteria for the first block artificiallylow to ensure that the student will reach the second block, but then he hasno check on the performance of the first block's material. With the presentarchitecture, one would probably resort to a third block in order to presentagain the first block's material at a time when performance evaluation hasbecome meaningful.

5.6 Randomized Presentation

As discussed in section 3, the randomized presentation strategy has bothgood and bad points, briefly, variety and confusion. As currently perceived,the requirements on the presentation strategy are fairly complex and onlypartially met by the existing program.

At the beginning of an exercise block, the normal randomization patternshould be suppressed so that the student can determine what to do and canhave an opportunity for some practice before moving off on a differentstrand. This was done in 1973. A similar criteria applies at the beginning ofa session. 8 To preserve the beneficial aspects of movement between strands,no one strand should be idle for too long. If one is not careful, at startuptime there will be a very long interval between strands as first one. thenanother exercise-block gets its long initial control. (An exactly similar prob­lem exists. at the beginning of a new time period in the curriculum.)Perhaps a mixed strategy is indicated. Start one strand, then a second.Before going on to a third, do an item or two from the first before thestudent forgets it entirely. This scheme unfortunately increases the timerequired to bring all strands to active status.

In many cases, randomization might usefully be controlled by the cur­riculum author. If two items follow one another closely in content, the

II The computer might check the date and time of the previous session and adjust the lengthof the initial segments accordingly. In particular, a student who interrupts his session for a fewminutes should not have a full startup when he returns.

704 BARSON, SMITH, D. LEVINE, SCHOLL, & SCHOLL

program could present them without interruption. A similar- suppressionof the randomizer should occur when the student types a "?" to ask for thecorrect response. He should be given another item from the same exerciseright away while the information from the "?" is fresh in his mind.

Proper strand switching becomes a necessity when dealing with wrongresponses. If a remedial item is indicated, it should probably follow im­mediately, without intervening items from other strands. This is currentlydone, though the ranrlomizer does not even "see" the remedial item and somay tend to present too many items in a row from that strand. If part or allof an exercise block is to be repeated, however, strand switching plays avital role in separating repeated presentations of the same material. Thestudent receives the correct answer when he misses an item and will merelycopy that answer if the item reappears too soon.

5.7 Implementation Details

In 1972, at the beginning of each time period the program drew up acomplete specification of the order of presentation based on the relativenumber of items in each strand. This specification was recomputed in anexercise block that had to be repeated. The algorithm insured adequaterandomization, but was not sensitive to the various instances in whichstrand switching should be suppressed.

In 1973, the program used a simpler, dynamic computation based onthe percentage of items remaining in each strand: the strand with thehighest percentage remaining is chosen next. This scheme has the advan­tage of permitting a temporary suspension of strand switching at any pointwithout elaborate recomputation of the sequence specification. However, itdoes not provide the broad control needed for gradual startup at thebeginning of a session, as indicated above. The percentage technique alsoperforms poorly when an exercise-block is repeated. The current programmerely adds to the number of repeated items the count of those remainingand adjusts the percentage accordingly. The result, unfortunately, is toboost the percentage remaining for that strand so high that it monopolizesthe presentation for nearly the entire remedial block which is the worstpossible thing to do. It would be better, perhaps, to add the extra items butalso increase the denominator to preserve the current percentage. Thatwould force more frequent presentation of the strand in question, but overa longer period.

6. WHAT WAS LEARNED FROM THIS PROJECT

This project was important in the evolution of CAl at Stanford since itindicated both the strengths and limitations of a number of techniquespreviously designed or suggested.

-:;>'-

FRENCH 705

The importance of audioin language instruction became more obviousas a result of this project and yet the limitations of the audio system in use atthat time became apparent. Anything more complex than drill-and­practice requires the ability to synthesize dynamically whole sentencescomplete with some reasonable prosodic features. These prosodiccapabilities were not available at that time. The interest in the MISS audioeffort and research associated with it can be traced to this.

The strand curriculum construction seems to meet its natural limits inthe French program. The strand is an excellent way of organizing a cur­riculum, but provides only limited structure for diagnosing student errorsand misconceptions. It rests not only on the careful structuring and syn­chronization of that curriculum, but also on the error analysis and evalua­tion algorithms that drive the decisions about correctness of responses andthe readiness of the student to proceed to another concept.

While the major contribution of the French program is, the erroranalysis algorithms, the very success of the program in identifying a largenumber of student errors points to the need for more knowledge-basedanswer evaluation. Recent work at the Institute has emphasized this con­cern for the evaluation of student responses, for example, in the work onGerman grading by Levine (1981), and, in a quite different domain, thework on the EXCHECK system for mathematics instruction.

REFERENCES

Atkinson, R. C., Fletcher, J. D., Lindsay, E. J., Campbell, J. 0., & Barr, A.Computer-assisted instruction in initial reading (Tech. Rep. 207). Stanford, Calif.:Stanford University, Institute for Mathematical Studies in the Social Sciences,1973.

de Sauze, E. B. The Cleveland plan for the teaching rf modern languages. New York:Holt, Rinehart, & Winston, 1961.

Levine, D. R. Computer-based analytic grading for German grammar instruction.In P. Suppes (Ed.), University-level computer-assisted instruction at Stanford: 1968­1980. Stanford, Calif.: Stanford University, Institute for Mathematical Studies inthe Social Sciences, 1981.

Pucciani, 0., & Hamel, J. Langue et langage. New York: Holt Rinehart & Winston,1967.

Reiser, J. F. SAIL (Stanford Artifical Intelligence Labortory Memo AIM 289).Stanford, Calif.: Stanford University, Stanford Artificial Intelligence Labora­tory, 1976.

Sanders, W. R., Benbassat, G. V., & Smith, R. L. Speech synthesisfor computer assistedinstruction: The MISS system and its applications. Proceedings of the ACM Comput­er Science Conference, Irvine, Calif.: 1976.

Tenczar, P., & Golden, W. Spelling, word, and concept recognition (X-35). Urbana,Ill.: University of Illinois, Computer-based Education Research Laboratory,1972.

-">