Study of My Word Coach

8/13/2019 Study of My Word Coach

1/36

1

WORD COACH DOES IT COACH WORDS?

Tom Cobb

Universit du Qubec Montral

ABSTRACT

This study reports on the motivation, design, development, and testing of a

research informed video game for vocabulary learning,My Word Coach,played

on a Nintendo D or !ii and distributed commercially since late "##$% The

learning effects of the game &ere tested over four months in "##' &ith (# age)

appropriate *nglish as a second language +*- students in a Montreal school% .

battery of observational and empirical tests trac/ed e0perimental and uasi)

control groups2 le0ical development on the dimensions of meaning recognition,

free production, and speed of access% T&o months2 game use coincided &ith an

average 1#3)"#3 increase in recognition vocabulary si4e5 6#)(#3 reduction in

code)s&itching in oral productions5 and 1'3)783 increase in speed of le0ical

access% 9ncreased fluency appears to be the most interesting outcome% Questions

are raised about the importance of post)game follo& up, and suggestions made for

principled modification of the game2s learner model

INTRODUCTION & BACKGROUND

:ideo games occupy more and more of the time and attention of school age learners, &ith

an effect on learning that almost definitely e0ists but is presently un/no&n% .rguments in

principle for the learning po&er of such games are many +;ee, "##75


2/36

"

learning effects, 9 had also been a linguistic consultant for its design and development,

and first presented the game to the public at the Getting the Word Outsymposium at the

.merican .ssociation of .pplied inguistics conference in "##$% The symposium &as a

collection of pro>ects &here some part of the voluminous vocabulary research of the past

"# years had been applied to a real)&orld problem and &ould potentially help large

numbers of real learners meet their vocabulary needs%

The vocabular !roblem

!hat are the vocabulary needs of real learners? 9f *nglish users are not reuired to read,

&rite or listen to lectures, these needs may not be all that great or difficult to meet%

@esearch has sho&n that the most freuent 1### &ord families of *nglish account for

almost A#3 of the individual &ords other than proper nouns in typical conversations

+.dolphs B chmitt, "##7-% These &ords can be learned fairly easily because of their

freuent recurrence in the environment% Cut if the learners have to read, &rite, or listen to

lectures, the picture changes dramatically% @esearchers have sho&n that the minimal

vocabulary needed to read a basic academic te0t in *nglish is probably about ',### &ord

families +Nation, "##8-% This figure presents little challenge to educated adult native

spea/ers of *nglish, &ho typically /no& roughly "#,### &ord families +;oulden, Nation

B @ead, 1AA#-% Cut for 1 children, there is a notable vocabulary divide along socio)

economic lines even before school begins +=hall B a/obs, "##7- &hich only &idens

over the school years% =hildren of immigrant families are particularly at ris/, particular

those &hose uic/ acuisition of conversational ability hides a vocabulary deficit that

only appears in high school &hen literacy s/ills become crucial and remediation is

difficult +@oessingh B *lgie, "##A-% aufer2s +"###- survey of seven vocabulary si4e

studies of *E learners on three continents found that many students attempting to study

academic sub>ects through *nglish medium &ere functioning &ith an average of only

",1## +D A$$- &ord families% 9t seems uncontroversial to conclude that vast numbers of

learners of *nglish both as 1 and " are several thousand &ords short of a functional


3/36

7

le0icon in the tas/s they have set for themselves% There is clearly a place in this &orld for

a cheap, principled, and effective means of rapid vocabulary e0pansion%

A role "or #ech$olo%

The idea that a computer)based training system could be a good &ay to meet the

vocabulary instruction challenge has been around for some time% The large amount of

material to be covered, the li/elihood of strong individual differences in both goals and

learning rates, and the need for recycling and record)/eeping &ere identified early on as

reasons to loo/ to computeri4ed instruction +e%g%, .t/inson, 1A$"-% @ecent developments

add strength to the argumentF increased capacity allo&s for the provision of

concordances or glossaries as learning tools, increased processing speed allo&s for fine)

tuned control of procedural interactions, and the advent of net&or/s frees the learner

from particular times or places of learning% The still more recent development of game)

playing devices carries the potential benefits of a computational medium to a logical

conclusion%


4/36

6

driven, resource)informed systems +=obb, "##A-% =urrent development &or/ involves

e0panding each of these approaches via game activities% Cased on


5/36

(

met% The games focus on one or more of the follo&ing s/ill areas as mentioned above J

form, form)meaning connection, and le0ical access%

The user loads Word Coachinto the Nintendo console, enters his or her name +four can

play on one machine, plus guests, and many more &irelessly- and begins to play the one

game available at this point, &hich is effectively a placement test, a sample item from

&hich appears in the leftmost screen of Eigure 1% The testing starts by pitching &ord sets

from a medium freuency level +the 6### &ord level-5 if these &ords are not /no&n, the

level goes do&n until a 4one containing at least 7#3 un/no&n &ords is determined +or if

mainly /no&n, the level goes up-% 9n this &ay, players are challenged at their o&n level%

More games are introduced as play proceeds% There are t&o form)based games5 Missing

etter, &hich involves using the stylus to supply the letter missing from a &ord +e%g%,

new_paper, see Eig% ", left screen-, and Cloc/ etters, &hich involves clic/ing on falling

letters to form one of four given &ords as unused letters pile up to&ard an e0plosion +in a

version of Tetris, see Eig% ", middle screen-% Eour of the games involve connecting &ords

and simplified " glosses,in various combinations of both receptive and productive

tas/s% 9n plit Decision, the upper screen displays a definition &hile the lo&er screen

presents &ords that players toggle through until they identify the one that matches the

definition% 9n !ord huffle a &ord from the bottom screen is dragged and dropped on

one of four definitions in the top screen% 9n


6/36


7/36

$

Eigure "F !ord =oach2s form)based games and session feedbac/ +7 split screens-

Eigure 7F !ord =oach2s form and meaning games +7 split screens-

RE+IEW O, THE -ITERATURE

Eollo&ing on this introduction to the game itself, &e turn to a revie& of the relevant

research in three parts ) the research behind the game components, the research behind

the learning choices, and the research precedents for vocabulary and literacy training

games%

Reearch beh'$) #he %ame. com!o$e$#


8/36

'

=ontent% nly relatively recently has it become possible to specify &hat the basic non)

specialist le0icon of *nglish consists in% This specification follo&s from three connected

research pro>ects over a 1#)year periodF the assembly of the 1## million &ord Critish

National =orpus +0ford =omputing ervices, "##(-5 its brea/do&n into a list organi4ed

by freuency and range +eech et al, "##1-5 and the pedagogical adaptation of the lists by

Nation, &hich involved recombining items from the &ritten and spo/en sections to Kmore

closely represented the order Oof learningP the &ords +Nation B Ceglar, "##$, p% 11- and

organi4ing them into 16 family)grouped thousand lists +Cauer B Nation, 1AA75 Nation,

"##8-% @andom items from the resulting lists are sho&n in Table 1 by &ay of illustration%

. final adaption of the lists for !ord =oach purposes &as a ran/ing of each list into

more and less difficult spellings according to number of syllables +si0 in

unreasonableness-, number of double consonants +t&o pairs in committed-, presence of

triple vo&els +agreeable-, and /no&n spelling conundrums +li/e admissible-, in order to

provide seuences according to difficulty for the form)oriented games +see

le0tutor%caHspellingHbncIconundrumsH-%

Table 1% !ords at five CN= freuency levels

Eirst 1### Third 1### Eifth 1### *ighth 1### Tenth 1###held

transport

point

lighten

degree

line

understand

high&ay

forty

sale

steam

adapt

stream

fiddle

urge

cheat

clip

trivial

polite

heal

diagnose

minimal

deer

gloomy

void

spine

captive

glossary

ra4or

&indscreen

garlic

bac/drop

mai4e

fret

draughtsman

bipolar

caption

tingle

moron

staunch

hairspray

beehive

vestry

into0icate

ban/note

deliverance

clang

fallible

temperance

disservice

.lso relatively recent is the publication of learner dictionaries &ith simplified definitions,

follo&ing the principle that as fe& &ords as possible in the definition should be less

common than the head&ord itself +as &as al&ays the case &ith the format Ka car is a

vehicle &hich-% The Cambridge Advanced Learners ictionary +"##(- &as used in
http://lextutor.ca/spelling/bnc_conundrums/http://lextutor.ca/spelling/bnc_conundrums/


9/36

A

!ord =oach mainly o&ing to its definitional format and adeuate number of entries%

Many of the definitions &ere further simplified by research assistants to match the screen

si4e available and to be comprehensible to as many learners as possible%


10/36

1#

The principles of the hand computer &ere programmed into !ord =oach e0actly as

presented in Mondria et al, such that all game &ords even if previously /no&n are

recycled at least five times, &hile un/no&n or difficult &ords are recycled possibly a

do4en or more times and are not dropped from the game +Ksucceeded, in the designers2

vernacular- until they have made it through five correct trials at increasing intervals%

;ames, learning, and motivation% The basic cognitive operations that promote initial

&ord retention and learning +moving &ords from &or/ing to long)term memory- are

fairly &ell /no&n% These are operations that reuire decisions about &ords, including

rehearsal, repetition, recall, retrieval, and use, in conditions of attention and arousal, as

discussed in Thornbury +"##", p% "6-% .ll of these operations are fairly straightfor&ard to

embody in a game format +apart from Kuse in the type of games &e are discussing-%

*ach of !ord =oach2s games gives learners practice in one or more of the basic

operations of &ord learning, from rehearsal of form, recall of form cued by meaning, to

recall of meaning cued by form, and all of them give learners motivation and opportunity

to increase their speed of le0ical access%

Reearch beh'$) #he lear$'$% cho'ce

!ord =oach is based on a classic paired associate learning model +&ords and glosses-,

&ith some e0tra attention paid to processing speed and forms of &ords% .s argued by

Nation +e%g%, "##1, pp% "A8)718-, paired)associate learning is +a- the learning model &e

/no& most about and +b- the best &ay to meet the goal of establishing a critical mass of

vocabulary as early as possible in language learning% The learning approach can also be

positioned on a number of schemes from the more recent research literature% 9n terms of

incidental or deliberate &ord learning +Nation, "##'-, the approach here is clearly

deliberate% 9n terms of conte0tual vs% definitional +=obb, 1AAA5


11/36

11

forms +aufer, "##8-, the approach is clearly focus on forms%


12/36

1"

le0icon- as &ell as being less amenable to e0plicit learning, as suggested by *llis +1AA6-

and Gulsti>n +"##"-% The games that give learners practice &ith forms alone are Missing

etter and Cloc/ !ords, and these are designed to &or/ in con>unction &ith form)

meaning games to establish the basis for high uality entries in the mental le0icon

+n, "##(-% 9t is often argued that

le0ical access speed li/e other implicit or procedural /no&ledge types can only be built

up over thousands of hours of e0posure to a language +e%g%, *llis, 1AA6-, but some recent

laboratory studies by nellings et al+"##"- and *lgort +"##$- suggests that le0ical access

may in fact be trainable using activities built &ith research soft&are ) and replicable in

principle on a game console such as My !ord =oach%

Third, on the matter of " definitions, a training study by *lgort +"##$- appears to sho&

that le0ical acuisition at all levels including the implicit level can be achieved through

e0plicit paired)associate training using " glosses% he gave advanced * learners

practice in acuiring 6' *nglish)li/e n, "##(-% *lgort e0plained these results in terms of iang2s +"##6-


13/36

17

semantic transfer model of adult " vocabulary acuisition% iang2s model postulates that

ne& " &ords are almost inevitably associated &ith old 1 concepts for e0tensive

periods, or forever, even if the &ord is learned through conte0tual inference, unless steps

are ta/en to prevent the association from forming% This can apparently be achieved

through focus on an " definition through several rehearsals, provided the definition is

readable, &hich is &hy as mentioned above that the game definitions &ere doubly

simplified%

(rev'ou reearch o$ %ame "or vocabular a$) l'#erac

.s mentioned above, the claims for learning from video games are many and the

empirical studies fe&% There are some good reasons for the lac/ of studies% :ideo games

are played on special machines that, unli/e computers, are not supplied by schools, so

that getting an e0perimental group together is difficult% .lso, fe& such games attempt to

teach school sub>ects, so school authorities might be reluctant to donate their learners2

time for purposes they see as unclear%

*ven &hen literacy instruction is the presumed ob>ective +as can be inferred from the title

of ;ee2s, "##7, What video games have to teach us about learning and literacy-,

empirical evidence for learning from games is still thin% This is because literacy is often

construed in some special &ay in this literature ) critical literacy, ne& literacy, and game

playing as a literacy, in ;ee2s te0t ) &hich does not generate falsifiable propositions%

!hat is the test that could establish that reading and &riting should not be vie&ed only

as mental achievements going on inside peoples heads but also as social and cultural

practices &ith economic, historical, and political implications +p% '-? :ideo game

research, especially for games in the narrative, immersive, and Massive Multiplayer

nline modes, appears to be at the stage of frame&or/ building and uestion forming, or,

as the editors of a recent gaming issue of!ducational "echnology #esearch and

evelopment+pector B @oss, "##', p% (1#- put it, is Kin its infancy% ;ee2s student

tein/uehler +"##'- describes the current games research agenda in this &ayF K.s part of


14/36

16

this success Oof ma/ing games and learning a serious topic on the intellectual and social

agendaP, &e have begun to move beyond mere plausibility arguments and are poised to

start as/ing more focused and empirically driven research uestions +p% "(1, emphasis

added-% .pparently &e must &ait a &hile for anything definite about learning from this

sort of gaming%

There is a legitimate uestion about K!hat is a game? The &ord &as !ittgenstein2s

+1A(7- classic e0ample of a concept &ith almost no common features across instances%

The games under discussion in the present study are clearly much simpler than the

multiplayer versions mentioned above% Go&ever, even simpler video games &ith clear

and even curricular ob>ectives have not generated strong research as yet% . study

referenced by


15/36

1(

1% !hat is the e0tent of game use, and are there any game or learner characteristics

that predict game use?

"% Go& many &ords are /no&n receptively before and after game use?

7% Go& many learned &ords are used productively after game use?

6% 9s there a difference in speed of le0ical access before and after game use?

-ear$er a$) Se##'$%

The learners &ho participated in this study &ere t&o intact classes of "( ;rade 8

Erancophone * learners in a middle)class suburban school in Montreal% The ethnicity

of both groups &as roughly 7#3 Quebec Erancophone children and $#3 immigrant

children mainly from Erancophone countries% The medium of instruction at the school

&as Erench, e0cept for t&o hours per &ee/ of *, &here the teacher used *nglish as

much as possible follo&ing a communicative language teaching approach% The children

had various amounts of *nglish e0posure out of class, from e0tensive to none, and &idely

varying levels of *nglish proficiency% The teacher reported that the parents of these

children had complained that non)educational video games &ere eating into their

children2s home&or/ time and that they &ere more than &illing to try the educational

variety +A'3 of parents supported this e0periment-% The school supported the research,

&hich too/ place over a four month period in the spring of "##$% n receiving the game,

each group received (# minutes training and roughly eual encouragement during *

class to use the game%

The role o" #he %am'$% com!a$

Ubisoft 9nc% of Montreal provided 8# Nintendo D players andMy Word Coachgame

dis/s, &ithout &hich the study could not have ta/en place% Cy mutual agreement &ith

Ubisoft, no conditions &ere placed on the /inds of results e0pected or on ho& or &here

these &ould be publici4ed%


16/36

18

Reearch De'%$

Cecause of the school2s reuirements that all children have a chance to use the game, and

that the research groups be intact classes, a pre)e0perimental, &ithin)sub>ects design &as

chosen for this study% ne group used the game for t&o months &hile the other served as

uasi)control, and then the roles &ere reversed% The same &ord /no&ledge tests &ere

administered to both groups at the beginning of the e0periment, at the changeover point,

and at the end of the four months% Eor learners &ho received the game first, the seuence

&as as follo&sF pre)test, t&o months &ith the game, post)test 1, t&o months of normal

classes &ithout the game, and post)test "% +The second post)test served as a measure of

delayed retention%- Eor learners &ho received the game second, the seuence &as pre)test

1, t&o months &ithout the game, pre)test ", t&o months &ith the game, and a post)test%

+The t&o pre)tests serves as a baseline for normal le0ical development against &hich

game inspired development could be compared%- The design is sho&n schematically in

Table "%

Table "F Diagram of research design

T1 T" T7

Eirst game

group


17/36

1$

game period% .ll measures &ere compared at the three test points using basic .N:.s,

since the usual arrivals and absences of an intact setting created slightly uneual groups

and made a repeated measure impossible%

Reearch I$#rume$#

The research instruments &ere as follo&sF

1% The game itself provides detailed trac/ing of &hich &ords &ere played and ho&

often, as &ell as session si4e and freuency +Eig 1-%

"% The recognition /no&ledge measure is Nation and Ceglar2s +"##$- revision of the

classic :ocabulary evels Test +Nation, 1AA#-% The levels for this version of the

test are sampled from the first fourteen CN= freuency lists, as elaborated by

Nation, and thus correspond precisely to the content of the game% The test

measures only recognition /no&ledge, in that the test ta/er is not as/ed to

produce the &ord or its meaning but merely to match the underlined &ord in a

short non)defining conte0t to one of five short glosses +Eigure 6-% The test glosses

are not the same as the glosses encountered playing !ord =oach e0cept

coincidentally%

3. PERIOD: It was a difficult period.a. questionb. timec. thing to do

d. book

Eigure 6F evels Test +"##$- format

There are ten test uestions at each 1###)family level, such that the score

multiplied by 1## gives an estimate of the number of &ord families /no&n at that

level +eight out of 1# suggests '## families /no&n-% Nation and Ceglar +"##$-

discuss the test2s sampling and reliability% 9n this e0periment only the first 1#

levels of the test &ere administered, in vie& of the learners2 predicted level and

institutional time constraints% ince only one validated version of the test e0isted


18/36

1'

at the time of the e0periment, the same test &as used &ith each learner three

times%

7% The production measure &as an oral telling of the "( line dra&ings of Mayer2s

+1A8$- &ordless storyA %oy, a og, and a &rogas told +untimed- to a research

assistant and recorded% The recordings &ere transcribed as te0t files for processing

by the CN= version of :ocabprofile +httpFHH&&&%le0tutor%caHvpHbncH-% This

program categori4es each &ord of an input te0t by 1### level, according to

Nation2s famili4ed lists as described above, resulting in a freuency profile of the

learner2s production +the percentage of &ord families, types, and to/ens at each

1### level-%

6% e0ical access speed &as measured &ith a simple instrument developed by

UN*= for literacy testing in developing countries% 9t is simply a list of 8#

&ords in order of decreasing freuency and increasing length +all &ithin the first

1### freuency 4one-% The learner is as/ed to read the list aloud to a research

assistant for one minute as uic/ly as possible% The assistant notes the last &ord

reached &ithin the time and stri/es out any mispronunciations that appear to

indicate unfamiliarity &ith the &ord, resulting in a tally of &ords read correctly in

one minute%

.ll four measures &ere administered in one hour in each of the three testing periods% 9n

addition, the teacher solicited &ritten comments using a uestionnaire format%

RESU-TS

Game ue

.ccording to Word Coach2s tally of K&ords succeeded +in the game2s vernacular- the

amount of game use in each of the t&o)month periods &as e0tensive and similar bet&een

the groups, although &ith considerable variance among individuals% ;roup 1 players
http://www.lextutor.ca/vp/bnc/http://www.lextutor.ca/vp/bnc/


19/36

1A

succeeded an average of "'6A &ords +DW1'$A-, &hile ;roup " succeeded "(78 &ords

+D 1A(A-, &ith the difference bet&een means not statistically significant +pW%('-% The

standard deviations are high, sho&ing that some too/ to the game more than others +range

for &ords succeeded is 86## do&n to "$"-%

These numbers translate into days and hours as follo&sF .t the high end, there &ere si0

players in each group &ho succeeded at over 6### &ords in 8# days, &hich at an

estimated average of eight appearances of each &ord needed to succeed it, totals roughly

7",### &ords played, or >ust over (## &ords per day, or about "( game sets of "# &ords

apiece% .t five minutes per game this &ould amount to roughly t&o hours of play per

day, or a total of 1"# hours over t&o months% .t the lo& end, there &ere seven in each

group &ho succeeded at fe&er than 1### &ords +',### recurrences, 176 &ords per day,

si0 or seven game sets-% .t five minutes per game, this amounts to about 7# minutes per

day, or total of 7# hours over t&o months%

*ea$'$% reco%$'#'o$

Coth groups too/ the first ten 1### levels of Nation2s +"##$- CN=)based evels Test of

meaning recognition at all three testing points% 9t does not appear that the learners had

learned the test +as opposed to its content- to any degree, despite ta/ing the same 1##)

item test, since the T1)T" scores for ;roup " &hen they did not use the game are not

significantly different%

The pre)test results at T1 &ere similar bet&een groups +sho&n in Eigure ( and Table 7-,

&ith the appearance of a slight advantage for the first game group +not significant for any

single level belo& the fifth, or for the first five levels ta/en together-% ome&hat

surprising in the results is the roughly eual numbers of &ords /no&n across the second

through fifth levels5 it is more normal to see a decline as the &ords become less freuent%


20/36

"#

Cecause the scores drop sharply after the fifth level, and because the first (### &ords are

a sufficient immediate goal for these learners, it &as decided to calculate learning gains

on the basis of only the first (### &ord families% The mean number of &ords /no&n from

the first (,### &as ",6"' +D $$1- for the first group, and ","78 +D '$7- for the second,

leaving a comfortable learning space of at least "(## &ords%

Eigure (F !ord families /no&n at Time 1

Table 7F Mean evels Test scores for first 1#,### &ord families at T1 +D-

S1 S" S7 S6 S( S8 S$ S' SA S1#

1stgame group 8%77

+1%(1-

(%#7

+"%7A-

6%17

+"%1$-

6%A#

+1%86-

6%'$

+1%A7-

"%$7

+"%#8-

"%A7

+1%($-

"%'#

+"%#$-

1%$#

+1%(7-

1%6$

+1%8$-

'ndgame

group

8%##

+"%#6-

6%78

+"%(7-

7%A8

+"%68-

6%66

+1%'8-

7%8#

+"%86-

"%##

+1%A6-

1%86

+1%6A-

1%(8

+1%$"-

#%'6

+1%#(-

#%(8

+#%A#-

Eollo&ing game play, learners in both groups acuired ne& &ords across the first five

levels of the test, more or less eually, as sho&n in Eigure 8 for the first group +the group

for &hich pre, post and delayed post scores are available in the present design-% Table 6

sho&s the number of &ords learned across five levels by both groups%


21/36

"1

Eigure 8F Mean evels Test scores by level for ;roup 1 at three times

Table 6F !ord families from (### /no&n at three times Xgain, +D-

Time 1 Time " Time 7

Eirst game group "6"'+$$1-

"("$+$7A-XAA

"A7"YY

+$#"-X6#(

$econd game

group

""78+'$7-

"77"+$$A-X"A

"((8Y+'7'-X""6

YY pZ%##15 YpZ%#15 underlining indicates game periods

The results at T7 sho& significant and e0tensive gro&th of &ord /no&ledge but in a

pattern that &ould have been difficult to predict% Eor the first game group, the average

number of meanings recogni4ed had increased by only AA immediately after the game

+n%s%d-, but then at a delay of t&o months increased by a further 6#( &ords +EWA%1A,

pZ%##1-, for a total of (#6 &ords over the four)month period, or a gain of "#%$3 &ith

respect to the starting /no&ledge of "6"' &ords% Eor the second group, the average

number of meanings recogni4ed had increased by only "A &ords through t&o months2

normal classroom e0perience +n%s%d%-, but follo&ing the game had increased by ""6 &ords

+significant at T1)T7, EW(%11, pZ%#1-, a gain of 1#3 &ith respect to the starting

/no&ledge of ","78 average &ords%


22/36

""

,ro% S#or'e 12or) '$ ue3

The story accounts &ere transcribed by research assistants, spell)chec/ed manually,

stripped of immediately repeated &ords and phrases, and run through :ocabprofile)CN=

both as corpora and individually% The uestion as/ed of the corpus &as &hether any of

the 7/, 6/, and (/ &ords that had been learned for meaning recognition &ould sho& up in

the tellings% 9f this &as not the case, it &ould not necessarily indicate that no learning had

ta/en place, since the transformation of receptive into productive /no&ledge is rarely

instantaneous +aufer, 1AA'-% The :


23/36

"7

Eam Typ To/ Eam

Typ To/ Eam

Typ To/

Eirst game group 6(+""-

(1+"8

-

18$+'$

-

6'+"1-X7

(6+"8-

X7

"#$YY

+A#-

X6#

66+"1-)6

(#+"8-

)6

1$A+$(-

)1(econd game group 6#

+17-6(+16

-

1($+6"

-

7'+18-)"

61+1A-)6

1(A+("-X"

66+1$-X7

6'+"1-

X"

1$A+8#-

X11

YYpZ%#15 off)list items are not included

Table ( sho&s :ocabprofile2s family, type, and to/en counts for the t&o groups at three

times% There are an average three additional &ord families for each group follo&ing the

game periods +at T" for ;roup 1, T7 for ;roup "-, but these gains are statistically

significant only at the pZ%1# criterion in both cases% There &ere ho&ever large increases

in the number of total &ords used to tell the stories, as represented by the to/en counts, at

T" for ;roup 1 +6# mean increase from an original mean of 18$, about "(3, EW(%$',

pZ%#1- and T7 for ;roup " +"# mean increase from an original 1(A at T", about 1"3 but

not significant-% There are declines for all units in repeated re)tellings follo&ing non)

game periods, &hether preceding game use +;roup " at T"- or follo&ing +;roup 1 at T7-%

-e4'cal acce !ee)

Coth groups made small gains in le0ical access in the non)game periods +presumably a

practice effect- and strong gains follo&ing game periods% Erom a similar starting point of

(6%A average &ords read correctly in one minute for ;roup 1 and 87%8 &ords for ;roup "

+n%s%d%-, ;roup 1 increased the number by 1A%( &ords or 783 from T1)T" and another 8%A

by T7 +EW"#%'", pZ%###1-% ;roup " increased its &ords read by an average '%" at T1)T",

and by 1"%" more &ords or 1'3 increase follo&ing the game period T")T7 +EW6%"A,

pZ%#(-% This information is summari4ed in Table $%

Table $F Number of error)free &ords read in 1 minute at three times +D-+Xgain-


24/36

"6

Time 1 Time " Time 7

Eirst game group (6%A+11%6-

$6%(YY+"$%A-X1A%(

'1%6Y+7#%6-X8%A

$econd gamegroup

87%8+"#%6-

$1%'+"(%"-X'%"

'6%#Y+"'%"-X1"%"

YY pZ%#1 YpZ%#(5 underlining indicates game period

S#u)e$# 5ue#'o$$a're

.fter each game period, the teacher too/ &ritten reports from the students on their

e0perience of the game% Most students e0pressed satisfaction &ith the game, seeming to

prefer the games &ith less to read +single &ords rather than definitions- and faster action

+li/e Cloc/ etters-% ne recurring comment stands out, that the players perceived the

systematic reappearance of &ords that &ere either ne& or had been involved in an error

as Kboring% o much for the careful recycling algorithm J and a reminder that there are

probably limits to the possibilities of edutainment%

(re)'c#or o" %ame ue

;iven the fact of very heavy and very light users, &ere there any obvious learner

characteristics that &ould predict game use? The first guess is that some learners might

not have enough *nglish to get started &ith the games% The participants &ere ran/

ordered by number of &ords succeeded and then divided into t&o groupings, those &ho

had succeeded in more than "### game &ords +86#7 do&n to "#'#-, and those &ho had

succeeded in fe&er than "### +1A8' do&n to 6(#-% The number of &ords succeeded &as

then compared to performance on the different levels of the evels Test at T1% The only

interesting relationship &as bet&een &ords succeeded and scores at the first 1### level%


25/36

"(

f the heavy game users, $#3 had 1/ evels Test scores of 8#3 to $#3, suggesting they

probably /ne& about 8## or $## of the first thousand &ords of *nglish% f the light

users, $(3 had scores that &ere either belo& 6## &ords or else above '## &ords% 9n

other &ords, too much or too little /no&ledge of the first 1### &ords predicted light

game use5 a moderate /no&ledge predicted heavy game use%

;iven the crucial role of the first 1### &ords in *nglish at large +comprising at least $#3

of an average te0t and far more of an average conversation- or !ord =oach in particular

+it is the language of all the game2s instructions, feedbac/, definitions, and tutorial

messages-, a /no&ledge of only 7## or 6## &ords in this 4one &ould reasonably predict a

tough time ma/ing sense of the game% =onversely, a score of over '## may have given

learners the impression they &ere already good at *nglish +in the conte0t of their school

and neighbourhood- and had little to learn%

DISCUSSION

. pattern to notice across the measures is that &hile the results are in the same direction

for the t&o groups &ith respect to the game periods, they are larger and more consistently

significant for the first group that used the game despite their eual starting points and

amount of game use% 9t is probably safe to say there &as an e0citement to the first run that

had dissipated slightly &hen the game had become habitual% .nother pattern is that

fluency gains +reaction speed, access to *nglish &ords rather than code s&itches- &hile

strong appear to &ane as soon as the game period is over% ectives%

Wor) lear$e)


26/36

"8

Cased on the performance of the t&o groups of learners, it appears that an average of

bet&een ""6 and (#6 of the &ords presumably met largely on !ord =oach &ere at least

remembered and understood, in that they could be matched to a definition on the evels

Test that &as not simply the one originally learned in the game% This &as an e0pansion of

vocabulary si4e of about 1#3 and "#3 respectively +giving some credibility to the

finding cited by ective of matching

game &ords to player levels%

The importance of these gains should be assessed against t&o bac/drops, +1- the mean

numbers of game &ords that had been through a full recycling +succeeded-, and +"- the

normal vocabulary gro&th that occurs &ith classroom learning% The average number of

&ords that had been succeeded &as roughly "(## and "'## &ords for the t&o groups,

such that (#6 &ords or >ust under "#3 of succeeded &ords &ere remembered by the first

group, and ""6 &ords or >ust under 1#3 &ere remembered for the second group%

The typical or habitual vocabulary gro&th for learners in Quebec schools has never been

measured, to my /no&ledge% The second group2s "A average &ords T1)T" is probably a

good start to&ard building a baseline% The average gro&th e0perienced in this period

appears plausible, in that "A &ords e0trapolates to "A# for a 1#)month school year and

about "7## for eight years of school *, and this tallies &ith studies &ith comparable

but older Quebec learners +Rahar et al, "##1-%

9t is interesting that the strongest /no&ledge gains +;roup 12s 6#( &ords T")T7- &ere

registered not right after the game period but at a t&o)month delay% Delayed appearance

of le0ical gro&th is not unprecedented +=obb, 1AAA- and may even be the norm

+unac/no&ledged o&ing to the general lac/ of delayed posttest measures in vocabulary

acuisition studies, as discussed in chmitt, in press-% 9t ma/es sense that &ords learned

inside one2s gro&th area are encountered again &ithin a short period and that further

consolidation and de)conte0tuali4ation ta/es place%


27/36

"$

The freuency levels of the &ords learned deserves comment% @esults sho& the

participants had learned &ords across the first five freuency levels, but not to the $#3

criterion at any level +Eig% (-% ;iven the high coverage of the first and second thousand

&ords +Nation, "##1-, it &ould seem an important element in any designed approach to

vocabulary gro&th to fill these levels to at least '#3 /no&n items +achieving &hat

;rabe, "##A, p% 7"( calls a Kcritical mass of le0is- before moving the focus to less

freuent items% !ord =oach2s placement test may have failed to do this because it

assumed that learners entering the game had been learning &ords up to then roughly in

order of freuency% 9f this &as the case, then once a level had been identified &ith (#)

$#3 of &ords /no&n, this &ould be set as the &or/ing level% These learners, ho&ever,

had several such gro&th 4ones J their mean percentages &ere 6#)(#3 at T1 right across

the second through fifth levels +Eig% (-, ma/ing any of them plausible candidates for

learning 4one% 9t is li/ely the learning algorithms &ere confused by these uneven profiles,

and in their ongoing assessment /ept assigning learners to ne& levels, &here indeed the

learners /ept on pic/ing up ne& &ords%

Wor) '$ ue

ral production results sho& that &hile only a fe& ne& &ord families &ent into

immediate use in the Erog tories, the stories nonetheless changed on other units% tory

si4e gre& significantly follo&ing game use, but &here did these e0tra &ords come from ?

The presence of e0tra &ord to/ens indicates some additional elaboration of incidents in

the story +not >ust repetitions as these had been removed-% Cut it also indicates a strong

but unanticipated reduction in code s&itching, i%e% recourse to Erench &ords to flesh out

the accounts% The off)list component of the :ocabprofiles &as mainly 1 +Erench- &ords,

and this component dropped by ("3 for ;roup 1 +EW'%A$, pZ%#1 T1)T", pZ%#( T1)T7-

and 6#3 for ;roup " +pW%#8-% The non)game periods produced a pattern similar to the

one &e have seen before, for ;roup 1 some loss of gains at T7, and for ;roup " a T1)T"

baseline sho&ing little movement%


28/36

"'

Table 8F n +"##(- found that ostensible bilinguals in Montreal often had

slo&er and more variant le0ical access times for common &ords than native spea/ers,

suggesting the use of attention)demanding rather than automati4ed le0ical processing in

reading% peed)up is not the same as automaticity, but it is presumably a step along the

&ay, and *lgort2s +"##$- strong finding for a similar training regime suggests that

gaming and le0ical access is a line &orth further development%

CONC-USION & ,UTURE

9s !ord =oach or something li/e it a solution to the problem &e started &ith, that many

learners are in serious need of a means of rapid vocabulary e0pansion? The results

presented above suggest it is a definite step along the &ay% 9n terms of ra& learning


29/36

"A

po&er, an interesting game set attached to the currently available freuency lists seems a

clear alternative or supplement to the slo& accumulation of a le0icon through e0perience

and chance encounter% Cet&een 1#3 and "#3 of fully played &ords seem to have been

remembered and transferred, at least as far as being attached to a novel definition% 9n the

case of the heaviest ten players in the present study +over 8### &ords fully recycled-, this

could amount to as many as 1"## reasonably stable acuisitions in a couple of months

&ith corresponding gains in accessibility% uch an augmentation could ma/e a big

difference to an academic learner2s le0icon, say from 1'## to 7### /no&n &ord families

+or, from about $#3 to A#3 coverage in average academic te0ts- provided the &ords

&ere the right ones% !as that the case here?

These learners &ere pic/ing up &ords from across the first five 1###)levels, meaning that

they &ould largely not e0perience a coverage gain in &hat they &ere li/ely to be reading

in *nglish at school or else&here% The ma>ority of school te0ts or even home ne&spapers

they &ould be reading comprise mainly &ords from the first t&o 1### levels, and at these

levels their gains &ere minimal% !ith $#3 of first 1### items and 8#3 of second /no&n

after game use +Eig% (-, a simplified reader still presents roughly 1(3 un/no&n le0ical

items, as can be calculated at httpFHHle0tutor%caHclo4eHvpH%!hat this means is that despite

/no&ing "(## and more &ords across the first five levels, these learners could still find

their school materials heavy going% !hat it also means is that some of the gains at the

lo&er freuency levels &ould probably be lost &ithin a fe& months through lac/ of

further reinforcement and learning from the environment%

The /ey to better targeting of game content lies in the design of !ord =oach2s placement

and progress tests, or learner modeling% .t present, the game begins by testing a player2s

&ord /no&ledge at a medium level +fourth thousand &ords li/e rigged, sigh, inhale,and

poach- and &or/s do&n +third thousand, second thousand- until it finds a level &ith (#)

$#3 of &ords /no&n% The game assumes this to be the primary gro&th 4one, since the

level belo& that is presumably largely /no&n items% Cehind this choice, ho&ever, is the

further assumption that le0icons gro& more or less linearly according to freuency% This

appears to be the case in 1 le0icon building, as research by Ciemiller and lonim +"##1-
http://lextutor.ca/cloze/vp/http://lextutor.ca/cloze/vp/http://lextutor.ca/cloze/vp/


30/36

7#

sho&s% 9t cannot be assumed in ", ho&ever, &here many factors +li/e accessibility of

lo&er freuency cognates and transfer of items from technical domains- could encourage

more top)heavy constructions% Eor many of the learners in this study, game play &ould

have begun at the third or even fourth 1###)level, &hen in fact there &ere further gro&th

4ones &ell belo& that, and more useful ones in terms of the greater coverage afforded by

higher freuency items and the greater gaps caused by not /no&ing them%

The alternative approach to learner modeling &ould be to start lo& and &or/ up, as doing

this &ould catch the most basic level of &ea/ness% 9t could, ho&ever, compromise the

allure of the game for 1 players to start &ith a test of basic &ords li/e meet, speed,

touch, andgrow)9t is a uestion of &hether a single game can serve both 1 and "

needs% 9t probably can, if the possibility of mi0ed profiles is incorporated into the learner

modeling process%

!or/ing at a less)than)ideal level does not, ho&ever, appear to affect implicit or

procedural level gains% Most of the &ords being played and learned &ere at the third

through fifth 1### levels, yet the le0ical access speed gains &ere achieved &ith first

1###)level &ords, and the e0tra &ord to/ens that replaced Erench &ords in the Erog

stories &ere also first)1### items% 9n other &ords, it appears that retrieving and rehearsing

medium freuency &ords under time pressure increases the access of high)freuency

items in the player2s true gro&th 4one, or 4one of productive ability% There appears to be

a generali4ed le0ical processing effect in the " that does not necessarily apply to the

e0act &ords played%

There is thus much more &or/ to do &ith this type of &ord game, and follo&ing the

resolution of the issues >ust mentioned still more testing is needed to determine &hether

learned game &ords are retained long term, &hether and &hen they go into active use,

and &hether fluency gains on the implicit level are permanent% The sense of previous

research &ould suggest that production and deeper comprehension depend on meeting

&ords not >ust matched to a gloss and recycled 1# times, although that is a reliable start,

but also in meaningful and varied conte0ts and situations% This of course could also


31/36

71

happen in a game conte0t% The challenge follo&ing the successful revision and further

testing of !ord =oach is the integration of vocabulary design into a more immersive,

narrative game format%

RE,ERENCES

.dolphs, %, B chmitt, N% +"##7-% e0ical coverage of spo/en discourse%Applied

Linguistics, '*+6-, 6"(J67'%

.t/inson, @%=% +1A$"-% ptimi4ing the earning of a econd)anguage :ocabulary%

+ournal o( !perimental -sychology, ./+1-, 1"6)1"A%

Cauer, %, B Nation, 9%%


32/36

7"

=obb, T% +1AA$-% Erom concord to le0iconF Development and test of a corpus)based

le0ical tutor% Unpublished


33/36

77

;oulden, @%, Nation, n, %G% +"##"-% To&ards a unified account of the representation, processing and

acuisition of second language /no&ledge% $econd Language #esearch, l


34/36

76

Lightspan #eport on Goodman !lementary $chool in Aldine, ">)+"##7-% @etrieved from

httpFHH&&&%plato%comHmediaH*valuation3"#tudiesH@H@alph3"#;%3"#;oodman

3"#*lementary%pdf

Mayer, M% +1A8$-%A boy, a dog, and a (rog)Ne& Lor/F


35/36

7(

0ford University =omputing ervices +"##(-% "he %ritish =ational Corpus) 9nformation

at httpFHH&&&%natcorp%o0%ac%u/


36/36

78

chmitt, N%, chmitt, D%, B =lapham, =% +"##1-% Developing and e0ploring the behaviour

of t&o ne& versions of the :ocabulary evels Test% Language "esting 3n, % +"##(-% .utomaticity in bilingualism and second)language

learning% 9n % Sroll B .% de ;root +*ds%-,:andboo9 o( %ilingualism5 -sycholinguistic

Approaches+pp% 7$1)7''-% 0ford, USF 0ford University

Study of My Word Coach

Documents

Transcript of Study of My Word Coach