The Ontario Institute for Studies in Education at theUniversity of Toronto

“Effects of Asynchronous CMC and Face-to-Face Discussion on Oral Production”

by Jennifer Claro

A thesis submitted to the Faculty of Graduate Studies inpartial fulfillment of the requirements for the degree of

Master of Arts

in

Education

Curriculum, Teaching, and Learning

Toronto, Ontario, CanadaFall 2008

1

ABSTRACT

Recent research suggests that there are many aspects of online discussion that may make it better for language learning than face-to-face discussion. In this quasi-experimental study, equality of participation and individual output in face-to-face discussion were measured after treatment. Treatment consisted of ten female university students non-randomly placed into two groups of five students each. The first group participated in an online discussion board for seven weeks. The second group discussed the same issues face-to-face for seven weeks. Pre-study and post-study measures of variables were taken and analyzed. Results showed; a) an increase in subsequent oral production of the target language by the face-to-face group but a decrease in production of target language by the ACMC group; b) an increase of TOEIC scores in both groups, and c) an increase in the use of the L1 (Japanese) in subsequent oral discussion by the ACMC group. Based on the results of this study, ACMC discussion appears to have an overall negative effect on oral production of the L2.

2

Table of Contents

Chapter 1: Introduction……………………………………………………… 1

1.1 Background of the study ………………………………………… 1

1.2 Purpose ………………………………………………………….. 2

1.3 Objectives ……………………………………………………….. 3

1.4 Hypothesis ………………………………………………………. 3

1.5 Professional Significance of the Study ………………………….. 3

1.6 Outline of Thesis …………………………...……………………. 3

Chapter 2: Review of the Literature ………………………………………… 5

2.1 Foreign Language Learning Theory …………………………….. 5

2.2 CMC and Foreign Language Learning Theory ………………….. 6

2.3 Benefits of Using CMC in Language Learning ………………….. 7

2.3.1 Equalization of Participation of Students ……………… 7

2.3.2 Increase in Production of Target Language …………… 11

2.3.3 Other Benefits of Online Discussion …………………. 13

2.3.4 Transfer of Benefits of Online Discussion to F2F

Discussion ………………………………………….... 14

2.3.5 Features of SCMC and ACMC ………………………… 16

3

2.4 Cultural Influences on Japanese Reluctance to Speak English ….. 17

2.4.1 Japanese Classroom Culture …………………………… 17

2.4.2 Japanese Culture ………………………………………. 18

2.5 The Learning Style of the Japanese ……………………………. 19

2.5.1 Japanese Learning Style and CMC …………………… 20

2.6 CMC and Japanese University Students Studying English ……… 21

Chapter 3: Methodology ……………………………………………………… 23

3.1 Type of Research and Specific Subtype …………………………. 23

3.1.1 Quantitative Analysis …………………………………… 23

3.1.2 Qualitative Analysis ……………………………………. 24

3.2 Research Plan …………………………………………………….. 24

3.2.1 Pre-study ……………………………………………….. 24

3.2.2 The Treatment Phase …………….……………………. 24

3.2.3 Post-study ………………………………………………. 26

3.3 Content and Access ………………………………………………. 26

3.4 Participants and How Selected …………………………………… 27

3.5 Groups ……………………………………………………………. 27

3.6 Data analysis ……………………………………………………… 28

4

Chapter 4: Results …………………………………………………………….. 29

4.1 Quantitative Analysis …………………………………………….. 29

4.1.1 Research Question 1 ……………………………………. 29

4.1.2 Research Question 2 ……………………………………. 30

4.1.2.1 Mean Number of Words ……………………... 31

4.1.2.2 Mean Gain Scores ……………………………. 33

4.1.3 Research Question 3 …………………………………… 34

4.1.4 Research Question 4 …………………………………… 34

4.1.5 Variables Affecting Output in Pre-study and Post-study

F2F Discussions ……………………………………… 37

4.1.6 Validity of Results …………………………………….. 39

4.1.6.1 External validity …………………………….. 40

4.1.6.2 Internal validity ……………………………… 40

4.2 Qualitative Results ………………………………………………. 40

4.2.1 Questionnaire 1 (Pre-study) Results …………………… 40

4.2.1.1 Previous Experience with Computer

Communication Technology ………………… 40

4.2.1.2 Travel and Contact with Native Speakers …… 41

4.2.1.3 Reasons for Studying English ………………. 41

5

4.2.1.4 Feelings about Speaking English ……………. 41

4.2.1.5 Expressing Opinions ………………………… 42

4.2.2 Questionnaire 2 (Post-study) Results …………………. 43

4.2.2.1 Feelings about the Course …………………… 43

4.2.2.2 Favorite Topics ……………………………… 43

4.2.2.3 English Improvement and Vocabulary

Notebook …………………………………….. 44

4.2.2.4 Feelings about their Group ………………… .. 44

4.2.2.5 Feelings about Speaking English ……………. 45

4.2.2.6 Discussion Environment Preferences ……….. 45

4.2.2.7 Thoughts on whether Online Discussion

Helped their Spoken English ………………… 45

4.2.2.8 Differences Regarding Online Discussion ….. 46

Chapter 5: Discussion………………………………………………………… 47

5.1 Participation and Production …………………………………….. 47

5.2 TOEIC Score Increase …………………………………………… 50

5.3 Variables that Affect English Output ……………………………. 50

5.4 ACMC – To What Purpose? …………………………………….. 55

6

Chapter 6: Conclusion ……………………………………………………….. 57

References ………………………………………………………………………… 59

Appendices ………………………………………………………………………… 67

Appendix A: Questionnaire 1 (Pre-Study) …………………………… 67

Appendix B: Questionnaire 2 (Post-Study) ………………………….. 70

Appendix C: Letter of Informed Consent ……………………………. 72

7

List of Tables

Table 1: Research Questions, Groups Tested, and Test to be used for

Measurement ………………………………………………… 23

Table 2: English Words per Student after Treatment …………………………. 29

Table 3: English and Japanese Words per Student after Treatment …………… 30

Table 4: Mean Number of Words after Treatment …………………………….. 31

Table 5: Mean Gain Scores after Treatment …………………………………... 33

Table 6: Factors that Affect Online Participation ……………………………… 35

Table 7: Variables Affecting Output in Pre-study F2F Discussions …………… 37

Table 8: Variables Affecting Output in Post-study F2F Discussions ………….. 38

8

Chapter 1: Introduction

1.1 Background of the study

In a recent white paper, MEXT (the Japanese Ministry of Education, Culture, Sports, Science,

and Technology), acknowledges that the acquisition of communication skills in English has become an

“extremely important issue” (MEXT, 2002, p. 1). Furthermore,

At present, though, the English-speaking abilities of a large percentage of the population are

inadequate, and this imposes restrictions on exchanges with foreigners and creates occasions

when the ideas and opinions of Japanese people are not appropriately evaluated. (MEXT, 2002, p.

1).

Thus, there is an evident need for the Japanese, including students, to improve their English-

speaking abilities. In Japan, students study English for a total of six years in junior high school and high

school. The primary methods used are the traditional grammar translation method and rote memorization

of lists of vocabulary (Littlewood, 1999). These methods have been chosen with the goal of enabling

students to achieve a high score on university entrance exams, which are entirely written. Thus there is

little to no perceived need for students to speak English. By the time students graduate from high school,

they have achieved a certain level of proficiency in reading and writing English, but most of them cannot

speak English beyond a very basic level.

In undergraduate English conversation classes, which are always requirements for English

majors, and often requirements for non-English majors, many students are reluctant to speak English. One

reason for this is that the Japanese are shy (Doyon, 2000). The desire of Japanese students to avoid

making mistakes in order to avoid embarrassment means that they are reluctant to speak unless they know

the exact way to say what they want to say. Student reluctance to speak English may be due to their

shyness, their passivity, and in part because they have so rarely spoken English before.

9

One of the benefits of online discussion is the oft-cited increase in production (Fitze, 2006) of the

target language. Since Japanese students have never discussed anything in English, or been asked to

express their opinion about anything in English, online discussion may be a bridge for students to

speaking English. There is some evidence for the possibility of transfer of online discussion skills to oral

English skills (Abrams, 2003).

CMC (Computer-mediated communication) is a largely untapped resource in Japan, and

computers in general are not yet well-utilized in schools. The main reason, according to Johnson & Brine

(1999), is that the use of computers is not mandated by the Ministry of Education, thus they are not

perceived as being central components of English language instruction.

Main research questions:

a) Does the individual output of Japanese students’ spoken English increase after treatment?

b) What factors affect participation in online asynchronous discussion?

1.5 Professional Significance of the Study

Depending on the results, this research could prove important to the field of language teaching. If

it can be shown that students gain communicative skills by interacting online, and that these skills can be

transferred to consequent face-to-face communication, online interaction may be recommended for

language students hoping to improve their oral skills.

Chapter 2: Review of the Literature

2.1 Foreign Language Learning Theory

Much theoretical work has been done on trying to discover how foreign language learning can be

facilitated. Krashen (1985) claims that second language acquisition depends almost entirely on the

amount of comprehensible input a student receives. However, without the chance for output, input alone

is insufficient to obtain high levels of proficiency (Payne & Whitney, 2002). According to the Output

10

Hypothesis (Swain 1985), learners must be given the chance to produce comprehensible output.

According to Swain (1985), the role of output is to test hypotheses about the target language and “to

move the learner from a purely semantic analysis of the language to a syntactic analysis of it” (p. 252).

Output can be used as a way for students to test their hypotheses about structures of the target

language as learners “stretch their interlanguage to meet communicative needs” (Swain, 1995, p. 126).

Swain (1995), writes that students, when producing comprehensible output, engage in metalinguistic

discussion with peers and teachers, discussing doubts and questions they may have about the target

language, which further leads to development of their interlanguage (Swain & Lapkin, 1995), which can

be briefly defined as the learner’s developing second language. As the roles of comprehensible input and

comprehensible output are so central to language learning, the use of an environment that gives students

access to input and allows students to focus on their output is crucial.

2.3 Benefits of Using CMC in Language Learning

Note: “Online discussion” and “CMC” refer to both synchronous CMC (SCMC, i.e. “chat”) and

asynchronous CMC (ACMC, i.e. discussion boards). When differences have been noted by cited studies,

the more precise SCMC and ACMC are used.

2.3.1 Increase in Production of Target Language

Because of the current focus on the importance of output in acquiring an L2 (Swain, 1985; Swain

& Lapkin, 1995), much attention has been paid to the quantity of language output in CMC discussion as

compared to face-to-face discussion. However, studies showing an actual increase in language production

in CMC versus face-to-face (F2F) discussion are hard to find. There is some anecdotal evidence

(Beauvois 1998, Kelm 1992) that supports the notion of increased production, but empirical evidence is

conflicting.

11

Kern (1995), whose study is one of the most-often cited studies on CMC versus F2F discussion,

found that the mean number of words produced in SCMC discussion exceeded that in F2F discussion.

However, he did not statistically analyze these results. According to Fitze (2006), if Kern (1995) had

performed statistical analysis of the total word counts, he would have found no significant difference.

Kobayashi (2006) found that language production decreased in SCMC mode. Two SCMC

discussions yielded 514 words in total, and two F2F discussions yielded 1505 words. However, as

Kobayashi (2006) did not hold the time allowance constant over the two conferences, we cannot conclude

that more words were produced in one conference over the other over the same amount of time. As well,

Kobayashi (2006) did not test this result to see if it was statistically significant.

Fitze (2006) counted the number of words generated in four sessions of 20 minutes of each of

SCMC discussion and F2F discussion and found no statistically significant difference between the total

number of words in either conference (SCMC or F2F) in the same amount of time. Fitze’s (2006) study

seems to be the only study that accurately measured quantity of output.

2.3.4 Transfer of Benefits of Online Discussion to F2F Discussion

Two research studies have investigated the possibility of transfer of benefits associated with

online discussion to F2F discussion. In both studies, students were split into CMC and F2F groups, and

were then videotaped having a pre-study F2F discussion. Students then engaged in discussion in solely

the environment chosen for them (CMC or F2F). Then all groups were videotaped a second time having a

post-study F2F discussion. Results of the second F2F discussion were compared to the first F2F

discussion to see if the environment had any observable effects on the production of the target language.

In the first of these two studies, Abrams (2003) studied 96 students of intermediate German,

which were assigned non-randomly (they were intact classes) to six sections. Two sections (the control

group) had no exposure to CMC throughout the duration of the study, and participated in small group F2F

discussion and group work only. Two sections participated in SCMC twice for 50 minutes on the day

12

immediately preceding the F2F discussion, and two sections participated in ACMC for one week

immediately preceding the F2F discussion. Abrams (2003) found that the F2F group and the SCMC

group significantly outperformed the ACMC group in oral interactions, as measured by syntactic

complexity and lexical richness. She concluded that small-group F2F discussions better promote oral

interactions than ACMC. She also found that the SCMC group produced significantly more language in

the same period of time than the ACMC and F2F groups, which she asserts shows a sign of increasing

fluency (Leeson 1975, cited in Abrams, 2003). Indeed, Abrams (2003) found that the ACMC group

produced significantly less output than the other two groups.

In the second of the two research studies, Kobayashi (2006) studied the participation patterns of

12 students of Japanese as a Foreign Language. Students were non-randomly assigned to two groups, a

control F2F group and a quasi-experimental ACMC group. Kobayashi (2006) found no significant

difference in volume of language output in the two groups. He also found no difference in syntactic

complexity, lexical richness, or accuracy.

The two studies to date that have attempted to determine whether benefits of online discussion

can be transferred to F2F discussion had conflicting results. Abrams (2003) found a significant increase in

volume of output in F2F discussion only in groups that had taken part in SCMC and F2F discussion. Thus

ACMC had no beneficial effect on the language output of subsequent F2F discussion. Kobayashi (2006)

found no significant difference in the volume of output in F2F discussion of groups who had participated

in previous F2F or SCMC discussions. Kobayashi (2006) did not investigate ACMC.

Thus the only study done to date that has tried to determine whether exposure to ACMC

discussion can result in greater output volume in subsequent F2F discussion (Abrams, 2003), found that it

did had a significantly negative effect. Because of the dearth of research in this area, the issue of whether

participation in ACMC discussion results in an increase in output volume in subsequent F2F discussion is

addressed in the current study.

13

2.3.5 Features of SCMC and ACMC

SCMC discussion is often compared to F2F discussion because of the immediacy of the need to

reply in both SCMC and F2F (see e.g. Kelm, 1992; Payne & Whitney, 2002). Written language in SCMC

also has features of spoken English, including speech acts such as questions, feedback, clarification

checks, requests for help, and self-correction (Chun, 1994; Lee, 2002a).

The most apparent difference between SCMC and ACMC is the time lag in ACMC. The increase

of time lag in ACMC means that students have more time and to look up words and plan what they want

to say, compared to the immediacy of SCMC. Students have the time to check what they write before

submitting it. This allows them time to reflect on what they write, to make sure that they are saying what

they mean to say.

Both SCMC and ACMC are environments that meet student need for access to comprehensible

input (Krashen, 1985) and the chance to produce their own comprehensible output (Swain, 1985).

However, SCMC and ACMC have their own unique features that promote different aspects of language

learning. While not mutually exclusive, it seems that fluency is best enabled by SCMC and accuracy by

ACMC. The goals of any research project involving online discussion need to take these differences into

account.

2.4 Cultural Influences on Japanese Reluctance to Speak English

Japanese reluctance to speak English is well-documented (see e.g. Anderson, 1993; Doyon, 2002;

Nimmannit, 1998). In a previously mentioned study by Warschauer (1996a), the least verbal students

were Japanese. Japanese are frequently described in research literature as being reluctant to speak English

(Umemoto, 2001), and it is also often the case with the Principal Investigator’s students. Japanese scoring

highly in the uncertainty avoidance category (Hofstede, 2003) means that Japanese tend to avoid

situations that they feel are uncertain, or make them feel unsure. Japanese feel nervous when they speak

English and tend to avoid it (Krowner, 2002).

14

In Japan, students spend years learning English in junior high school and high school, but rarely

have the chance to speak English. Since they have not had much opportunity to practice speaking English,

students are often shy when asked to converse in English. Japanese students also worry that they will

make mistakes when they speak English, and this creates more stress.

This stress can be lessened by moving from a face-to-face environment to a computer-mediated

environment. In this way, students are still using their developing English interlanguage to communicate

with each other, but barriers to communication are lessened by the faceless environment and the

opportunity to reflect on and modify what they will say before they say it. It is perhaps the face-saving

quality of CMC that can make it potentially such an appropriate learning environment for Japanese.

Thus CMC has many things to offer language students in general, and students in Japan in particular.

According to language learning theory, CMC provides an environment rich in input, as well as the

opportunity to produce their own output, from which students can learn from each other, improving their

interlanguage. Asynchronous CMC gives students time to reflect before responding and reduces the

chances of losing face, both aspects which are particularly relevant for Japanese students, who have

difficulty in F2F discussion because of cultural and educational influences (Doyon, 2000). Individual

student participation may increase in online discussion, and it is usually those who contribute the least in

face-to-face discussions whose participation increases the most in online discussions. There are

indications that skills acquired in CMC discussion may be transferred to F2F discussion. If this is

possible, then oral skills of Japanese students may benefit from CMC discussion.

Chapter 3: Methodology

3.1 Type of Research and Specific Subtype

This research project used a combination of quantitative and qualitative research methods.

3.1.1 Quantitative Analysis

15

Table 1.

Research Questions, Groups Tested, and Test to be used for Measurement

Research Question Groups Tested Test

1. After treatment, will the individual output

of Japanese EFL students’ spoken English

increase (i.e. more words per student than at

the beginning of the study)?

Both groups word count, then independent

sample t-tests of both total word

count and gain scores

2.Will TOEIC scores be affected by the

treatment?

Both groups independent samples t-test and

paired samples t-test

3. Which variables have a correlation with

ACMC output?

Group 2 only bivariate Pearson product-

moment correlation analyses

Group 1 – Students who interact face-to-face only for 7 weeks

Group 2 – Students who interact online for 7 weeks

3.2 Research Plan

3.2.1 Pre-study

a) Letters of consent were read and signed by students.

b) A baseline measure of the proportion of time each student contributes when speaking their own

native language was collected, by putting students together in groups of 3-4 and having them discuss a

certain topic together in Japanese. All conversations were videotaped and tape-recorded.

16

c) An initial measure of what proportion of time each student contributes in their L2 (English)

was collected. Groups of 3-4 students first read an easy article by themselves (topic – the heir to the

Japanese throne, of high interest to most Japanese). They then discussed the topic naturally. Discussions

were videotaped and tape-recorded.

d) Students’ level of English was tested using the TOEIC 1 week before the beginning of the

project.

e) Students also answered pre-study and post-study questionnaires.

3.2.2 The Treatment Phase

Each of the two online groups and the two face-to-face groups consisted of 2-3 students each,

depending on how many students were absent that day. Groups changed regularly and were necessarily

formed around present students.

To prepare for discussion, students read an article in their course textbook, titled Topics for

Global Citizenship by David Peaty. Students then chose 10-15 English words from the reading. They

wrote these in their vocabulary notebook along with either a translation of the word in Japanese or an

English definition. Then students wrote one example sentence to show the meaning of the word. These

notebooks were corrected regularly by the instructor with an emphasis on correct word usage rather than

grammar.

Students were separated for the actual discussions. Students doing ACMC discussion met in the

computer classroom, where they sat slightly separated from each other. Initially some students spoke to

each other in Japanese during their online discussions, so they were moved to more remote locations in

the room. Students used Moodle for CMC discussion. The discussion board was used seven times and the

chat was used once.

The students having face-to-face discussions met in a classroom. At the beginning and end of

17

class, the class met together in the computer room too for briefings by the instructor.

3.2.3 Post-study

a) A post-study measure of what proportion of time each student contributes in their L2 (English)

was collected. Groups of three students first read an article by themselves. They then discussed the topic

naturally. Discussions were videotaped and tape-recorded.

b) Students’ post-study level of English was tested using the TOEIC.

c) Questionnaire 2 (see Appendix B) was handed out and later collected.

d) All tape-recorded sessions were transcribed. One bilingual research assistant was hired for this

purpose. The assistant transcribed the video recordings, counted the words per utterance, and determined

the total number of words contributed per student. Data from two asynchronous online discussions was

analyzed qualitatively. All data was rendered anonymous for the assistant.

3.3 Participants and How Selected

The participants were third year English majors – selected because many of them are

reluctant to speak English despite their English ability being adequate for basic conversation.

They were all English majors at Shokei University in Kumamoto, Japan, in a conversation class

taught by the Principal Investigator.

3.4 Groups

Beauvois (1992) advocates the use of small conferences, and Bohlke (2003) found that groups of

five students did not show the equalizing effect on participation, while groups of four students did.

Bohlke (2003) thus recommends keeping group size below five in order for students to benefit from a

possible increase in participation. There were 12 students in this study, thus 4 groups of three students

each kept group members at less than five. Students were not randomly divided in to CMC and F2F

18

groups. Instead, the researcher attempted to keep groups balanced on the basis of a) results on the TOEIC,

a standard test of English ability (with no spoken component) and b) level of shyness, as expressed by

students on the pre-study questionnaire.

3.5 Data Analysis

Data was tape-recorded during the face-to-face discussions and transcribed and the

number of words spoken per student was counted. Data was collected from the discussion board.

The contributions made during 2 online discussions were calculated (by word count per student).

A series of qualitative analyses was carried out on the questionnaires and discussions (online and

face to face) along with frequency data on discussion word counts in the two groups.

Chapter 4: Results

4.1 Quantitative Analysis

4.1.2 Research Question 1

After experiencing the equalizing effects of an online environment, will Japanese EFL students

contribute more in face-to-face discussion (i.e. more words per student than at the beginning of the

study)?

Note: Students often reverted to their L1 (Japanese) while discussing topics face-to-face. Because

of this, the total numbers of words spoken in the L1 and L2 were analyzed separately as well as together.

4.1.2.1 Mean Number of Words

To determine if there was a difference in the mean number of English words spoken per student

after treatment in the two types of environments (F2F and ACMC), an independent samples t-test was

19

calculated on the total number of words in each environment. Please see Table 4 below for a summary of

the results.

Table 4

Mean Number of Words after Treatment

Mean Number of Words

St. Dev.

t p Eta squared

Group 1 –English

word count

156.20 58.92 1.192 .274 0.150821

Group 2 – English

word count

119.20 36.68

Group 1 – Japanese

word count

57.80 59.95 -1.440 .204 0.205845

Group 2 – Japanese

word count

152.20 133.72

Group 1 -

Eng. & Jap. word

count

214.00 77.65 -.802 .446 0.074417

Group 2 - 271.40 139.93

20

Eng. & Jap. word

count

Group 1 – Group that participated in F2F discussion only for seven weeks, prior to the final F2F discussion.

Group 2 – Group that participated in ACMC discussion only for seven weeks, prior to the final F2F discussion.

It is important to point out here that traditional boundaries of statistical significance of p<.05 or

p<.01 do not automatically imply a meaningful or practical effect (Schuele & Justice, 2006).

With a small sample size, statistical comparisons may show there to be no statistically significant

difference between two groups, even when the means of the two groups seem quite different based on

informal inspection of the data (Schuele & Justice, 2006, p. 3).

Because of this, the APA suggests that authors “both report and interpret effect size estimates”

(APA, 2001, cited in Schuele, C.M. & Justice, L.M., 2006, p. 3). Schuele & Justice (2006) suggest using

Cohen’s d eta squared (or Cohen’s d ) as an index of effect size. Accordingly, in this study, eta squared

values have been included in all analyses of relationships between groups. The basis for comparing

groups will therefore be values for eta squared and not values of significance of p<.05 or p<.01, because

of a) the small sample size of this study (N=10) and; b) recommendations of the APA (2001).

The test on mean number of English words spoken after treatment in the 2 groups

revealed no statistically significant effect across environment condition, t=1.192. The difference

in the means for total number of English words was 156.2 – 119.2 = 37 words, which comprised

21

23.7% of the ACMC total number of English words, and 31.0% of the F2F total number of

English words. The eta squared1 statistic (0.150821) indicated a large effect size.

When we examine the total number of Japanese words spoken by the two groups (See

Table 4, Column 1, rows 3 and 4), we can see that the mean number of Japanese words spoken

by Group 2 after treatment is 152.20/57.80 = 2.63 times as high as the mean number of Japanese

words spoken by Group 1. The eta squared statistic (0.205845) indicated a large effect size.

When we examine the total number of words, including both English (L2) and Japanese

(L1) words, the eta squared statistic (0.074417) indicated a moderate effect size.

4.1.2.2 Mean Gain Scores

To determine if there was a difference in the gain scores of words spoken per student after

treatment in the two types of environments (F2F – Group 1 and ACMC – Group2), an independent

samples t-test was run on the gain scores in each environment. See below for a summary of the results.

Table 5

Mean Gain Scores after Treatment

Mean gain score

St. Dev. t p Eta

squared

Group 1- 21.4 47.73 1.289 .234 0.171973

1 Eta squared is a common effect size statistic, which provides an indication of the differences between groups (Pallant, 2005). Guidelines are (proposed by Cohen, 1988, cited in Pallant, 2005, p.126): .01=small effect, .06=moderate effect, .14=large effect. The values for eta squared are not provided by SPSS for independent samples t-tests, therefore the following formula was used;

Eta squared = t 2 t2 + (N1+N2-2)

22

English

Group 2-

English

-49.0 112.44

Group 1-

Japanese

4.2 46.91 1.479 .209 0.214719

Group 2-

Japanese

-173.4 264.40

Group 1-

Eng. & Jap.

25.6 45.88 1.497 .207 0.218827

Group 2-

Eng. & Jap.

-222.4 367.48

Gain score is defined as the score of the post-study evaluation minus the score of the pre-study evaluation.

An examination of gain scores (English words only) shows that Group 1 gained a mean number

of 21.4 English words per person when comparing the post-project word count with the pre-project word

count. Group 2 lost a mean number of 49 English words. The eta squared statistic (0.171973) indicated a

large effect size.

An examination of results shows a mean gain score of 4.2 Japanese words for Group 1 and a

mean gain loss of 173.4 for Group 2. The eta squared statistic (0.214719) indicated a large effect size.

23

An examination of gain scores (Japanese words only) shows a mean gain score of 25.6 Japanese

and English words for Group 1 and a mean loss of 222.4 Japanese and English words for Group 2. The

eta squared statistic (0.218827) indicated a large effect size.

In summary, there were large effect sizes in the gain scores in the L1 and the L2 (when

considered separately and together) after treatment.

4.1.4 Research Question 3

Will TOEIC scores be affected by the treatment?

A paired samples t-test was conducted to evaluate the impact of the seven weeks discussion in the

two environments on TOEIC scores. The eta squared statistic2 (0.389847) indicated a large effect size in

the increase in TOEIC scores from the pre-test to the post-test.

To determine whether this increase in TOEIC scores was related to the treatment, an independent

samples t-test was conducted to compare the TOEIC scores for Groups 1 and 2. The magnitude of the

differences in the means was =0.000072, indicating no effect size.

Thus, there was an increase in TOEIC scores from pre-test to post-test but this increase was not

affected by the treatment. Students TOEIC scores increased regardless of which environment they were

in.

4.1.4 Research Question 4

Which variables have a relationship with online participation?

To determine which variables have a relationship with online participation, Pearson product-

moment correlation analyses were performed. The results can be seen below in Table 6.

2 For paired samples t-tests, the following formula for eta-squared was used (Pallant, 2005);

Eta squared = t 2 t2 + N - 1

24

Table 6

Factors that Affect Online Participation

Level of Significance (p)

1st variable

2nd

variable

Pearson Correlation (r)

Strength of Correlation

Coefficient of Determination (CoD)

Shared Variance (%)

.008 wpf toeic1 1.000** 1-to-1

correlation

1.000000 100.00

.031 wpf toeic 2 .999* almost 1-to-1

correlation

.998001 99.80

.189 wpf type .956 very strong .913936 91.36

.133 wpf ebetot .978 very strong .956484 95.64

.627 wpf eaetot .553 strong .305809 30.58

.577 wpf shy .616 strong .379456 37.95

N=3

wpf = words per forum;

type=typing score;

toeic1=TOEIC score pre-test;

toeic2=TOEIC score (post-test);

ebetot = total number of English words spoken per student in F2F discussion (pre-study);

eaetot = total number of English words spoken per student in F2F discussion (post-treatment);

shy=level of shyness perceived by subject

**p<.01

*p<.05

25

A 1.00 value for r and CoD indicates a one-to-one correlation between the mean number of words

typed per student (in Group 2) per forum discussion and students’ TOEIC1 score (beginning of project).

This one-to-one correlation indicates that the TOEIC score explains 100% of the variance in students’

mean number of typed words per forum. As this result is significant at p<.01, even with such a small

sample size of N=3, this is a very significant result.

The .999 value for r and .998001 value for CoD indicate a near one-to-one correlation for wpf

and TOEIC2 score (end of project). Thus 99.8% of the variance can be explained by the TOEIC2 score

(significant at p<.05). When considered with the result of the TOEIC1 analysis (see previous paragraph),

TOEIC scores can predict the mean number of words students will type per form with very close to 100%

accuracy. Thus the number of words students type may depend entirely on the student’s level of English

ability.

It is important to point out here that, according to Pallant (2005, p. 127),

The significance of r is strongly influenced by the size of the sample. In a small sample (e.g.

N=30), you may have moderate correlations that do not reach statistical significance at the

traditional p<.05 level… Many authors in this area suggest that statistical significance should be

reported but ignored, and the focus should be directed at the amount of shared variance.

Because of the small sample size of this study (N=10, and for some analyses, N=3), the significance level

will be reported, but the focus will be on the amount of shared variance.

The score for the typing test (type) also had a very high correlation [r=.956, shared

variance=91.36%] with the mean number of words contributed in the online forums. However, correlation

does not indicate causality, and in this case the typing score may be a side effect of the actual level of

English ability. Students with a higher level of English ability would likely type more English words in a

given amount of time than students with a low level of English ability.

26

The variable eaetot (total number of English words spoken per student in F2F discussion, post-

treatment) showed a strong correlation with wpf [r=.553, shared variance=30.58%], though not as strong

as the correlation between ebetot (total number of English words spoken per student in F2F discussion,

pre-study) and wpf [r=.978, shared variance=95.64%]. Indeed, the shared variance dropped by 65.06%

from the pre-study measure to the post-study measure. Why? As we will see in the next section (4.1.7, p.

22), eaetot (total number of English words spoken per student in F2F discussion, post-treatment) has a

number of variables affecting it that ebetot (total number of English words spoken per student in F2F

discussion, pre-study) does not.

The variable shy (level of shyness perceived by subject) also showed a strong correlation with

wpf [r=.577, shared variance=37.95%].

It was not possible to investigate whether these factors also affected F2F participation during the

seven weeks of treatment, as Group 1 discussions were not videotaped. However, it is possible to

investigate which factors affected the F2F output (by both groups) in the pre-study and post-study F2F

discussions.

4.1.5 Variables Affecting Output in Pre-study and Post-study F2F Discussions

Table 7

Variables Affecting Output in Pre-study F2F Discussions

Level of

Sig. (p)

1st

variable

2nd

variable

Pearson

Correlation

(r)

Strength of

Correlation

CoD Shared

Variance

(%)3

3

27

.018 ebetot toeic14 .726* strong .527076 52.71

.313 ebetot japtot .356 medium .126736 12.67

.473 ebetot mistake .257 small .066049 6.60

.549 ebetot pron .216 small .046656 4.67

.906 ebetot shy -.043 none .001849 .18

.616 ebetot conf -.181 small .032761 3.28

.799 ebetot comp -.092 none .008464 .84

.650 ebetot speng .164 small .026896 2.69

N=10

ebetot = total number of English words spoken per student in F2F discussion (pre-study);

eaetot = total number of English words spoken per student in F2F discussion (post-treatment);

toeic1=TOEIC score (pre-test);

japtot=total number of Japanese words spoken in Japanese (L1) F2F pre-study discussion

mistake=level of fear of making mistakes (e.g. grammar, vocabulary) while speaking English, as perceived by subject

pron=level of fear of making pronunciation mistakes, as perceived by subject

conf=level of confidence in speaking English, as perceived by subject

comp=level of comfort using computers

speng=level of ability in spoken English, as perceived by subject

shy=level of shyness as perceived by subject

**p<.01

*p<.05

4 toeic1 was used for this analysis because it is the pre-study measure of English ability

28

Toeic1 had a strong correlation with ebetot, and japtot had a medium correlation with ebetot.

Thus the two variables which had the most influence on the number of English words spoken in the pre-

study F2F discussion were English ability and the number of words contributed per student in the L1

(Japanese) pre-study F2F discussion. A standard multiple regression analysis of ebetot showed that toeic1

and japtot gave an R value of .781 and an R-square value of .610. Thus these two variables accounted for

61.0% of the variance in ebetot. A second standard multiple regression analysis showed that all of the

variables shown in Table x gave an R value of .903 and an R-square value of .815. Thus all the tested

variables accounted for 81.5% of the variance in ebetot.

Table 8

Variables Affecting Output in Post-study F2F Discussions

Level of

Sig. (p)

1st variable

2nd

variable

Pearson Correlation (r)

Strength of Correlation

CoD Shared Variance (%)5

.594 eaetot toeic26 .193 small .037249 3.72

.248 eaetot japtot .403* medium .162409 16.24

.104 eaetot mistake -.545*

medium

negative

.297025 29.70

.091 eaetot pron -.562*

medium

negative

0.315844 31.58

.961 eaetot shy .018 none .000324 .03

5

6 toeic2 was used for this analysis because it is the post-study measure of English ability

29

.742 eaetot conf -.120 small 0.014400 1.44

.606 eaetot comp .187 small 0.034969 3.50

.878 eaetot speng -.056 none 0.003136 .31

N=10

ebetot = total number of English words spoken per student in F2F discussion (pre-study);

eaetot = total number of English words spoken per student in F2F discussion (post-treatment);

toeic2=TOEIC score (post-test);

japtot=total number of Japanese words spoken in Japanese (L1) F2F pre-study discussion

mistake=level of fear of making mistakes (e.g. grammar, vocabulary) while speaking English, as perceived by subject

pron=level of fear of making pronunciation mistakes, as perceived by subject

conf=level of confidence in speaking English, as perceived by subject

comp=level of comfort using computers

speng=level of ability in spoken English, as perceived by subject

shy=level of shyness as perceived by subject

**p<.01

*p<.05

No variables had a strong correlation with eaetot, but three variables had a medium correlation

(japtot, mistake, and pron). Thus the three variables which had the most influence on the number of

English words spoken in the pre-study F2F discussion were a) the number of words contributed per

student in the L1 (Japanese) pre-study F2F discussion; b) the level of fear of making mistakes in grammar

and vocabulary, and; c) the level of fear of making mistakes in pronunciation. A standard multiple

regression analysis of eaetot showed that these three variables gave an R value of .871 and an R-square

value of .759. Thus these variables accounted for 75.9% of the variance in eaetot. A second standard

multiple regression analysis showed that all of the variables shown in Table y gave an R value of .971 and

an R-square value of .942. Thus all of the variables tested accounted for 94.2% of the variance in eaetot.

30

In summary, pre-study F2F output (ebetot) depended largely on English ability (toeic1) whereas

post-study F2F output (eaetot) depended largely on other variables, the ones with the greatest effect on

eaetot being japtot, mistake, and pron.

4.1.6 Validity of Results

4.1.6.1 External validity

External validity can be defined as, “The degree to which results of a study with a sample of

subjects can be generalized to make statements about a much larger population of subjects” (Keppel,

Saufley & Tokunaga, 1992, p.241). Because of the small sample size of this study, the results do not carry

a great deal of external validity.

4.1.6.2 Internal validity

Four possible threats to internal validity were identified. A violation of the independence of

observations assumption is assumed because students worked together in small groups. According to

Pallant (2005), the behavior of each member of the group influences all other group members. Therefore

independence of observations cannot be assumed. An observation effect was also identified. Students

were aware that they were being observed and recorded, and this may have led them to participate

differently than they would have if left unobserved. A topic effect was identified. Students may have been

more or less interested in the topics they were discussing, and therefore their participation in the

discussion of those topics may have been affected. Finally, because students took the same TOEIC test

twice, they may have gotten a higher score on it the second time because of a repeated testing effect.

Chapter 5: Discussion

5.1 Production

Because students often reverted to their first language (Japanese) in post-study F2F

discussion, word counts of English, Japanese, and both were analyzed. There was no statistically

31

significant difference between the two groups in the total number of English words, nor in the

total number of English and Japanese words. However, Group 2 produced more Japanese words

in the post-study oral discussion than Group 1 (t=-1.440, eta squared=0.205845, indicating a

large effect). While it is not possible to precisely determine the reason for this, it can be

postulated that because although students in Group 2 got a lot of practice writing English in their

weekly ACMC discussion forums, they did not get any practice speaking English (with the

exceptions of the pre-study and post-study videotaped face-to-face discussion). It may therefore

have been more difficult for Group 2 students to speak English than students in Group 1, and

perhaps this is why so much Japanese was used. When asked in the post-study questionnaire if

they thought their spoken English had improved, six of the ten students answered that they

thought it had. An independent samples t-test showed that the two groups differed significantly

(p=.243) in their response to this question, with Group 1 (the F2F group) students thinking that

their spoken English had improved more than Group 2 students thinking so. When we look at the

results of the post-study questionnaire, six of eight students replied that they prefer online

discussion to F2F discussion. Reasons included having more time to think about one’s answer,

being able to observe one’s ideas, and being less tense in an online environment. Students

recognized some benefits to ACMC discussion, and may have found it to be motivating, a

variable that was not tested in this study.

When we look at mean gain scores after treatment, it becomes obvious that Group 2 did not

benefit from the treatment. Please see Table 5 (page 14) for results pertinent to this discussion. Group 1

gained a mean of 21.4 English words in comparison to the pre-study discussion, whereas Group 2 lost a

mean of 49 words. The results from the two groups are significantly different (p=.234). This clearly

shows that while Group 1, who spent seven weeks speaking English face-to-face, benefited from the

32

treatment, Group 2, who spent seven weeks in online discussion, actually decreased the number of

English words that they used in post-treatment face-to-face discussion.

It is interesting to notice that Group 2 students decreased the number of Japanese words too, but

they still spoke a significantly higher number of Japanese words than Group 1 (see discussion on page

22). When we look at the gain scores for both English and Japanese words together, Group 1 gained in

number of total words, and Group 2 lost in number of total words. Thus we cannot say that ACMC

discussion increases the word count in either the L1 or the L2; indeed, there were decreased number of

words in both. We must conclude that ACMC discussion was detrimental for these students to subsequent

production of spoken English. This finding is consistent with the results of Abrams (2003) study, which

found that the ACMC group produced significantly less output than either the F2F group or the SCMC

group. Abrams (2003) concluded that small-group F2F discussions better promote oral interactions than

ACMC discussion, and that is our main conclusion here.

5.2 TOEIC Score Increase

The eta squared statistic indicated a large effect size in the increase in TOEIC scores of both

groups from the pre-test to the post-test. This indicates that the English ability of both groups of students

benefited from the treatment. In other words, both ACMC discussion and F2F discussion have a positive

effect on English ability. However, this increase was not necessarily related to the treatment. These

students are English majors, and took other English courses throughout this study, thus it is not possible

to ascertain whether the increase in English ability is due solely to the seven weeks of English discussion.

5.3 Variables that Affect English Output

Please see Table 6 (page 16) for results pertinent to this discussion. Six factors that were tested

for correlation with online participation, as measured by mean number of words per student per forum

showed a significant correlation. Those six variables are toeic1 (pre-treatment TOEIC score) and toeic2

33

(post-treatment TOEIC score), typing speed, the total number of English words spoken per student in

F2F discussion (both pre-study and post-study), and level of shyness (as perceived by student).

Pearson correlations of 1.000 (toeic1) and .999 (toeic2), with associated shared variances

of 100.00% and 99.80% indicate a one-to-one relationship7 between TOEIC scores and the mean number

of words per student per forum. These results are statistically significant at the p<.01 (toeic1) and

p<.05(toeic2) levels. Therefore the number of words a student types would seem to depend entirely on

their level of English ability.

But four other variables are also significantly correlated with ACMC output. The first is typing

speed, as measured in words per minute. When we consider the impact of English ability on ACMC

output, which looks like a one-to-one causal relationship, we can postulate that the higher the level of

English ability, the faster a student can type, and the faster a student can type, the more words they

produce. In other words, we can postulate that the main influence on ACMC output is English ability,

which drives both typing speed and the number of words produced (ACMC output).

The fourth variable to consider is the total number of English words spoken in the pre-study F2F

discussion. The strength of correlation is very strong between this variable and the ACMC output (mean

number of words per student per forum). When we look at causal relationships, however, it is again

necessary to consider English ability, as measured by TOEIC score. As we saw in Table 6, the

correlations between TOEIC scores (English ability) and F2F output are significant. It is not, however,

the one-to-one relationship we see between English ability and ACMC output. Drawing on the above

argument for English ability possibly being directly responsible for ACMC output, it seems logical that

English ability may also be responsible (i.e. the causative factor) for F2F output. It may seem intuitive

that English ability drives both ACMC output and F2F output; after all, we can expect that students who

7 In the case of toeic2, results indicate a near one-to-one relationship. For the purposes of this study, and because of the exact correlation of toeic1, the very slight difference in the results of toeic1 and toeic2 will be ignored.

34

have better English ability will produce more output than students with lesser English ability. The

interesting point to notice here is that the correlation between ACMC output and English ability seems is

one-to-one. The only other variable that has any impact on ACMC output is shyness (as perceived by

students), with a shared variance of 37.95%, although as we saw in section 4.1.7, (p. 22), that there is no

correlation between shyness and F2F output. Although we cannot state a causative effect here, it would

seem from these results that shyness affects the number of words typed in an online discussion board, but

it does not affect the number of words spoken in either pre-study or post-study F2F discussions. However,

the relationship between shyness and the number of words per forum is negatively correlated, with higher

numbers of words per forum being contributed by shyer students8. This is what we would expect, as CMC

has been shown to increase the participation of the shyest students (Bump, 1990, Warschauer, 1996a), and

we could therefore expect that shyer students would contribute more in online discussion.

In the case of English ability and F2F output, the relationship is not one-to-one. The shared

variance between English ability and F2F output is 52.71% in the case of the pre-study TOEIC test, but

only 3.72% in the case of the second TOEIC test. This implies that other variables are impacting

significantly on F2F output, especially with respect to the second TOEIC test (post-study). Results of a

bivariate Pearson correlation analysis (see Tables 8 and 9) showed that the pre-study F2F output had a

strong correlation with English ability, as measured by the pre-study TOEIC test. This is something we

would expect, that students who had a higher level of English ability would produce more output (speak

more) than students who had lesser ability. As well, the pre-study F2F output also had a medium

correlation with the number of words spoken in Japanese in the pre-study L1 (all Japanese) discussion.

This result was unpredicted, and itwas arrived at simply by testing all measured variables for

relationships. This result is interesting, as it suggests there may be a “talkativbility” factor, where which

postulates that students who speak a lot in Japanese may also speak a lot in English. Future studies could

investigate the viability of a talkativity variable that could be defined by the amount of output in student’s

8 Shyness was measured by a Likert scale of 1 to 5 with 1=extremely shy and 5=not shy at all.

35

native language (L1). It would be interesting to investigate this further, by first establishing the viability

of this as a construct through international studies measuring talkativity in students’ native language, and

then comparing it to talkativity in the L2 to see if talkativity is transferable across a wide variety of

languages and speakers. It would also be interesting if other variables affect talkativity in the L1 and

whether these variables affect talkativity in the L2 in similar ways.

Talkativity, defined as the number of words native speakers spoke in their own L1, is the only

variable in this study that affected both the pre-study F2F English output and the post-study F2F English

output. In both cases, talkativity (japtot) had a medium correlation with the number of words produced. In

this respect, talkativity is the most stable variable affecting output.

5.4 ACMC – Tto Wwhat Ppurpose?

Results showed that English ability, as measured by TOEIC scores, increased significantly over

the course of this study. However, according to the results of this study, F2F oral output decreased as

result of seven weeks of ACMC discussion. Can we therefore recommend ACMC discussion for language

students?

As students need to improve all four skills in their L2 – speaking, listening, reading, and writing –

any treatment that impacts negatively on any of these four skills must be reconsidered. We must

remember the words of Hippocrates, on cautioning physicians in their treatments, “First, do no harm.”

The amount of output is crucial, because it is in producing output that students test hypotheses

about the target language (Swain, 1985). Through testing hypotheses and restructuring their own

interlanguage, students progress in both their understanding of the L2 and in their skill to use it

productively. Therefore, the more output a student produces, the more chances they have to improve their

language skills. If, by being exposed to ACMC discussion, students decrease their subsequent oral output,

this is indeed a negative effect, serious enough for us to question the role of ACMC in oral language

practice.

36

The benefits of CMC discussion from prior research are a) an increased equality of participation

(Beauvois, 1992, 1998; Bohlke 2003, Chun 1994; Kelm, 1992; Kern, 1995; Sullivan & Pratt, 1996; Tella,

1992, Warschauer 1996a); b) increased production in the L2 (anecdotal evidence only, see (Beauvois

1998, Kelm 1992); c) a decrease in production of the L1 in L2 discussion (anecdotal evidence only, see

Beauvois 1998; Chun 1994; Kelm, 1992); d) a reduction in language anxiety (Beauvois 1998; Beauvois &

Eledge, 1996; Chun 1994; Warschauer 1996a, but see Kobayashi, 2006); e) an increase in motivation

(Beauvois 1992; Warschauer, 1996b); and f) an increase in interactive competence (Chun 1994; Fitze

2006; Kelm 1992; Kern 1995; Sullivan and Pratt 1996).

The results of this study in relation to these previously found benefits showed; a) no increase in

production of the L2; b) an increase of TOEIC scores in both groups, and c) an increase in the use of the

L1 in subsequent oral discussion by the ACMC group. The first two results show no positive effect of

ACMC discussion on subsequent oral discussion. The last result indicates a negative effect. Therefore, we

can conclude that, based on the results of this study, ACMC discussion has an overall negative effect on

oral production of the L2. However, several benefits of CMC were not tested in this study. The last three

benefits described in the previous paragraph; d) a reduction in language anxiety (Beauvois 1998;

Beauvois & Eledge, 1996; Chun 1994; Warschauer 1996a, but see Kobayashi, 2006); e) an increase in

motivation (Beauvois 1992; Warschauer, 1996b); f) an increase in interactive competence (Chun 1994;

Fitze 2006; Kelm 1992; Kern 1995; Sullivan and Pratt 1996). To date, no studies that the researcher has

found have investigated these possible benefits in an online discussion board, and so it is recommended

that more studies be conducted which would a) confirm the results of this study; and b) investigate

language anxiety, motivation, and interactive competence in both online discussion boards and in

subsequent oral production.

Chapter 6: Conclusion

CHAPTER 6 - CONCLUSIONS

37

In the rush to integrate computerConclusions

technology into every facet of our lives, it is good to take a moment to stand back and ask the

question, “What is it all for?” There are many positive effects of technology on our lives, one

being the instant access to information that the Internet has brought us, and the ability to

communicate with people far away in a number of ways that were not possible before. Many

people believe that the Internet is bringing us together, enabling us to connect with numbers of

people through websites, online chat groups and discussion boards, and the use of e-mail.

However, technology also separates us, from each other (in the case of two people sitting

together text messaging other people instead of interacting with each other) and from our

surroundings (for example, the use of cell phones while driving distracting us from our driving).

Technology, for all its benefits, is not a cure-all. Not everything touched by technology is

enhanced by it. In this study, the addition of technology to one group in a language class actually

decreased student oral ability, as measured by an English word count. The other group, which

discussed topics face-to-face, increased their word counts, indicating increasing fluency (Leeson

1975, cited in Abrams, 2003). These results support the research findings of Abrams (2003), who

found that the ACMC group produced significantly less output than the other two groups (SCMC

and F2F).

This brings up the question of why SCMC would have a positive impact on spoken

language, while ACMC has a negative impact (in terms of amount of output). The answer may

lie in the fact that communicating by SCMC, or “chat” is more similar to speaking than ACMC

is. Beauvois (1998, p. 2) writes of the “conversational quality of writing” of SCMC. Chun (1994,

p. 17) writes “the interactional structures resemble spoken conversation”. Perhaps it is because it

is similar to spoken language that SCMC discussion impacts positively on the oral skills of

38

students. Similarities between SCMC and talking may make transfer natural. In fact, Abrams

(2003) found that the SCMC group produced significantly more language in the same period of

time than the ACMC and F2F groups. Thus the benefits of SCMC discussion were more

beneficial to the oral ability of language students than F2F discussion. This points clearly to the

place of SCMC in language instruction, at least as far as oral skills are concerned. However,

Abrams (2003) found that F2F discussion had more positive impact on student oral skills than

ACMC did. And in the current study, student oral skills were negatively affected by time spent in

ACMC discussion.

The goal of this study was to investigate the effects of ACMC discussion on subsequent

oral discussion. Results of this study showed; a) an increase in subsequent oral production of the

target language by the face-to-face group but a decrease in production of target language by the

ACMC group; b) an increase of TOEIC scores in both groups, and c) an increase in the use of the

L1 (Japanese) in subsequent oral discussion by the ACMC group. Based on the results of this

study, ACMC discussion appears to have an overall negative effect on oral production of the L2.

However, we must not conclude that ACMC discussion has no benefits at all for language

learning. It may be that ACMC has positive effects on student writing, or on other aspects. In this

study, student TOEIC scores rose in both groups, suggesting that ACMC has a positive effect on

overall English ability. ACMC may share SCMC’s positive effects on motivation (Beauvois

1992; Warschauer, 1996b) or on language anxiety. All possible effects must be measured and

evaluated before deciding to accept or reject any new technology.

Technology is a tool, and tools are neither good nor bad within themselves. It is the

results that tools produce, the impact that they have on learning, which must be evaluated and

then selected or rejected. In the case of CMC, teachers and researchers must be aware that there

39

is a variety of forms of CMC – ACMC, SCMC, voice chat, video conferencing, podcasting, etc.,

all with their own set of affordances and constraints, and while all of these technologies have

possible benefits for students, there are negative effects to be discovered as well. It is therefore of

the utmost importance that researchers continue to evaluate the impact of various technologies

on student learning. Only in this way can we be sure that we are choosing technology because it

is the right tool for the job.

40

References

Abrams, Z. (2003). The effect of synchronous and synchronous CMC on oral performance in German.

The Modern Language Journal, 87(2), 157-167.

American Psychological Association (2001). Publication Manual of the American Psychological

Association (5th ed.). Washington, DC: Author.

Beauvois, M. (1998). Conversations in slow motion: Computer-mediated communication in the

foreign language classroom. The Canadian Modern Language Review, 54(2), 198-217.

Beauvois, M. H. (1997). Computer-mediated communication: Technology for improving speaking and

writing. In M. D. Bush (Ed.), Technology enhanced language learning (pp. 165-184).

Lincolnwood, IL: National Textbook Company.

Beauvois, M. H., & Eledge, J. (1996). Personality types and megabytes: Student attitudes toward

computer-mediated communication (CMC) in the language classroom. CALICO Journal, 13,

27-45.

Beauvois, M. H. (1992). Computer-assisted classroom discussion in the foreign language classroom:

Conversation in slow motion. Foreign Language Annals, 25(5), 455-464.

Berge, Z. (1997). Computer conferencing and the online classroom. International Journal of

Educational Telecommunications, 3 (1), 3-21.

Blake, R. (2000) Computer mediated communication: a window on L2 Spanish interlanguage.

Language Learning & Technology, 4 (1),120-136.

41

Bohlke, O. (2003). A comparison of student participation levels by group size and language stages

during chatroom and face-to-face discussions in German. CALICO Journal, 21(1), 67–88.

Brown, H. (2004, 4th ed.). Principles of Language Learning and Teaching. New York: Prentice Hall.

Chapelle, C. (1997) CALL in the year 2000: Still in search of research paradigms? Language Learning

& Technology, 1 (1),19-43.

Chaudron, C. (1988). Second language classrooms: Research on teaching and learning. Cambridge:

Cambridge University Press.

Chun, D. (1994). Using computer networking to facilitate the acquisition of interactive competence.

System, 22 (1), 17-31.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.

Doyon , P. (2000) Shyness in the Japanese EFL Class. The Language Teacher, 24 (1) 11-17. The

Japanese Association for Language Teaching. [Electronic version]. Retrieved August 18, 2008,

from http://www.jalt-publications.org/tlt/articles/2000/01/doyon

Fitze, M. (2006). Discourse and participation in ESL face-to-face and written electronic conferences.

Language Learning and Technology, 10 (1), 67-86.

Flores, M. (1990). Computer conferencing: Composing a feminist community of writers. In C. Handa

(Ed.), Computers and community: Teaching composition in the twenty-first century (pp. 107-

139). Portsmouth, NH: Heinemann.

Gass, S. (1997). Input, interaction, and the second language learner. Mahwah, NJ: Lawrence Erlbaum

42

Associates.

Gray, R. & Stockwell, G. (1998). Using computer-mediated communication for language and culture

acquisition. On-CALL, 12 (3), 2-9.

Hoffman, R. (1996). Computer networks: webs of communication for language teaching. In M.

Pennington (Ed.), The power of CALL (pp. 55-78). Houston: Athelstan.

Hofstede, G. (2003). Geert Hofstede Cultural Dimensions. [Electronic version].

Retrieved on August 18, 2008 from http://www.geert-hofstede.com/hofstede_japan.shtml

Hofstede, G. (1991). Cultures and organizations: Software of the mind. New York: McGraw-Hill.

Kazdin, Alan E. (1992). Research Design in Clinical Psychology. Massachusetts: Allyn and Bacon.

Johnson, K. E. (1995). Understanding communication in second language classrooms. Cambridge:

Cambridge University Press.

Johnson, E. & Brine, J. (1999). Design and development of CALL courses in Japan. CALICO Journal,

17, (2), 251-268

Kelm, O. R. (1992). The use of synchronous computer networks in second language instruction: A

preliminary report. Foreign Language Annals, 25, 441-545.

Keppel, G., Saufley, W. H. Jr., & Tokunaga, H. (1992). Introduction to design and analysis. A

student's handbook. New York: W.H. Freeman and Company.

Kern, R. G. (1995). Restructuring classroom interaction with network computers: Effects on quantity

43

and characteristics of language production. The Modern Language Journal, 79, 457-476.

Kinginger, C. (2000). Learning the pragmatics of solidarity in the networked foreign language

classroom. In J. K. Hall & L. S. Verplaetse (Eds.), Second and foreign language learning

through classroom interaction (pp. 23-46). Mahwah, NJ: Erlbaum. 

Kobayashi, U. (2006). Effects of computer-mediated communication (CMC) on learner’s language

production and attitude towards oral communication in Japanese. Unpublished master’s thesis,

University of Alberta.

Komiyama, Y. (2005). Japanese learning styles in the online learning environment: A review of

literature. Unpublished manuscript, University of British Columbia.

Krashen, S. (1985). The input hypothesis. London: Longman

Kupelian, M. (2001). The use of e-mail in the L2 classroom: an overview. Second Language Learning

and Teaching, 1(1). [Electronic version]. Retrieved June 29, 2006, from

<http://www.usq.edu.au/opacs/sllt/1/Kupelian01.htm>.

Lamy, M.-N., & Goodfellow, R. (1999). "Reflective Conversation" in the Virtual Language

Classroom. Language Learning and Technology, 2(2): 43-61.

Lebra, T. (1976). Japanese patterns of behavior. Honolulu: University of Hawaii Press.

Lee, J., & Van Patten, B. (1995). Making communicative language teaching happen. New York:

McGraw-Hill, Inc.

44

Lee, L. (2002a). Synchronous online exchanges: A study of modification devices on non-native

discourse. System, 30, 275-288.

Lee, L. (2002b). Enhancing learners’ communication skills through synchronous electronic

interaction and task-based instruction. Foreign Language Annals 35(1), 16–24.

Littlewood, W. (1999). Defining and developing autonomy in East Asian contexts. Applied

Linguistics, 20(1), 71-94.

Liu, N.F. & Littlewood, W.T.(1997). Why do many students appear reluctant to participate in

classroom learning discourse? System, 25(3), 371-384.

Long, M. H. (1996). The role of linguistic environment in second language acquisition. In W. C.

Richie & T. K. Bhatia (Eds.), Handbook of research on language acquisition, Vol. 2: Second

language acquisition (pp. 413-468). New York: Academic Press.

Long, M. H. (1991). Focus on form: A design feature in language teaching methodology. In K. de Bot,

R. Ginsberg, & C. Kramsch (Eds.), Foreign language research in cross-cultural perspective

(pp. 39-52). Amsterdam: John Benjamins.

Long, M. (1980). Input, interaction and second language acquisition. Unpublished Ph.D. dissertation,

University of California at Los Angeles.

Long, M., & Robinson, P. (1998). Focus on form: theory, research and practice. In C. Doughty & J.

Williams. Focus on Form in Classroom Second Language Acquisition (pp. 15-41). Cambridge:

45

Cambridge University Press.

McCarty, S. (2000). Reforms in Computer Education that are Possible for Japan. Child Research Net,

Tokyo: Educational Visions.

MEXT (the Japanese Ministry of Education, Culture, Sports, Science, and Technology), (2002).

Developing a strategic plan to cultivate "Japanese With English Abilities". [Electronic

version]. Retrieved September 12, 2008, from

http://www.mext.go.jp/english/news/2002/07/020901.htm

Nelson, G. (1995). Cultural differences in learning styles.  In J. Reid (Ed.), Using learning styles in the

ESL classroom (pp. 3-18).  Boston: Heinle & Heinle.

Ortega, L. (1997). Processes and outcomes in networked classroom interaction: Defining the research

agenda for L2 computer-assisted classroom discussion. Language Learning & Technology,

1(1), 82-93.

Pallant, J. (2005). SPSS survival manual. Maidenhead: Open University Press.

Payne, J. S., & Whitney, P. J. (2002). Developing L2 oral proficiency through synchronous CMC:

Output, working memory, and interlanguage development. CALICO Journal, 20(1), 7-32.

Pellettieri, J. (2000). Negotiation in cyberspace: The role of chatting in the development of

grammatical competence. In M. Warschauer & R. Kern (Eds.), Network-based language

teaching: Concepts and practice. New York: Cambridge University Press.

Pellettieri, J. (1996). Network-based computer interaction and the negotiation of meaning in the

46

virtual foreign language classroom. Unpublished manuscript, University of California at

Davis.

Rao, Z. (2002). Bridging the gap between teaching and learning styles in East Asian contexts. TESOL

Journal, 11(2), 5-11.

Sato, C. (1982). Ethnic styles in classroom discourse. In Mary, E.H. & William, R (Eds.). On TESOL

'81. Washington, DC: Teachers of English to Speakers of Other Languages.

Schuele, C.M. & Justice, L.M. (2006). The importance of effect sizes in the interpretation of research.

The ASHA Leader, 11(10), 14-15, 26-27.

Selfe, C. L. (1990). Technology in the English classroom: Computers through the lens of feminist

theory. In C. Handa (Ed.), Computers and community: Teaching composition in the twenty-

first century. Portsmouth, NH: Heinemann.

Sherry, L. (2000). The nature and purpose of online discourse: A brief synthesis of current research as

related to the WEB project. International Journal of Educational Telecommunications, 6(1),

19-51.

Shimizu, J. (2006). Why are Japanese students reluctant to express their opinions in the classroom?

[Electronic version]. Retrieved September 6, 2006, from

http://www.review.keioup.co.jp/PaperFolder/pdf/01/2006/0104802.pdf

Smith, F. (1988). Joining the literary club: Further essays into education. Portsmouth, NH:

Heinemann.

47

Sotillo, S. (2000). Discourse Functions and Syntactic Complexity in Synchronous and Asynchronous

Communication. Language Learning & Technology, 4 (1) 82-119.

Sproull, L. & Kiesler, S. (1991). Connections: New ways of working in the networked organization.

Cambridge, MA: MIT Press.

St. John, E., & Cash, D. (1995). Language learning via e-mail: Demonstrable success with German. In

M. Warschauer (Ed.), Virtual connections: Online activities and projects for networking

language learners (pp. 191-197). Honolulu, HI: University of Hawai'i, Second Language

Teaching and Curriculum Center.

Stevens, J. (1996). Applied multivariate statistics for the social sciences (3rd edn.) Mahway, NJ:

Lawrence Erlbaum.

Sullivan, N. (1993). Teaching writing on a computer network. TESOL Journal 3, 34-35.

Sullivan, N. & Pratt, E. (1996). A comparative study of two ESL writing environments: a computer-

assisted classroom and a traditional oral classroom. System, 4, 491-501.

Swain, M. (2005). The output hypothesis: Theory and research. In E. Hinkel (Ed.), Handbook on

research in second language teaching and learning (pp. 471-484). Mahwah, NJ: Lawrence

Erlbaum.

Swain, M. (1985). Communicative competence: Some roles of comprehensible input and

comprehensible output in its development. In S. M. Gass & C. G. Madden (Eds.), Input in

second language acquisition (pp. 235–253). Rowley, MA: Newbury House.

48

Swain, M. (1995). Collaborative dialogue: Its contribution to second language learning. Annual

conference of the American Association for Applied Linguistics, Long Beach, CA.

Swain, M., & Lapkin, S. (1995). Problems in output and the cognitive processes they generate: A step

towards second language learning. Applied Linguistics 16, 371-391.

Tella, S. (1992). Talking shop via e-mail: A thematic and linguistic analysis of electronic mail

communication. (Research report No. 99). Department of Teacher Education, University of

Helsinki. (Also available as ERIC Document No. ED352015).

Toyoda, E. & Harrison, R. (2002) Categorization of text chat communication between learners and

native speakers of Japanese. Language Learning & Technology, 6 (1), 82-99.

Tudini, V. (2004). Virtual immersion: Native speaker chats as a bridge to conversational Italian.

Australian Review of Applied Linguistics. Series S, no. 18, 63-80.

Umemoto, M. (2001). Uncertainty avoidance and classroom interaction: Implications for language

teaching. [Electronic version]. Retrieved August 18, 2008, from

http://www.cels.bham.ac.uk/resources/essays/Umemoto%201.pdf

Vygotsky, L. (1978). Mind in society. Cambridge, Massachusetts: Harvard University Press.

Warschauer, M. (2001). Online communication. In R. Carter & D. Nunan (Eds.), The Cambridge

guide to teaching English to speakers of other languages (pp. 207-212). Cambridge:

Cambridge University Press.

49

Warschauer, M. (1998). Interaction, negotiation, and computer-mediated learning. In V. Darleguy, A.

Ding, & M. Svensson (Eds.), Educational Technology in language learning: Theoretical

reflection and practical applications. Lyon, France: National Institute of Applied Sciences,

Center of Language Resources, p. 125-136.

Warschauer, M. (1997). Computer-mediated collaborative learning: Theory and practice (Research

Note #17). Honolulu: University of Hawai'i, Second Language Teaching & Curriculum Center.

Warschauer, M. (1996a). Comparing face-to-face and electronic discussion in the second language

classroom. CALICO Journal, 13, 7-25.

Warschauer, M. (1996b). Motivational aspects of using computers for writing and communication. In

M. Warschauer (Ed.), Telecollaboration in foreign language teaching: Proceedings of the

Hawai’i symposium. Honolulu, HI: University of Hawai’i, Second Language Teaching and

Curriculum Center.

Zimbardo, P. (1977). Shyness: What it is - what to do about it. Reading, Massachusetts: Addison-

Wesley.

50