Teachers of English to Speakers of Other Languages, Inc. (TESOL)

Qualitative Research in Applied Linguistics: A Progress ReportAuthor(s): Anne LazaratonSource: TESOL Quarterly, Vol. 29, No. 3, Qualitative Research in ESOL (Autumn, 1995), pp. 455-472Published by: Teachers of English to Speakers of Other Languages, Inc. (TESOL)Stable URL: http://www.jstor.org/stable/3588071Accessed: 11/07/2010 19:56

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available athttp://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unlessyou have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and youmay use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained athttp://www.jstor.org/action/showPublisher?publisherCode=tesol.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printedpage of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

Teachers of English to Speakers of Other Languages, Inc. (TESOL) is collaborating with JSTOR to digitize,preserve and extend access to TESOL Quarterly.

http://www.jstor.org

TESOL QUARTERLY Vol. 29, No. 3, Autumn 1995

Qualitative Research in Applied Linguistics: A Progress Report ANNE LAZARATON The Pennsylvania State University

This article describes the status of qualitative research in applied linguistics and ESOL. It identifies trends by reporting on an informal survey of published journal articles, highlighting relevant published qualitative studies and research methods texts, and relating the views of research methodologists working within and outside the qualitative tradition. Several of the unresolved, persistent issues raised are prog- ress toward a definition of qualitative research, the role of quantifica- tion in qualitative research, and the generalizability of qualitative research.

This special-topic issue on Qualitative Research in ESOL can be seen as evidence of a second coming of age of the research in

applied linguistics.' The first was noted by Henning (1986) in his survey of published articles in the TESOL Quarterly and Language Learning; he concluded that the great majority of articles exemplified quantitative research and that this was "a positive development-a kind of coming of age of a discipline" (p. 704). This issue of the TESOL Quarterly contains two overview articles on qualitative research as well as three full-length papers and two shorter reports selected from more than 50 abstracts submitted in response to the original call for papers. In addition, Qualitative Research Guidelines are now set forth in the Information for Contributors section of this journal "to ensure that

Quarterly articles model rigorous qualitative research"; the Statistical Guidelines with which they appear have been in place for about 2 years.

'I use applied linguistics, rather than (T)ESOL, throughout this article as the term that encapsu- lates the scope of our field because it seems to be a more inclusive label, covering not just (second) language learning and/or language teaching but also cognition, comparative rhetoric, language planning and policy, language testing, and the broad area of language use. It also seems to be a better label for the discipline of which we are a part as opposed to the profession in which we are engaged. Obviously, this crucial issue cannot be taken up here; see Kaplan and Grabe (1992) and Strevens (1992) for more on the historical development and emerging definitions of applied linguistics, and Pennycook's (1990) argu- ment for a more critical applied linguistics.

455

Another sign that qualitative research is attaining more prominence in applied linguistics is the inclusion of both quantitative and qualitative topics in the Research Issues column that appears in every other issue of the TESOL Quarterly.

These facts suggest that qualitative research has made significant gains in terms of visibility and credibility in recent years, yet the pur- poses, assumptions, and methods of qualitative research are still de- bated, misunderstood, and/or ignored by some in our profession. One broad-based survey of 121 applied linguists (Lazaraton, Riggenbach, & Ediger, 1987) acknowledged that "qualitative approaches to data collection and analysis are clearly important for the types of questions asked in applied linguistics research" (p. 264); however, the survey only assessed statistical literacy. Surprisingly, only a few respondents complained that such a view of research was narrowly conceived and that it reflected neither the expertise nor the interests of the field at large.

An informal survey of four major journals in the field (Applied Lin- guistics, Language Learning, Studies in Second Language Acquisition, and the TESOL Quarterly) over the past 10 years reveals a growing interest in qualitative research issues and studies, but in terms of sheer numbers the "domination of the psychometric model" (Eisner & Peshkin, 1990) is still apparent. The proportion of empirical TESOL Quarterly articles that employ qualitative, and especially ethnographic, techniques has slowly increased over the past 10 years (e.g., Benson, 1989; Canagara- jah, 1993; Clair, 1995; Cleghorn & Genesee, 1984; Crago, 1992; Hark- lau, 1994), but these contributions represent only a fraction of the total articles published in that time period. Empirical articles in Applied Linguistics, Language Learning, and Studies in Second Language Acquisition cover a broad range of research topics and represent various qualitative research traditions (e.g., text analysis, discourse analysis, and case stud- ies), although ethnographic methods are still underrepresented in terms of total numbers; only three such studies were published in these journals in the past 10 years (Holliday, 1992; Poole, 1992; Ramp- ton, 1991).2 Of course, quality qualitative research has appeared in other professional publications (see, e.g., Gilmore & Glatthorn, 1982; Trueba, Guthrie, & Au, 1981). Encouragingly, a number of qualitative studies have appeared in the last two special-topic issues of the TESOL

2These findings are consistent with Nunan's (1991) critical analysis published in Studies in Second Language Acquisition of 50 published articles in applied linguistics. His findings showed that almost 40% of these articles used experimental techniques and that elicitation was the most common method of data collection. Although he found the classroom studies to be more interpretive (he notes, however, that all studies require some sort of interpretation), most of the research was narrow in focus and scope and was not situated within a defined social context.

TESOL QUARTERLY 456

Quarterly, one on adult literacies (Vol. 27, No. 3, 1993) and the other on K-12 (Vol. 28, No. 3, 1994).

We are indeed fortunate to have at our disposal a number of useful, user-friendly reference texts on research methods in applied linguis- tics, given the relatively short history of our discipline. However, these texts display a distinct bias toward quantitative research methods (I take the term quantitative methods to include the application of descrip- tive and/or inferential statistical procedures) and a consistent lack of attention to qualitative research, with two notable exceptions. Although Hamp-Lyons (1989) notes that the methodological choices available to the applied linguist have increased and that qualitative research is now common, none of the three research methods books she reviews (Brown, 1988; Butler, 1985; Woods, Fletcher, & Hughes, 1986) deals with qualitative research in any meaningful way. Even though Brown (1988) states that "statistical research is neither the only kind of re- search . . . nor even necessarily the best type" (p. 5), he deals exclusively with it in his text. Hatch and Lazaraton (1991) also treat only statistical research. Seliger and Shohamy (1989) attempt to "describe what can be considered paradigmatic types and principles of second language research" (p. 1) but provide only a brief, general discussion of qualita- tive research design, data collection, and analysis. Larsen-Freeman (1985), in a chapter on research methodology, contends that the (sub)- field of second language acquisition (SLA) embraces several different approaches to conducting research, yet the criteria for evaluating re- search that she presents display a distinct bias toward approaches that require the analysis of quantified data.

Two more recent texts, Nunan's (1992) Research Methods in Language Learning and Johnson's (1992) Approaches to Research in Second Language Learning, do devote considerable space to qualitative research concepts and methods. Nunan states that "two alternative conceptions of the nature of research provide a point of tension within the book" (pp. xi-xii) and dedicates much of the first chapter to a discussion of this issue. However, two reviewers of this text were unhappy with this approach. Galguera (1993), in his review of Nunan's book in Language Learning, contends that Nunan displays a bias toward his stated prefer- ence for nonexperimental research despite his attempts to provide a balanced and objective view. Another reviewer (Fang, 1994) criticizes Nunan for devoting too much space to comparing qualitative and quantitative research and suggests that the section on experimental design should have been expanded. Johnson's book also strives for balance and objectivity in the presentation of six research approaches (correlational, case study, survey, ethnography, experimental, and multisite/multimethod/large scale); the ordering of these chapters is helpful in achieving this goal (Lazaraton, 1994). Nunan's andJohnson's

QUALITATIVE RESEARCH: A PROGRESS REPORT 457

coverage of case studies is particularly important because there is an established case study tradition in linguistics and child language acquisition research; discussions of other qualitative methods that have their roots in anthropological and educational research (e.g., ethnogra- phy) may overlook this qualitative approach.

Notwithstanding the chapter-length coverage that qualitative re- search receives in these two books, the fact remains that there are to date no qualitative research methods texts written for and by applied linguists. In fact, only recently has qualitative research methodology been discussed independently of the quantitative methods with which it contrasts. Watson-Gegeo (1988) and Wolfson (1986) are two excep- tions; recent Research Issues contributions in the TESOL Quarterly by Blot (1991), Davis (1992), Johnson and Saville-Troike (1992), and Ulichny (1991) have also begun to rectify this disparity in coverage and to provide a forum for voices within the community of qualitative researchers. Perhaps the conclusion to draw from these observations is that we are in the same position today, with large gaps in resources and reference coverage on qualitative research, that we were in 15 years ago with quantitative research, when the field used teaching texts and reference books from the allied disciplines of education, linguistics, psychology, and sociology until an applied linguistics model became available (e.g., Hatch & Farhady, 1982). Bogdan and Biklen (1992) make a similar point in the introduction to the second edition of their text: When their first edition appeared in 1982, few texts on qualitative research in education were available. Today, there are many choices available to the applied linguistics researcher; some of the more frequently cited references in texts on research methods in applied linguistics and in published empirical studies include Fetterman (1989), Goetz and LeCompte (1984), Lincoln and Guba (1985), Miles and -luberman (1994), Patton (1990), and Spradley (1979, 1980). Heath

(1983) remains the most widely recognized (and cited) example of an applied linguistics ethnography. Note also that a growing number of specialized research methods texts are linked to specific topics such as discourse analysis (e.g., Cook, 1989; Hatch, 1992; McCarthy, 1991; see also Schiffrin, 1994, for a very thorough treatment of six prominent discourse-analytic approaches).

The shortage of material on qualitative research for and by the applied linguist undoubtedly accounts at least in part for the lack of consensus on and the confusion about what qualitative research is and what it can and cannot do. Although it is beyond the scope of this article to do justice to the numerous issues surrounding qualitative research methodology in applied linguistics (a book waiting to happen, we can hope), the remainder of this article highlights a few of the

TESOL QUARTERLY 458

issues about which we urgently need further dialog and examination. I hope that by bringing up these issues again, in this forum, we will be obligated to examine our assumptions and our biases about the procedures and applications of all kinds of research, and a more informed debate about them will be possible.

TOWARD A DEFINITION OF QUALITATIVE RESEARCH

Statisticians try to measure IT. Experimentalists try to control IT. Evaluators value IT. Interviewers ask questions about IT. Observers watch IT. Participant observers do IT. (Patton, 1990, p. 7)

In our research, it is easy to be misled into believing there is one superior method for understanding "IT." Jacob (1987) notes that the qualitative-quantitative dichotomy leads one to conclude that only two methodological alternatives are available to the educational researcher. This conclusion is clearly false when it comes to data collection and analysis in statistical research (see, for example, Hatch & Lazaraton, 1991, pp. 544-545); it is also untrue for qualitative research, as there is no one qualitative approach but a "variety of alternative approaches" (Jacob, 1987, p. 1). In fact, Denzin and Lincoln (1994) distinguish six interpretive paradigms and perspectives that guide the research process: positivism/postpositivism, constructivism, feminism, ethnic models, Marxist models, and cultural studies models.

Anthropologists, educators, evaluators, and sociologists (e.g., Berg, 1989; Bogdan & Biklen, 1992; Jacob, 1987; Miles & Huberman, 1994; Patton, 1990) distinguish qualitative research traditions, approaches, or types that have one or more distinctive disciplinary roots, as shown in Figure 1. Each of the traditions in the figure employs one or more qualitative research strategies (e.g., nonparticipant observation, partici- pant observation, interviews, and archival strategies); there are also qualitative themes associated with these traditions that embody the central assumptions of qualitative research (e.g., naturalistic, descrip- tive, emic, interpretive, inductive, holistic, contextualized, etc.; see Rei- chardt & Cook, 1979, for a complete discussion of the applicability of these thematic concepts and their quantitative counterparts). Of course, each of the aforementioned qualitative approaches has its own philosophy, literature, and guidelines for conducting research and

QUALITATIVE RESEARCH: A PROGRESS REPORT 459

FIGURE 1

Qualitative Research Traditions and Their Disciplinary Roots

Holistic ethnography Anthropology Ethnography of communication Anthropology, sociolinguistics Cognitive anthropology Anthropology, linguistics Discourse analysis Linguistics Phenomenology Philosophy Ecological psychology Psychology Symbolic interactionism Social psychology Heuristics Humanistic psychology Ethnomethodology Sociology Hermeneutics Theology, philosophy, literary criticism

reporting outcomes; obviously, the researcher who adopts any such approach needs a firm grounding in the literature and procedures.

The applied linguistics literature, however, tends to blur these quali- tative approaches and attempts instead first to distinguish between qualitative and quantitative research traditions themselves3 and then to discuss their attributes (or themes). This is not to say that qualitative research is never considered on its own or that a blending of qualitative and quantitative approaches is not addressed, but the general trend is one of contrast between traditions. Figure 2 gives an initial frame of reference for considering the basic distinctions regarding research traditions present in a number of texts and articles on research method- ology that were considered in preparation of this article.

Unfortunately, because these applied linguistics sources do not all define and delineate types of qualitative research, some ambiguity and confusion remains in terms of understanding what counts as qualitative research and what does not. As mentioned, ethnographies and case studies are the two types of qualitative research discussed by Johnson (1992) and Nunan (1992); the latter also covers interaction analysis. Although properly conducted anthropological or educational ethno- graphies are clear examples of qualitative research, the status of other qualitative methods is less clear. For example, even though case studies are frequently employed in applied linguistics research, the approach "in and of itself does not constitute ethnographic [italics added] re- search" (Heath, 1982, p. 36), as ethnography requires a deeper and broader philosophical and methodological commitment than does sim- ple participant observation; a case study may, in fact, be not an analytic

3As should be clear from the discussion so far, there is little consistency in, and perhaps unnecessary confusion caused by, the use of the terms paradigm, tradition, method, design, technique, strategies, and so on in the literature thus far cited.

TESOL QUARTERLY 460

FIGURE 2

Distinctive Research Traditions

Source Distinctive Research Tradition Oschner, 1979a Nomothetic versus Hermeneutic Henning, 1986 Nonquantitativeb versus Quantitative Grotjahn, 1987 Exploratory-interpretive versus Analytical-Nomonological Brown, 1988 Case study versus Statistical' Chaudron, 1988 Ethnographic versus Interactional versus Discourse Analytic

versus Experimental/Psychometric Seliger & Shohamy, Qualitative versus Descriptive versus Experimental

1989 Larsen-Freeman & Longitudinal versus cross-sectional

Long, 1991 Johnson, 1992 Constructivist/interpretive versus Scientific/Interventionist Nunan, 1992 Ethnographic versus Psychometric Cumming, 1994 Descriptive versus Interpretive versus Ideological

aOschner (1979, p. 55) presents a set of distinctions similar to the ones in the figure but derived from a much broader historical and disciplinary perspective. bHenning attempts to provide a definition of quantitative research, "as opposed to qualitative or anecdotal research" (1986, p. 701). cIn a later publication, Brown (1991) carefully shuns the term empirical when discussing statistical research, stating that "there are other, nonstatistical studies that could be called empirical (e.g., ethnographies, case studies, etc.), since, by definition, empirical studies are those based on data (but not necessarily quantitative data)" (p. 570).

approach at all but a data collection technique. And classroom interac- tion analysis (e.g., Spada, 1994), the interpretive orientation with which ethnography (Hornberger, 1994) is grouped in Cumming's recent (1994) article on TESOL research orientations, is considered by some to represent a significant deviation from true qualitative research in the sense that ethnography is (Hymes, 1982; Mehan, 1981; Watson- Gegeo, 1988). It is also debatable whether the other orientations de- scribed in Cumming (1994) should be considered qualitative. Is the critical ethnography authored by Canagarajah (1993) to be viewed as an interpretive orientation, an ideological one, or both? As for the aforementioned Qualitative Research Guidelines now in the TESOL Quarterly, they are clearly designed with ethnographic research in mind and are difficult to apply to many discourse-analytic studies (e.g., Bar- dovi-Harlig & Hartford, 1990; Ernst, 1994; Lazaraton, in press; Strodt- Lopez, 1991; Tyler, 1992), which use data sources (especially carefully transcribed recorded interactions) that present the researcher with a different set of data analysis and presentation concerns than does ethnography. Therefore, it remains to be seen whether a more inclu- sive or a more exclusive view of qualitative research will be necessary to conceptualize the traditions to which applied linguistics subscribes and to understand the research undertaken.

QUALITATIVE RESEARCH: A PROGRESS REPORT 461

THE ROLE OF QUANTIFICATION IN QUALITATIVE RESEARCH

What does one make of an approach to the study of the educational world that depends upon the unique aptitudes or proclivities of the investigator, that possesses no standardized method, that focuses upon nonrandomly selected situations, and that yields questionable generalizations by conven- tional research criteria? Indeed, are we justified in referring to the use of such a collection of procedures as "research"? (Eisner & Peshkin, 1990, p. 10)

In addition to the problem of defining what constitutes qualitative research (and not just applying it as a catchall term for studies that are not quantitative), a fair amount of controversy exists about the scientific rigor of qualitative research. This controversy is not unique to applied linguistics, and the debate about it appears in essentially all the relevant historical literature from education (see, e.g., Eisner & Peshkin, 1990, who trace the history of qualitative research in the American Educational Research Association, and Bogdan & Biklen, 1992, who trace qualitative research back to its roots). The rigor argu- ment seems to encompass two related issues: that quantification of qualitative data is not only possible and desirable but necessary in order to make generalizable claims to and about other contexts.

The strong position on quantification is unequivocally expressed by Henning (1986), who maintains that "without some recourse to quantitative methods, some marriage of words and numbers, it is inconceivable that the investigation of language acquisition will ever be said to belong to the realm of scientific [italics added] inquiry" (p. 702). For Henning, quantitative methods allow the researcher to go beyond the data themselves (the "tyranny of the single case," Erickson, 1981, p. 27) and to generalize to other instances not studied. Adopting a similar but somewhat weaker position, Chaudron (1988) points out that

Process-oriented qualitative researchers explore the intersubjective and context-dependent nature of classroom events as they occur, noting the regularities and idiosyncrasies in the events. In order for researchers to derive the implicit rules governing the participants' behavior, however, regularity of particular events or sequences in the discourse must be ob- served. This regularity then will support reliable claims about rules of interaction. It also allows for counting and other quantitative analyses; the ultimate need for generality and for comparisons across classroom contexts inevitably requires such quantification of events. Regrettably, too few re- searchers with an ethnographic orientation have provided the validation necessary for generalization to other contexts. (p. 49)

TESOL QUARTERLY 462

Elsewhere in the same book, Chaudron claims that "almost every ethno- graphic or discourse analytical study refers to the frequency, magni- tude, or proportion of occurrences of analytical units observed" (p. 15) and that "most researchers adopting qualitative or ethnographic tech- niques have recognized the need to continue their analysis with some quantification of events" (p. 47). However, this statement is not sup- ported by much of the published research cited in this article. Although most of the published discourse-analytic studies analyze essentially qualitative data,4 these studies were arguably designed to collect such data and then to analyze them via quantification;5 in other words, the intent may not have been qualitative analysis at all. In any case, the data presented in these studies are usually in the form of descriptive statistics (frequency counts, percentages) that are not analyzed statisti- cally at all.

The fact that some qualitative researchers themselves employ or recommend quantification further complicates the situation. Watson- Gegeo (1988) claims that in a hypothesis-oriented mode, qualitative research may "involve quantification in the form of frequency counts, tests of significance, or multivariate analyses of patterns and themes" (pp. 584-585). In fact, four of the seven empirical studies in Trueba et al. (1981) quantified at least some of their data. And, to my surprise, Heath's (1983) ethnography of language learning and use and Mehan's (1979) study of classroom discourse do contain several tables of de- scriptive statistics, although these quantified data appear to be pre- sented for the purpose of making sense of the phenomena noted in those contexts rather than for the purpose of generalizing to or comparing with other contexts not studied.

In fact, very few (too few, perhaps) researchers design studies that employ both qualitative and quantitative approaches, despite the fact that, today, "bimethodologicalism" may be "a true mark of scholarly sophistication" (Eisner & Peshkin, 1990, p. 7). Although obtaining data from multiple sources using multiple collection techniques is not uncommon in applied linguistics, triangulation of analytic approaches appears to be, perhaps because multimethod studies require the re- searcher(s) to be trained in each of the analytic methods; and such

4Confusion over this term may also arise when one considers that some experimentalists (e.g., Kirk, 1982) refer to nominal data as qualitative.

5Although it is safe to say these data were quantified, these researchers present no evidence that such data were, in fact, quantifiable. In other words, actual counting may be a simple, straightforward matter, but it is often much more difficult to justify the counting and coding of features that was done. This issue is explored in depth by Schegloff (1993), who maintains that quantification of data from naturally occurring conversation is premature, given our incomplete understanding of both the structures we may wish to count and the environments in which they relevantly occur. See Scholfield (1995) for another opinion.

QUALITATIVE RESEARCH: A PROGRESS REPORT 463

studies tend to be both time consuming and expensive. A number of such studies probably exist, but I located only the few examples cited here. In an ethnographic study of culture, environment, and cognition among children in Puerto Rico, Jacob (1982) generated a set of vari- ables that were later subjected to path analysis, a correlation-based statistical procedure that examines both direct and indirect effects of a set of variables on some outcome variable (Hatch & Lazaraton, 1991). Johnson's (1987) evaluation of a migrant education program analyzed both coded observational data and qualitative interview data. In an- other study, Lazaraton and Saville (1994) used qualitative discourse analysis of the oral interview process and multifaceted Rasch analysis (Linacre, 1993) of interview outcome ratings to validate an interlocutor support rating scale in the Cambridge Assessment of Spoken English; both techniques generated the same conclusions about certain oral examiners but in different ways. Other studies have combined qualita- tive discourse-analytic methods with frequency counts and descriptive statistics (see, e.g., example, Rounds, 1987, on international teaching assistants' talk and Goldstein & Conrad, 1990, on writing conferences).

Therefore, it seems clear that researchers do not always follow the prescriptions about what one should, or should not, do when utilizing a particular research approach and that the rigid dichotomies pre- sented and discussed in the literature do not necessarily match the reality of the research undertaken in the field. The observations here may not represent an adequate response to the points raised by Hen- ning and Chaudron, researchers who can be considered to represent the quantitative paradigm and who may not be the ones to whom qualitative researchers should respond. As the body of work by qualita- tive researchers in applied linguistics grows, we can hope to find further guidance on methodological and analytic issues such as quantification.

THE GENERAl IZABILITY OF QUALITATIVE RESEARCH

"So what?" is a question sometimes asked of the detailed descriptions pro- vided by anthropologists of minutiae. To what extent is material and the sense of a particular phenomenon developed for one social group generaliz- able to other social groups? The same question can certainly be asked of studies of a single school or classroom or situation within a formal educa- tional setting. (Heath, 1982, p. 41)

Perhaps the most frequent criticism leveled against qualitative re- search is that the results obtained are not generalizable to other con- texts (see, e.g., the arguments in Larsen-Freeman & Long, 1991; Long, 1983; Seliger & Shohamy, 1989). According to Davis (1992), qualitative

TESOL QUARTERLY 464

researchers strive for transferability of findings, and "the degree to which working hypotheses can transfer to other times and contexts is an empirical matter, depending on the degree of similarity between the two contexts" (p. 606); thus the need for sufficiently thick descrip- tions of the study context (see also Schofield, 1990, for some sugges- tions on increasing the generalizability of qualitative research).

Although it is not possible to do justice to the issue of generalizability in this article, several points should be made. First, generalizability in research is more than a matter of counting. Quantification of any set of data does not ensure generalizability to other contexts, nor does a large sample size: Population characteristics must be carefully consid- ered when selecting a sample from which to make statistical inferences. Although the vast majority of the published studies in applied linguis- tics may employ quantification of data, a much smaller number can be considered to have used a large sample size, and even fewer still randomly select and assign subjects to treatment conditions, the tradi- tional prescriptive requirement for generalizability to some population at large. In other words, generalizability is a serious problem in nearly all the research conducted in our field. Second, even meeting these stringent criteria does not guarantee meaningful interpretation of results:

Even statistically significant findings from studies with huge, randomly selected samples cannot be applied directly to particular individuals in particular situations; skilled clinicians will always be required to determine whether a research generalization applies to a particular individual, whether the generalization needs to be adjusted to accommodate individual idiosyn- crasy, or whether it needs to be abandoned entirely with certain individuals in certain situations. (Donmoyer, 1990, p. 181)

Finally, critical theory has made a significant contribution to our profession in that we have begun to question the meaning of concepts that we take for granted (e.g., Pennycook, 1989 on method; 1994 on alternative approaches to research). We should probably view the arguments about quantification and generalizability in the same way. According to Donmoyer (1990), matters of research methodology are not just abstract, epistemological issues about the way we view the world: They are also issues of legitimacy and power. Theorists and researchers tend to have greater access to data sets that lend themselves to quantification and generalization; practitioners, especially teachers, deal with individuals and do not normally have the resources to access or to analyze large aggregates of data. In other words, we must be reminded and we must remember that arguments about the character- istics of rigorous research cannot be divorced from the political reali- ties, and the ideological biases, of our profession.

QUALITATIVE RESEARCH: A PROGRESS REPORT 465

WHY SO LITT'l'LE PUBLISHED QUALITATIVE RESEARCH?

What qualitative research does best and most essentially is to describe key incidents in functionally relevant descriptive terms and place them in some relations to the wider social context, using the key incident as a concrete instance of the workings of the abstract principles of social organization. (Erickson, 1981, p. 22)

Clearly, no research approach is suitable for every situation or question. Nevertheless, we might ask why qualitative research is not more preva- lent than it is in applied linguistics, given our interest in the social and/ or sociocultural context of language learning and use. Watson-Gegeo (1988) suggests that one reason ethnography is not more widely used in SLA studies is that it views language learning from a language socialization rather than language acquisition perspective, crediting context and culture for much of what happens in the learning environ- ment. Because many of the studies that use elicited, experimental data rarely consider these factors, it is understandable why the approach has not been more widely adopted. Training is probably another factor. An examination of the listings in the Directory of Professional Preparation Programs in TESOL in the United States (Kornblum, 1992) suggests that graduate students are normally exposed to a traditional research de- sign and statistics course within their own departments or programs; such a course is often required. Departments of ESOL and applied linguistics less commonly teach general qualitative methods courses, so students must seek out such courses in other departments (although specialized research methods courses, such as discourse analysis, may be offered). The faculty who supervise such students may themselves be trained as quantitative researchers, and logically they would feel more comfortable advising students working within that tradition. Al- though there are books available to use for self-study and reference, it is not an easy task to train oneself in any research methodology. In fact, without rigorous training in qualitative methods, "blitzkrieg" qualitative research, in which the researcher conducts a few interviews and/or observations, then labels the project a qualitative study, is an unfortunately common occurrence. Finally, anyone who has completed a qualitative research project is familiar with the sheer size of the resulting document. This makes publication in most journals, which normally limit contributions to about 25 pages, difficult in terms of providing a thick description of the context and a comprehensive account of the results. Editorial board members or outside reviewers of such journals, who may or may not be conversant in qualitative research, may recommend changes to qualitative manuscripts that ac- tually violate principles of the particular approach. Although these

TESOL QUARTERLY 466

facts cannot fully account for the status of qualitative research in the field, they surely play a part in the situation today.

CONCLUSION

The pendulum metaphor, which is frequently invoked in literature on language teaching methodology, applies to the dialectic on qualita- tive and quantitative research methodology as well. According to Rei- chardt and Cook (1979), in part because quantitative research methods were taught and employed zealously and exclusively in the past, we now see that they are fallible; they are not always best suited for a particular purpose. But it would be a mistake to assume that the past overapplication of quantitative techniques can only be rectified by an equally drastic swing toward qualitative methods. As Reichardt and Cook point out, once qualitative methods are given the acid test, they will prove no better than the ones they were meant to improve on; thus the pendulum swings back in the other direction and "the current debate keeps the pendulum swinging between extremes of methods and extremes of dissatisfaction" (p. 27). Researchers should choose an approach in light of the purpose of the study:

What is important for researchers is not the choice of a priori paradigms, or methodologies, but rather to be clear about what the purpose of the study is and to match that purpose with the attributes most likely to accomplish it. Put another way, the methodological design should be determined by the research question. (Larsen-Freeman & Long, 1991, p. 14)

It remains to be seen whether 10 years hence qualitative research will be on equal footing with quantitative research in how frequently it is employed and how it is received by the profession. A special forum such as this one probably does a better job of serving as a podium for previously unheard voices than of persuading its readers that such new voices are worth listening to; surely the latter is the ultimate goal of this special-topic issue. If graduate students pursuing degrees in applied linguistics and related fields are primarily or exclusively ex- posed to quantitative techniques in research methods courses and to articles employing these methods in professional journals, how will they ever come to respect the qualitative voice as equally important as and, in some cases, more appropriate than other methodological choices?

An important responsibility facing the qualitative research commu- nity is to ensure that qualitative methods and the studies that employ them are relevant, and accessible, to the practitioner-as a consumer or a producer of qualitative research. This task should not be difficult,

QUALITATIVE RESEARCH: A PROGRESS REPORT 467

given that one strength of qualitative research is the rich descriptions of context that result, contexts such as classrooms, schools, and commu- nities that are often familiar in a general sense from personal experi- ence. It is hoped that those engaged in pedagogy will not see qualitative research as removed, irrelevant, or too abstract, a complaint that one often hears about much of the quantitative research published in our journals. For this reason I eagerly await publications such as Bailey and Nunan (in press) that promise to bridge the theory/research-prac- tice gap.

Perhaps consensus on the definitions, principles, and value of quali- tative research is not necessary, desirable, or even possible. Neverthe- less, I believe it is incumbent on those of us who consider ourselves research methodologists representing any particular orientation to strive for balance, objectivism, and open-mindedness in presenting and evaluating the myriad of choices available to the applied linguistics researcher. As Johnson and Saville-Troike (1992) point out, our com- mitment should always be to quality research, not just to research that represents one particular paradigm.

ACKNOWLEDGMENTS

I thank Sandra McKay, Nancy Hornberger, and Donna Johnson for their helpful comments on an earlier draft of this article.

THE AUTHOR

Anne Lazaraton is Assistant Professor of Speech Communication at the Pennsylva- nia State University. Her research interests include conversation analysis, native speaker-nonnative speaker and nonnative speaker-nonnative speaker interaction, qualitative approaches to oral proficiency interview validation, and issues of re- search methodology.

REFERENCES

Bailey, K. M., & Nunan, D. (Eds.). (in press). Voices from the language classroom: Qualitative research in second language education. New York: Cambridge University Press.

Bardovi-Harlig, K., & Hartford, B. S. (1990). Congruence in native and nonnative conversation: Status balance in the academic advising session. Language Learning, 40, 467-501.

Benson, M. J. (1989). The academic listening task: A case study. TESOL Quarterly, 23, 421-445.

TESOL QUARTERLY 468

Berg, B. L. (1989). Qualitative research methods for the social sciences. Boston: Allyn & Bacon.

Blot, R. K. (1991). The role of hypothesis testing in qualitative research. TESOL Quarterly, 25, 202-205.

Bogdan, R., & Biklen, S. K. (1992). Qualitative research for education: An introduction to theory and methods (2nd ed.). Boston: Allyn & Bacon.

Brown, J. D. (1988). Understanding research in second language learning. Cambridge: Cambridge University Press.

Brown, J. D. (1991). Statistics as a foreign language-Part 1: What to look for in reading statistical language studies. TESOL Quarterly, 25, 569-586.

Butler, C. (1985). Statistics in linguistics. Oxford: Oxford University Press. Canagarajah, A. S. (1993). Critical ethnography of a Sri Lankan classroom: Ambi-

guities in student opposition to reproduction through ESOL. TESOL Quarterly, 27, 601-626.

Chaudron, C. (1988). Second language classrooms. Cambridge: Cambridge University Press.

Clair, N. (1995). Mainstream classroom teachers and ESL students. TESOL Quar- terly, 29, 189-196.

Cleghorn, A., & Genesee, F. (1984). Languages in contact: An ethnographic study of interaction in an immersion school. TESOL Quarterly, 18, 595-625.

Cook, G. (1989). Discourse. Oxford: Oxford University Press. Crago, M. (1992). Communicative interaction and second language acquisition:

An Inuit example. TESOL Quarterly, 26, 487-505. Cumming, A. (Ed.). (1994). Alternatives in TESOL research: descriptive, interpre-

tive, and ideological orientations. TESOL Quarterly, 28, 673-703. Davis, K. A. (1992). Validity and reliability in qualitative research on second

language acquisition and teaching. TESOL Quarterly, 26, 605-608. Denzin, N. K., & Lincoln, Y. S. (Eds.). (1994). Handbook of qualitative research.

Thousand Oaks, CA: Sage. Donmoyer, R. (1990). Generalizability and the single-case study. In E. W. Eisner

& A. Peshkin (Eds.), Qualitative inquiry in education: the continuing debate (pp. 175-200). New York: Teachers College Press.

Eisner, E. W., & Peshkin, A. (Eds.). (1990). Qualitative inquiry in education: The continuing debate. New York: Teachers College Press.

Erickson, F. (1981). Some approaches to inquiry in school-community ethnogra- phy. In H. T. Trueba, G. P. Guthrie, & K. H. Au (Eds.), Culture and the bilingual classroom: Studies in classroom ethnography (pp. 17-35). Rowley, MA: Newbury House.

Ernst, G. (1994). "Talking circle": Conversation and negotiation in the ESL class- room. TESOL Quarterly, 28, 293-322.

Fang, Z. (1994). [Review of the book Research methods in language learning]. TESOL Quarterly, 28, 212-213.

Fetterman, D. M. (1989). Ethnography step by step. Newbury Park, CA: Sage. Galguera, T. (1993). [Review of the book Research methods in language learning].

Language Learning, 43, 289-294. Gilmore, P., & Glatthorn, A. A. (Eds.). (1982). Children in and out of school: Ethnogra-

phy and education. Washington, DC: Center for Applied Linguistics. Goetz, J. P., & LeCompte, M. D. (1984). Ethnography and qualitative design in educa-

tional research. Orlando, FL: Academic Press. Goldstein, L. M., & Conrad, S. M. (1990). Student input and negotiation of mean-

ing in ESL writing conferences. TESOL Quarterly, 24, 443-460. Grotjahn, R. (1987). On the methodological basis of introspective methods. In

QUALITATIVE RESEARCH: A PROGRESS REPORT 469

C. Faerch & G. Kasper (Eds.), Introspection in second language research (pp. 54- 81). Clevedon, England: Multilingual Matters.

Hamp-Lyons, L. (1989). Recent publications on statistics, language testing, and quantitative research methods: I. TESOL Quarterly, 23, 127-132.

Harklau, L. (1994). ESL versus mainsti-eam classrooms: Contrasting L2 learning environments. TESOL Quarterly, 28, 241-272.

Hatch, E. (1992). Discourse and language education. New York: Cambridge University Press.

Hatch, E., & Farhady, H. (1982). Research design and statistics for applied linguistics. Rowley, MA: Newbury House.

Hatch, E., & Lazaraton, A. (1991). The research manual: Design and statistics for applied linguistics. New York: Newbury House.

Heath, S. B. (1982). Ethnography in education: Defining the essentials. In P. Gilmore & A. A. Glatthorn (Eds.), Children in and out of school: Ethnography and education (pp. 33-55). Washington, DC: Center for Applied Linguistics.

Heath, S. B. (1983). Ways with words: Language, life, and work in communities and classrooms. Cambridge: Cambridge University Press.

Henning, G. (1986). Quantitative methods in language acquisition research. TESOL Quarterly, 20, 701-708.

Holliday, A. (1992). Tissue rejection and informal orders in ELT projects: Collect- ing the right iniformation. Applied Linguistics, 13, 403-424.

Hornberger, N. H. (1994). Ethnography. In A. Cumming (Ed.), Alternatives in TESOL research: Descriptive, interpretive, and ideological orientations. TESOL Quarterly, 28, 688-690.

Hymes, D. (1982). What is ethnography? In P. Gilmore & A. A. Glatthorn (Eds.), Children in and out of school: Ethnography and education (pp. 21-32). Washington, DC: Center for Applied Linguistics.

Jacob, E. (1982). Combining ethnographic and quantitative approaches: Sugges- tions and examples from a study on Puerto Rico. In P. Gilmore & A. A. Glatthorn (Eds.), Children in and out of school. Ethnography and education (pp. 124-147). Washington, DC: Center for Applied Linguistics.

Jacob, E. (1987). Qualitative research traditions: A review. Review of Educational Research, 57, 1-50.

Johnson, D. M. (1987). The organization of instruction in migrant education: Assistance for children and youth at risk. TESOL Quarterly, 21, 437-459.

Johnson, D. M. (1992). Approaches to research in second language learning. New York: Longman.

Johnson, D. M., & Saville-Troike, M. (1992). Validity and reliability in qualitative research on second language acquisition and teaching. TESOL Quarterly, 26, 602-605.

Kaplan, R. B., & Grabe, W. (1992). Introduction. In W. Grabe & R. Kaplan (Eds.), Introduction to applied linguistics (pp. 1-9). Reading, MA: Addison- Wesley.

Kirk, R. E. (1982). Experimental design (2nd ed.). Belmont, CA: Wadsworth. Kornblum, H. (with Garshick, E.). (1992). Directory of professional preparation pro-

grams in TESOL in the United States, 1992-1994. Alexandria, VA: TESOL. Larsen-Freeman, D. (1985). Considerations in research design in second language

acquisition. In M. Celce-Murcia (Ed.), Beyond basics: Issues and research in TESOL (pp. 125-136). Rowley, MA: Newbury House.

Larsen-Freeman, D., & Long, M. H. (1991). An introduction to second language acquisition research. London: Longman.

Lazaraton, A. (1994). [Review of the book Approaches to research in second language learning]. TESOL Quarterly, 28, 208-209.

TESOL QUARTERLY 470

Lazaraton, A. (in press). Interlocutor support in oral proficiency interviews: The case of CASE. Language Testing.

Lazaraton, A., Riggenbach, H., & Ediger, A. (1987). Forming a discipline: Applied linguists' literacy in research methodology and statistics. TESOL Quarterly, 21, 263-277.

Lazaraton, A., & Saville, N. (1994, March). Process and outcome in oral assessment. Paper presented at the 14th Language Testing Research Colloquium, Washing- ton, DC.

Linacre, J. M. (1993). Many-faceted Rasch measurement. Chicago: Mesa Press. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage. Long, M. H. (1983). Inside the "black box": Methodological issues in classroom

research on language learning. In H .W. Seliger & M. H. Long (Eds.), Classroom oriented research in second language acquisition (pp.3-36). Rowley, MA: Newbury House.

McCarthy, M. (1991). Discourse analysis for language teachers. Cambridge: Cambridge University Press.

Mehan, H. (1979). Learning lessons: Social organization in the classroom. Cambridge, MA: Harvard University Press.

Mehan, H. (1981). Ethnography of bilingual education. In H. T. Trueba, G. P. Guthrie, & K. H. Au (Eds.), Culture and the bilingual classroom: Studies in classroom ethnography (pp. 36-55). Rowley, MA: Newbury House.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thou- sand Oaks, CA: Sage.

Nunan, D. (1991). Methods in second language classroom-oriented research: A critical review. Studies in Second Language Acquisition, 13, 249-274.

Nunan, D. (1992). Research methods in language learning. Cambridge: Cambridge University Press.

Oschner, R. (1979). A poetics of second language acquisition. Language Learning, 29, 53-80.

Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage.

Pennycook, A. (1989). The concept of method, interested knowledge, and the politics of language teaching. TESOL Quarterly, 23, 589-628.

Pennycook, A. (1990). Towards a critical applied linguistics for the 1990s. Issues in Applied Linguistics, 1, 8-28.

Pennycook, A. (1994). Critical pedagogical approaches to research. In A. Cumming (Ed.), Alternatives in TESOL research: Descriptive, interpretive, and ideological orientations. TESOL Quarterly, 28, 690-693.

Poole, D. (1992). Language socialization in the second language classroom. Lan- guage Learning, 42, 593-616.

Rampton, M. B. H. (1991). Second language learners in a stratified multilingual setting. Applied Linguistics, 12, 229-248.

Reichardt, C. S., & Cook, T. D. (1979). Beyond qualitative versus quantitative methods. In T. D. Cook & C. S. Reichardt (Eds.), Qualitative and quantitative methods in evaluation research (pp. 7-32). Beverly Hills, CA: Sage.

Rounds, P. L. (1987). Characterizing successful classroom discourse for NNS teaching assistant training. TESOL Quarterly, 21, 643-671.

Schegloff, E. A. (1993). Reflections on quantification in the study of conversation. Research on Language and Social Interaction, 26, 99-128.

Schiffrin, D. (1994). Approaches to discourse. Cambridge, MA: Blackwell. Schofield, J. W. (1990). Increasing the generalizability of qualitative research. In

E. W. Eisner & A. Peshkin (Eds.), Qualitative inquiry in education: The continuing debate (pp. 201-232). New York: Teachers College Press.

QUALITATIVE RESEARCH: A PROGRESS REPORT 471

Scholfield, P. (1995). Quantifying language: A researcher's and teacher's guide to gather- ing language data and reducing it to figures. Clevedon, England: Multilingual Matters.

Seliger, H. W., & Shohamy, E. (1989). Second language research methods. Oxford: Oxford University Press.

Spada, N. (1994). Classroom interaction analysis. In A. Cumming (Ed.), Alterna- tives in TESOL research: Descriptive, interpretive, and ideological orientations. TESOL Quarterly, 28, 685-688.

Spradley,J. (1979). The ethnographic interview. New York: Holt, Rinehart & Winston. Spradley, J. (1980). Participant observation. New York: Holt, Rinehart & Winston. Strevens, P. (1992). Applied linguistics: An overview. In W. Grabe & R. Kaplan

(Eds.), Introduction to applied linguistics (pp. 13-31). Reading, MA: Addison- Wesley.

Strodt-Lopez, B. (1991). Tying it all in: Asides in university lectures. Applied Linguistics, 12, 117-140.

Trueba, H. T., Guthrie, G. P., & Au, K. H. (Eds.). (1981). Culture and the bilingual classroom: Studies in classroom ethnography. Rowley, MA: Newbury House.

Tyler, A. (1992). Discourse structure and the perception of incoherence in interna- tional teaching assistants' spoken discourse. TESOL Quarterly, 26, 713-729.

Ulichny, P. (1991). The role of hypothesis testing in qualitative research. TESOL Quarterly, 25, 200-202.

Watson-Gegeo, K. A. (1988). Ethnography in ESL: Defining the essentials. TESOL Quarterly, 22, 575-592.

Wolfson, N. (1986). Research methodology and the question of validity. TESOL Quarterly, 20, 689-699.

Woods, A., Fletcher, P., & Hughes, A. (1986). Statistics in language studies. Cam- bridge: Cambridge University' Press.

TESOL QUARTERLY 472