Quality and Quantity, 19 (1985) 353-373 Elsevier Science Publishers B.V., Amsterdam - Printed in The Netherlands

353

Appropriate Criteria of Trustworthiness and Goodness for Qualitative Research on Education

Organizations *

CATHERINE MARSHALL

Peabody School of Education, Vanderbilt University, Nashville, TN, U.S.A.

Introduction

Research on organizations, particularly on schooling, has an emerging and viable tradition of incorporating qualitative methods. This tradition has progressed beyond qualitative versus quantitative debates to a recognition that researchers must match research methods with research questions and the choice of qualitative methods is appropriate for some fundamental questions about education. However, the strengths of qualitative research may still be undermined by well-intentioned efforts to apply the canons of quantitative research. to improve the trustworthiness of qualitative research, as with Huberman and Miles’ techniques for “drawing valid meaning from qualitative data” (Huberman and Miles, 1983).

This article questions whether this is the right time to concentrate on improving trustworthiness. At present, organization theory may be in the midst of paradigm shift, dealing with unanswered questions. Qualitative research has unique qualities for exploration, a “goodness” that allows researchers to explore and generate hypotheses and researchers who apply organization theory to schooling must expend energy searching and explor- ing and generating hypotheses.

The article shows the need for two separate strands of qualitative research. The first strand tests hypotheses in context, in the field, using many sites to identify the range and intermixing of variables, to build current knowledge and improve current practice. Huberman and Miles’ scheme provides excel- lent guidance for research in this strand and should be adopted for research

* A paper prepared originally for a panel presentation for the Special Interest Group on Organizational Theory at the Annual Conference of the American Educational Research Association, New Orleans, April 1984.

0033-5177/85/$03.30 0 1985 Elsevier Science Publishers B.V.

354

on topics guided by well-developed and accepted theory. The second strand explores, without knowing what theory will explain phenomena, without assuming any particular groupings and categories, without assuming a par- ticular world view or hierarchy of needs and interpretations.

The Evolution of Qualitative Research and the Quest for Trustworthiness

Qualitative research on social life evolved historically from anthropology and sociology, with frank clear justification for the inductive, open, dis- covery mode and varying approaches for managing questions of trustworthi- ness of findings. Sennett and Cobb (1972) and Rubin (1976) for example, said they were making composite sketches, pulled together from what they found, and reporting, novel-like. Recently, Lightfoot (1983) abandoned all pretenses and presented “portraits” of high schools. Becker et al. (1961, p. 17) said “in one sense our study had no research design,” but they then described the research approach for 30 pages. They described the procedures and perspectives for developing a chain of evidence, systematically accu- mulating proof of findings, compiling frequencies and distributions of pat- terns of activities, sentiments, interactions in order to judge the degree of confidence in the conclusions. Glaser and Strauss (1967), and Schatzman and Strauss (1973), described and illustrated the constant comparative method of analytic induction and theoretical sampling. Their method of collecting, coding, analyzing data, deciding on next steps based on initial analysis would lead to the emergence of grounded theory. Further data collection would search for negative instances, integrating them, expanding explanations until all phenomena are explained.

Others emphasized triangulation of various types of data, and taking interpretations to the research subjects as a sort of validity check (e.g., Jick, 1979; Guba, 1983). These approaches allowed for the use of various perspec- tives, the discovery of the question from immersion in the real world, the exploration of issues with no theoretical framework settled upon a priori, the exploration for appropriate theoretical analysis, with real-world data creat- ing or justifying the framework.

However, the qualitative versus quantitative debate continued, with criti- cism of the trustworthiness of qualitative research. Critics, particularly those from the positivist tradition, challenged the qualitative method, asking “how do you know that you know, how do you deal with replicability, validity, interrater reliability, controls, degrees of significance?” Miles (1979) worried that, without renewed efforts at improving methodological inquiry, “qualita- tive research on organizations cannot be expected to transcend story-telling.” Yin (1982), noting the difficulty in having causal theories and cumulative

355

knowledge from qualitative studies, using different definitions, called quali- tative research “a messy enterprise.”

Huberman and Miles, first in Quality and Quantity and then in their book (Miles and Huberman, 1984), provided a scheme for approximating positi- vist canons and for managing and controlling qualitative data in a large study with numerous researchers and sites. They specified the process that others had spoken about mystically. They provided a means for bureaucrati- zation of field work. Previous researchers had demanded such a scheme noting that we have insufficient methods and traditions for comparing and aggregating qualitative studies, that humans, as the research instruments in qualitative research, need tools to assist them in organizing and assuring trustworthiness.

Researchers have to find ways to control qualitative data-it has a lot of spring. Huberman and Miles provided answers. procedures for controlling for bias, devising tests to check data collection and analytic procedures and built-in ways for testing alternate hypotheses and interpretations. This is useful, particularly for large, multi-site, multi-personnel studies.

Researchers using qualitative methods could now demonstrate how their methods parallel the standards applied to quantitative research. Also, quantitative and qualitative research could marry and be helpmates (Sieber, 1978). However, in many marriages one partner dominates, overtly or subtly, and the other accepts the terms of the dominant partner. Qualitative researchers, in attempts to measure up to the terms and criteria for trust- worthiness developed by quantitative research, accept the submissive role, sacrifice some of the uniqueness and value of the method. Huberman and Miles have accepted the submissive role, renaming but directly applying the quantitative canons of reliability, validity, generalizability, and replicability. There are sacrifices that qualitative research makes by this submissive acceptance.

Criteria of Trustworthiness

There are standards that can be applied appropriately for judging the trustworthiness of qualitative research. They include the following: (1) the procedures for data collection are explained; (2) data are displayed and used to flesh out meaning of concepts; (3) negative instances are shown and explained; (4) the logic of the biases are displayed; (a) biases of interest (personal, professional, and policy-related) and the biases for direct applica- tion are stated; (b) theoretical biases and assumptions are stated; (5) data collection and analysis are semi-public, not magical, as in Huberman and Miles’ procedure for “self-documentation” (1983, pp. 334-336) and as when

356

research teams check each others’ interpretations; (6) the “in-field-work” analyses are documented (e.g., in logs); (7) the points where competing hypotheses appear are laid out and the reader can see them, and the choice made, and the evidence for that choice; (8) the data are preserved, available for reanalysis; (9) methods are devised for checking data quality, e.g., informants’ knowledgableness, ulterior motives, truthfulness; (10) transfer- able knowledge is identified by connecting with previous knowledge, and the distinction is made between site-specific findings and findings that are generalizable to other settings.

Huberman and Miles’ scheme contributes a systematic approach for validity in qualitative research and provides specific tools. These make the research researcher-proof, eliminate the human biases, control the dif- ferences in interpretation, and ensure that there is a demonstrated chain of agreed upon evidence for the conclusions in qualitative studies. They set up a system whereby checks on interpretation are built into the research design, where there is a mapping of data collection and analysis, where data are kept organized and controllable by identifying and fitting them into categories. They sanitize the process.

This is laudable, but limited. Questions and cautions remain; for example, can this be done by one researcher? Does the complex, convoluted mapping tell us anything? Does mapping constitute goals-displacement by defensive qualitative researchers? Does mapping become an end in itself? Most im- portantly, do we lose the insights, the unanticipated research questions as we work so hard in control our data? Is it good to devalue, eliminate the human biases and insights and metaphors as we researcher-proof the process? Finally, should this approach be used when research questions require discovery, questions where existing theory is being challenged and existing structure and practice appears to be based on fundamentally wrong assump- tions?

The Search for Trustworthiness Could Undermine the “Goodness” of Qualitative Research

It is important to recall the rationale and value of qualitative studies to examine whether the quest for trustworthiness sacrifices something valuable. Researchers choose qualitative methodology for the following situations: (1) research that cannot be done experimentally for practical or ethical reasons; (2) research that delves in-depth into complexities and processes; (3) re- search questions where the relevant variables and interconnections have yet to be identified and described; (4) research that seeks to understand in-

357

stances where policy, folk wisdom, and current practice do not work; (5) research on unknown societies or innovative systems; (6) research on infor- mal and unstructured linkages and processes in organization and research on real, as opposed to stated, goals; (7) research that seeks to discover and explore how previous theory fits, explains and predicts, with an openness to several theoretical perspectives; and, (8) research that seeks the subjective understandings of individuals, the “emic” view.

Such research must be systematic and meet criteria of trustworthiness, but also “criteria of goodness” (derived from Wolcott, 1975, and translated and amended). In “good” qualitative research the following criteria are observed: (1) the problem or question is discovered out of personal curiosity and out of observation in the real world; (2) the researcher is a valuable research instrument; (3) the analysis uses cross-cultural perspectives; is open to competing paradigms; (4) there are ways to maintain the objective, curious, stable stance of the researcher; checks against ethnocentrism, going native, researcher fatigue, and methods to maintain role and reciprocity; yet the methods can make use of the personal talents and experiential biases and insights of the researcher; (5) the research is ethical and sensitive to its impact on subjects and settings; (6) there is an appropriate match between the information sought and the various modes of gathering and reporting data; (7) original data are included so the reader can get a sense of the complexities; (8) the analysis is “tied into the big picture.”

These lists of particular reasons for using qualitative methods and the criteria of “goodness” show that positivists’ definitions of validity, demands for replicability and generalizability and standards for reliability are often inappropriate for qualitative research.

Sometimes, in the effort to have systematically collected and analyzed data and valid analyses, a researcher could reduce and confine a study within valid, but insignificant parameters. Rigidly trustworthy research may be limited by the methods devised to ensure trustworthiness. It could make qualitative researchers into nothing more than objective observers and coding specialists. It could lead to premature coding, forcing data within a theoretical framework, closing off alternate conceptualizations and preclud- ing discovery of hidden, secret, unrecognized, subtle, “ unimportant” data, connections, and processes. Research that tests hypotheses has the built-in ethnocentrism of its field. It uses established theory and previous literature to define the categories and expected linkages. If that is the research goal, then qualitative methods are used in test-in-context, to test existing theory with elaborate systems that make qualitative research approximate the positivist criteria of trustworthiness. However, many research questions require an exploration-for-meaning approach, and therein lies the strength of qualitative research.

358

A Modest Proposal For Two Separate Strands of Qualitative, Research

If we look at the criteria for goodness along with the criteria for trust- worthiness, we may see a tension between researchers working on being systematic and controlled, to fit findings within a theoretical framework and derive recommendations to improve organizational life, on the one hand, and researchers who want to explore, uncover hidden meanings and connec- tions and structures and boundaries in organizational life. Can qualitative researchers do both at the same time? The real question is, can we discover and develop emerging new theory and explore unanswered puzzles while we’re working vigorously to control and systematize? Trustworthiness is important, but perhaps not as important as discovery. Forcing and extract- ing data during analysis is appropriate only if there are identified, workable hypotheses to test. The pressure to control and manage data may seriously undermine the value of qualitative research. Yin and Givaltney (1981, p. 34) note that “premature extraction of factors can oversimplify and misrepre- sent the following phenomena: (1) the temporal sequence among organiza- tional events, (2) the substitutability of factors in any given individual case, (3) the potential interactions among factors, and, ultimately, (4) the estab- lishment of causal relationships.”

Matrices, boxes, quasi-statistics, charts, do have a role to play in analysis of qualitative research, but they are used to check the promise of emerging patterns, to see how frequently, how strong, with what variation a phenome- non occurs once that phenomenon has been observed and described and once its connections with other phenomena are observed.

However, can researchers ever expand theory or challenge existing inter- pretations, do anything more than reify existing theory if they stay tied to schemes for standardizing data analysis? This question shows how Huber- man and Miles part company with Glaser and Strauss and other proponents of exploratory research.

There are two different strands of qualitative research. Both strands strive toward trustworthy, systematic analyses. Qualitative researchers must de- termine which strand fits with their research goals and learn to work with the strengths and limits of their methodology.

Huberman and Miles represent the strand that seeks to test in context. They present an approach to collecting and analyzing qualitative data, with clear mapping of the chain of evidence, incorporating rich and context-laden data systematically, meeting all criteria for trustworthiness, making qualita- tive research parallel the procedures of positivists.

Glaser and Strauss would not object to testing in context, but would assert that qualitative research is at its best for hypothesis generating, exploring in context, developing theory from real world activities, interactions, and sentiments. They would support an exploration-for-meaning strand.

359

New Paradigms and Unresolved Problems Require Exploration for Meaning

Before choosing the testing-in-context approach the researcher must real- ize it may disallow surprises, new leads, openness to competing paradigms, cross-cultural perspectives, gaining from the researcher’s special insights, discovering the subjective understandings, the hidden structures, the perspec- tives of the unvalued, the elements of power and conflict. The testing-in- context approach incorporates an ethnocentrism, the assumptions of the dominant paradigm, the accepted ways of viewing organizations’ goals and processes. It does not allow research that is tied into the big picture, including the picture put forward by the new sociology of organizations, the radical critique, and the subjective meanings that organizations have for those who see fundamental persistent problems with the current structure of organizations. This section on new paradigms outlines arguments asserting the fallacies in the dominant paradigm in organization theory. These argu- ments demonstrate the need for exploration-for-meaning qualitative re- search.

Critiques of the Dominant Paradigm

There is an inherent bias for administrative control in organizational theory and educational administration research. The assumption is that the study of organizational life has had the purpose of identifying problems and finding ways to fix them. As Hack et al. (1971, p. 5) said, about educational administration; “administration gives order to the specialized and interde- pendent activities which characterize an organization. It provides and coordi- nates these so as to facilitate goal determination and achievement.” The purpose of research is to explain and predict what happens in the social world by searching for regularities and causal relationships between its constituent elements. The bureaucratic conception of the school creates assumptions that problems can be repaired, bureaucracies can be made more rational and productive and equitable by policy alterations, increased coordi- nation, special programs.

The assumption is that administration and policymaking are aimed at fixing the imperfections in organizations. “Liberals argue that there are system imperfections in terms of family and cultural norms that may develop “culturally deprived” or socially disadvantaged individuals whose deficits may be ameliorated through actions in the public sector, especially educa- tion. Liberals expend energy on making the system more meritocratic” (Papagiannis et al., 1982, p. 80). The assumption is that good organizations can work toward achieving social efficiency; schools are for skills training, for developing attitudes favorable to achievement/ success income, con- sumption.

360

The dominant paradigm assumes that our organizational problems come from: (1) ignorance or lack of motivation of the populace; (2) ignorance or lack of motivation of public sector professionals and bureaucrats; or, (3) the lack of sufficient resources. Therefore, these problems can be solved by adjusting incentives, resources, by staff development, or by public relations. Policies and programs can be devised and implemented to adjust goals, and refine the essentially meritocratic system so that those with neutral technical competence can make the organization move smoothly toward meeting its agreed upon goals. When those in the dominant paradigm look at educa- tional innovations, they look at the individual participants and the systems structures and they look for the right combination of relevant variables that support the adoption of innovation and solve problems.

Challenges to the Dominant Paradigm: Currently Unanswerable Questions

Some have proposed that we have an emerging paradigm, a shift in organization theory and its application to education. Paradigm shifts occur when: (1) theorists reject old theory; (2) practitioners cannot find guidance from old theory; (3) real world facts/problems/dilemmas cannot be analyzed; (4) policymakers, following older assumptions, cannot accomplish what they intend; (5) subgroups persistently are dissatisfied with the answers, the explanations, the services and the policies that come from those who work under old assumptions (Kuhn, 1962).

Are our current ways, the ways of “the dominant paradigm,” sufficient for looking at organizations and applying organization theory to education? The following sections give evidence of a paradigm shift in organization theory, as applied to schooling, by outlining the arguments, evidence and theory of the radical critics, the new sociology of organizations, and the persistent unresolved problems.

The Radical Critique

The radical critique attacks the dominant paradigm for ignoring com- pletely many aspects of our social reality, for example: (1) throughout the world poorly educated and cheap workers are quite useful to maintain capitalist profits; (2) motivation signifies little without power; (3) there continues to be greater concentration of power, and persistent social prob- lems; (4) the lack of resources devoted to solving social problems represents a political choice (Papagiannis et al., 1982, p. 130).

The dominant paradigm fails to look at the social system wherein innova- tion emerged as the choice way for change. By focusing on questions such as

361

who adopts innovations and what structures facilitate adoption of innova- tions, the dominant paradigm ignores issues like why educational innova- tions do little about poverty, racism, sexism and inequality in general. Focusing on issues such as who adopts first and most and for the longest time, are not as important as why a particular innovation was chosen, why particular strategies are promoted, legitimated, funded, and disseminated. Radicals say that problems are not due to system imperfections, irrationality, mindlessness, but rather due to the fact that the education system functions in the interests of more powerful social classes (Papagiannis et al., 1983, p. 90).

The radical critique says that we must look at who benefits from a structure or policy, that the beneficiaries of education are the capitalists since the social system produces a docile, fragmented, competing, and powerless working class and people who believe in meritocracy and strive to move up within the system. The radical critique says that the education system disseminates this ideology and maintains the myths of meritocracy and democracy while creating and reinforcing different skills and attitudes among different classes of students: rich vs. poor, men vs. women, black vs. white, urban vs. rural, etc. The radicals’ emphasis is on power and conflict of interests and this leads to a focus on how power accumulates and how education systems allow and facilitates this. For example, “expertise” is not neutral technical competence but rather an agreed upon and developed myth created by those accumulating power and positions to control the hierarchi- cal system. The education system requires and legitimizes experts.

Real Problems/Facts/Dilemmas that Cannot be Analyzed

Critics raise fundamental challenges to the dominant paradigm, citing facts that defy assumptions of meritocracy, showing that conditions of organizations in decline cannot be managed under old assumptions, and showing the political nature of decision making. Critics also list persistent problems and unexplained patterns in organizations.

The real world challenges the liberal view that meritocracy is attainable. “Blacks and females get lower rewards even after controlling for the in- fluence of ‘relevant characteristics.’ Success in educational systems reflects this same race, sex, and class bias.. . and even educational reforms targeted at the disadvantaged often help the advantaged most” (Papagiannis et al., 1982, p. 80).

Real world problems cannot be managed. Boyd (1983, p. 8) points out that “educators are now confronted with four kinds of decline, simulta- neously- declining enrollments, declining economic-budgetary cir-

362

cumstances, declining public confidence in schooling, and declining legi- timacy of administrative authority” and says that education management must be re-thought, that our applications of theory in educational adminis- tration may be inappropriate because they were developed before these declines. Our theory of leadership may not fit the situation of schools in fiscal stress, fighting for public confidence, motivating elderly staffs, and implementing policy without adequate supports, incentives, and resources.

Boyd points out that our organizational structure is based on assumptions of neutral technical competence and rational problem solving (getting politics out of education and modelling the rational bureaucracy). But this ignores several real world observations: (1) technical competence is not neutral; it is inevitably tied to the values of professionals, which may conflict with public values; (2) decision making and problem solving are deciding “who gets what when and how” and that means politics.

Efforts at reform must be viewed in a values and power context. As Paulston (1978, p. 2) said “ideology, power, and perceived group self-inter- est.. . (are) key factors influencing planning and implementation of basic educational reforms”. People see this, particularly in a time of decline when tough choices must be made and a time when people are seeing that educators do not have the systematic technology to deliver the goods, when people are making demands for participation and coming up against inflexi- ble bureaucratic structures and the education professionals who are adept at manipulating and preventing parent involvement-thus we have the legi- timacy crisis in education.

There are subgroups who persistently are dissatisfied with the explana- tions, services, supports, and policies promulgated under old assumptions. In the past, in a time of growth, the times when most of our applications of organizational theory to education were devised, we could promulgate as- sumptions like neutral technical competence and rational decision making. But now some conflicts have become unmanageable and visible-teacher strikes, parents picketing for decentralization in New York city, curriculum and textbook controversies, the demands of women and minorities for equal access to jobs in school organizations.

In a time of decline, management and policies (and theory and assump- tions) must be different because: (1) there are clear winners and losers in a time of shrinking budgets; (2) participation is intense, from the sense of personal stake; (3) equity and entitlement pressures limit choices for re- trenchment; and (4) morale, dependent upon new incentives and a desire for moving up and forward, plummets. School management by manipulations and offering new incentives and by short-term planning will not work. In decline, there will be ‘hard questions about cuts, about equity, for example, when there is no new money and no political support for equity, will not

363

schools with low-socioeconomic-status (SES) students be harder hit by decline than affluent ones? When there are few job openings, will those who implement affirmative action policy achieve steps toward equity?

Minorities and clients with special needs focus on the inputs and outputs of school systems, raising questions about equity and about special services or special needs, and persistently say that educational organizations are failing in these areas. Women are finding it hard to accept that the selection process and the criteria for mobility in school administration result in the mobility of the best qualified candidates.

Parents, particularly minority and lower-class parents, are not participat- ing in the various structures devised purportedly to increase their access and participation in school functioning (Izu, 1980). There are fundamental assumptions built into the school system that make people resist, not bother, give up, laugh at participation. There are people who refuse to believe, cooperate in, be motivated in, and participate in a system that perpetuates something that benefits others but not them. They no longer believe that the public sector acts in a neutral, technocratic manner to perform objectively valuable functions for society as a whole. They persistently cast a vote of ‘no confidence’ in the explanations of policymakers and professionals in edu- cation, dropping out, withholding funds, no longer even bothering to protest. Perhaps they suspect that new programs and policies to improve the system actually function to perpetuate the power of authority, the jobs, and the facade of attainable social efficiency and they have seen enough to know that they will not be the main beneficiaries. They sense that the experts are not solving problems but rather creating diversions, new programs and relying on personal communication and ceremonial gestures to cover funda- mental problems.

Some students, in spite of our continued tampering with the technology of teaching, the mix of programs and policies, still find school irrelevant, and they drop out, literally or figuratively. These dropouts are often the most disadvantaged, the ones who, in theory, schooling will help the most. Sometimes these dropouts are the very bright students who, in theory, should love learning, schooling.

Are there assumptions, theories, that have guided the structures, the policies, the people, the organizational culture of education that just do not accord with the facts that these people know? If yes, then we are seeing challenges to the dominant paradigm. We’re seeing people who know that the school superintendent is deluding herself in thinking that the new program and the extra money for low SES schools will alleviate racism. We are seeing people who eschew the choice of teaching as a career, not just because of low salary (a quickly fixable item) but because of the loss of legitimacy and authority of expertise.

364

The Importance of Power and Politics

“The dominant perspective neglects the importance of and the nature of power differences in society” (Papagiannis et al., 1982, p. 80). But now there is a growing realization that goes beyond the knowledge that education and politics are intertwined. Political values are built into the rules and structure of school organizations. Decision making structures ensure that certain values will prevail. There are built-in fundamental conflicts in school organi- zations. Most importantly, many policies devised to alter values, equalize opportunity, and smooth over the conflict (Head Start, staff development, school/community task forces) are band aids holding together a facade of the rational orderly bureaucracy.

Who are the people who are saying and seeing this? They are people like poverty activists who notice that the block grant policy appears to be rational policy, giving discretion at the state and local level for setting priorities, but, that it has the effect of eliminating programs that meet their needs. They are women who notice that the formal selection process for school administration appears to be fair competition, particularly with affirmative action assertions, but still chooses white men, having the effect of devaluing women’s competence.

Who is articulating theory that offers possible explanations? Those who study the politics of education; those who say upfront that superintendents are politicians, that educational administration includes power issues, in- cludes the act of moving people to do what they would not ordinarily do in the exercise of power for the advancement of certain values; those who see that education is at the center of fundamental values conflicts-that edu- cation includes the political system, “the authoritative allocation of values” (Easton, 1966, p. 126).

In studying the politics of organizations, one sees how assumptions of organizational management do not explain irrationality, the conflict of values. One also sees how the formal structure and elites’ assumptions and strategies conflict with informal coalitions and one sees the ritual, symbolic action juxtaposed with observed real values, intents, outcomes. As Minogue (1983, p. 73) points out, in his critique of the limits of administration, “nothing gets done which is unacceptable to dominant or influential politi- cal groups, which may be defined to include the ‘bureaucratic leadership’ group.”

Researchers and activists observe how community values do or do not get into school board decision making, how values conflicts are played out in budget hearings in state legislatures, and how subgroups coalesce, organize into interest groups and develop strategies so that their needs and values are incorporated into the curricula of new programs. They look at any arena

365

where values are converted into policy and they look first at the formal, recognizable structures for this (school boards, state legislatures), the macro- politics. But they now look farther and closer at the informal structures for conversion of unstated values into informal policy, the micropolitics. These occur in the “hidden curriculum,” the informal selection system in school administration, or the use of professional jargon as a way of presenting an aura of authority/power, the authority of expertise.

The New Sociology of Organizations: Theorists of a New Paradigm

Besides “real” people, those who study the politics of education, and the radical critique, several theorists have presented challenging ways of viewing organizations. Their ideas have important implications for research method- ology: they challenge researchers to look beyond the bureaucratic/ structural/functional to the subjective meanings and social construction of shared realities in organizational life.

Pfeffer posits that organizations can be viewed as systems of shared meanings and that one key task of managers is the construction and maintenance of those systems, those shared paradigms and shared languages and cultures. Much administrative work involves using language, symbolism, and ritual to develop shared meanings and beliefs in organizations. He says “One of the elements of competition and conflict among social organizations involves the conflict between paradigms, defined as systems of beliefs or points of view” (Pfeffer, 1981, p. 9). Management, therefore, means develop- ing a consensus of beliefs and a shared construction of social realities in organizations. Ceremonies and talk are essential tools for projecting the symbols, values, and rationalizations that function to reduce conflict, assess productivity, and create an image of authority, control, and expertise.

Meyer and Rowan (1977) add to this notion, saying that in our imperfect organizations, with imperfect technologies, a major goal of leadership and a major function of the organization is to maintain its image, its myth of service and special mission. Meyer (1977, p. 66) saying “education systems are thus, in a sense, ideologies,” is suggesting that there will never be policies for reform that liberate/separate education goals. from dominant social goals. Reforms that contradict the social structure will not fly.

Weick (1976), also emphasizes cognitive processes of individuals in organi- zations as they selectively attend, order, and discredit information. Thus, individuals’ values and limitations and meaning-making are key elements in organizational life.

Greenfield (1975, 1979) criticizes dominant organization theory that re- ifies organizations and serves professors who can do research and training

366

about how to make organizations work well. He says that organizations exist as agreements among people who are served by them, whose values and interests are served by the sort of social order and control promulgated by the organization.

These theorists directly challenge the paradigm that assumes that, with some tinkering, educational organizations will achieve harmony as a work- place, affect equity through intensified efficacious programs, and meet the needs of their various clientele. They alert researchers to study meaning-mak- ing process in interactions, the use of symbols and rituals for organizational maintenance, and the information and meanings-management process in the organizational structures which reinforce the dominant values of larger society.

Exploration-For-Meaning Qualitative Research in the Midst of a Paradigm Shift

Theorists are rejecting old theories and proposing new ways to conceptual- ize what an organization is. Educational administrators and policymakers are facing real world problems that are unmanageable if they work under their earlier assumptions derived from theory. Subgroups are persistently thumb- ing their noses at explanations that are supposed to make them satisfied with the policies and services promulgated in organizations. These are the indica- tions of paradigm challenge.

There are several ways to react. One reaction is to work within the dominant paradigm, construct higher standards, more exact definitions, and more refined techniques for proving how what we know is so. This is what Huberman and Miles have done. An alternative reaction is to grasp the ideas of the challenging paradigm, seek appropriate methods, questions, data sources, and insights and develop suitable methods to communicate findings. This section projects and illustrates what this second reaction could be like.

Research Designs for Exploring in a New Paradigm

The challenges of the emerging paradigm call for an exploration-for-mean- ing strand of qualitative research, for researchers to look at the following: (1) the subjective interpretations of events by the participants in the setting; (2) the connections between and among organizations, e.g., the connections between the economic and political structure and the education system, focusing on systems of power; (3) the processes by which certain subgroups are continuously excluded from the benefits of organizations; (4) the func-

367

tions of organizational rituals and ceremonies; the process of meaning- and consensus-making in organizations, including a focus on the process by which some people are excluded from that process; (5) the ways of managing organizations that are losing legitimacy, resources, and morale-conditions that did not exist in the past.

To conduct such research, we need research designed to capture the words of the underclass, the hidden structured connections among organizations, the informal policies or the unanticipated outcomes of policies on particular populations.

Often researchers will not know the exact research question, set of guiding hypotheses, or theory. They will have to devise research designs for exploring openly, without the constraints of pre-determined theory.

This sort of research would recapture the qualities of earlier work. Geer (1969) gives a fine explanation of the process by which a researcher enters a setting with a sense that interesting questions and surprises will emerge from observing people in the first days in the field. Whyte’s Street Corner Society (1955) and Liebow’s Tulley’s Corner (1967) did not display systematic methodology, but their descriptions of subgroups of society made people reconsider assumptions that were driving social policy. Another fine example is provided by Wax; in a study of Indian American children’s achievement in school, her immersion in their schools and communities allowed her to observe and identify a research question about the gradual withdrawal of these children from interactions with the teacher. Her openness to various strands of theory and research allowed her to discover that the way the students withdrew was connected to their traditional sense that competing with others from your tribe was wrong. Wax recalled, “(We) did not invent, search for, or discover our problem: it forced itself upon us” (Wax, 1971, p. 249).

Other examples: when I started my research on women in school adminis- tration, I did not know which theoretical framework (theory of sex structur- ing, discrimination, motivation, role strain) would guide my analysis and I waited for the women’s words and career experiences (the data) to make the connections. Similarly, a student of mine is exploring the experiences of mothers who are graduate students, drawing from theory of professional socialization, family structure and adult development, based on her own sense that there was a good research question there; she did not start with a question and a theory with relevant categories, rather, she started with her own experience and a feeling that some undefined interconnection existed which had heretofore been seen as relatively unimportant, and an area where policymakers were applying bandaids such as scholarships for women and a few day-care centers.

A few more examples illustrate research that begins with a search for

368

meaning in context. West (1982) found a hidden barrier to women’s career mobility by focusing on the turn-taking and negotiation in talk in the workplace. Gronn (1983), by searching for meaning in administrator talk, uncovered power dynamics. Finally, I offer the example of Bates’ project in Australia (Angus, 1984), where a school was studied by several researchers looking at different questions, using participant observation, and pulling together descriptions and analyses of the same site from different perspec- tives to derive a composite challenging picture of schools as institutions. (This is reminiscent of the way Lawrence Durrell (1958) presented different characters’ interpretations of each other and of the same incidents, showing how, in one person’s eyes a murder was the key event while intrigues, adulteries, sunsets and other happenings were interpreted differently when viewed through the subjective meaning-making of individuals.)

This exploration-for-meaning strand of qualitative research would be cramped by strict adherence to Huberman and Miles’ scheme for trust- worthiness and inhibited by the attempt to make research researcher-proof and sanitized. This strand would require some different approaches to goodness and trustworthiness that would preserve the following elements of qualitative research:

(1) The acceptance of the possibility that all organizations are unique, with unique employees, histories, and environments (McKelvey and Aldrich, 1983). Burlingame (1981, p. 87) proposes that “schools are best viewed as a collection of extremely diverse social settings.. . and by inspecting the simi- larities and differences in sites (within schools) a more subtle view is possible. . . .”

(2) The awareness that ‘applied ethnography’ may be a contradiction in terms since it may require researchers to “assist in further rationalizing the social world so that administration of it is more feasible, efficient.. .” (Noblit, 1984, p. 97).

(3) The strengths of qualitative research must be guarded against utilitarian thrusts; the “ utilitarian culture . . . will select, because of interest in power, deterministic causal chains as the subjects of interest” leading to “reification of rationalistic models that seek to explain struggle, power, and emotion as technical problems in prediction and control, and thus propose administra- tive solutions over political ones” (Noblit, 1984, p. 99).

Thus it is essential to be explicit about the political and power issues built into applied research.

(4) That qualitative research should search for negative instances and alternative explanations from the challenging paradigm, perhaps by includ- ing proponents on research teams.

(5) That qualitative research must recognize that “the study of reality creation in organizations is a study of power, in that definition of reality,

369

normalcy, rationality and so on serve as paradigms that in some sense govern the conduct permissible within them” (Brown, 1978, p. 371).

(6) That ideas of the radical critique require further exploration of the parallels between schooling and the power, authority systems, uniformity and regimentation of industry and that exploration will require discovery modes in qualitative research (e.g., Jackson, 1969).

(7) That qualitative research must incorporate people’s unstudied talk and behavior in order to interpret social action, power, intention and individuals’ meaning-making; the focus on language, on speech in social groups is promising.

(8) That research must explore dynamics of power that are covered by organizational structures-that organizations are arenas of conflict but many organizational structures may serve the purpose of subduing and covering conflict and convincing organizational members to accept the myth-making of those in control.

(9) That exploratory qualitative studies should not have presumptions of boundaries, should be open to discovering sources of values conflict to uncover the values assumptions that hinder the implementation of policies aimed at providing equity (e.g., Rist (1978) and Ogbu (1974)).

(10) That qualitative research must explore the macro level in order to explore the connections among educational and political and economic institutions.

(11) That the new sociology of organizations requires qualitative re- searchers to search assertively for the varying subjective interpretations of things, events, people; they must value the differing data recognizing that they are snapshots of varying perspectives.

(12) That qualitative researchers need to discover how underlings, sub- strata, powerless, “ unimportant” people in organizations interpret events and actions.

(13) That, if the concept of organization is a socially constructed one, then qualitative researchers need to discover “ the procedures and considerations actors invoke in relating common-sense construction to things in the world” (Bittner, 1974, p. 75) and they may need to abandon current organizational theory to make that discovery.

(14) That qualitative research can benefit from the creative intuitive insights of human researchers; “ to isolate one psychological function, in this case the rational, from all others, is to limit the effectiveness of that function, diminish the quality of its product” (Bargar and Dugan, 1982, p. 12). Researchers who do retrospective reworking can purposefully bring an event into central awareness, ruminate and free associate to uncover patterns of significance in data analysis (Schwartz and Schwartz, 1955) but attempts to eliminate researcher bias may eliminate researcher insight. Further, objective

370

observers who stringently monitor themselves against “going native” can never understand from the inside, the insider’s view.

(15) That qualitative researchers’ tasks include revealing the multiplicity of meanings of an event, not to exhaust the singular meaning; reality is knowable in an infinite number of ways, each of which is revealed by a shift in perspective, method or purpose. Many equally valid descriptions are possible and no standard format for collecting and presenting data could be appropriate for all settings and purposes.

(16) That qualitative researchers cannot assume that they capture general truths; knowledge is relational, evolving in the interaction among people and “valid knowledge is held to reside neither in the subject nor in the object, but in the transactions that unfold between them.. . . What is real for men (sic) is real for all practical purposes” (Rock, 1979, pp. 61 and 66).

To explore these possibilities, qualitative researchers need to spend val- uable time exploring the multiple constituencies in organizational life, taking advantage of multiple competing professional biases, curiosities and insights of the humans who are the research tools, challenging their own observations by seeking alternate explanations offered by their subjects, people from other disciplines, cross-cultural comparisons. Team research should be en- couraged, but not necessarily with the goals of achieving a consensual generalizable picture of organizational life with recommendations on how to solve problems.

This may appear anarchistic; it also sounds productive, fun and ap- propriate for a time when oganization theory and the very assumptions that have guided practice and policy are under serious challenge. If it sounds impractical, keep in mind that practicality is defined by those who hold power and funds, but who have not necessarily solved fundamental problems and who are being challenged by “real” people, the radical critique, and the new sociology of organizations. As Sennett (and Cobb, 1972, p. 11) said: “I am aware that both the methods.. . and the style of presentation are vulnerable to criticism from colleagues in the social sciences. The small sample, not randomly chosen makes generalizability suspect. The anecdotal presentation raises the question of representativeness in the use of the data. The only answer to these criticisms lies in the quality of the work itself-in its ability to persuade by appealing to a level of ‘knowing’ that exists in all of us is not very often tapped.. . to generate an ‘aha experience’.”

Qualitative research must preserve its unique goodness, its ability to explore for meaning, for the “aha” experience of discovery. Qualitative research cannot be negated by submitting to the criteria set by positivists, as in this vignette, borrowed from Van Maanen (1979): Qualitative Researcher: “Many people these days are bored with their work

and are. . . .”

371

Quantitative Researcher: “What people, how many, when do they feel this way, where do they work, what do they do, why are they bored, how long have they felt this way, what are their needs, when do they feel excited, where did they come from, what parts of their work bother them most, which. . . .” Qualitative Researcher: “Never mind.”

References

Angus, L. (1984). Ideology, social structure and collaborative critical research. Paper pre- sented at the annual conference of the American Educational Research Association, New Orleans, April.

Bargar, R.R. and Dugan, J.K. (1982). “Cultivating the creating endeavor in doctoral research”, Journal of Higher Education 53: l-31.

Becker, H.S., Geer, B., Hughes, E.C. and Strauss, A. (1961). Boys in White: Student Culture in Medical School. Chicago: University of Chicago Press.

Bittner, E. (1974). “The concept of organization”, pp. 69-81 in Roy Turner, ed., Ethnomethodology. Markham, Ontario: Penguin.

Boyd, W.L. (1983). Political Science and Educational Administration: Rethinking Educational Policy and Management in the 1980s. Deakin, Australia: Deakin University Press.

Brown, R.H. (1978). “Bureaucracy as praxis: Toward a political phenomenology of formal organizations”, Administrative Science Quarterly 23: 365-382.

Burlingame, M. (1981). Social settings in educational organizations: An exploratory study of deliberate segregation and change in schools.” Unpublished manuscript, University of Illinois, Urbana, August.

Durrell, L. (1958). Balthazar, a novel. New York: E.P. Dutton. Easton, D., ed. (1966). Varieties of Political Theory. Englewood Cliffs, NJ: Prentice-Hall. Geer, B. (1969). “First days in the field,” in G. McCall and J.L. Simmons, eds., Issues in

Participant Observation. Reading, MA: Addison-Wesley. Glaser, B. and Strauss, A. (1967). The Discovery of Grounded Theory: Strategies for Qualitative

Research. Chicago: Aldine. Greenfield, T.B. (1975). “Theory about organization: A new perspective and its implication

for schools”, in M.C. Hughes, ed., Administering Education: International Challenge. London: Athlone Press.

Greenfield, T.B. (1979). “Organization theory as ideology.” Curriculum Inquiry, 15: 43-65. Gronn, P. (1983). “Talk as the work: The accomplishment of school administration,”

Administrative Science Quarterly 28: l-21. Guba, E. (1983). “Remarks as discussant on ‘What is the best conceptualization of design

validity?‘.” Symposium at the Annual Meeting of the American Educational Research Association, Montreal.

Hack, W.G., Gephart, W.J., Heck, J. and Ramseyer, J.A. (1971). Educational Administration: Selected Readings (2nd edn). Boston: Allyn & Bacon.

Huberman, A.M. and Miles, M. (1983). Drawing valid meaning from qualitative data: Some techniques of data reduction and display, Quality and Quantity 17: 281-339.

Izu, J. (1980). “Myths and realities of policy implementation: The case of California’s School Improvement Program.” Unpublished working paper, Stanford University.

312

Jackson, P. (1969). Ltfe in Classrooms. New York: Holt, Rinehart & Winston. Jick, T. (1979). “Mixing qualitative and quantitative methods: triangulation in action”,

Administration Science Quarterly 24: 602-611. Kuhn, T. (1962). The Structure of Scientific Reuolutions. Chicago: University of Chicago Press. Liebow, E. (1967). Talleys Corner: A Study of Negro Men. Boston: Little, Brown. Lightfoot, S.L. (1983). The Good High School. New York: Basic Books. McKelvey, B. and Aldrich, H. (1983). “Populations, natural selection, and applied organiza-

tional science”, Administration Science Quarterly 28: 101-128. Meyer, J.W. (1977). “The effects of education as an institution”, American Journal of

Sociology 83: 55-77. Meyer, J.W. and Rowan, B. (1977). “Institutionalized organizations: formal structure as myth

and ceremony”, American Journal of Sociology 83: 340-363. Miles, M.B. (1979). “Qualitative data as an attractive nuisance: the problem of analysis”,

Administrative Science Quarterly 24: 590-600. Miles, M. and Huberman, A.M. (1984). Qualitative Data Analysis. Beverly Hills, CA: Sage

Publications. Minogue, M. (1983). “Theory and practice in public policy and administration”, Policy and

Politics 2: 63-85. Noblit, G.W. (1984). “The prospects of an applied ethnography of education: a sociology of

knowledge interpretation”, Educational Evaluation and Policy Analysis 6: 95-101. Ogbu, J. (1974). The Next Generation, An Ethnography of Education in an Urban Neighbor-

hood. New York: Academic Press. Papagiannis, G.J., Klees, S. and Bickel, R. (1982). “Toward a political economy of educa-

tional innovation”, pp. 74-117 in W.L. Boyd ed., Political Science and Educational Administration. Deakin, Australia: Deakin University Press.

Paulston, R.G. (1978). Changing Educational Systems: a Review of Theory and Experience. Washington, DC: Development Economics Department, World Bank.

Pfeffer, J. (1981). “Management as symbolic action”, pp. l-52 in L.L. Cummings and B.M. Staw eds., Research in Organizational Behavior, Vol. 3, JAI Press.

Rist, R.C. (1978). The Invisible Children. Cambridge: Harvard University Press. Rock, P. (1979). The Making of Symbolic Interactionism. London: Macmillan. Rubin, L.B. (1976). Worlds of Pain: Life in the Working-class Family. New York: Basic

Books. Schatzman, L. and Strauss, A. (1973). Field Research: Strategies for a Natural Sociology.

Englewood Cliffs, NJ: Prentice-Hall. Schwartz, M.S. and Schwartz, C.G. (1955). “Problems in participant observation”, American

Journal of Sociology 60: 343-354. Sennett, R. and Cobb, J. (1972). The Hidden Injuries of Class. New York: Random House. Sieber, S.D. (1978). “The integration of field methods and survey research”, American

Journal of Sociology 78: 1335-1359. Van Maanen, J., ed., (1979). “The seminar”, Administrative Science Quarterly 24 (4): 519. Wax, R. (1971). Doing Fieldwork: Warnings and Advice. Chicago: University of Chicago

Press. Weick, K.E. (1976). “Educational organizations as loosely coupled systems”, Administrative

Science Quarterly 21: l-19. West, C. (1982). “Why can’t a woman be more like a man?: an interactional note on

organizational game-playing for managerial women”, Work and Occupations 9: 5-29. Whyte, W.F. (1955). Street Corner Society. Chicago: University of Chicago Press.

313

Wolcott, H.F. (1975). “Criteria for an ethnographic approach to research in schools”, Human Organization 34: 111-127.

Yin, R. (1982). “Studying phenomena and context across sites”, American Behavioral Scientist 26: 84-100.

Yin, R.K. and Givaltney, M.K. (1981). Organization Collaborating to Improve Educational Practice. Cambridge, MA: Abt Associates.