Analiza de Discurs_learning Tech_Morissey

7/30/2019 Analiza de Discurs_learning Tech_Morissey

1/20

http://evi.sagepub.com

Evaluation

DOI: 10.1177/135638909500100206

1995; 1; 217EvaluationOrla Morrissey

Shifting Paradigms: Discourse Analysis as an Evaluation Approach for Technology Assessmen

http://evi.sagepub.com/cgi/content/abstract/1/2/217The online version of this article can be found at:

Published by:

http://www.sagepublications.com

On behalf of:

The Tavistock Institute

can be found at:EvaluationAdditional services and information for

http://evi.sagepub.com/cgi/alertsEmail Alerts:

http://evi.sagepub.com/subscriptionsSubscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

1995 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.at INDIANA UNIV on July 9, 2008http://evi.sagepub.comDownloaded from
http://www.tavinstitute.org/index.phphttp://www.tavinstitute.org/index.phphttp://evi.sagepub.com/cgi/alertshttp://evi.sagepub.com/cgi/alertshttp://evi.sagepub.com/subscriptionshttp://evi.sagepub.com/subscriptionshttp://www.sagepub.com/journalsReprints.navhttp://www.sagepub.com/journalsReprints.navhttp://www.sagepub.com/journalsPermissions.navhttp://www.sagepub.com/journalsPermissions.navhttp://evi.sagepub.com/http://evi.sagepub.com/http://evi.sagepub.com/http://evi.sagepub.com/http://www.sagepub.com/journalsPermissions.navhttp://www.sagepub.com/journalsReprints.navhttp://evi.sagepub.com/subscriptionshttp://evi.sagepub.com/cgi/alertshttp://www.tavinstitute.org/index.php


2/20

217

Shifting ParadigmsDiscourseAnalysis as an EvaluationApproachforTechnologyAssessment

ORLAM O R R I S S E Y

University College at Cork

This article examines the application of discourse analysis in the empiricalcontext of learning technology innovation. It outlines the various theoreticaland methodological principles, as well as the technical procedures whichhave been followed in relation to the operationalization of an evaluation

design using discourse analysis. The study itself is concerned withstakeholders subjective and experiential knowledge structures whichunderlie a culture of technology innovation. The article examines how

technology innovation is discursively constituted through the creation ofshared meanings and images among stakeholders.

Introduction

This article seeks to outline an approach to discourse analysis which supports a

learning evaluation paradigm in technology assessment. Within such an approach theevaluation agenda is explicitly concerned with the communication strategies, self-formation and self-reflection of social actors.A learning model implies a concern withthe transformation of individual learning and actions into collective learning and

actions; this requires the effective communication of different interests from thestarting point of how social actors understand and evaluate their own experiences,values, beliefs and actions. This model is associated with the social constructivisttradition of Guba and LincolnsFourth Generation Evaluation (1989) and represents a

paradigmatic shift from assessment-type models of evaluation. Such a model also

challenges the practice of program-oriented evaluation where the normative interest isin social engineering.The discourse analysis design has been conceptualized to incorporate the stakeholder

perspective and it begins with the premise that stakeholders have different construc-tions of reality. Like Guba and Lincolns hermeneutic cycle, the discourse analysis is

designed to help make explicit the process of reflexivity for stakeholders. Guba andLincoln (1989: 52) define stakeholders as groups at risk; they share a direct stake inthe evaluation, whether this is status, power or financial resources. Evaluation

http://evi.sagepub.com/http://evi.sagepub.com/http://evi.sagepub.com/http://evi.sagepub.com/


3/20

218

methodologies designed from the stakeholder stance are concerned with the subjectiveinterests invested in the evaluand-the area of activity to be evaluated.

Given the uncertain environment of technology innovation, there is a need forevaluation methodologies which capture baseline experiences. The emphasis of thediscourse analysis design, developed originally as part of a wider evaluation study, is towork directly with stakeholders accounts of their actions and beliefs, and how these

define the social context of technology innovation. The analysis of discourse opens upthe possibility of understanding the symbolic dimension of change in specific socialcontexts of innovation. The learning value of such an approach is defined by theoperational context, empirically defined as the local context of interpretation andaction. The focus of the evaluation exercise thus moves away from the traditional

evaluation criteria of causality and predictability to a search for an understanding of the

meanings subjects bring to a particular context.Ahermeneutic or interpretative approach is adopted in the analysis of stakeholdersdiscourse. The central focus or objective of the study is to facilitate a research agendawhich is concerned with subjective and experiential knowledge structures, i.e. howsocial actors make sense of their own realities. Epistemologically, the researcher isconcerned with the question of how everyday practice is understood.A critical theoryof discourse analysis is used to inform the interpretative framework, i.e. to identify the

combined effects of the voices and the global relevance of the discourse outside itspurely semantic context. This framework addresses three levels of discourse produc-tion : (1) how stakeholders come to share a common appraisal of their actions; (2) how

technology innovation is discursively constituted; and (3) the communicative strategieswhich are encoded in patterns of meanings and action orientations.

Context of the Research

This article arises from an empirical study carried out as part ofARTICULATE, a

trans-European evaluation project on learning technology innovations funded under anaction program of the European Commission. The program is part of an ongoing EUinitiative in R&D on Telematic Systems for Flexible and Distance Learning; it consistsof 22 projects involving trans-European partnerships.2 The program is divided intothree strands: R&D, pilot testing and experimentation and implementation. Its definedobjective is to support learning technology solutions across the European Union

through the development, validation and diffusion of learning technologies. The role ofARTICULATE and its associated evaluation activities within the program is tocontribute to the understanding of the actual and potential contribution of learningtechnology innovation, and to promote an embedded evaluation culture within the

programs area of activity. Guba and Lincoln point out that most stakeholders areunaware either that they are being evaluated or of the value of information produced inevaluations. The notion of embedding an evaluation culture simply means bringingevaluation closer to the human subject and internal decision-making processes; makingexplicit the role of individual stakeholders in the evaluation process and the value oftheir own knowledge and experiences. Such an advocacy agenda suggests an action-research approach to evaluation.



4/20

219

Part of the task of the overall evaluation project is to develop a multimethodologicalmonitoring evaluation system which captures change at the micro-social and organiz-ation level of the innovation activities. Specific phenomena are singled out for inquiry;these include the self-formation of actors in organizations, the social construction of

images of innovation and the role of formal evaluation actions. The emphasis on

change defines a traditional evaluation function, but when designed from the per-

spective of a learning model it is intended to capture the dynamic and evolvingrelationship between the above elements. Hence, the focus of the development of a

monitoring system is to map the propositional field of education technology to thecultural logic of the pilot and project activities,4 i.e. the cultural patterns of relevancewhich pilot and project actors internalize, reproduce or transform. Cultural logic refersto cultural meanings which become manifest through cultural discourses and whichhave a structuring effect on identity formations, actions and cognitions. In other

words, what is noteworthy about how actors come to construct and represent theirrealities.

There are a number of features of the monitoring system which influence thediscourse analysis design. First, the monitoring system is envisaged as a reflexive tooland the notion of self-assessment is pertinent (in line with the learning/communicationmodel). Second, the methodological objective is to combine a model of under-

standing with a model of explication, i.e. the subjective systems of self-under-standing and meanings and objective ascribed criteria. The notion of a reflexive tool

implies a learning evaluation approach, and the methodological distinctions categor-ized herein as an interest in understanding and an interest in explication may be

interpreted as a synthesis between a bottom-up approach to evaluation and a top-down approach.Alternatively, when we consider the objective of the evaluation to

promote reflexivity within learning technology pilots and projects, an approach whichhas the potential of dialogical communication (openness to negotiation) and learning(understanding the norms that influence individual and collective action) becomescentral to the action-research model implied.The concepts of stakeholders and cultural logic thus serve as key referents to a

discourse analysis approach. The notion of the stakeholder in evaluation research refersto those interested parties who constitute the constituency of the evaluand (in a direct

sense); cultural logic implies a cultural reference for understanding the collectiveclaims made by different stakeholders which allows us to identify a particular discourse

ecology, meaning the interaction and synthesis of different constructs produced and

interpreted through discourse.The objects of analysis of the evaluation are the programs pilots and projects-

viewed as collective stakeholders-on technologies and systems development. Part ofthe their task is to explore the social conditions of advanced learning technology (ALT)which implies (1) an understanding of the social effect ofALT and (2) that a collectiveclaim can be made which is communicable and potentially realizable. The backgroundof the participating actors is indicated by the technological profile of the projects andthe technical imperative which defines the goals, rationale and objectives of the

program taken as a whole.



5/20

220

Principles of DiscourseAnalysis

Before outlining the design of a discourse analysis and its application in a specificempirical context, the principal tenets of a critical discourse analysis theory need to beaddressed.A theory of discourse analysis has been developed within a range of

disciplinary fields and theoretical traditions. Within sociology alone one can identify

different methodological paradigms in the application of a discourse analysis theory,for example: critical hermeneutics, frame analysis, textual analysis, ethnography andconversational analysis, speech act theory and theories of hegemony and ideology.Similar paradigmatic distinctions are made in linguistics; for example, Schiffrinidentifies speech act theory, interactional sociolinguistics, ethnography of communica-

tion, pragmatics, conversational analysis and variation analysis as distinct approachesin the linguistic analysis of discourse (Schiffrin, 1994). The mix of disciplinary,methodological and theoretical traditions has meant that discourse, and discourse

analysis in particular, is not clearly understood as a definable concept. The problemprimarily stems from the adoption of techniques without explicit reference to

methodological concerns. The sociological interest in discourse and its analysis is inthe production, interpretation and reception of meanings, i.e. cultural reproduction andtransformation. Textual and frame analysis based on social semiotics and critical

hermeneutics are combined in this study as techniques of analysis.Discourse is understood as language in action, in that it contributes to the

production, transformation and reproduction of objects and subjects of social life. It isnot simply a descriptive tool for representing reality, but constructs meaning for reality(Fairclough, 1993: 41). The following definitions assume the constituting and structur-

ing effect of discourse: according to Donati (1992: 138) discourse is a language event...

the act through which ideative and symbolic constructs are actualized and made realin the human world; Van Dijk (1985: 5) defines discourse as a central and manifestcultural and social product in and through which meanings and ideologies are

expressed or (re-)produced; Fairclough (1993: 64) understands discourse as a practicenot just of representing the world, but of signifying the world, constituting and

constructing the world in meaning.The different theoretical presuppositions entailed in these definitions conceive of

language as a social process, as a locus of ideology, power, agency and socialtransformation. They also imply a communication paradigm, or what in speech act

theory is referred to as the illocutionary act, i.e. the communication that is made

possible in saying.The positions (i.e. paradigms) entailed in these definitions focus on different

constitutive elements of discourse and the semantic space in which social meanings are

produced and contested. For example, in the study of political and media discourse themain emphasis is on the systematic categorization of events and participants, as well as

production and interpretative processes. Methodological distinctions are drawnbetween micro and macro or structural and contextual sensitive approaches to

discourse, and models which integrate along this continuum. Cicourel, for example,distinguishes between top-down and bottom-up approaches as a general categoriza-tion of models of discourse

analysis.Top-down approaches are concerned with higher



6/20

221

levels of abstraction or deep structure which indicate the context of goals and beliefs,etc. Bottom-up approaches emphasize structural features of a discourse, for example,syntactic structure and propositional content (see Corsaro, 1981: 24-8).

The approach adopted for this study is an integrated model in that it seeks tocombine a textual and frame analysis into its analytical framework. Kress distinctionbetween discourse and text provides a useful reference for an integrated model. Textual

analysisconcerns the

materiality,form and structure of

languageand

belongsto the

linguistic domain, whereas the study of discourse is concerned with the content,function and social significance of language (Kress, 1985: 27). The analytical tools oftextual analysis allow us to identify and understand the structural cohesion of a text

beyond the mere descriptive reference to its propositional content. For example, the useof textual signifiers enables the analyst to establish a relationship between agency,causality and responsibility in relation to propositional claims and the .structure of

argumentation and interactive communication. Textual signifiers also enable the

analyst to identify a set of meaning relations, including ideational and interpersonalmeanings.5 This is the micro level of analysis, whereas the use of frame analysisfacilitates the macro level of abstraction, which is primarily concerned with discursive

procedures. The identification of frames of reference allows us to move from discourseto the narrative structures which underpin meaning formations. These are inferred from

the ecology of utterances which make up a discourse, or which structure competingdiscourses.Aframe analysis is concerned with the deep structure of language use and thecommunicative dimension of discourse. Frames are understood as the macro-cognitivestructures which social actors mobilize in the creation of shared meanings and

understandings and which give meaning and reference beyond vocabulary style;discourse is more than a mere collection of words, phrases and sentences. The conceptof a frame originated in the cognitive sciences to denote a higher-order pattern orschema-that known structure which gives meaning to perceptions. In summarizingDonatis discussion of framing the following definition is offered: a frame is a

standardized, predefined structure of perception which enables the social actor to makesense of his or her reality, not just in the sense of describing reality, but identifying it

(Donati, 1992: 141). Similarly, Goffman uses the concept of frame to indicate

interpretative schemata which enable social actors to locate, perceive, identify andlabel their experiences of the world (Snow et al., 1986: 464). Gamson (1988: 220-1)develops the notion of frames as cultural themes which are mobilized in the process of

constructing meanings.Although we may identify a dominant frame of reference in any given text, the

dynamics of communication are determined by how frames are symbolically consti-tuted, i.e. what they appeal to in terms of collective identity construction. Socialmovement theorists point out that frames are strategically used in the creation ofcollective identities and in competing for symbolic opportunities, frames may bealigned or played off against each other. This is indicative of the permeability ofinstitutional discourses and shifting discursive procedures.Aframe analysis enablesthe analyst to move beyond propositional claims and how they are articulated in termsof the mode of discourse, to incorporate knowledge claims as cultural constructs. It is



7/20

222

only by identifying the framing of a theme that we can begin to look at the symbolicmessages which a particular frame, or frame alignment, encodes and how these

symbolic messages are carried. Other than in the use of textual features such as

metaphors and analogies, which act as discursive signifiers in indicating a frame of

reference, the task of identifying a frame is an interpretative process.Adopting a frame or macro analysis enables the analysis to move beyond mere

referential procedures (lists of linguistic and textual signifiers) to asserting an interestin language in action where agency is the central sociological tenet. Part of this task isto identify internalized representations or schemes of the external environment and re-

present them according to the discursive procedures which structure their meaning. For

example, how do narratives function in creating shared meanings? How do discursive

procedures maintain ideological closure or open up the possibility of a reflexivediscourse? This is particularly pertinent to evaluation approaches which focus on thecollective actor as the object of study, as well as the subject.

Supporting a Communication Paradigm inAction Research

Promoting reflexivity defines the action dimension of the research in that it assumesthat stakeholders have something meaningful to say about the world they occupy. The

discourse analysis combines the pragmatic goal of facilitating an evaluation culturewith a clear methodological focus. This is achieved by making explicit symbolic,cognitive and other processes which play a key role in decision-making structures andin defining norms of action for innovation activities. The discourse analysis also

explains the symbolic mechanisms, particularly in terms of values and belief structures

(i.e. predefined expectations), which appeal to constructs such as technical proof andwhat is socially desirable and acceptable from the broader discourse domain of new

communication and information technologies.There is an explicit link between the application context of the discourse analysis

and the promotion of reflexivity within the reference group of the study. Reflexivityessentially concerns the idea that everyday practical accounts are not only self-

referring, but that these accounts reproduce and transform the social situations to which

they refer. The analysis of discourse allows us to reconstruct from sense making thecontext in which actors find themselves. The discourse analysis design is comparable toGuba and Lincolns hermeneutic dialectic cycle in that it is designed to make explicitthe creation of shared understandings about what it means to be involved in innovationactivities. The critical element of this approach is that it avoids the abyss of evaluativerelativism by reconstructing the discourse(s) to a higher-level synthesis. This opens upthe opportunity for stakeholders to critique, reflect on and/or reformulate theirconstructions as part of their own self-evaluation work (Guba and Lincoln, 1989: 149).This is analogous to the learning-by-doing model valued by innovation actors in

dealing with technical problems.Collective learning and meaning formations are made possible through communica-

tion. The coherence of cultural meaning systems (cultural logic) is constituted bydiscourse. The discourse analysis allows us to identify the constraints of the commu-

nicationprocess

on

reflexivityand self-formation. Since the discourse is the

productof



8/20

223

an interactive and communicative event involving the social exchange of meanings, its

analysis reveals something about the dynamics of what is unfolding in relation to thecultural logic of the focus groups and how this translates into normative lines of

action.

Operationalization

The substantive area of inquiry involves two trans-European project teams: ProjectAconcerns the development of a new dimension in computer-aided teaching andeducation and Project B concerns the development of an integrated system forsimulation-based training in small and medium enterprise environments. (The names of

the projects and identities of the partners and their organizations remain anonymous inall public presentations of this study. Identity tags were, however, used in .the internal

reporting structures in order to elicit feedback from participants.)The design of the discourse analysis is operationalized through self-evaluation fora,

otherwise known as round-table discussions. The intention behind these constructed

events is to provide a forum which would have the potential to facilitate a process ofself-evaluation for the participating actors. This design follows an action-research

framework; at an action level, the discourse analysis attempts to answer the question

how are shared understandings emerging among project participants?; at the level ofdescriptive findings, it seeks to answer the question, are shared meanings beingconstructed? The methodological approach, hence, attempts to incorporate two

epistemological positions-combining a service role with a research agenda. The pointof intersection between these two agendas is the pursuit of a constructivist orhermeneutic approach to evaluation, hence the focus of the study on naturalistic

settings (natural language events) in a time/context frame which captures the process ofinnovation.

The round-table discussions were set up as part of each projects concertation

activities, i.e. management project meetings which involve all the participatingpartners. Two hours are set aside for a round-table discussion which is facilitated in

both cases by a fieldworker, otherwise known as a liaison officer, who has experiencein the field and acts as a mediator between the analyst and the focus groups involved in

the study. Each round-table is audiotaped and transcribed in detail, although not in asmuch detail as one would require for a conversation analysis design, where attention

may be given to paralinguistic features.Approximately 4 hours of discussion involving23 participants across both round-table fora amount to 64 pages of text and over 3000coded lines to be analysed.

The idea of setting up round-table discussions as defined contexts for the participa-tion of stakeholders limits what can be meant in a discursive situation, and thus

contains the focus of the analysis on the evaluation question of the construction ofshared meanings in highly uncertain environments. Otherwise the analysis wouldbecome too arduous to be meaningful. The facilitator of the round-table discussions is

given a set of questions which serve as loose indicators of the type of data the researchdelineates as possible topic frameworks. The format of the round-table, however, isunstructured and open to the participants shaping the event for their own needs. The



9/20

224

facilitators discursive contributions are not excluded from the analysis, since she or heacts as a probe suggesting propositions and introducing topic areas for discussion and

metaphors or specific meaning contexts, for example, for such nebulous concepts aslearners or users, which are frequently found in the rhetorical structures of learningtechnology discourse.

Process ofAnalysisThe investigative process moves iteratively between three levels of analysis which are

categorized as descriptive, analytical and interpretative. Each level is referred toas a separate process although they are analytically related. The descriptive levelconcerns the prepositional content of the text and gives the first snapshot of the overallcontent of the discourse; it is also the starting point for identifying themes and topicsand the intertextual context of the discourse, i.e. how assumptions are carried over fromone speaker to another, in circumventing what can be said in relation to a topic area.For the purposes of analysis the propositional content is organized sequentially.

The analytical level involves the deconstruction and coding of the text. Codes areconstructed for textual features which give coherence to the discourse. The framework

begins with features of intertextuality. In its broadest definition intertextuality refers to

how utterances are shaped by prior utterances to which they are responding and thesubsequent utterances that they anticipate (see Fairclough, 1993: 84-5). Intertextualityserves as an analytical tool in understanding how subjects are constituted through texts,and how texts are reproduced and transformed. From the point of view of structuralcohesion, intertextuality allows the analyst to look at relational processes; in other

words, how social practices are reproduced, maintained or transformed.An easy way of

approaching this in dealing with discourse is to ask of the data: how do subjectsposition themselves vis-a-vis other stakeholders, including those stakeholders such asusers who are indirectly constituted in the discourse? This dimension of the analysisis further expanded by identifying other textual functions which indicate ideational and

interpersonal meanings. These correspond to the cognitive and action functions ofdiscourse (Halliday and Hasan, 1985: 44), and refer respectively to how discourse isused to reflect experientially and how the roles of participants are encoded in the

discourse. This enables the analyst to identify the symbolic meaning of how individualor collective actors represent their actions to themselves from the standpoint of otherstakeholders. This is particularly pertinent in a context where a culture of communica-tion and shared meanings is assumed.

The round-table fora are an appropriate setting for the type of interaction which in its

very essence is intertextual, in that participants recreate their personal narratives as partof the historical and collective narrative of the project. This is particularly useful wherethere are time constraints placed on the evaluation research. For example, Guba andLincolns hermeneutic cycle requires an extended timeframe in order to create suchintertextual contexts. In order to reconstruct stakeholders constructions, expandingcircles of interviews are conducted with different stakeholders in a given socialnetwork, and these are staggered to allow for the analysis of each individual interviewas a reference for

conducting subsequentinterviews with other stakeholders.



10/20

225

In the first step of the analysis salient themes are identified and mapped out as

macro-propositional indicators of the discursive exchange among the focus group. Thisis the descriptive level and it serves as a key reference for the analyst as she or hemoves within and between the other two levels. Salient themes, once identified, serve

as indices of meaningful speech acts. These are subject to a deeper structural analysisby unpacking the framing mechanisms adopted.Although we may identify a dominant

frame of reference in any given text, the communication dynamic is structured by thesymbolic packaging of frames, in other words, what they appeal to and how the act of

appealing is constructed. Frames are not literally outlined in a text, but textual

signifiers such as metaphors and analogies may suggest a frame. The full sense offrame analysis, however, depends on a critical interpretative framework which offers a

deep-structure analysis of the text. Such a framework often requires many iterationsand reconceptualizations of the analytical categories adopted. The mode of discourse

(scientific, natural, etc.) is also important in signifying the context of meaningformations and the role of the active subject.

The third level refers to the reconstruction of the texts in relation to the state of

knowledge, beliefs, wants and attitudes that are articulated. The global relevance of thetext refers to how the discourse of learning technology innovation is constituted and thesocial significance attached to technology innovation, i.e. its symbolic meaningstructure.At this level we are ready to address the role of discursive practices increating the institutional context for leaming technology innovation and their structur-

ing effect on shared meanings. Figure 1 illustrates the relationship between thedifferent levels of analysis. The circular representation (1) refers to the centrality of

reflexivity to the discourse analysis as a whole and (2) indicates the many iterations and

reformulations required in a hermeneutic approach to discourse analysis.

Figure 1. Levels ofAnalysis

The investigation process is constructed as iterative, qualitative, heuristic and open.As the analyst moves from one level to another she or he is forced to reflect back andforth. Level 2 is done step by step as the analyst codes the text according to each

analytical category, but coding is only a general categorical device and in forming the



11/20

226

links between, for example, the thematic structure, propositional claims and frameswith the expression of affinity and relational processes (social identities and events/

actions), the analyst has to move cyclically between the categories. The movementbetween and within levels involves a critical review of assumptions, the appropriate-ness of different analytical tools and the use of conceptual models. While the differentlevels are conceived of as analytically distinct, they are also interrelated. Hence, the

methodological decisions made at each stage have to be addressed in terms of theirimplications for the final interpretation. The process itself allows for this type of

openness and flexibility.Once the text is transcribed it is loaded on to a software program known as the

Ethnograph.8 This numbers the lines of the text; the numbered text is then hand codedaccording to the analytical framework described earlier and computed. The programallows the analyst to search for key sections, for example, according to theme, frame,modality. The coding and search arrangements enable the analyst to carry out acontent analysis which helps to clarify her or his interpretation of the dominant modeof discourse adopted by the participants in the round-table.The categories, however, are not designed purely for a traditional content analysis,

the purpose of which is to yield frequency of occurrence of certain categories. The

coding is designed instead to facilitate the descriptive and explanatory dimensions of

the analysis; to enable the analyst to work her or his way through different levels of theanalysis process without turning the analysis into an arduous task, particularly giventhe quantity of text involved in the study. The Ethnograph software program is

designed around the twin concepts of decontextualizing and recontextualizing data. Itenables the analyst to embed contextual information into the data sets. The usefulnessof this organization feature becomes more apparent when the analyst is dealing with

multiple data sets. The coding format is flexible and allows for the possibility of

overlapping between boundary segments and nesting codes into larger units of

meaning. This type of flexibility makes the ethnograph suitable to the type of open anditerative model of analysis suggested here.

The final interpretation, however, does not belong to the researcher alone. Thediscourse analysis has to have some practical input into the projects that have

participated in the round-table discussions, and their feedback on the findings will be a

valid input into the final interpretation and critical to the development of discourseanalysis as an evaluation methodology appropriate to a learning model. The facilitator/fieldworker also has a role in terms of providing general observations and contextualcomments to guide the analysis.

Observations and InterpretationSince the textual analysis allows for structural elements in the discourse to be captured,we can extrapolate the meaning of the context in which the data are gathered for this

study. How the subjects in the study interpret this event gives us an indication of the

potential of the pragmatic goals of discourse analysis when designed from the

perspective of an evaluation strategy which stresses the stakeholder stance. The degreeof openness of the participants to this event is indicated by how they position



12/20

227

themselves vis-a-vis each other in the context of the round-table fora and relationally interms of the dominant assumptions encoded in the projects official text,9 i.e. therhetoric of goals and objectives. This, for example, can be captured by intertextualrelations between the immediate talk in action and other discursive events constituted

as texts. This is what usually makes up the narrative structure which participantsdevelop through a dynamic discourse. The analysis also looks at how the participants

position themselves subjectively or objectively in terms of their affinity to proposi-tional claims by using textual signifiers such as transformation and modality.o

In both cases the overall mode of discourse is informal and textual meanings areencoded in explanatory accounts with participants positioning themselves subjectivelyand normatively vis-a-vis each other in seeking shared meanings. In those sections ofthe discussion where technical topics are foregrounded the objective mode of ascientific discourse is adopted and a didactic mode emerges at times as a more forceful

type of text as participants seek to establish their own identity in rating a certain kind of

knowledge (expertise) or perspective over another.

Reflexivity is the self-organizing principle of R&D projects which function as

temporal organization units. The discourse analysis shows that this is the case when

partners operate from the standpoint of all the stakeholders involved, even when this is

experienced as constraining on individual actions or interests. The round-table fora are

presented and consequently perceived as self-evaluation or self-reflexive exercises.When the discussion shifts in one of the round-table discussions to the value of

participating in this type of event, one participant points to reflexivity as a discursive

procedure which has benefit for the social relations of a project and the realization of a

learning process:

... if youre asking what did we discover in the last one-and-half hours, I think we are forcedalso to do some self-reflection, and I really believe that self-reflection is quite often veryuseful, and because of the always going frame of time ... the time question ... you often

forget to do that self-reflection.And it is clearly because also, it is an opportunity to show that

you respect the other person, because it looks like if you are always in an argument situationthat you are always arguing with one another. But at the same time, we have quite a lot ofideas coming and this self-reflection, not necessarily in the shape it is done now, you certainlyrealize that you are learning.

The discourse analysis shows that reflexivity is not only a significant factor for thecommunication culture of a transnational project (made up of different stakeholderswith different interests), but that it is also central to the learning process: it entails the

possibility of transforming practical actions. One of the key observations of this study(based on subjective accounts of the innovation process) is that when project actorsdeal with specific and transient problem contexts over the lifecycle of a project, there isno added value unless learning can be realized in the first place and then transferred toother social contexts. The self-reflective mode noted in the discourse analysis is thebasis of how stakeholders account for their own positioning in events. The discourse

analysis also suggests that reflexivity lies outside formal evaluation criteria. Indeed,when we look at how the field of innovation is discursively constituted, evaluation isnot perceived as having a formative influence on development projects.

Let us turn

brieflyto the discourse

producedon the evaluation function, since this



13/20

228

emerges as a salient concern in both round-table discussions. The discourse on this

theme follows official evaluation discursive procedures which distinguish betweenformative and summative evaluation, as if these were the only forms of evaluation.This is not only found in propositional claims and argumentation structures, but in themanner in which these are framed. Summative evaluation is perceived to requirestabilized features to evaluate and this is translated to mean the evaluation of outcomes.

A

distinction is made between the profit-only agenda of stakeholders (interpreted asan indirect reference to big business interests) who require this type of evaluation (cost-benefit, etc.) and stakeholders who participate in R&D for the sake of the future of

technological innovation and knowledge (assuming more altruistic motives). The

alignment of a technological and moral frame of reference appeals to the scientific

episteme (structures of knowledge which enable us to view the world in a certain way),but within the context of a set of a priori assumptions which communicate the notionthat technological innovation is always socially desirable.

While evaluation is not perceived to have a formative influence on developmentprojects, it is apparent from the discourse constructed that the term evaluation isinternalized more as a measurement/assessment tool than a processual tool (identifyingthe process of decision-making over the lifecycle of a project, in particular the processof transforming individual action into collective action); that it is concerned more with

project outcomes (outcome driven) such as products, for example, rather than areflexive and leaming tool to enable stakeholders to reconstruct their thinking. Yet, itbecomes clear from the narrative structure of the discourse that as projects move

through their lifecycles partners are led to recontextualize the ongoing process ofinnovation. Outside what is internalized as the official requirements of evaluation-

represented by the audit metaphor-reflexivity is identified as a key process in

understanding the change dynamic involved in processes of innovation. Because

projects fail to transfer learning and knowledge from one context to another, changebecomes increasingly difficult to identify and channel in terms of stated objectives and

goals according to the official texts which define the projects. These official texts,which are part of the intertextuality of the round-table discussions, function as

supranarratives circumscribing the factual or scientific world of innovation activitiesand what stakeholders may say in relation to its episteme.

Evaluation, then, is perceived more as a measurement or assessment tool for hardproject outcomes; a prototype, for example, would not fit this category.A learningmodel which would enable a formal appraisal of actions such as decision-makingprocesses is not part of the projects repertoire. The process by which these data are

gathered also indicates that without appropriate discursive procedures, cognitive andcultural frameworks for new institutional alliances which seek to traverse national,sectoral and disciplinary boundaries lag behind sophisticated technical and marketframeworks. This creates specific problems for projects in developing concrete

projections for the future; in terms of the everyday realities, it creates problems for

partners who have to transcend cultural and spatial distances in order to communicate.Communication aimed at dialogical consensus, therefore, should be considered an

important ingredient in organic project cultures where partnerships are based on the

autonomy of the

expert.This

pointis bome out in the

followingdiscussion.



14/20

229

The project actors make a distinction between communication as a conduit for

sending and receiving information (the transmission metaphor) and communication asa process of organizing. The communication model is implied in the salience of

negotiated consensus as a theme at both round-table discussions. The alignment of acultural and moral frame of reference appeals to the notion of a project culture which

presupposes a communicative rationale based on shared norms and consensus building.

This appeal is most salient in the context of grievances which are articulated in relationto the governance structure of a project; for example, the difficulty in marrying thetechnical and management coordination of the projects. In ProjectA, for example, thelevel of reflexivity which takes place towards the end of the discussion is indicated bythe comparison drawn between issues pertinent to the leaming environment which

technology development seeks to enhance and the need to develop appropriate culturaland social frameworks for technology development:

Maybe theres a parallel to our approach in the multiple training system, because in the

system we say what we actually would like to have is, say, audio and video communications

between, say, the trainee and trainers or among the different trainees. But they can actuallycommunicate on line with each other, and its not enough just to send an e-mail and then waitfor an answer.And maybe, its the same here in the project.As soon as you spoke to the other

guy sitting at the same table or on the telephone, problems can be solved out, but if you just

have off line communication with fax and e-mail the communication sometimes gets stuck, soits the same in development like in the leaming environment. You actually need realcommunication with your partners.

Projects are integrated in terms of (1) the notion of collective risk taking, and (2) the

metaphorical constitution of learning technologies as The Future (see later). Bothconstructs are carriers of cultural messages about European identity and are dominated

by a moral frame of reference, which appeals to the social desirability of technologyinnovation and development.

Because change is the most fundamental concept of innovation, there must be acollective cognitive understanding about what is different about the innovation. While

propositional claims found in the discourse indicate that stakeholders have different

images of learning technology innovation, and over the course of the projectslifecycles these images may change, there is a cognitive level underpinning normativeexpectations which enable shared meanings to be created and communicated. This isthe level of leitbilder (Tepper, 1993: 337) or collective vision.According to Tepper,the function of leitbilder is to create the notion of a shared vision and collective

projection for the future application and diffusion of new technologies. The informa-tion society and knowledge society metaphors create the context for a shared visionof the future and this is dominated by a market frame of reference. For example,education, training and knowledge production are metaphorically constituted asmarkets through the use of commodity metaphors. Know-how is a marketable

commodity, and participation in European transnational projects is constructed as ashared opportunity structure for future markets and European competitiveness. I havechosen a few exemplars to illustrate this normative relationship between technologyinnovation and economic possibility, which is guided by a market frame of reference.These

exemplarsare taken from that

partof a discussion

thematicallyidentified as



15/20

230

opportunity structures and where participants discuss the opportunities which havebeen opened up through their involvement in trans-European actions for innovation and

competitiveness in the field of learning technology.

Well, especially for our department, it gives us the opportunity to work and handle the new

technology and now we have the time for more than 2 years to handle the networking stuffand how to handle how to get across platforms, portability, things like that.... So it gives

them a good chance to go to industry and tell people, look thats the waywe can do it and we

have a chance to be successful and so, within this consortium with all the problems we raninto, that gives us special advantage to be competitive.

They arent selling products, they are selling know-how.

...

now more and more companies come to us and ask us, how can we use this new

technology. Now we have the experience how you use it and it took us quite a while to be ableto handle it. But now we can tell people, okay, if you buy this component, or this componentand if you program it like that, this architecture, you have a good chance to be successful andthat is 1 year or 2 years that they were doing their own work.And thats what theyre paidfor.

...

this is going in the direction of out-sourcing, light sizing, character architecture and not

only using the workstation, but also using the PCs in order to address the small and mediumsize

enterprises,and this is our new

targetmarket within the

projects community.The importance attached to the imperative of the economy and the post-industrialcontext for new markets or market niches is used as both a cognitive and normativestandard. Similarly, opportunity structures metaphorically constitute collective risk

taking and this is dominated by a market frame of reference indicated by the discursive

procedure of reification.At the level of symbolic integration collective identities are bound up with the role of

the expert. The notion of cross-fertilization between expert domains carries a culturalvalue which resonates with the official cultural agenda of EU initiatives in informationand communication technology. This creates a symbolic context for transnational

cooperation which is rationalized in terms of building a European research profile todetermine future lines of action for new technology and new institutional arrangementsin the areas of education and training. The difficulties commonly experienced byproject participants in such loosely organized research networks are qualified by a setof shared assumptions about what it means to be involved in European innovation. For

example, the construction of the collective agent is mediated by the grand metaphor:The Future.

The Future is a cognitive construction which defines the context of meaning.Although future-oriented goals underpin stakeholders involvement in trans-Europeanprojects and serve as cohesive mechanisms at both project and program level,

propositional claims indicate that there is no clear objective for future developments.Hence, the discourse indicates the need for reflexive learning to assess the differencebetween cognitive and normative frameworks for change. This kind of learning lies atthe intersection between a futuristic vision and what stakeholders could hope to achievewithin the context of their immediate actions.

Drawingon both anecdotal and more

systematicevidence

generatedas

partof the



16/20

231

broader evaluation study of which the discourse analysis is a part, many of the claimsof R&D practitioners in the field of learning technology innovation about userinvolvement-an issue close to the heart of social scientists and education evalua-

tors-are dismissed as rhetoric. There is little explicit in the discourse produced as partof this study which indicates that technology innovation projects are driven by userneeds. However, metaphors such as virtual classroom, workbench, network and

gruyere cheese are used by the focus groups to configure and communicate a vision ofthe future of leaming technologies in terms of techniques and social structures whichare familiar and accessible, particularly to end-users. The manner in which metaphorsare used to convey the image of a specific innovation or the generic concept, i.e.

technology innovation per se, indicates that, at a conceptual level, user orientation isassimilated as a central feature and determinant of the future of learning technologies.The centrality of the user is also indicated by the alignment of a market and

pedagogical frame of reference; the user is both a discerning customer and a leamer.Learners are discursively constituted as active subjects in technology innovation

through consumer metaphors:

And I mean we are conscious that things change quite rapidly in this area, but um ... and wehad our own sort of experience and knowledge. But we supplemented that by doinginterviews with three

major companiesand the sort of work

theywere

doingto

geta

better,you know to confirm and to, this issue of the user profile-the user being these sorts of

designers. We are conscious that because the thing is constantly developing; we are consciousin a sense the people who are developing, say authorware or any similar package, or a

modelling environment such as Lap View which is one weve mentioned. Even in the historyof this project of two-and-half years, there have been two or perhaps three versions of those,so in other words those packages have been upgraded by the commercially, commercial

judgements of the people who make them to reflect the needs of that, of their user group.

The particular example occurs in the context of a discussion about the value of theuser as a yardstick for measuring the success of one particular product over another. It

appeals to the importance of commercial actors as both users and knowledgeproducers; it also appeals to the significance of the transfer of expert knowledge aboutmarkets to an R&D context.

Communicating a vision of the future forALT innovations is part of the everydaydiscourse of innovators; it structures a collective narrative to create a focus for shared

meanings to emerge. The Future is constructed as a family of images which functionas interpersonal stabilizers at the level of project participation. For example, thesymbolic meaning structure of networks, which is part of the technical lexicon of

technology texts, appeals to a vision of the future. The network concept is a

metaphorical reference to loose organizations (less formal, quasi-institutional arrange-ments) such as trans-European projects. Constructed to mean material resources,networks create future opportunity structures.As technical constructs, networks arecommunication media. Their symbolic significance, however, is found in a cultural textwhich appeals to the notion of developing appropriate cognitive and social frameworksfor new and future modes of knowledge production and delivery.At the level of the

symbolic integration of the innovation projects, the network concept is part of the self-



17/20

232

image of the projects. This is also found in propositional claims about the significanceof being able to transfer knowledge across expert domains and using communicationand information technology in place of paper-and-pencil techniques, or pedagogicalclaims about replacing chalk-and-blackboard techniques. The cultural subtext of

European cohesion is also appealed to in the use of the network concept, i.e. acollective European identity which is facilitated through a culture of technology. This

creates the moral narrative. For example, in one project the cooperative goal is used inappealing to the notion of partnership as something more than a means-end

relationship. In other words, practical rationality is given precedence over technical

rationality:

Yes, we talked about it in the morning, about the efforts we have spent in the project and wesaid, well if we give all of the money just to one company, this company may get a better jobdone than by spreading the money all over Europe. But these European projects are not onlyfor getting a job done, but also for forming the links between the different partners.

In another project propositional claims about the difficulties involved in building ateam with partners who come from such disparate institutional and cultural back-

grounds are redefined by one partner in terms of the unique intellectual experiencewhich a

partnershipculture endows:

I would actually suggest that theres an interesting sort of collaboration with colleaguesaround the table here, which is rather ... has a lot of stimulation ... intellectual stimulation

that doesnt have perhaps some of the overlay of company politics. Or ... you know internal

pressures of ... that might exist in organizations. I mean, for example I come from auniversity background, so I dont have to cope with the politics of my boss and myprofessor.

In the absence of a hierarchical organizational symbol system where consensus iscoerced to fit a managerial framework, processes of negotiation and integration are

foregrounded as the key constituting organizational structures. In one of the round-table discussions a marriage analogy is used to exemplify an alternative organizationalculture. The grievances articulated in relation to the managerial and coordination

problems which arise in such loosely bound organizational forms are redefined andtransmuted by an appeal to the active involvement of partners in self-formation freefrom organizational constraints and managerial frames of reference. This in turn

appeals to the action-shaping influence of innovation actors and the relative autonomyof innovation activities.

The discursive procedure of referentiality is used to indicate what is new byappealing to what is already known within the realm of human experience. This isachieved through the use of metaphors and analogies. For example, at one of the round-tables the social and technical background of the telephone is used to construct anarrative about the meaning of innovation, the initial uncertainty which surrounds itsinvention and the realization of its potential social impact.At the level of textual

meaning the properties of a new system are explained in terms of a structure which isfamiliar and hence can be communicated.At the level of

symbolic meaningstructures,



18/20

233

a moral message is encoded to suggest (1) the uncertainty of the future and (2) thedeterminate nature of technology:

But if you remember in the case of the telephone, when it was, oh! was it the telephone orsome other device. I remember reading that when it was first invented, it was agreed that thiswould be of no use except in a very limited area, which as you know has been proved totallyunreal.

Concluding Remarks

This interpretation does not claim to be a definitive reading of the texts it addresses. Forthe purposes of this article, the author has set out to develop an argument for the needfor processes of analysis which could explicate and invoke communication and

reflexivity as a key referents of a learning evaluation approach in technologyassessments. The article is essentially concerned with the question of how shared

understandings emerge in a context where institutional boundaries have yet to developclear meaning formations. The methodological approach attempts to develop a pictureof technology innovation from the point of view of the stakeholder.

While claims and counterarguments are developed over the course of the round-table

discussions, macro propositions emerge which constitute thematic links for discussingtechnology innovation in the institutional spheres of education and training in Europe,for example, opportunity structures, idea formation and evaluation of experiences interms of future actions, etc. These form part of the narrative structure which, along with

key frames of reference, constitute a collective social identity (at least temporarily).Stakeholders operate from the point of view of the other stakeholders even in looselybound and temporal organizational structures, because without shared knowledge andbelief structures communication becomes impossible; the communicative dimension

acquires specific importance in cross-cultural temporally bound settings. This is not to

underplay the ideological constructs found in the discursive constitution of technologyinnovation, but rather to point out the need for the explication of discursive procedureswhich delimit processes of understanding and action.

The identification of key frames of reference enables the analysis to reconstruct the

supranarrative form as a discursive procedure which functions to encode ideologicaland cultural preferences. Narrative structures may serve to valorize dominant images inorder to negate or gloss over varying and possibly antithetical interests and, hence,function as strategies of closure.

In conclusion, the relationship between a learning model of evaluation and discourse

analysis needs to be asserted. The application of a discourse analysis design in anaction-research context makes possible the opening up of discursive procedures and,hence, facilitates the meeting of the theoretician and the practitioner in praxis. The

analysis itself follows a hermeneutic model in emphasizing how stakeholders come toshare a common appraisal of their actions and, in so doing, generate intersubjectivelymeaningful contexts of action. This is achieved through working directly with theresearch participants discourse and their understandings of the social context of

learningtechnology

innovation.



19/20

234

Notes

1. The task of developing a coherent evaluation framework and strategy for the program in

question is carried out by a project consortia entitledARTICULATE, which exists as ahorizontal activity within the overall program.

2. Trans-European partnerships are constituted by research institutions, including universities,small and medium size enterprises and large businesses from EU Member States working

cross-culturally to support European R&D efforts. The official aim of trans-Europeanpartnerships is to transfer expertise between institutions across the different Member Statesas a means of increasing European market competitiveness.

3. The Study reported in this article was undertaken as part of the overallARTICULATEevaluation project sponsored by the Delta Office of the European Commission, andcoordinated by the EDRU unit at the Tavistock Institute, London.

4. Pilot and project activities are distinguished according to two distinct purposes of the

program. Pilot activities are mainly concerned with experimentation and/or validation of

existing technologies, while project activities are involved in technologies and systemsdevelopments.

5. The seminal work of Halliday on social semiotics and textual analysis, and its use in an

integrated model (see Fairclough, 1993) have provided useful analytical tools for developinga critical interpretative framework in this study.

6. The environment of the evaluand is theoretically described as uncertain because of the

quasi-institutional context of innovation activities and the loosely bound organizationalsettings in which this activity is taking place. In fact it is this uncertaintyloosely associatedwith the concept of riskwhich legitimates the need for technology assessment.

7. See Guba and Lincoln (1989: Ch. 7) for a detailed description of the methodology ofhermeneutic cycles as a constructivist inquiry.

8. The Ethnograph software is described in Tesch (1990).9. Official texts refer to documentation outlining projects contractual obligations under theDELTA Program; official events such as technical audits at the European Commissionwhere projects are annually assessed in terms of meeting their contractual obligations, and

public reports which are contractually commissioned for reporting progress and outcomes atintermittent periods over the lifecycle of a project.

10. Textual signifiers such as modality and transformation have an ideological significance in the

production and delivery of discourse. For example, scientific discourse removes or mystifiesthe role of agents and processes. This is marked by features such as nominalizations,passivization and objective modality (see Fairclough, 1993: 236).

References

Corsaro, W.A. (1981) Communication Processes in Studies of Social Organization: Socio-

logicalApproaches to DiscourseAnalysis, Text:An Interdisciplinary Journal for the Study ofDiscourse 1(1): 5-62.

Donati, P. R. (1992) Political DiscourseAnalysis, in M. Diani and R. Eyerman (eds) StudyingCollectiveAction, Sage Modem Politics Series Vol. 30. London: Sage.

Fairclough, N. (1993) Discourse and Social Change. Cambridge: Polity.Gamson, W.A. (1988) Political Discourse and CollectiveAction, in B. Klandermans et al.

(eds) From Structure toAction: Comparing Social Movement Research across Cultures.

Greenwich, CT: JAI Press.

Guba,E. G. and Yvonna S. Lincoln

(1989)Fourth Generation Evaluation. London:

Sage.



20/20

235

Halliday, M.A. K. and R. Hasan (1985) Language, Context, and Text:Aspects of Language in aSocial-semiotic Perspective. Oxford: Oxford University Press.

Kress, Gunther (1985) Ideological Structures in Discourse, in T.A. van Dijk (ed.) Discourse

Analysis in Society: Handbook of DiscourseAnalysis, Vol. 4. London:Academic Press.Schiffrin, D. (1994)Approaches to Discourse. Oxford: Blackwell.Snow, D.A., R. E. Burke, S. K. Worden and R. D. Benford (1986) FrameAlignment Processes,

Micromobilization and Movement Participation,American Sociological Review 51: 464-81.

Tepper,A. (1993) FutureAssessment by Metaphors, Behaviour and Information Technology12(6): 336-45.

Tesch, R. (1990) Qualitative Research:Analysis Types and Software Tools. New York: TheFalmer Press.

Van Dijk, T.A. (1985) Introduction: DiscourseAnalysis in (Mass) Communication Research,in T.A. van Dijk (ed.) Discourse and Communication: NewApproaches to theAnalysis ofMass Media Discourse and Communication. New York: Walter de Gruyter.

O R L A M O R R I S S E Y isa member of the research team at the Centre for

European Social Research, University College Cork, Ireland. Her academicinterests are in qualitative methodologies, critical sociology and feminism. Herresearch experience ranges from projects in telematics through to the field of

biotechnology, in particular, bioethics and the construction of public discourses.Her interest in qualitative methodologies and action research has led her to focuson discourse analysis as a reflexive and communicative approach to social

enquiries. (email: [email protected])

Analiza de Discurs_learning Tech_Morissey

Documents

Transcript of Analiza de Discurs_learning Tech_Morissey