From Inner Speech to Dialogic Semiosis:A Semiotic Approach to AudiovisualMultimedia CommunicationUlla Oksanen, Helsinki

AbstractIn the continuum of audiovisual narration from the early days of cinema untiltoday there has at times emerged the intention to study in depth this “language”of images and sounds and to develop it radically. The inquiry intothese kinds of meanings, and the dynamics connected to them, has in particularbeen approached from the semiotic point of view. The attempt tocomprehend and construct audiovisual narration as inner speech and dialogismoffers an attractive frame of reference for perceiving the nature ofcommunication in modern multimedia interaction as semiosis, the action ofsigns. In this article the change in audiovisual narration, from the semioticpoint of view, can be reduced to the following tendencies: from monologismto dialogism, from diachronic (successive) to synchronic (simultaneous) andfrom syntagmatic (co-ordinative) to paradigmatic (associative). These tendenciesare exemplified by the views of Lev Vygotsky, Mikhail Bakhtin,Sergei Eisenstein and Jean-Luc Godard.In my argument C.S. Peirce´s concept of communicational dialogismwill be of particular importance. It is a central concept in audiovisual narrationas well as in hyper- and multimedial interactive communication, and inthe dynamic process of the function of signs, dialogic semiosis, it is a fundamentaldimension.Keywords: audiovisual; multimedia; dialogism; synchrony; paradigm; semiotics;dialogic semiosis.194 Ulla Oksanen

1 INTRODUCTIONFrom the birth of audiovisual narration until today, there havebeen attempts to reach reality or produce illusions of it throughreal-like phenomena or more or less immersive experiences. Thiskind of transition from magic lantern stories to multimedia1 canalso be traced as a continuum of audiovisual narration, where attimes other kinds of aspirations have also emerged, namely theintentions of studying in depth the meaning of this “language” ofimages and sounds and to develop it radically. The inquiry intothese kinds of meanings, and the dynamics connected to them, hasin particular been approached from the semiotic point of view.The attempt to comprehend and construct audiovisual narration asinner speech and dialogism offers an attractive frame of referencefor perceiving the nature of communication in modern multimediainteraction as semiosis, the action of signs.

The top level of the multidimensional model by Tella &Mononen-Aaltonen (2000), introduced in this volume, consists ofthe background flows of communication and mediation, whose influencesare reflected to all levels of the model. These flows areimportant in audiovisual communication as well. Along withthem, however, there are other unique flows of changes, which,from the semiotic point of view, can be channelled to the followingtendencies: from monologism to dialogism, from diachronic(successive) to synchronic (simultaneous) and from syntagmatic(co-ordinative) to paradigmatic (associative). In this article thesetendencies are illustrated with the views of Lev Vygotsky, MikhailBakhtin, Sergei Eisenstein and Jean-Luc Godard.1 Hypermedia is here defined as an interactive manner of expression based on links. It mayconsist of texts, images, moving images and sounds. By multimedia I mean digitally workedup interactive programs, which may consist of texts, images, moving images and sounds andwhich are distributed as CD-ROMs, DVDs or network products.From Inner Speech to Dialogic Semiosis 195

In my argument, C.S. Peirce´s concept of communicationaldialogism in the communication background flow is of particularimportance. Communicational dialogism is a central concept inaudiovisual narration as well as in hyper- and multimedial1 interactivecommunication. In the dynamic process of the function ofsigns, dialogic semiosis, it is a fundamental dimension. Even thecontext and the medium itself are always essentially present in thissemiotic process and appear as a cultural interaction, intermediation(Lehtonen 1998) or to use Tella & Mononen-Aaltonen´s term,as intermediality.This article will now concentrate on certain instances ofchange flows, which, together, seem to characterise the transitiontowards hypermedial narration. The chapter “Inner Speech andEisenstein´s Idea of Montage” will deal with the concept of dialogismas well as with its obvious counter poles: Eisenstein´s innermonologue and Vygotsky´s inner speech. In the chapter “FromMonologue to Dialogue: Bakhtin´s Inner Dialogism”, dialogismwill be described in its Bakhtinian meaning. The chapter “FromDiachrony to Synchrony, and Syntagm to Paradigm” will examinethe change towards multimedial narration on these axes using theideas of Eisenstein´s and Godard´s cinematic theory. Finally, inthe chapter “Total Mental Image and Dialogic Semiosis” communicationaldialogism will be defined as a fundamental dimensionof dialogic semiosis in the light of total mental image and Peirce´spragmatic idea of signs.2 INNER SPEECH AND EISENSTEIN´S IDEA OF

MONTAGE

At the beginning of the 1920s, Sergei Eisenstein, film theorist anddirector, was fascinated by the “attractions” from the early age ofcinema, by the principles of showing and making perceptible. He196 Ulla Oksanen

started to emphasise what he saw as the fundamental essence ofthe appeal and impressiveness of the cinema, indeed all the arts,namely the principle of conflict. “In the intersection of nature andindustry there is art” (Eisenstein 1978, 106). On the basis of thiskind of Hegelian thinking, the world is seen as a continuous developmentof two conflicting opposites in interaction, in otherwords, as a continuum of thesis, antithesis and synthesis. Eisensteineven compared montage cutting to an efficient combustionengine in a car, where the “attraction” shots, defined by cutting,form a series of explosions, whose dynamic energy runs thecar—or in his case the film. The same principle of conflict wasvisualised, said Eisenstein, in Japanese pictography, where twoseparate signs are summed up and “explode” into abstract ideas.eye + water = to crymouth + bird = to singdoor + ear = to hear (Eisenstein 1978, 108)Likewise, in Eisenstein´s thinking when two still pictures areplaced next to each other, they produce a new idea: “... becausesuccessive things are not actually placed adjacent to each other,but one upon another” (Eisenstein 1978, 109). According to JuriLotman (1989, 67), Eisenstein´s montage, the so-called “intellectualmontage”, activates the boundary of thought and changes itinto a principal medium of meaning (e.g. Kerensky mounting thestairs in the film “October”).In 1927 a well-known formalist Boris Eikhenbaum presenteda view in which he emphasised the close connection of the “language”of the cinema and the so-called inner speech. In his opinion,montage, as well as common speech, consists of fragmentaryexpressions rather than complicated and logical sentences. Manyscholars (among others Wollen 1977; Broms 1984; Hietala 1994)see the decisive role of Eikhenbaum in the development of Eisenstein´s theory of montage and the inner monologue. However, theFrom Inner Speech to Dialogic Semiosis 197

concept of inner speech has been analysed more profoundly byLev Vygotsky in his writings (1920) and later in his book“Thought and Language” (1934). One of the basic principles ofVygotsky in this work is that the genetic roots of language andspeech are separate (Vygotski 1982, 99). The decisive meaning inthe development of thought is in inner or silent speech, by whichVygotsky refers to the junction where thinking is transformed into

the lingual, and language into thought. According to Vygotsky innerspeech is primarily thinking in terms of pure meanings, or itfunctions on the semiotic level of speech. In other words, what inVygotsky’s ideas is expressed in language mainly as diachronic,successive, appears in thought as synchronic, simultaneous. (Vygotski1982, 243–244)The inner speech, “hovering” between word and thought,also has according to Vygotsky its characteristic features. Innerspeech, by contrast with outer speech, essentially reflects a clearlydifferent, new and independent function of speech: poetry-like silentspeech to oneself. Inner speech is also simplified and compressedas it “opens up” with difficulty to others and is hardly intelligiblewithout context. It consists of apparent fragmentariness,which makes it elliptic, including “open” gaps. So inner speechdeviates by its syntax from written speech by being predicativeand often even idiomatic, like a dialect. (Vygotski 1982, 46-47,178–179, 230–244)After 1928 Eisenstein started to move away from strict antagonisticdialectics and emphasised montage as an emergingsynthesis of the different senses. After familiarising himself withJames Joyce´s novel “Ulysses”, Eisenstein´s thinking began tomove towards a more organic, “softer” view of cinema, in which,instead of conflicts, he started to stress harmony and holism.(Malmberg 1974, 34) The filmic “inner speech” or inner monologuenow became the cornerstone of Eisenstein´s new idea ofcinema. This kind of “monologue”, which as a structure forms a198 Ulla Oksanen

stream-of-consciousness-like reconstruction of the thinking process,was a continuation of the earlier mentioned idea of “intellectualmontage” from the beginning of the 1920s. Though Eisensteineaninner “cinematic” speech, “the inner monologue”, is fundamentally“thinking with pure meanings”, the nature of it is describedas flexible, pictorial, non-logical and mythic. Even thoughit is reminiscent of language, it uses, not only natural language asits material, but also pictures, sounds and writing. (Willemen1983, 155; cited in Hietala 1994, 49) Eisenstein, too, was convincedof the fact that inner speech was, unlike outer verbalspeech, closer to sense- and image-based thinking (Wollen 1977,29). As Eisenstein´s view of montage approached stream-ofconsciousness-like “speech”, he also started to emphasise counterpointor polyphony (a term he was to use later) as a centralmethod and goal in cinema. Eisenstein compares “harmonic polyphony”to a developing consciousness, which while creating contact

between separate phenomena of reality, experiences everythingas a simultaneous “great unity” (Jalander 1990, 79). On thebasis of psychoanalytic theory this kind of “inner monologue” canalso be understood as a meeting and “negotiation point” or intersectionof textual surfaces.3 FROM MONOLOGUE TO DIALOGUE:BAKHTIN´S INNER DIALOGISMUnlike Vygotsky, his contemporary Mikhail Bakhtin consideredthe nature of the whole of human consciousness as dialogic. Aword, a language or a whole culture can become dialogised. Becauseof this it is at the same time an orientation of any living discourse(Holqvist 1990, 427–429). According to Bakhtin, the literatureof Dostoevsky broke the form of the established linear andmonologic novel in an epoch-making manner. Whereas GoetheFrom Inner Speech to Dialogic Semiosis 199

strove to see the consciousnesses of the same person as successive(diachronic), Dostoevsky´s work was characterised by a tendencyto describe those phases of consciousness as simultaneous (synchronic)and parallel. Different persons seem to meet each otherin a spatial dimension. As the events are mainly placed in the“present”, or as if in the “eternity”, his novels lack causality, allusionsto the past, the environment and education. The fact thatDostoevsky viewed life as an interaction, as voices that sing indifferent tones of the same theme, Bakhtin regarded this as polyphonicand at same time thoroughly dialogic. Bakhtin also describedthis kind of dialogue as an almost universal phenomenonthat covered all human speech and relations and all expressions ofhuman life, indeed, everything that matters. No idea, he said, issimply able to survive in the isolated consciousness of a humanbeing, “...if it remains there only, it degenerates and dies” (Bahtin1991, 51–71, 132; translated by C. Emerson). In other words, onlyby bringing ideas into connection with other consciousnesses andstrange ideas can thoughts live and give birth to new ideas.In Bakhtin´s views dialogisation can also occur in the humanconsciousness in an internal monologue. This he calls inner dialogue2,(“microdialogue” in the revised edition in 1963), a dialoguewith oneself. Dialogue can also be part of the whole structureof words, and intervene at all semantic and expressive layers.Dialogue penetrates into every single word and causes a struggleand resonance between the voices (Bahtin 1991, 115, 279,392–397). The addresser, the author, and the addressee, the reader,2 Unlike Bakhtin, Vygotsky considered the written language and inner speech as the monologicform of language, and only oral speech was dialogic. Vygotsky saw dialogism as achain of reactions that presupposes that partners know the essence of the matter and whose

looks, gestures and tones add a zest to the conversation. (Vygotski 1982, 235) However, acloser study of Vygotsky´s writings on inner speech suggests, according to Wertsch, thatVygotsky in effect saw the dialogic (in Bakhtian terms) nature of speech. A more appropriateterm for inner speech would thus be inner dialogue. (Wertsch 1980, 151–152; cited in Tella& Mononen-Aaltonen 1998, 25) Accordingly, on the same grounds, Eisensteinean “innermonologue” could also be interpreted as inner dialogue.200 Ulla Oksanen

are both part of the dialogue. According to Söderbergh Widding(1997, 14) the spectator of cinema is also an active partner in thecinematic experience, because he or she has the decisive role as anaesthetic subject, as an interpreter of meanings. Answering to theimpulses produced by the film (or novel) the addressee ultimatelycreates the true essence of the film (or novel) and in a fundamentalway gives birth to his or her subjective experience.Bakhtin understands the thoughts and voices of the parties tothe dialogue as a unity of intersecting levels, as a polyphonic systemthat creates harmony and resonance or produces dissonances(Bahtin 1991, 358). Similarly, Kristeva sees texts as multidimensionalintersections that endlessly bear traces of other texts. Thisintersection is at the same time the dialogue among several writings,where the author, the addressee and the characters, as well asthe present and the past cultural context meet (Kristeva 1993,22–26). So to Kristeva the Bakhtinian dialogism also means intertextuality,the mutual communication and quotation which compriseslanguage itself. In the age of the Net these kinds of ideas,from the cognitive semiotics viewpoint, can also be considered apolysemic network of signification, where the signification itself isa wholeness composed of several links and junctions, a synthesisof meanings3. It seems that in the debate concerning the nature ofnetwork-based media and multimedia narration, these kinds ofviews of the network character of meaning will increase in thefuture.3 A polysemic network of meanings refers to the theory of network-based meaning and cognitivesemiotics (Norvig & Lakoff 1987, 197–198; cited in Ylä-Kotola 1998, 57).From Inner Speech to Dialogic Semiosis 201

4 FROM DIACHRONY TO SYNCHRONY,AND SYNTAGM TO PARADIGMInner speech or dialogue proceeding in the Eisensteinean streamof-consciousness way, will in Broms´ ideas above all be manifestedas emotional and logical narration. This is, in his view,characterised by repetition of lingual and cinematic expression,“psychic drumming”. This kind of repetition refers, according toBroms to mythic “oriental” consciousness, which, unlike itscounterpole, natural-science-like “western” consciousness, ischaracterised by synchrony, a tendency to see the world as simultaneous

and repetitive. It also means a tendency to see the worldas devoid of linear continuity, a place where causality is abolishedand a new sense of time created: the overlapping of its levels andcyclicality will gain power. (Broms 1984, 67–75, 133) Juri Lotmanalso views the form of Eisensteinean montage as a “system ofjump-like transitions”, which as such foregrounds rather thestructure of life than the logic of incidents. Rhythmic repetitionlessens the visible and emphasises the abstract, logical or associativemeanings. (Lotman 1989, 51, 69) Broms (1984, 146, 169),referring to Lotman and Russian semioticians, also sees thechange of scientific paradigm in the long run leading towards unlinearways of thinking, to “oriental consciousness”, “nocturnalknowledge”, where a deep-reaching change of course towardspictorial language and symbolic knowledge has begun. Ylä-Kotola, too, holds that as an alternative to linearity the time exeriencecan be converted into the fragmentary, the curvilinear andthe cyclic (e.g. Eisensteinean “repetition”). He accordingly suggeststhat the fundamental questions of unlinearity be approachedrather by means of the concept of paradigm. Eisenstein´s experimentsat the beginning of 1920 (e.g. the “God sequence” and the“stair sequence” in “October” which were based on repetition)could, in his opinion, be considered early attempts to break away202 Ulla Oksanen

from syntagmatic4, process-like thinking (e.g. classic cinematiccontinuity) and direct itself towards paradigmatic, associative narration,and towards structure. (Ylä-Kotola 1998, 249)The tendency towards a new kind of associative and paradigmaticthinking did not begin to take its audiovisual form on alarge scale until the time of alternative or counter cinema and themusic video—and later with hyper and multimedia narration experiments.Ylä-Kotola suggests that especially in films directed byJean-Luc Godard after 1980 radical signs of change can be distinguished.He considers the “Histoire(s) du Cinéma” (produced inthe 1990s) in particular to be a kind of audiovisual hybrid and arepresentative of a transition period from the traditional audiovisualnarration to hypermedial narration. Ylä-Kotola, unlike Baudrillard,considers the fragmentariness of texts (meaning widelyany significant element in culture) to be a key factor in interactiveness,as it makes these texts suit the information network. Theabundance and richness of references in Godard´s texts, he argues,function as built-in-encouragement to “associative navigation”.(Ylä-Kotola 1998, 214–218, 281–282) In contrast with classicalcinema in its temporal and spatial closeness and clear causal relations,

it is a typical tendency in Godard´s5 thinking to offer thesyntagmatic continuity to paradigmatic fragmentariness and associativeness(Ylä-Kotola 1998, 90).Fragmentariness and interactivity have appeared as conscioustargets among other arts, too. In theatre for example, the tradi-4 Syntagm can be defined as a combination of signs that fill a certain space: e.g. shoes, trousers,jacket, cap... (Tarasti 1990, 17). A syntagm consists of all the elements that are placedadjacently to each other (in “both—and” relation), elements that are “present” in a picture(Kuusamo 1990, 49). Paradigm is defined as an associative, alternative relation (Tarasti1990, 17) that combines the present sign with the store of other possible signs (an “either—or” relation). It characterises absent relations (Kuusamo 1990, 49), e.g. cap could besubstituted by following (absent) alternatives: fur cap, brimmed hat, swimming cap, etc.5 In the fragmentary “style” of Godard this is expressed by following characteristics: the narrationis formed of “jump cuts”, monologues spoken to the camera, interviews, intertextualloans, “scratching”, and radical separation of image and sound. Time is described in motion,it is manipulated e.g. by slowing down. (Ylä-Kotola 1998, 224)From Inner Speech to Dialogic Semiosis 203

tional “Brechtian” methods of alienation were directed towardspedagogic goals. Similarly, in the 1970s Gorin anticipated thecognitive and constructivist phase, and rejected the idea of behaviourist“forced feeding” in cinema. In his view a narration thatmakes the spectator think for himself and offers versatile constituentsfor this is progressive. (Ylä-Kotola 2000) Also Eisenstein,especially in his early “attraction” montages of the 1920s, developedthe theory and forms of fragmentary narration. Parallel toEisenstein, Godard also emphasised the activity of the spectator inthe cinematic experience. To Godard the blank association spacesbetween the fragments (pictures, texts and other hints) have a particularstimulating effect on thinking, helping the spectator to dealwith the film and to construct the whole. These kinds of intentions,which connect together all these experiments viewing thefragmentariness of texts as a new dimension in audiovisual narration,mean at the same time an endeavour to see their connectionon the mental level, as dialogic. Ylä-Kotola, too, refers to the dialogic(or polylogic) nature of the spectator´s experience when heemphasises the text itself, i.e. the film, as a place of study and selfreflectionwhere the film and the spectator on the basis of his orher own consciousness and outlook on life “converse” (Ylä-Kotola 1998, 214–215). When the spectator´s experience in thisway offers the viewer a possibility to observe critically his ownattitudes to the films and ultimately empowers the spectator tochoose, Ylä-Kotola considers this kind of dialogism to be ethicallyjustified. Godard, as well as other counter cinema advocates, seeclassical Hollywood films as “narcotic”, like a “veil of fog” thatoften succeeds in immersing the spectator into the story, but reducesthe interaction to an automatic response. According to Bakhtin,

the essence of dialogic relation is integrally tied to the conceptof “otherness”. When a thought in a monologic world is eitherapproved or denied, in a true dialogic and conversational re204Ulla Oksanen

lationship it also becomes possible to understand other kinds ofviews.5 TOTAL MENTAL IMAGE AND DIALOGIC SEMIOSISIf an audiovisual experience is viewed as a multimedia interactivewholeness, worked out by a subject, a spectator, reader, listener,interpreter, writer, from fragmentary images, texts and sounds, itis, according to constructivist ideas, revived only after the synthesiswhich is produced by the subject´s earlier experience and action.In the dynamic interactive dialogue/polylogue of the textsand the mental structures of the subject, the experience of thesubject is referred to as a mental image. Astrid Söderbergh Widding,while actually engaged in studying the mental image inEisenstein´s cinematic ideas, comes near the fundamental questionsof the multimedia narration of today. She sees, like Eisenstein,that the basis for cinematic experience is formed by fragmentswhich have presented themselves to be modified by theconsciousness of the subject and which offer a possibility ofbuilding a total mental image on account of what is seen and experienced.In this action the user-subject (or spectator) constructsa total mental image or experience6 using his or her existing mental(cognitive) structures. The user-subject thus becomes an activepartner in the dynamic drama, where he or she answers the impulsesof the message and fundamentally gives birth in a dialogicrelationship to his or her own mental image, and his or her ownexperience. When the message in the constructivist process isviewed as if it extended outside its own subject matter, it can6 Söderbergh Widding does not define the concept of total mental image, but speculates atthe end of the article on the possibility of expanding the concept to mean cinematic experiencein totality. It is possible to assume that by total mental image she means implicitlycinematic experience.From Inner Speech to Dialogic Semiosis 205

never be considered final but instead indeterminate and open, alwayscapable of being converted into a new (total) mental image.(Söderbergh Widding 1997, 11, 14) A multimedia message, too, isalways the result of a construction process where the subject onboth the physical and the mental level builds a synthesis out offragments. This kind of “multimedial experience” is also a labileconstruction, which like a cinematic message is born and changedin a dialogic relationship.As said earlier, the concept of the mental image (viewed by

Eisenstein as dynamic and interactive), also enables one to see theconcept as dialogic. Gilles Deleuze described the relations createdand reflected by it as the concept of “la tiercité”, “the triad”, referringto the pragmatist semiotician C.S. Peirce (SöderberghWidding 1997, 11). According to Peirce: “Thinking always proceedsin the form of a dialogue …” and ”… it is essentially composedof signs, as its matter….” (Peirce 1931–1958, 4.6; cited inJohansen 1993, ix, 189). A sign is for Peirce by its nature a mediatingrelationship (see Figure 1: Peirce´s concept of sign), whichbrings together three things: representamen (the sign-vehicle),object (the object to which it refers) and interpretant (its influenceor the interpretation of it). For Peirce, the sign-vehicles, objectsand interpretants may as well be material as mental, as well athought as an action. (Veivo & Huttunen 1999, 41)So a sign for Peirce is a general and wide concept, “anythingthat is related to another thing” (Kuusamo 1990, 45): a picture,word, piece of music, symptom that represents something and canbe interpreted somehow.206 Ulla Oksanen

Figure 1 illustrates Peirce´s triadic conception of signs:Figure 1. Peirce´s Triadic Conception of Signs (Huttunen & Veivo 1999,41).In Peirce´s pragmatist sign model, the topic of particular interest,from the viewpoint of dialogism, is its processual nature, whichstresses the action of signs, as well as the contextuality of themodel, which emphasises the temporal and spatial connections.The ability to extend beyond the apices of the triangle is also anessential part of the dynamics of Peirce´s model. Any interpretationmay turn to a sign, which again can be interpreted (cf. SöderberghWidding´s mental image). The process is continuous, as isour whole culture, which, according to Octavio Paz, consists of anendless chain of interpretations. If we were to define one sign, itwould be necessary to refer to another sign that again would referto a third one, etc. (Tarasti 1990, 29) Meaning expands continuallyand is to be attained at the end points of these diverging paths.Söderbergh Widding (1997) seeks for such a dialogic modelthat would be capable of uniting audiovisual texts and multipleelements of spectatorship and letting them interact and bring newFrom Inner Speech to Dialogic Semiosis 207

kinds of information concerning the richness and complexity ofthe cinematic experience. Jörgen Dines Johansen, too, seems toaim at a similar goal as he has undertaken to develop consistentlyPeirce´s triadic model and complete it on the basis of Peirce´s ownwritings. This model will be presented in the following as an

uniting view of semiosis, in which the central theme of the article:the communicational dialogism is realised and illustrated. Moreover,this reconstruction, based on Peirce´s triad, offers to act as atentative visualisation of the dialogic semiosis which is active inany audiovisual narrative process.immediateobjectOBJECTDYNAMICALDYNAMICAL INTERPRETANTOBJECTFINALINTERPRETANTimmediateinterpretantUTTERERaddresserINTERPRETERaddresseeSIGNtokentype

Figure 2. Johansen´s Dialogic Model of Semiosis, (“The Semiotic Pyramid”)(Johansen 1993, 246).Johansen´s new model is based essentially on C.S. Peirce´s dynamicand expanding view of the apices of the triangle and RomanJacobson´s concepts of communication (see Figure 2: Johansen´s Dialogic Model of Semiosis, “The Semiotic Pyramid”) (Johansen1993, 246). The sign-vehicle (here: “sign”) presented inthis figure may dynamically appear as an occasional concrete “token”,but it may also adopt the nature of a “type”, a rule. Similarly,a sign may refer to an object applying two aspects: ”dy208Ulla Oksanen

namic” and ”immediate”. By a dynamic object Peirce means thereal factor or reason affecting the sign, which is not usually seenby the addressee of the sign. Concerning the dynamic object (e.g.the reality of which a photograph is taken) affecting ”behind” thesign, some kind of idea is, however, mediated in the sign processon the basis of the sign. This idea is called the immediate object(e.g. the reality as such that the photograph presents). (Veivo &Huttunen 1999, 43–44)Johansen´s view of Peirce´s interpretant is also dynamic sothat the immediate and dynamic objects also have their parallels:the immediate interpretant is the immediate and potential effect ofthe sign, whereas dynamic is its actual effect. In the former theintention of the addresser/utterer is strongly manifested, whereasthe latter is dominated by the intention of the addressee/interpreter.The final interpretant, where the interpretation in theory isended, includes the nucleus of Johansen´s interpretation process:the communicational interpretation, the dialogic act, where both

the parties addresser/utterer and addressee/interpreter “negotiate”,and where they can reach a mutual agreement. (Johansen 1993,169–174)The worth of the model lies basically in its ability to illustrateand in this way add to the development of Peirce´s pragmaticsemiotics and his view of semiosis. The three-dimensional pyramidalso visualises in the semiosis the dialogic or communicationalinterpretation implied in Peirce´s thinking, which was mademore widely known by his admirer Roman Jacobson, the developerof the classical communication model, in his own work (Johansen1993, 190). It is also possible to see Söderbergh Widding´saudiovisual experience and the total mental image illustrated inthis model that unites the three parts: interpreter/addressee (e.g.the receiving function of the user-subject), the utterer/addresser(e.g. the writing function of the user-subject) and Peirce´s triadic(Figure 1) idea of semiosis (e.g. the message).From Inner Speech to Dialogic Semiosis 209

Similarly, her idea of the continual change, openness andambiguousness of the interpretation and its tendency to expandoutside itself is visualised by the arrows at the triangle apiceswhich refer to their dynamism.6 CONCLUDING WORDSIf the future is viewed from the viewpoint of Greimas´s three dimensions:temporality, spatiality and actoriality that determine thebasic nature of audiovisual narration, it is possible to arrive at thefollowing ideas: According to Ylä-Kotola when the time andspace experience in the telematic society continues to becomeweaker as principles that determine observing and representing theworld, correspondingly unlinear ways to analyse data will becomestronger (Ylä-Kotola 1998, 321–322, italics added). In this articlein fact the same basic idea has been outlined by three permeatingtendencies or “threads”, which are the changes (i) from monologismto dialogism, (ii) from diachrony to synchrony and (iii) fromsyntagm to paradigm. The concepts have been illustrated in separateconnections with the verbal and audiovisual ideas of Vygotsky,Eisenstein, Bakhtin and Godard. These ideas can withoutexaggeration also be seen as profound displacements in narrativestructures and as kinds of bridges to a new type of associativeaudiovisual narration and polysemic multimedial expression.“Dialogue in its Bakhtinian and Vygotskyan sense is the undividedorigin of all knowledge, thought, and thus also learning”(Tella, Mononen-Aaltonen & Kynäslahti 1998, 38). It is also thecentral idea of the semiotic approach to audiovisual communication

in this article. Johansen´s model of dialogic semiosis or “thesemiotic pyramid” can be considered as such a pragmatic semioticview of how meanings come into existence and change in thedialogic interaction and communication process, even in the210 Ulla Oksanen

studying and learning process while working and reflecting withan educational multimedia program.7 In the same chapter thismodel is also presented as a dialogic total mental image, which thesubject constructs. In the frame of reference of media education,the presented model could tentatively be considered a junction ofthe communication and mediation flows that stream on the highestlevel of the multidimensional model, a semiosis where the signs ofthe background flows intersect and create new signs and newmeanings.7 REFERENCESAarseth, E. 1999. Kyberteksti – näkökulmia ergodiseen kirjallisuuteen (Cybertext:Views on Ergodic Literature). Parnasso 3, 264. (in Finnish)Bahtin, M. 1991. Dostojevskin poetiikan ongelma. (Problems of Dostoevsky´s poetics; translated by C. Emerson.) Helsinki: Orient Express.(in Finnish)Broms, H. 1984. Alkukuvien jäljillä: Kulttuurin semiotiikka. Porvoo: WSOY.(in Finnish)Eisenstein, S. 1978. Elokuvan muoto. Helsinki: Love. (in Finnish)Hietala, V. 1994. Tunteesta teesiin. Helsinki: Kirjastopalvelu. (in Finnish)Holquist, M. 1990. The Dialogic Imagination: Four Essays by M. M. Bakhtin.Austin: University of Texas Press.Jalander, Y. 1990. Eisensteinin elokuvateorioista. (On Eisenstein´s CinematicTheories.) Teoksessa Kinisjärvi, R., Lukkarila, M. & Malmberg, T.(toim.) Elokuvateorian historia. (History of Cinematic Theory.)Helsinki: Like, 58–84. (in Finnish)Johansen, J.D. 1993. Dialogic Semiosis. Bloomington & Indianapolis: IndianaUniversity Press.Kristeva, Julia. 1993. Sana, dialogi ja romaani. (Word, Dialogue, and Novel.)Teoksessa Puhuva subjekti. (The Speaking Subject.) Helsinki: Gaudeamus,21–50. (in Finnish)7 The model also pays attention to the new possible user functions in cybertextual communicationallowing the subject to adopt e.g. a writing function. (Aarseth 1999).From Inner Speech to Dialogic Semiosis 211Kuusamo, A. 1990. Kuvien edessä. Helsinki: Gaudeamus. (in Finnish)Lehtonen, M. 1998. Merkitysten maailma. Tampere: Vastapaino. (in Finnish)Lotman, J. Merkkien maailma. (The World of Signs.) 1989. Helsinki: SN-kirjat.(in Finnish)Malmberg, T. 1974. Vertov ja Eisenstein: Neuvostoelokuvan perinteidenajankohtaisuudesta. Helsinki: Suomen elokuvakerhojen liitto ry:njulkaisusarja 3. (in Finnish)Norvig, P. & Lakoff, G. 1987. Taking a Study in Lexical Network Theory. InProceedings of the Thirteenth Annual Meeting of the Berkeley LinguisticSociety. Berkeley California, 195–206. (Cited in Ylä-Kotola

1998)Peirce, C. S. 1931–1958. Collected Papers I–VIII. C. Hartshorne, P. Weiss &A. Burks (eds.). Cambridge MA: Harvard University PressSöderbergh Widding, A. 1997. Mentaalinen kuva – mietteitä elokuvaelämyksestä(The Mental Image—Visions of Cinematic Experience). Lähikuva2–3, 7–14. (in Finnish)Tarasti, E. 1990. Johdatusta semiotiikkaan. Helsinki: Gaudeamus. (in Finnish)Tella, S. & Mononen-Aaltonen, M. 1998. Developing Dialogic CommunicationCulture in Media Education: Integrating Dialogism and Technology.Media Education Centre. Department of Teacher Education. Universityof Helsinki. Media Education Publications 7.[http://www.helsinki.fi/~tella/mep7.html]Tella, S. & Mononen-Aaltonen, M. 2000. Towards Network-Based Education:A Multidimensional Model for Principles of Planning and Evaluation.(In this volume)Tella, S., Mononen-Aaltonen, M. & Kynäslahti, H. in collaboration withNummi, T., Passi, A., Ristola, R., Sariola, J., Vahtivuori, S. & Wager,P. 1998. Towards a Communal Curriculum: Strategic Planning and theEmerging Knowledge of Media Education. In Tella, S. (ed.) Aspects ofMedia Education: Strategic Imperatives in the Information Age. MediaEducation Centre. Department of Teacher Education. University ofHelsinki. Media Education Publications 8, 1–84.[http://www.helsinki.fi/~tella/mep8cc.html]Veivo, H. & Huttunen, T. 1999. Semiotiikka: merkeistä mieleen jakulttuuriin. Helsinki: Edita. (in Finnish)212 Ulla OksanenWillemen, P. 1983. Cinematic Discourse: The Problem of Inner Speech. InHeath, S. & Mellencamp, P. (eds.) Cinema and language. Los Angeles:The American Film Institute. (Cited in Hietala 1994, 49)Wertsch, J.V. 1980. The significance of dialogue in Vygotsky´s account ofsocial, egocentric, and inner speech. Contemporary Educational Psychology5, 150–162. (Cited in Tella, S. & Mononen-Aaltonen, M.1998)Wollen, P. 1977. Merkityksen ongelma elokuvassa. (Signs and meaning inthe Cinema.) Helsinki: Gaudeamus. (in Finnish)Vygotski, L. 1982. Ajattelu ja kieli. (Thought and Language.) Espoo:Weilin+Göös. (in Finnish)Ylä-Kotola, M. 1998. Jean-Luc Godard mediafilosofina: rekonstruktio simulaatiokulttuurinlähtökohdista. Mediatieteen julkaisuja B 1. Lapin yliopisto.Taiteiden tiedekunta. (in Finnish)Unprinted ReferenceYlä-Kotola, M. 2000. Uusmedian semiotiikka -luennot Helsingin yliopistossa13–14.4.2000. Semiotiikan opintosuunta. (in Finnish)