Examining Information Processing on the World Wide - School of

Examining Information Processing on theWorld Wide Web

Using Think Aloud Protocols

William P. Eveland, Jr.School of Journalism and Communication

The Ohio State University

Sharon DunwoodySchool of Journalism and Mass Communication

University of Wisconsin–Madison

Some theorists argue that the node-link design of the Web mimics humaninformation storage and that Web use encourages individuals to processinformation efficiently and effectively, potentially increasing meaningfullearning. However, critics claim that Web navigation increases cognitive loadand often produces disorientation. This reduces the processing devoted tomeaningful learning, and, thus the Web may potentially inhibit learning. In anexamination of information processing on the Web using a quantitativeanalysis of think aloud protocols, we found that users spend a substantialproportion of their cognitive effort orienting to the content and structure of theWeb, and this effort comes at the expense of elaborative and evaluativeprocessing. Additional findings suggest that, at least during a single relativelyshort session, time spent in a given site does not reduce the processing devotedto orientation. Finally, this paper offers a theoretically informed strategy foranalyzing information processing activities that may be of use to otherscholars.

Requests for reprints should be sent to William P. Eveland, Jr., School of Journalismand Communication, The Ohio State University, 154 North Oval Mall, Room 3016,Columbus, OH 43210. E-mail: [email protected]

MEDIAPSYCHOLOGY, 2, 219–244.Copyright © 2000, Lawrence Erlbaum Associates, Inc.

The purpose of this article is to delineate how individuals process informationpresented to them via the World Wide Web (or “Web”). Some theorists havesuggested that the design of hypermedia systems, such as the Web, can facilitateuseful information processing activities that lead to learning. Others have arguedthat some factors, such as disorientation, can hinder effective informationprocessing of Web content. An important area of inquiry, then, is to betterunderstand the information processing that takes place when individuals browsethe Web.

Unfortunately, few studies have attempted to empirically determine theprevalence of different types of information processing using observationalmethods. Instead, most studies either experimentally manipulate informationprocessing to determine its effects (e.g., Craik & Tulving, 1975; Hamilton, 1997;Johnsey, Morrison, & Ross, 1992; Mayer, 1980; Pressley, McDaniel, Turnure,Wood, & Ahmad, 1987) or attempt to measure information processing via selfreports (e.g., Eveland, 1997a, 1997b; Kardash & Amlund, 1991; Perse, 1990;Salomon, 1981, 1983; Schmeck, Ribich, & Ramanaiah, 1977; We i n s t e i n ,Zimmermann, & Palmer, 1988). Although these are certainly useful approaches,they do leave out direct assessment of naturally occurring variation ininformation processing over time or across content.

This article presents an exploratory and descriptive study of informationprocessing of Web content and structure. The study uses a think aloudmethodology to provide relatively direct observation of patterns of informationprocessing compared to experimental manipulations or self-reports. We concludefrom this study that a majority of processing while using the Web is focused onmaintaining orientation to the structure and content of the site, thus reducingother forms of information processing that have been demonstrated to producemeaningful learning.

INFORMATION PROCESSING IN THE CONTEXTOF HYPERMEDIAAND THE WORLD WIDE WEB

More than 50 years ago, Vannevar Bush (1945) proposed the creation of amachine called a memex that would allow instantaneous access to multiplesources of information through associational links. He believed that this machinewould increase learning because it would function in the same way thatindividuals’ brains worked—as an associative network. In Bush’s vision, theinformation would be stored on microfilm and presented on multiple viewersmounted into a large desk. The technology underlying this idea was laterupdated, and the resulting product was labeled hypertext (now hypermedia) byTed Nelson (Bevilacqua, 1989; Heller, 1990; Nelson, 1993; Tsai, 1988–1989).

2 2 0 EVELAND & DUNWOODY

The defining feature of hypermedia is the use of nodes (packets ofinformation, typically in the form of a “page”) connected by links that may beeasily traversed at the whim of the user (Horney, 1991; Shirk, 1992). As such,hypermedia is distinguished from other media, such as television and radio, by ahigh level of user control over the pace, order, and content. This control allowsuse of this medium to be nonlinear or nonsequential (Duchastel, 1990; Horney,1993; Shin, Schallert, & Savenye, 1994), although individuals may still chooseto use the medium in a linear or sequential manner (Eveland & Dunwoody, 1998,in press).

Nearly five decades after Bush’s (1945) classic article, the idea of thememex—in the form of hypermedia—took the United States by storm in theguise of the Web (The Internet, 1997). The Web is, technologically, a massivehypermedia system (Astleitner & Leutner, 1995) created by thousands ofdifferent authors across the globe. Recent statistics on the popularity of theInternet—of which the Web is a major component—reveal the massive growthin this medium over the past few years. A poll conducted in the fall of 1998 (PewResearch Center, undated) found that more than 40% of American adults used theInternet, with nearly half of those beginning during the prior year. A series ofstudies conducted by Bimber (1999) found that Internet access among Americanadults increased from 26% in October 1996 to 46% in February 1998 and 55%in March 1999. Although exact figures depend in part on how Internet use isdefined, it is clear that a substantial proportion of the U.S. population is makinguse of the Internet today, and that use has been increasing rapidly over the pastseveral years.

PROMISES AND PROBLEMS OF USING HYPERMEDIA FOR LEARNING

Theorists interested in the uses and effects of hypermedia frequently argue thatthe structure of hypermedia and the process of its use mimics the associativestructure of human memory and the function of human information processing(e.g., Bieber, Vitali, Ashman, Balasubramanian, & Oinas-Kukkonen, 1997;Churcher, 1989; Jonassen & Wang, 1993; Kozma, 1987; Lucarella & Zanzi,1993; Marchionini, 1988; Shin et al., 1994; Shirk, 1992). For instance, Jonassen(1988) noted that “because hypertext is a node-link system based upon semanticstructures, it can map fairly directly the structure of knowledge it is representing”(p. 14). Tergan summarizes this perspective by noting the following:

I N F O R M ATION PROCESSING ON THE W E B 2 2 1


Some researchers have argued that structural and functional features ofhypertext/hypermedia technology match very well with cognitive networktheories of the human mind, constructivist principles of learning, and multiplemental modes for representation of knowledge. The suggested match hasnourished expectations that hypertext-based technologies may overcomedeficiencies inherent in the traditional reading comprehension and informationprocessing approach of teaching and learning and may even revolutionizelearning. (Tergan, 1997, pp. 257–258)

Thus, advocates hypothesize that hypermedia systems can serve as superiorlearning tools compared to other, more constrained and linear media that do notrepresent a knowledge domain so precisely. Churcher (1989) argued that “wherehypertext is highly structured and indeed is the structure of the domain ofknowledge and that structure/system is to eventually become the users’conceptual model it strongly suggests hypertext as a more effective learningenvironment” (p. 245). Thus, the argument made by many hypermedia advocatesis that, because hypermedia can be designed to emulate the appropriate (based ondomain experts) links among concepts in a particular knowledge domain,learners will more easily be able to build their own mental models from themodel used in the hypermedia system (e.g., Churcher, 1989; Jonassen, 1988;Jonassen & Wang, 1993). In effect, in most theoretical approaches the user isassumed to employ the hypermedia system to shape his or her own mentalrepresentations of the domain of knowledge—both in terms of content andstructure—thereby emulating the knowledge structure of the domain expertwhose input influenced the design of the hypermedia system itself.

However, some argue that there are important differences between thestructure and use of hypermedia systems and those of human memory. Nelsonand Palumbo (1992) argued that

at present, most hypermedia systems support linkages indicating only that oneunit of information is somehow related to another unit of information, withoutspecifying the nature of this relationship and a rationale for its existence. . . . Incontrast, human memory supports a much stronger linking mechanism that bothestablishes a relationship and conveys information about the associational natureof the link. (p. 290)

In addition, Tergan (1997) criticized the assumption that hypermedia use isanalogous to human information processing and thus raises questions about thesuperiority of hypermedia as a learning tool. Despite these and other criticismsof the conceptual ties between human memory and hypermedia, manyhypermedia researchers who take a stance on the issue seem to agree that the

similarities between the two are many and theoretically important. If accurate,this would suggest that hypermedia may facilitate information processing,particularly if the design of the hypermedia system is structured in a meaningfulway.

Although many hypermedia theorists focus on the benefits of usinghypermedia for information processing and learning, there are those who seeanother, darker side of hypermedia use. One of the most common concerns abouthypermedia use expressed by these individuals is its propensity to causedisorientation (McDonald & Stevenson, 1996). Disorientation is likely to reducelearning and, potentially, even lead users to abandon use of the system altogether.

From this perspective, the relevant metaphor for hypermedia use is nothuman processing of information but navigation through unfamiliar physicalspace (Kim & Hirtle, 1995). Based on this metaphor and formal observations(e.g., Dias & Sousa, 1997), as well as informal reports of users, this perspectivepoints out that people often get confused and even lost in virtual spaces withwhich they are unfamiliar. This is particularly true when these spaces are poorlydesigned.

To avoid getting lost, people must engage in orienting techniques, such asidentifying landmarks and exploring the relationship of one location to another.McDonald and Stevenson (1998) argued that nonlinear hypermedia systemsproduce “a high cognitive burden on users such that they must simultaneouslyfocus on their information retrieval goals and on orienting themselves within thehypertextual space” (p. 24). Under the assumption of a limited cognitive capacity(e.g., Kahneman, 1973), the effort spent orienting oneself to the informationspace—sometimes called cognitive overhead (Conklin, 1987; T h ü r i n g ,Hannemann, & Haake, 1995)—consumes some or all of the cognitive effort thatmight otherwise be invested in more meaningful processing of the content. Thus,the focus of information processing on efforts to orient oneself suggests that,even if the user never actually becomes disoriented, the cognitive overheadproduced by hypermedia may potentially inhibit, instead of encourage, otherinformation processing activities that lead to fruitful learning.

One means of addressing this debate would be to compare the relative amountof learning from hypermedia systems with more traditional media, such as print.A number of researchers have followed this route, with findings that arepotentially important but still somewhat ambiguous (Chen & Rada, 1996; Dillon& Gabbard, 1998; Eveland & Dunwoody, 2000). This study takes an alternateapproach by examining not the product of information processing—learning—but the processing of information itself. This allows us to elaborate on thefindings of learning experiments by describing the processing of information that



may have produced the results of those learning experiments. Thus, this studywill examine the relative proportions of information processing devoted toorienting to the information space compared to other forms that may be moreconducive to meaningful learning.

INFORMATION PROCESSING ON THE WEB

We focus on what we believe to be four basic, distinct, and meaningful categoriesof information processing that would occur after attention to content had alreadybeen established: maintenance, orientation, elaboration, and evaluation. These areall forms of processing information that one would expect to find in most formsof media use and in everyday activity. Orientation is likely to be particularlyprevalent for those using hypermedia systems such as the Web compared totraditional media, although this study was not designed to test this expectation.

Maintenance

Simply put, maintenance is the repetition of information in short-term memory.The quintessential example of maintenance is the mental rehearsal of a phonenumber or name over and over in an attempt to remember it. An importantcharacteristic of maintenance is that it does not include any attempts to connectthe information to existing knowledge or to interpret it in light of otherinformation. Estes (1988) suggested that “maintenance of items in active workingmemory simply by what is termed primary or maintenance rehearsal . . . serves toincrease the probability of later recognition but within wide limits has nodetectable effect on later recall” (p. 356). Others have drawn similar conclusionsabout the relatively weak effect of simple maintenance on recall and learning(e.g., Craik & Tulving, 1975; Haberlandt, 1994). Thus, for purposes of this studymaintenance is not considered an effective form of information processing forlearning, although it should be noted that it may have some limited positivee ff e c t s .

Orientation

Orientation is of particular concern for those interested in the use of hypermediasystems. Kim and Hirtle (1995, p. 241) argued that

while browsing a hypertext database, the user must carry out multiple tasksconcurrently. These tasks can be clarified into three categories: (1) navigationaltasks: planning and executing routes through the network; (2) informational tasks:

reading and understanding contents presented in the nodes and their relationships,for summary and analysis; and (3) task management: coordinating informationand navigational tasks (e.g., keeping track of digressions to incidental topics).Performance of these tasks exacts a high cognitive load upon the user.

It is the first and third of these cognitive activities that we consider orientation inthis study. Orientation, although potentially useful for learning the overallstructure of information (and thus valuable only if the information is structuredin a meaningful manner), also robs precious cognitive resources from otherinformation-processing activities that may be more valuable for learning. Hilland Hannafin (1997) claimed that “significant disorientation may hinder theu s e r’s ability to reference relevant prior subject knowledge as well asmetacognitive knowledge” (p. 58). If true, this would limit the amount ofmeaningful learning that takes place. Indeed, they noted that “it may be criticalto reduce perceived (or real) system discomfort and disorientation prior toadvancing open learning applications” (Hill & Hannafin, 1997, p. 61).

Elaboration

Perse (1990) stated that elaboration of media content “relates the incominginformation to existing knowledge and images and attaches connotative andassociative meanings” (p. 19). In effect, elaboration is the process through whichconnections are made between new and existing bits of information in memoryor between two or more existing bits of information (Hamilton, 1997).Elaboration serves to connect new information into existing schema as well as tocreate greater interconnectedness within schema. Both of these processes areintegral to learning, and are consistent with the purported benefits of hypermediafor learning.

Experimental research in cognitive and educational psychology hasconsistently upheld the connection between elaboration and greater learning fromstimulus materials (e.g., Hamilton, 1989; Mayer, 1980; Pressley et al., 1987;Woloshyn, Paivio, & Pressley, 1994; Woloshyn, Wi l l o u g h b y, Wood, & Pressley,1990). In their reviews of the literature, Estes (1988), Greene (1992), andHaberlandt (1994) concluded that recall is substantially greater when participantsengage in elaborative rehearsal than when they engage in simple maintenancerehearsal. In addition, there is ample evidence for a strong relationship betweensurvey measures of elaboration/deep processing and knowledge of specific topicsor academic achievement (e.g., Eveland, 1997a, 1997b; Kardash & A m l u n d ,1991; Perse, 1990; Schmeck, 1980; Schmeck & Grove, 1979; Schmeck &Phillips, 1982; Schmeck et al., 1977; Watkins & Hattie, 1981a, 1981b).


Evaluation

The final type of information processing we will consider is evaluation—assessing the value or worth of a given object or piece of information. On theWeb, even more so than in traditional informational media, assessments of thecredibility of the source and the accuracy of individual bits of information is animportant skill. At any moment the specific source of information, such as thesponsor of the Web site, may change, and each of these changes require a newassessment of credibility.

Some have suggested that evaluation is merely an extension or a subset ofelaboration (e.g., Gould, Trevithick, & Dixon, 1991), in part because nearly allevaluations require making connections to existing information like standards orexemplars. However, we argue that evaluation adds an affective judgment to anyelaboration—that is, good or bad, true or false—that is not an essential feature ofelaboration more generally. In effect, then, evaluations are elaborations thatinclude an affective tag and should therefore contribute to learning.

METHODS

Think Aloud Interviewing

The think aloud method has been most prominently advocated by Ericsson andSimon (1993). This method requires participants to engage in some task andexpress the thoughts going through their minds as they do so. It is a nondirectivetechnique, such that the only probe used after initial instructions is whenparticipants stop verbalizing for some time, at which point they are simplyreminded to think aloud. Given the large quantity of data obtained from eachindividual, think aloud interviews are normally conducted with small samples ofbetween 10 and 30 participants (see Calvi, 1997; Carmel, Crawford, & Chen,1992; Crampton, 1992; Darken & Sibert, 1996; Hill & Hannafin, 1997). T h eparticipant pools for think aloud interviews are typically students and are rarelydrawn from the general population. The products of think aloud interviews areoften coded quantitatively (Carmel et al., 1992), as we do in this study, althoughsome researchers analyze them qualitatively instead (e.g., Hill & Hannifin, 1997).

The purpose of the think aloud method is to make observable at least someproportion of the information processing that takes place during a given task.Researchers assume that the source of the think aloud output is informationcurrently in short-term memory. By quantitatively coding the think aloudprotocols, researchers should be able to develop a better understanding of


cognitive processes. Like most other nontraditional methods, the use of thinkaloud protocols has gone through a stage of attack by critics and defense byproponents (e.g., Ericsson & Simon, 1993; Kellogg, 1982; Nisbett & Wilson,1977; Russo, Johnson, & Stephens, 1989; Smith & Miller, 1978; Turner, 1988;Wright & Rip, 1981). Responses to the critics generally have been persuasive, asthe use of the think aloud protocols is accepted practice in fields such aseducational psychology, geography, computer science, and engineering (e.g.,Calvi, 1997; Carmel et al., 1992; Crampton, 1992; Darken & Sibert, 1996; Hill& Hannafin, 1997).

The Why Files

The Why Files Web site (http://whyfiles.news.wisc.edu), created by the NationalInstitute for Science Education and initially funded by the National ScienceFoundation, was designed to convey the “science behind the news.” This site hasalso served as a test-bed for research on the communication of scientificinformation to the general public. Our think aloud interviews, althoughconcerned with the processing of information on the Web generally, were alsodesigned to help us evaluate the processing of scientific information in The WhyFiles in particular. Therefore, we began all think aloud participants on the homepage of this site. The implications of this decision are described in later sections.

Participants

In the spring and early summer of 1997 a sample of Dane County, WI, residentswere contacted via telephone for a screening interview.1 The first question in theinterview asked respondents if they had used the World Wide Web in the pastmonth; those who did not were thanked for their time, and the interview wasdiscontinued.2 For those who had used the Web in the past month, several otherquestions were asked regarding the following: personal interest in four differenttypes of scientific information, each measured on a 1–10 scale, and whether theyhad used the Web more than five times versus five times or less in the past 30days.3 The gender of each respondent was also identified. If the sum of the fourscience interest questions was 20 or greater, the respondent was asked toparticipate in the think aloud interview. Then, to ensure representation acrosspotentially important correlates of information processing in Web sites, and thusmore generalizability of our findings, we selected equal numbers of high and lowWeb users distributed evenly between males and females.4 This left us with fourhigh-Web-use males, four high-Web-use females, four low-Web-use males, and


four low-Web-use females as participants in our think aloud interviews. At theconclusion of the session, each participant was paid $50.

Procedures

Each participant was run individually in a session that lasted approximately 90minutes. First, participants engaged in several practice tasks to familiarizethemselves with the process of thinking aloud (Ericsson & Simon, 1993).Specifically, they were asked to think aloud while engaging in more and morecomplex tasks: mental addition of two 3-digit numbers, solving anagrams, andreading a brief article from a print magazine. The final practice task—lastingfrom five to fifteen minutes—was to surf a science-related World Wide Web site(“The Exploratorium”—http://www.exploratorium.com) in order to make theparticipant comfortable with our computer setup and with the process ofexpressing thoughts while engaging in a task very similar to the primary thinkaloud task.

The primary task for the think aloud interview was to surf the Web using aMacintosh computer, either Internet Explorer or Netscape Web browser software(depending on the participant’s preference), a 14” color monitor, and either a14.4 modem or a direct Ethernet connection (depending on the participant’stypical connection speed when using the Web). The task initially placedparticipants on the home page of The Why Files Web site, but participants wereinformed that they were free to navigate from there to anywhere on the Web. Thetask lasted about 30 minutes for most participants.

An audiotape recording was made of the complete interview, beginning withthe first practice task. We also produced a synchronized, picture-in-picture videorecording of the following: (a) the facial expression of the individual during thepractice and formal Web think aloud tasks; and (b) the images on the computerscreen during this time using a direct feed from the computer.A transcript of theaudio portion of the interview was used for unitization and categorization tasks.Due to a technical problem, there was no video information available for one ofthe participants (a low-Web-use male); thus, the final number of interviewsanalyzed was reduced to 15.

Operationalizations and Intercoder Reliability

Intercoder reliability was assessed by having two trained coders independentlycode the practice Web site think aloud protocols.


Unitization of Thoughts. The raw transcripts from the think aloudinterviews were segmented into “thought” units. Typically, a thought may beoperationalized as a sentence, a clause of a sentence, or a phrase. Sometimes asingle word sentence may represent a thought. Nonessential verbiage such as“um” or “OK” was ignored; an exception was when “OK” was clearly anindication of “I understand” and not just unnecessary verbiage. Explicit notationsof “pause” and “long pause” inserted by transcriptionists unfamiliar with thegoals of the project were one means of determining dividing points betweenthought units. Indications by transcriptionists of “reading to self” or theoccasional phrase or sentence in quotation marks (meant to indicate readingaloud) were considered individual thoughts. Coders made use of contextualinformation such as the video images and voice inflection of participants (fromthe picture-in-picture videotape) to determine when a participant was merelyreading the content of the site. Any reading of the content (verbatim or slightlyrephrased) not broken by a pause or nonreading verbalizations was considered asingle thought. Two coders independently unitized all think aloud commentsfrom the practice Web site task (Krippendorff’s α = .86).

Categorization. Three categorical variables, plus one ratio-level variable,were coded for each thought unit. The three categorical variables wereinformation processing, source, and domain. The final variable coded was thenumber of words used for the thought.

Information Processing. This refers to the qualitative type of cognitiveprocess suggested by the thought. The main categories of this variable weremaintenance, orientation, elaboration, and evaluation. Maintenance wasoperationalized as either verbatim reading aloud, description of content, orrestating with slight rewording. In order to determine if content was being read,coders used information from the transcript, visuals from the videotape, and thetone and inflection of the participant’s voice. All verbatim reading that was notprefaced by an intention within the same thought unit (e.g., “Let’s try …”) wasconsidered maintenance.

Orientation was defined as expressed attempts to understand the content andstructure of the information space, often taking the form of a rhetorical questionor a prediction about content or structure. Orientation was also indicated by astatement of intention to navigate or seek particular information. When verbatimreading was prefaced (within the same thought unit) by “Let’s go. . .” or “I wantto see. . .” or some other similar phrase, this was considered orientation.Orientation codes were categorized as (a) neutral, as in statement of intentions


and some rhetorical questions; (b) indicative of misunderstanding ordisorientation (the participant didn’t understand what he or she had done wrongor wasn’t sure how to proceed); or (c) indicative of understanding or epiphany(when the participant figured out what he or she did wrong). When disorientationor epiphany was not obvious, the neutral code was used as the default.

Elaboration was defined as the integration or connection of currentinformation with content viewed elsewhere during the task, personal experience,or background knowledge. It was also represented by the participant answeringa (rhetorical or literal) question posed by the Web content. Reference to one’spersonal interests was also considered elaboration.

Evaluation was defined as an affective response or judgment and was codedas either negative or positive. Expressions of interest in the content or structurewere also forms of evaluation. In the case where the evaluation was a judgmentabout the accuracy or credibility of information in the site, a perception ofaccuracy or credibility was considered positive, whereas a perception ofinaccuracy or a lack of credibility was considered negative.

In the extremely rare situation in which a given unit could fall under morethan one category of the information processing variable, a hierarchy of codingwas used, such that orientation was always coded if present, evaluation wascoded unless orientation was present, elaboration was coded unless evaluation ororientation was present. Maintenance was never coded if any other code waspresent. Intercoder reliability for the information processing variable was .77 asmeasured by Cohen’s κ, .76 as measured by Krippendorff’s α, and .88 asmeasured by Perreault and Leigh’s I.5

The source variable represents whether the thought appears to have beengenerated by (a) The Why Files; (b) some other Web site; (c) the Web moregenerally; (d) the browser software; or (e) the computer hardware (computer,monitor, modem). If a referent was not explicit, the coder had to infer from thep a r t i c i p a n t ’s current location based on the videotape or other contextualinformation. Intercoder reliability for this variable was .64 as measured byCohen’s κ, .64 as measured by Krippendorff’s α, and .98 as measured byPerreault and Leigh’s I.

The domain of a thought refers to its focus, either content or structure. Acontent reference is defined as a reference to information, whethercommunicated via text or graphics. Reading of text was considered a contentreference. A content reference was the default code; that is, if a structurereference could not be justified, the thought was coded as a content reference.

A structure reference was defined as a reference to format or organization,whether it pertained to text, graphics, or higher order concepts, such as


navigation or links between Web pages. So, comments regarding color, size,italics, and so forth were considered structure references. When thoughts referredto the speed of loading a page or some other facet of navigation or linking, thiswas coded as a structure reference. Reading of a link title alone was considereda structure reference. However, when both text and link information were readaloud, the determination was based on whether the link that was read was thenselected for navigation. If it was, it was coded as a structure reference; otherwise,it was coded as a content reference. Intercoder reliability for the domain ofthoughts was .72 as measured by Cohen’s κ, .71 as measured by Krippendorff’sα, and .85 as measured by Perreault and Leigh’s I.

Finally, the number of words was counted for each thought unit. Intercoderreliability was very high at .99 (as measured by Krippendorff’s α).

RESULTS

Overall there were 2,790 thoughts across all participants, with individuals rangingfrom 65 to 393 thoughts each. As shown in Figure 1, our results indicate thatslightly more than 23% of thoughts were maintenance (individuals ranging from14.8% to 36% of all thoughts). These thoughts included participants literallyrepeating the information on the screen (both audibly and in barely comprehensiblemumbles) as well as stating the information in a slightly revised fashion.

Figure 1. Distribution of Information Processing Types (N = 2790)

The plurality of thoughts were of an orienting nature (39%). The number oforientation thoughts that could be classified as either disorientation (7% overall,


Positive Evaluation7%

Negative Evaluation6%

Maintenance23%

Disorientation7%

Neutral Orientation26%

Epiphany Orientation6%

Elaboration25%

individuals ranging from 0% to 21.5% of all thoughts) or understanding (6%overall, individuals ranging from 0% to 13.8% of all thoughts) wereapproximately evenly split, with the remainder classified as neutral (26% overall,individuals ranging from 11.7% to 36.4% of all thoughts). Disorientation wasindicated by comments such as, “Oops, that’s not what I meant to do,” “I haveno idea how to get the information that I want,” and “Why did that take meback?” Understanding (or epiphany) was indicated by participants when theyapparently solved a problem and thought “That’s better,” or “That should bringthe image up.” Neutral orientation thoughts were expressed by one participantafter he had elaborated about a volcano he knew about; he said, “I want to findout about the Mexican, Mexico City volcano.” Another indicated neutralorientation when he thought “Let’s see what else is in these files,” and “Let’s goback to the file and see what else is here.”

About a quarter of the thoughts expressed by the participants were elaborative(individuals ranging from 8.9% to 62.5% of all thoughts). For example, oneparticipant, reading about potential causes of asthma, noted both that “I recentlywas in Saudi Arabia and met a woman who’s concerned about asthma in childrenfrom air pollution there” and, regarding dust mites as causes of asthma, “Peoplesay they get in your beds and then you can’t get rid of them.” Another participant,while reading about mad cow disease, recalled, “Someone told me that Oprahhad a show on this, and the farmers were feeding the other cows parts of infecteddead cows.” Information about the impact of tornadoes prompted one participantto remark, “I remember that one in 1977. Took down all those trees. I don’t thinkthey really called it a tornado, though.” All of these participants were connectinginformation from past experience or prior knowledge to new information on theWeb and thus they were in engaging in elaborative processing.

Relatively few thoughts were evaluative (13%), and these were dividedevenly between negative evaluations (6% overall, individuals ranging from 0%to 12.9% of all thoughts) and positive evaluations (7% overall, individualsranging from 1.1% to 15.4% of all thoughts). One participant cycled betweencriticism of site design characteristics (e.g., “that’s boring,” “really f---ingannoying,” “that’s stupid”) and compliments for the level of interest the sitegenerated (e.g., “cool,” “interesting”). In addition to issues of design, there werealso thoughts about the veracity—or perceived lack thereof—of the informationprovided. One participant challenged the definition of a Latin word. Another,reading about the uses of solar energy to power airplanes, argued, “I’ve alwaysassumed that you couldn’t have a solar airplane, unless you made hydrogen gasfirst and burned that.” Later, when this participant had linked to a government-funded site on alternative energy, he assessed its credibility, noting, “I’m


suspicious this is all propaganda instead of anything worthwhile. If it’s put outby the government.”

Most of the thoughts generated by the think aloud procedure referred to thecontent of the sites (78%, individuals ranging from 55.3% to 95.5% of allthoughts) instead of their structure. Similarly, the source of the vast majority ofthoughts was either The Why Files site (79%, individuals ranging from 25.9% to99.5% of all thoughts) or some other site (18.5%, individuals ranging from 0%to 74.1% of all thoughts). Only about 2.5% of comments pertained to the Webgenerally, the browser software, or the computer hardware.

Correlates of Information Processing

Our exploratory correlational analyses were able to examine primarily contextualfactors that may be related to information processing. One possibility is that thedistribution of the information processing variable would change over time.First, as individuals became more comfortable with the process of browsing theWeb during the course of the think aloud session, there would be less need todevote effort to orienting, and thus more effort could be directed at elaboration.In order to test this possibility, correlations between two dichotomousinformation-processing variables (orientation and elaboration) and the order ofthoughts (numbered 1 through n during the session) were computed. Becausethere was variation in the total number of thoughts across participants,correlations were computed within participants. Although there were significantcorrelations suggesting both increased and decreased elaboration and orientationover time, most relationships were small and nonsignificant, and no meaningfulpatterns emerged (orientation: mean r = .01, range of –.14 to .21; elaboration:mean r = .03, range of –.19 to .32).

One explanation for the lack of a meaningful pattern in these correlations isthat some individuals moved outside of the initial site (The Why Files) later intheir sessions. This would potentially increase their need to orient and decreasetheir ability to elaborate at the same time that orientation would have decreasedand elaboration would have increased had they remained in a single site.Therefore, correlations were recomputed for only those thoughts related to TheWhy Files under the assumption that later thoughts within a single site would bemore elaborative and less orienting than earlier thoughts in the same site. Again,most correlations were small and nonsignificant, and there were no clear patternsamong the significant correlations (orientation: mean r = .03, range of –.14 to.27; elaboration: mean r = .06, range of –.13 to .51).

Our results can be summarized in one central and one related finding. First,our participants spent a substantial amount of time trying to orient themselves to


the content and structure of the Web sites they explored. Web critics note thatsuch a heavy expenditure of effort on orientation may be a common byproductof Web use and, given limited cognitive capacities, increased effort devoted toorientation cannibalizes effort that could be devoted to more meaningful formsof information processing for learning. This is consistent with the negativerelationship (measured at the individual level using nonparametric correlations)between the proportion of thoughts devoted to orientation and the proportion ofthoughts devoted to elaboration (τb= –.49, p < .05) and evaluation (τb = –.40, p< .05).6 Our second and related finding is that there was little evidence of over-time variation in the proportion of thoughts devoted to orientation.7 Our analysesindicated that orientation was no more or less likely early in the think aloudsession than it was at the end, and this pattern held even when examined withina single site.

DISCUSSION

This study sought to quantify the processing of science information on the Webusing the think aloud method. A large literature indicates that whether or notlearning takes place—from media or other sources—is largely based on how theinformation is processed. Theorists have argued that hypermedia systems like theWeb encourage individuals to process information in a more efficient andeffective manner because of the freedom they allow users to achieve fit with theirown mental models, and because the information can be structured in such a wayas to make its presentation more closely resemble the true form of the knowledgedomain. However, critics have pointed out that user navigation of hypermediasystems exacts a large cost in cognitive load and often produces disorientation.This suggests that the Web is, at least currently, not the panacea for learning thatsome would wish it to be. Our data are more consistent with the latterinterpretation than the former.

This study was predicated on the assumption—supported in the literature—that certain types of information processing are more useful for learning thanothers. We developed a coding scheme to tap into the processing of informationon the Web through the use of think aloud interviews and sought the prevalenceof four types of information processing: maintenance, orientation, elaboration,and evaluation.

One advantage of this study was the development of a coding scheme forinformation processing activities during media use that can be reliablyimplemented. Also, because this coding scheme is founded in theory andresearch instead of being created inductively, it makes findings more easily and


directly applicable to theoretical questions about information processing andlearning from media and thus more appropriate for researchers from a number offields.

Future research could profitably apply this coding scheme to other media,specifically by comparing processing of information presented on the Web withsimilar (or identical) information presented by paper (in the form of newspapersor magazines) or using some other linear medium like narrative television. It issuch a study that will ultimately be able to answer the question of whether or notindividuals process information on the Web differently than they do using moretraditional media of communication. However, potential differences in howpeople perform in think aloud tasks across media—that is, is it simply easier tothink aloud while using the Web or television than traditional print media?—willbe important to answer for this type of comparative research.

The coding scheme designed for and used in this study may also be used toexperimentally determine the impact of content and design features oninformation processing and learning. This could help answer such questions as,might different linking strategies and site organization, different design cues likein-text links versus icons, or the use of maps or overviews, influence how theinformation in the site is processed? Might different writing styles—such as thetraditional inverted pyramid from newspapers, more narrative forms often usedin magazines, or a hierarchical format—encourage or discourage different typesof information processing?

In addition to its strengths, this study has a number of limitations. First, thesmall samples necessary for think aloud studies make generalizing difficult. Infact, our sample of 15 was unlike the typical repeat user of The Why Files, thesite in which all participants began their browsing. Specifically, the think aloudstudy contained a higher proportion of females and lower proportions ofexperienced Web users and individuals strongly interested in science topics. Wemight, then, expect to find more confusion and disorientation among our thinkaloud respondents than would be typical of repeat users of The Why Files,because all of the factors that would contribute to this outcome aredisproportionately found in the think aloud participants. However, we shouldnote that our participants are actually much more representative of the generalpopulation than those in most studies of new technologies and most studies usingthink aloud methods, which often rely on convenience samples of students. Oureffort to sample from a general population made our participant pool less likecurrent users of The Why Files, but more like the real-world and potential usersof the World Wide Web.

Another limitation of this study was the instruction to begin in a science-related site and engage in browsing behavior. Although our participants did


divert themselves to nonscientific information at times, it may be that thedistribution of thoughts across the categories would have been different had weexamined different Web content, such as entertainment sites or current eventsnews sites. In addition, having participants engage in browsing behavior is likelyto have produced different cognitions than if we had asked them to achieve amore specific goals because past research has revealed different navigationpatterns based on the goal of the user (for a review, see Eveland & Dunwoody,in press). Future research should attempt to broaden the type of content examinedto determine how information processing of media content may differ acrossgenres and also examine processing when different motivations (i.e., search vs.browse, information vs. entertainment) are activated in users.

In addition, over the long term the findings of this study—specifically thosepertaining to orientation—are likely to change. Although we did not finddifferences in orientation over the progression of our study—about 30 minutes—over longer periods of time it is likely that the need to engage in orientingbehaviors may decrease. This provides a wonderful opportunity to beginlongitudinal research—using either a panel or repeated cross-section design—tostudy this process over time. In the late 1990s, it was likely that disorientationand a focus on orienting cognitions more generally was a function of at leastthree factors: (1) socialization: most people, including all of our participants,have been socialized to use linear media such as books and television andtherefore are not trained in the use of potentially nonlinear media such as theWeb, which can be more confusing because the Web requires almost constantdecision making; (2) expertise: many people, including some of our participants,are not experienced with the Web in particular and thus need to orient themselvesto this new medium; and finally (3) design issues: there is great variation in thequality of Web site design and organization, and both the lack of quality designand variations in organization of information across sites can increase the needfor orientation cognitions.

Over time, however, changes in all three of these factors may reduce users’focus on orientation. As with socialization to television, there will soon come atime when children grow up with nonlinear media such as the Web, and thosechildren, when they become adults, should be less confused by nonlinearity thanare adults today. Similarly, as time passes and more people gain access to andmake use of the Web on a regular basis, levels of expertise will increase to a highand uniform level, potentially reducing much of the existing cognitivepreoccupation with orientation. Finally, again as with television, as thosedeveloping Web sites gain experience, the quality of Web site design shouldincrease and become more uniform, further reducing the disorientationexperienced by users at this time.


AUTHOR NOTE

The research reported in this paper was supported by a cooperative agreementbetween the National Science Foundation and the University of Wisconsin-Madison (Cooperative Agreement No. RED-9452971). At UW-Madison, theNational Institute for Science Education is housed in the Wisconsin Center forEducation Research and is a collaborative effort of the College of Agriculturaland Life Sciences, the School of Education, the College of Engineering, and theCollege of Letters and Science. The collaborative effort is also joined by theNational Center for Improving Science Education, Washington, DC. Opinions,findings, and conclusions in this manuscript are those of the authors and do notnecessarily reflect the views of the supporting agencies.

NOTES

1 Although we used random digit dialing to contact our participants, our intention was notto produce a representative sample so much as a sample meeting our quota requirementsas described later.

2 Although it was our intention to convey “World Wide Web” and not “Internet,” it ispossible that some individuals, particularly novices, might have misunderstood ourquestion.

3 We used this dichotomous coding scheme only for the purposes of filling our quota, andfor that purpose we refer to these groups as high and low Web users. During the time thatthis study was conducted (1997), using five as the cut point made sense given therelatively small number of individuals that had access to the Web and the relatively lowlevels of use of the Web. In 2000 this cut point would likely need to be set somewhathigher. For the correlational results reported later, a continuous measure of the number oftimes the Web had been used in the past 30 days was used for analyses in place of thisdichotomous version. The median of the continuous measure among the study participantswas eight times in the past month, with two being the median for those who fell into the“five or less” dichotomous category, and 23 being the median for those who fell into the“more than five” category.

4 In a review of the hypermedia literature, Eveland and Dunwoody (in press) noted thatgender, content expertise, and domain expertise are related to hypermedia navigationpatterns and can even moderate the effects of hypermedia use.

5 These are all competing measures of intercoder reliability for categorical data withdifferent strengths and weaknesses (see Brennan & Prediger, 1981; Cohen, 1960;Krippendorff, 1970, 1980; Perreault & Leigh, 1989; Rust & Cooil, 1994).


6 Although one might expect a negative relationship for purely methodological reasons(because putting more in one category of a single variable must take away from othercategories), it is interesting to note that a significant negative relationship does not existbetween proportion of thoughts devoted to elaboration and proportion of thoughts devotedto evaluation (τb = –.04), nor between proportion of thoughts devoted to orientation andproportion of thoughts devoted to maintenance (τb = –.02). This implies that when oneengages in orienting thoughts, it disproportionately reduces elaboration and evaluation.

7 Analyses at the individual level using nonparametric correlations suggest that orientationdid not vary significantly by Web expertise (measured by frequency of Web use; τb = –.26,p = .18) or gender (τb = .37, p = .11), as we had expected when developing our quotasampling design. However, the correlations were in the correct direction, and our smallsample size at the individual level (N = 15) makes Type II error a realistic concern.Therefore the interpretation of these nonsignificant findings should be extremelytentative.

REFERENCES

Astleitner, H., & Leutner, D. (1995). Learning strategies for unstructuredhypermedia: A framework for theory, research, and practice. Journal ofEducational Computing Research, 13, 387–400.

Bevilacqua, A. F. (1989). Hypertext: Behind the hype. American Libraries, 20,158–162.

Bieber, M., Vitali, F., Ashman, H., Balasubramanian, V., & Oinas-Kukkonen, H.(1997). Fourth generation hypermedia: Some missing links for the World WideWeb. International Journal of Human-Computer Studies, 47, 31–65.

B i m b e r, B. (1999). Data on Internet users and on political use of the Internet [ O n -line]. Available: http://www. p o l s c i . u c s b . e d u / ~ b i m b e r / r e s e a r c h / d e m o s . h t m l[2000, April 19].

Brennan, R. L., & Prediger, D. J. (1981). Coefficient kappa: Some uses, misuses,and alternatives. Educational and Psychological Measurement, 41, 687–699.

Bush, V. (1945, July). As we may think. Atlantic Monthly, 176(1), 101–108.

Calvi, L. (1997). Navigation and disorientation: A case study. Journal ofEducational Multimedia and Hypermedia, 6, 305–320.

Carmel, E., Crawford, S., & Chen, H. (1992). Browsing in hypertext: A cognitivestudy. IEEE Transactions on Systems, Man, and Cybernetics, 22, 865–884.

Chen, C., & Rada, R. (1996). Interacting with hypertext: A meta-analysis ofexperimental studies. Human-Computer Interaction, 11, 125–156.


Churcher, P. R. (1989). A common notation for knowledge representation,cognitive models, learning and hypertext. Hypermedia, 1, 235–254.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational andPsychological Measurement, 20, 37–46.

Conklin, J. (1987). Hypertext: An introduction and survey. IEEE Computer,20(7), 17–41.

Craik, F. I. M., & Tulving, E. (1975). Depth of processing and the retention ofwords in episodic memory. Journal of Experimental Psychology: General,104, 268–294.

Crampton, J. (1992). A cognitive analysis of wayfinding expertise.Cartographica, 29(3&4), 46–65.

Darken, R. P., & Sibert, J. L. (1996). Navigating large virtual spaces.International Journal of Human-Computer Interaction, 8(1), 49–71.

Dias, P., & Sousa, A. P. (1997). Understanding navigation and disorientation inhypermedia learning environments. Journal of Educational Multimedia andHypermedia, 6, 173–185.

Dillon, A., & Gabbard, R. (1998). Hypermedia as an educational technology: Areview of the quantitative research literature on learner comprehension,control, and style. Review of Educational Research, 68, 322–349.

Duchastel, P. C. (1990). Examining cognitive processing in hypermedia usage.Hypermedia, 2, 221–233.

Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports asdata (revised ed.). Cambridge, MA: MIT Press.

Estes, W. K. (1988). Human learning and memory. In R. C. Atkinson, R. J.Herrnstein, G. Lindzey, & R. D. Luce (Eds.), S t e v e n s ’ handbook ofexperimental psychology (2nd ed., Volume 2: Learning and cognition; pp.351–415). New York: Wiley.

Eveland, W. P., Jr. (1997a, May). Motivation, information processing, andlearning from the news: Toward a synthesis of paradigms. Paper presented atthe annual meeting of the International Communication Association, Montreal,Canada.

Eveland, W. P., Jr. (1997b). The process of political learning from the news: Theroles of motivation, attention, and elaboration. Unpublished doctoraldissertation, University of Wisconsin-Madison.

Eveland, W. P., Jr., & Dunwoody, S. (1998). Users and navigation patterns of ascience World Wide Web site for the public. Public Understanding of Science,7, 285–311.


Eveland, W. P., Jr., & Dunwoody, S. (2000, June). A test of competing hypothesesabout the impact of the World Wide Web versus traditional print media onl e a r n i n g . Paper presented at the annual meeting of the InternationalCommunication Association, Acapulco, Mexico.

Eveland, W. P., Jr., & Dunwoody, S. (in press). Applying research on the uses andcognitive effects of hypermedia to the study of the World Wide Web. In W.Gudykunst (Ed.) Communication yearbook 25.

Gould, O. N., Trevithick, L., & Dixon, R. A. (1991). Adult age differences inelaborations produced during prose recall. Psychology and Aging, 6, 93–99.

Greene, R. L. (1992). Human memory: Paradigms and paradoxes. Hillsdale, NJ:Lawrence Erlbaum Associates.

Haberlandt, K. (1994). Cognitive psychology. Boston, MA: Allyn and Bacon.

Hamilton, R. (1989). The effects of learner-generated elaborations on conceptlearning from prose. Journal of Experimental Education, 57, 205–217.

Hamilton, R. J. (1997). Effects of three types of elaboration on learning conceptsfrom text. Contemporary Educational Psychology, 22, 299–318.

Heller, R. S. (1990). The role of hypermedia in education: A look at the researchissues. Journal of Research on Computing in Education, 22, 431–441.

Hill, J. R., & Hannafin, M. J. (1997). Cognitive strategies and learning from theWorld Wide Web. Educational Technology Research & Development, 45(4),37–64.

Horney, M. (1993). A measure of hypertext linearity. Journal of EducationalMultimedia and Hypermedia, 2, 67–82.

Horney, M. A. (1991). Uses of hypertext. Journal of Computing in HigherEducation, 2(2), 44–65.

The Internet: Bringing order from chaos. (1997). Scientific American, 276(3),50–51.

Johnsey, A., Morrison, G. R., & Ross, S. M. (1992). Using elaboration strategiestraining in computer-based instruction to promote generative learning.Contemporary Educational Psychology, 17, 125–135.

Jonassen, D. H. (1988). Designing structured hypertext and structuring access tohypertext. Educational Technology, 28(11), 13–16.

Jonassen, D. H., & Wang, S. (1993). Acquiring structural knowledge fromsemantically structured hypertext. Journal of Computer-Based Interaction, 20,1–8.

Kahneman, D. (1973). Attention and effort. Englewood Cliffs, NJ: Prentice-Hall.


Kardash, C. M., & Amlund, J. T. (1991). Self-reported learning strategies andlearning from expository text. Contemporary Educational Psychology, 16,117–138.

Kellogg, R. T. (1982). When can we introspect accurately about mentalprocesses? Memory & Cognition, 10, 141–144.

Kim, H., & Hirtle, S. C. (1995). Spatial metaphors and disorientation inhypertext browsing. Behaviour & Information Technology, 14, 239–250.

Kozma, R. B. (1987). The implications of cognitive psychology for computer-based learning tools. Educational Technology, 27(11), 20–25.

Krippendorff, K. (1970). Bivariate agreement coefficients for reliability of data.In E. F. Borgatta (Ed.) Sociological methodology 1970 (pp. 139–150). SanFrancisco, CA: Jossey-Bass.

Krippendorff, K. (1980). Content analysis: An introduction to its methodology.Newbury Park, CA: Sage.

Lucarella, D., & Zanzi, A. (1993). Browsing and searching in hypertext systems.Journal of Computing in Higher Education, 4(2), 79–105.

Marchionini, G. (1988). Hypermedia and learning: Freedom and chaos.Educational Technology, 28(11), 8–12.

Mayer, R. E. (1980). Elaboration techniques that increase the meaningfulness oftechnical text: An experimental test of the learning strategy hypothesis.Journal of Educational Psychology, 72, 770–784.

McDonald, S., & Stevenson, R. J. (1996). Disorientation in hypertext: Thee ffects of three text structures on navigation performance. A p p l i e dErgonomics, 27, 61–68.

McDonald, S., & Stevenson, R. J. (1998). Effects of text structure and priorknowledge of the learner on navigation in hypertext. Human Factors, 40,18–27.

Nelson, T. H. (1993). Literary machines 93.1. Sausalito, CA: Mindful Press.

Nelson, W.A., & Palumbo, D. B. (1992). Learning, instruction, and hypermedia.Journal of Educational Multimedia and Hypermedia, 1, 287–299.

Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbalreports on mental processes. Psychological Review, 84, 231–259.

Perreault, W. D., & Leigh, L. E. (1989). Reliability of nominal data based onqualitative judgments. Journal of Marketing Research, 26, 135–148.

Perse, E. M. (1990). Media involvement and local news effects. Journal ofBroadcasting & Electronic Media, 34, 17–36.


Pew Research Center. (Undated). Online newcomers more middle-brow, lesswork-oriented: The Internet audience goes ordinary [On-line]. Available:http://www.people-press.org/tech98sum.htm [2000, April 19].

Pressley, M., McDaniel, M. A., Turnure, J. E., Wood, E., & Ahmad, M. (1987).Generation and precision of elaboration: Effects on intentional and incidentallearning. Journal of Experimental Psychology: Learning, Memory, andCognition, 13, 291–300.

Russo, J. E., Johnson, E. J., & Stephens, D. L. (1989). The validity of verbalprotocols. Memory & Cognition, 17, 759–769.

Rust, R. T., & Cooil, B. (1994). Reliability measures for qualitative data: Theoryand implications. Journal of Marketing Research, 31, 1–14.

Salomon, G. (1981). Introducing AIME: The assessment of children’s mentalinvolvement with television. In H. Kelly & H. Gardner (Eds.) Viewing childrenthrough television (pp. 89–101). San Francisco, CA: Jossey-Bass.

Salomon, G. (1983). The differential investment of mental effort in learning fromdifferent sources. Educational Psychologist, 18, 42–50.

Schmeck, R. R. (1980). Relationships between measures of learning style andreading comprehension. Perceptual and Motor Skills, 50, 461–462.

Schmeck, R. R., & Grove, E. (1979). Academic achievement and individualdifferences in learning processes. Applied Psychological Measurement, 3,43–49.

Schmeck, R. R., & Phillips, J. (1982). Levels of processing as a dimension ofdifference between individuals. Human Learning, 1, 95–103.

Schmeck, R. R., Ribich, F., & Ramanaiah, N. (1977). Development of a self-report inventory for assessing differences in learning processes. AppliedPsychological Measurement, 1, 413–431.

Shin, E. C., Schallert, D. L., & Savenye, W. C. (1994). Effects of learner control,advisement, and prior knowledge on young students’ learning in a hypertextenvironment. Educational Technology Research and Development, 42(1),33–46.

Shirk, H. N. (1992). Cognitive architecture in hypermedia instruction. In E.Barrett (Ed.), Sociomedia: Multimedia, hypermedia, and the socialconstruction of knowledge (pp. 79–93). Cambridge, Mass: MIT Press.

Smith, E. R., & Miller, F. D. (1978). Limits on perception of cognitive processes:A reply to Nisbett and Wilson. Psychological Review, 85, 355–362.

Te rgan, S. O. (1997). Misleading theoretical assumptions in hypertext/hypermediaresearch. Journal of Educational Multimedia and Hypermedia, 6, 2 5 7 – 2 8 3 .


Thüring, M., Hannemann, J., & Haake, J. M. (1995). Hypermedia and cognition:Designing for comprehension. Communications of the ACM, 38(8), 57–66.

Tsai, C. J. (1988–1989). Hypertext: Technology, applications, and researchissues. Journal of Educational Technology Systems, 17, 3–14.

Turner, C. K. (1988). Don’t blame memory for people’s faulty reports on whatinfluences their judgments. Personality and Social Psychology Bulletin, 14,622–629.

Watkins, D., & Hattie, J. (1981a). The internal structure and predictive validityof the inventory of learning processes: Some Australian and Filipino data.Educational and Psychological Measurement, 41, 511–514.

Watkins, D., & Hattie, J. (1981b). The learning processes of Australian universitystudents: Investigations of contextual and personological factors. BritishJournal of Educational Psychology, 51, 384–393.

Weinstein, C. E., Zimmermann, S. A., & Palmer, D. R. (1988). Assessinglearning strategies: The design and development of the LASSI. In C. E.Weinstein, E. T. Goetz, & P. A. Alexander (Eds.) Learning and studystrategies: Issues in assessment, instruction, and evaluation (pp. 25–40). NewYork: Academic.

Woloshyn, V., Paivio, A., & Pressley, M. (1994). Use of elaborative interrogationto help students acquire information consistent with prior knowledge andinformation inconsistent with prior knowledge. Journal of EducationalPsychology, 86, 79–89.

Woloshyn, V., Willoughby, T., Wood, E., & Pressley, M. (1990). Elaborativeinterrogation facilitates adult learning of factual paragraphs. Journal ofEducational Psychology, 82, 513–524.

Wright, P., & Rip, P. D. (1981). Retrospective reports on the causes of decisions.Journal of Personality and Social Psychology, 40, 601–614.


APPENDIX A

Summary of Domain and Information Processing Coding Typology

Maintenance Elaboration Evaluation OrientationContent

Structure


Readingaloud; slightlyrephrasing the

contentwithout

adding newinformation

Makingconnections to

priorknowledge,

externalinformation,

or pastexperience

Expressinginterest ordisinterest;

makingjudgments ofthe accuracy

of infoAdditional

code:+ / –

Askingrhetoricalquestionsabout the

informationavailable in

the site

Stating thename of linksbefore or as

they aretraversed

Makingconnections to

pastexperiencewith other

sites or othermedia formats

Expressingaffect for theorganization

of informationor noncontentfeatures of site

Additionalcode:+ / –

Expressingconfusion

aboutnavigationthrough thesite; askingrhetoricalquestionsabout the

structure ofthe site;

synthesizingthe structure

of siteAdditional

code:epiphany /

disorientation

Examining Information Processing on the World Wide - School of

Documents

Transcript of Examining Information Processing on the World Wide - School of