Measuring the Flow Experience Among Web Users · Measuring the Flow Experience Among Web Users...

Measuring the Flow Experience Among Web Users

Thomas P. Novak ([email protected]) Donna L. Hoffman ([email protected])

Project 2000, Vanderbilt Universityhttp://www2000.ogsm.vanderbilt.edu/

July 1997

Paper Presented at Interval Research Corporation, July 31, 1997

Draft 1.0

elab.vanderbilt.edu

1

Measuring the Flow Experience Among Web Users

Thomas P. Novak and Donna L. HoffmanProject 2000, Vanderbilt University

http://www2000.ogsm.vanderbilt.edu/

Abstract

The flow construct has recently been proposed as essential to understanding consumer navigation behaviorin online environments. We review definitions and models of flow, and describe an empirical study whichmeasures flow in terms of respondents’ skills and challenges for using the World Wide Web. Skills andchallenges are shown to correlate in anticipated ways with scales measuring constructs of flow, control,arousal, and anxiety that underlie previous models of flow. By taking the sum and difference of skills andchallenges as axes of a two dimensional space, we derive a simple conceptualization of flow. The sum anddifference of skills and challenges for using the Web relates in hypothesized ways to measures of consumersearch and purchase behavior in online and traditional media.

2

1) Introduction

Objectives

The flow construct (Csikszentmihalyi 1977) has been recently proposed by Hoffman and Novak (1996) asessential to understanding consumer navigation behavior in online environments such as the World WideWeb. Previous researchers (e.g. Csikszentmihalyi 1990; Ghani, Supnick and Rooney 1991; Trevino andWebster 1992; Webster, Trevino and Ryan 1993) have noted that flow is a useful construct for describingmore general human-computer interactions. Hoffman and Novak define flow as "the state occurring duringnetwork navigation which is: 1) characterized by a seamless sequence of responses facilitated by machineinteractivity, 2) intrinsically enjoyable, 3) accompanied by a loss of self-consciousness, and 4) self-reinforcing." To experience flow while engaged in an activity, consumers must perceive a balance betweentheir skills and the challenges of the activity, and both their skills and challenges must be above a criticalthreshold. Hoffman and Novak (1996) propose that flow has a number of positive consequences from amarketing perspective, including increased consumer learning, exploratory behavior, and positivesubjective experience.

Over the past 20 years, numerous researchers have attempted to measure flow for a wide variety ofactivities, including composing music, sports, work activities, hobbies, and computer usage. We willsummarize the results of many of their efforts, and the models of flow that have been proposed. Ourinterest is in measuring and modeling flow specifically for consumer behavior on the Web. Our ownmeasurement effort builds upon previous research by operationalizing flow in terms of the combination of aWeb user’s perception of their skills at using the Web and how challenging they find using the Web to be.

Specifically, we seek to:

1) Review previous definitions and models of flow.

2) Describe our empirical study which measures a series of constructs related to flow.

3) Develop and evaluate the reliability of a series of Likert Scales for components of the eight channelflow model proposed by Massimini & Carli (1988): skill, challenge, and bipolar constructs offlow/apathy, control/worry, arousal/relaxation and anxiety/boredom.

4) Evaluate the convergent and discriminant validity of the above, in terms of the relationships among

these constructs as predicted by the eight channel flow model.

5) Develop a simple two-dimensional model of flow, where the dimensions underlie the four eightchannel flow models, and are defined by a) the sum of skills and challenges, and b) the difference ofskills and challenges. We will investigate demographic and usage differences on these twodimensions.

6) Specify and test a series of hypotheses regarding the relationship of consumer search and purchase

behavior on the Web and in traditional media to the sum and difference of skills and challenges usingthe Web.

We begin with a literature review of the flow construct, and models of flow, and then consider the topicslisted above.

The Flow Construct

What is flow? Table 1 provides definitions of flow from 16 different studies. As one reads through thislist, the phrases listed here seem to make intuitive sense - flow is "a holistic sensation where one acts with

3

total involvement, with a narrowing of focus of attention," and so forth. However, the exercise of readingthrough these phrases in an attempt to define flow is deceptive. One is not left with a central definition offlow, but rather a wide variety of constructs which tend to be experienced when one experiences flow.Some of these constructs define or cause flow, and some of these are experienced as a result of being in theflow state. Hoffman and Novak (1996) propose, for example, that centering of attention is a necessarycondition for achieving flow, as are congruent skills and challenges that are above a critical level.

Consider two definitions that have been proposed by Trevino & Webster (1992) and by Csikszentmihalyi &Csikszentmihalyi (1988). Trevino and Webster (1992) operationally define flow as the linear combinationof four characteristics: control, attention, curiosity, and intrinsic interest. But, it is not clear why these fourcharacteristics should be used. Do these define flow, or are they better thought of as antecedents orconsequences of flow?

The other definition, from Csikszentmihalyi & Csikszentmihalyi (1988) is quite different, focusing uponthe congruence of a person's skills in a given activity, and their perceptions of the challenges of the activity.The person's evaluation of the skills and challenges is typically thought of relative to other activities theperson performs, rather than on an absolute scale. That is, how challenging is surfing the Web, comparedto other ways the person has available of spending their time? The definition also states that there is acritical value that skills and challengers must be above. Thus, it is not simply the fact that skills andchallenges are congruent, they must also be high. This is different than many early definitions of flow interms of skills and challenges, which considered low skill and low challenge activities (take chewingbubble gum as an extreme example) to also produce flow (Csikszentmihalyi 1977).

In Hoffman and Novak (1996), we define flow in terms of the experience of flow (intrinsic enjoyment, lossof self-consciousness), structural properties of the flow activity (seamless sequence of responses facilitatedby interactivity with the computer and self-reinforcement), and antecedents of flow (skill/challengebalance, focused attention, and telepresence).

In this research, we focus upon high skills and challenges for using the Web as a requirement for theexperience of flow while using the Web. In addition, we use the construct of computer playfulness, asoperationalized by Webster and Martocchio (1992) as an indicator of the experience of flow while usingthe Web. The role of skills and challenges in defining flow is a central portion of the majority of thedefinitions in Table 1. We do not address the larger question in this paper of the exact configuration ofantecedents and consequences of flow, focusing instead upon the more limited research question of the roleof skills and challenges. Research in progress (Novak, Hoffman and Yung 1997) addresses the larger issueof the structure of antecedents and consequences of flow.

Table 1 - Definitions of Flow

Reference: Conceptual or Operational Definition:

Csikszentmihalyi (1977) "the holistic sensation that people feel when they act with total involvement" (p36)

when in the flow state "players shift into a common mode of experience when theybecome absorbed in their activity. This mode is characterized by a narrowing of thefocus of awareness, so that irrelevant perceptions and thoughts are filtered out; by loss ofself-consciousness; by a responsiveness to clear goals and unambiguous feedback; andby a sense of control over the environment...it is this common flow experience thatpeople adduce as the main reason for performing the activity" (p72)

Privette and Bundrick(1987)

"Flow..., defined as an intrinsically enjoyable experience, is similar to both peakexperience and peak performance, as it shares the enjoyment of valuing of peakexperience and the behavior of peak performance. Flow per se does not imply optimaljoy or performance but may include either or both." [p316]

Csikszentmihalyi and "The flow experience begins only when challenges and skills are above a certain level,

4

Csikszentmihalyi (1988) and are in balance." [p260]

Mannell, Zuzanek, andLarson (1988)

"Csikszentmihalyi (1975) describes the flow experience as 'one of complete involvementof the actor with his activity' (p. 36), and he has identified a number of elements that areindicators of its occurrence and intensity. These indicators include: the perception thatpersonal skills and the challenges provided by an activity are imbalance, centering ofattention, loss of self-consciousness, unambiguous feedback to a person's actions,feelings of control over actions and environment, and momentary loss of anxiety andconstraint, and enjoyment or pleasure." [p291]

"Flow was operationalized by measuring the affect, potency, concentration, and theperception of a skill/challenge balance ." [p292]¨

Massimini and Carli(1988)

congruent skills and challenges that are above each subject's average weekly levels

LeFevre (1988) "a balanced ratio of challenges to skills above average weekly levels" (p307)

Csikszentmihalyi andLeFevre (1989)

"When both challenges and skills are high, the person is not only enjoying the moment,but is also stretching his or her capabilities with the likelihood of learning new skills andincreasing self-esteem and personal complexity. This process of optimal experience hasbeen called flow."

Csikszentmihalyi (1990) we feel "in control of our actions, masters of our own fate...we feel a sense ofexhilaration, a deep sense of enjoyment" (p3)

"the state in which people are so intensely involved in an activity that nothing else seemsto matter; the experience itself is so enjoyable that people will do it even at great cost, forthe sheer sake of doing it."

Ghani, Supnick andRooney (1991)

"two key characteristics of flow: the total concentration in an activity and the enjoymentwhich one derives from an activity...the precondition for flow is a balance between thechallenges perceived in a given situation and skills a person brings to it" (p230) "arelated factor is the sense of control over one's environment" (p231)

Trevino and Webster(1992)

"flow characterizes the perceived interaction with CMC technologies as more or lessplayful and exploratory"..Flow theory suggests that involvement in a playful, exploratoryexperience - the flow state - is self-motivating because it is pleasurable and encouragesrepetition. Flow is a continuous variable ranging from none to intense." [p540]

"Flow represents the extent to which (a) the user perceives a sense of control over thecomputer interaction, (b) the user perceives that his or her attention is focused on theinteraction, (c) the user's curiosity is aroused during the interaction, and (d) the user findsthe interaction intrinsically interesting.." [p542]

Webster, Trevino andRyan (1993)

"the flow state is characterized by four dimensions...(a) the user perceives a sense ofcontrol over the computer interaction, (b) the user perceives that his or her attention isfocused on the interaction, (c) the user's curiosity is aroused during the interaction, and(d) the user finds the interaction intrinsically interesting. [p413]

Clarke and Haworth(1994)

"the subjective experience that accompanies performance in a situation where thechallenges are matched by the person's skills. Descriptions of the feeling of 'flow'indicate an experience that is totally satisfying beyond a sense of having fun." [p511]

Ellis, Voelkl and Morris(1994)

"..an optimal experience that stems from peoples' perceptions of challenges and skills ingiven situations. Situations in which challenges and skills are perceived to be equivalentare thought to facilitate the emergence of such indicators of flow as positive affect andhigh levels of arousal, intrinsic motivation, and perceived freedom" [p337]

Ghani and Deshpande(1994)

"The two key characteristics of flow are (a) total concentration in an activity and (b) theenjoyment which one derives from an activity...There is an optimum level of challengerelative to a certain skill level. ...A second factor affecting the experience of flow is asense of control over one's environment." [p383]

5

Lutz and Guiry 1994 "Psychologists use the term 'flow' to describe a state of mind sometimes experienced bypeople who are deeply involved in some event, object or activity...they are completelyand totally immersed in it...Indeed, time may seem to stand still and nothing else seemsto matter while engaged in the consump†ion event."[from respondent instructions]

Hoffman and Novak(1996)

"the state occurring during network navigation which is 1) characterized by a seamlesssequence of responses facilitated by machine interactivity, 2) intrinsically enjoyable, 3)accompanied by a loss of self-consciousness, and 4) self-reinforcing"

Flow Measurement Methodology

Table 2 summarizes the 16 articles shown in Table 1 according to the approach taken to measuring flow.There are three major approaches to measuring flow that have been taken in empirical research:

1) Respondent provides a narrative description of a flow experience and then evaluates the experienceusing a survey instrument (Narrative/Survey).

2) Respondents who have participated in a selected activity are asked to retroactively evaluate their

experience using a survey instrument (Activity/Survey). 3) In the Experience Sampling Method (ESM), respondents are paged throughout the day for a one week

period, and evaluated their activity at the time of being paged using a survey instrument.

Let us consider a few examples of these three approaches. First consider the Narrative/Survey method.This method was used by Privette and Bundrick (1987) to measure six "construct events" - eventscharacterizing constructs from the theoretical and research literature. Subjects are given short descriptionsof peak performance, peak experience, flow, average events, misery and failure, and asked to write a shortnarrative description of one of each of these experiences.

From the point of view of flow measurement, a major weakness of Privette's work is that flow is alwaysoperationalized as "describe the last time you played a sport or game." This is a very weak definition offlow. Each of the six experiences is then evaluated on a 47 item "Experience Questionnaire". Theobjective is to understand the nature of the differences among these six construct events. Thus, the analysisof flow is at a very general level. This methodology is not used to identify the extent to which flow occursin different people for different types of events.

The second method we call the Activity/Survey method. Webster, Trevino and Ryan (1993) describe twostudies which provide examples of this method. In the first, 133 subjects attended a one-day Lotus 1-2-3course, and in the second 43 subjects who were regular users of electronic mail were selected. In bothstudies, a 12-item scale measured flow as a combination of items for control, attention focus, curiosity andintrinsic interest.

Table 2 - Methods for Measuring Flow

Method Reference Description

Narrative +Survey

Privette & Bundrick1987

Subjects provide narrative descriptions of six "construct events”: peak performance(one incident in your life characterized by functioning at your best); peak experience(one incident in your life characterized by highest happiness); flow (the last time youplayed a sport or game); an average event (something you did between 3 pm and 6pm yesterday); misery one incident in your life characterized by deepest misery;failure (one incident in your life characterized by total failure).

6

These construct events were each evaluated on the "Experience Questionnaire"consisting of 47 rating scales.

Lutz & Guiry 1994 Subjects provided narrative descriptions of four categories of consumptionexperiences: peak experiences, peak performance, flow, ordinary consumption

These consumption experiences were rated on Privette's Experience Questionnaireconsisting of 42 ratings scales, plus single item measures of the challenges and skillsof the event.

Activity +Survey

Trevino & Webster1992

154 respondents at a midwestern health care firm completed surveys. Employeeswere users of email (average of 8.5 months) and voicemail (average of 7 months).

A voluntary one-day sample of email and voicemail was obtained and coded for eachrespondent. Survey data was collected on flow (four items measuring control,attention focus, curiosity, and intrinsic interest) and a large number of antecedentsand consequences.

Webster, Trevino &Ryan 1993

Study 1: Subjects (133 MBA students) attended a one-day Lotus 1-2-3 course.

Study 2: Subjects (43 employees in the accounting department of a large firm) who were regular users of electronic mail.

Survey: Both studies used a 12-item scale which measured flow as a combination of1) control, 2) attention focus, 3) curiosity, and 4) intrinsic interest.Correlates of flow were measured with scales for flexibility, modifiability,experimentation, expected voluntary use, and communication effectiveness.

Ghani, Supnick &Rooney 1991

59 undergraduate business students were randomly assigned to three-person groupsand completed three group activities: 1) an orientation to computer-mediated (CM)conferencing, 2) a CM group exercise, and 3) a face-to-face group exercise.

Respondents completed questionnaires measuring flow (operationalized as four itemsfor enjoyment and four for concentration), challenges, skills, and control.

Ghani & Deshpande1994

Sample was 62 managers from a variety of manufacturing, service, and governmentorganizations who used computers as part of their day-to-day work.

Respondents completed questionnaires measuring flow (operationalized as four itemsfor enjoyment and four for concentration), challenge, control, exploratory use, andextent of use

ExperienceSamplingMethod

Csikszentmihalyi &Csikszentmihalyi1988

Objective is to measure flow and other states of consciousness occurring in activitiesencountered in everyday life. For each respondent in the study, day-to-day activitiesare sampled by paging the respondent 8 times a day for a week, for a total of 56times. For example, Csiksentmihalyi & LeFevre 1989 paged respondents randomlywithin 2-hour periods from 7:30 am to 10:30 pm. The Experience Sampling Form,completed at each paging, measures challenges and skills, respondent's mood,motivation for doing the activity, and many other correlates of flow.

Mannell, Zuzanek& Larson 1988

Sample was 92 retired adults. Participants carried pagers for one week. Signals weresent between 8 am and 10 pm, with one signal occurring at a random time withinevery 2-hour block. ESF was filled out for each signal. Participants responded to76% of signals, providing 3,412 self-reports.

Massimini & Carli1988

Sample was 47 students between 16 and 19 years old living in Milan, Italy. Anaverage of 7 signals a day were sent between 8 and 10 pm for one week, and ESFfilled out. Activities were coded into categories.

Carli, Delle Fave & Replicated Massimini & Carli with a sample of 75 U.S. adolescents from a diversified

7

Massimini 1988 community high school.

LeFevre 1988 Sample was 107 workers recruited from five large companies in the Chicago area thatagreed to cooperate in a study of work satisfaction. Participants carried pagers for aweek, with signals sent between 7:30 am and 10:30 pm, with one signal occurringrandomly within 2-hour periods. Participants responded to 85% of all signals,providing 4,800 self reports.

Csikszentmihalyi &LeFevre 1989

(The data are from LeFevre 1988)

Clarke & Haworth1994

Sample was 35 students aged between 16 and 18 years old from a college in the UK.Signals were sent between 11:30 am and 11:30 pm. Eight signals were issued at set,but randomly determined times. A total of 1745 self reports were obtained.

Ellis, Voelkl &Morris 1994

Study 1 - sample was 12 older adults residing in nursing homes between ages 73 to95. Signals were sent six times a day between 7 am and 7 pm for one week.Participants responded to 74% of all signals, providing 307 responses.

Study 2 - sample was 59 student volunteers. Signals were sent 5 times a day between8 am and 10 pm for one week. Participants responded to 51% of all signals,providing 1057 responses.

The Activity/Survey method can also be used in laboratory experiments. Ghani, Supnick & Rooneyrandomly assigned undergraduate business students into face-to-face or computer-mediated groups, andhad respondents complete questionnaires measuring flow after their group experience.

The Activity/Survey method is, in principle, useful for either concurrently or retrospectively determiningthe experience of flow for specific events. One question is whether respondents can reliably evaluate flowafter rather than during an activity. It may be that surveys completed immediately after completion of anactivity have greater validity than surveys describing an activity which the person has engaged in for a longperiod of time.

The most commonly used method of measuring flow is Csikszentmihalyi's "Experience Sampling Method."The ESM is uniquely suited to measuring flow and other states of consciousness occurring in activitiesoccurring in everyday life. From the distinction between flow, peak experience and peak performance(Privette and Bundrick 1987), flow experiences are the sorts of experiences that occur on a regular, ongoingbasis, rather than being unusual or atypical events of peak experience or peak performance.

In the ESM, respondents are randomly paged 8 times a day for a period of a week, for a total of 56 times.When paged, respondents complete a two-page Experience Sampling Form, which measures the challengesand skills of the activity engaged in at the time of paging, plus a variety of measure of respondent's moodand motivation. The skills and challenges of each respondent are standardized after data collection, so theyrepresent, for each respondent, whether the skills or challenges were above or below the average level of allactivities rated during the one week period.

Models of Flow

Now, let us consider approaches to constructing models of flow. These models relate to both thedefinitions we have discussed and also to the methodologies for measuring flow. Models of flow gobeyond a definition of flow by describing causes, effects, and correlates of flow. We identify three broadapproaches to constructing models of flow:

(1) conceptual models(2) causal models

8

(3) flow channel segmentation models

We discuss each in turn.

Conceptual Models. Our general conceptual model of flow in computer-mediated is described in detail inHoffman and Novak (1996). Key features of this model are that flow is determined by high skills andchallenges and focused attention, and is enhanced by interactivity and telepresence. A simplified version ofHoffman and Novak’s conceptual model is shown in Figure 1.

S K IL L

C H A L L E N G E

T E L E P R E S E N C E

FOCUSED ATTENTION

When first used Web

Time per day use Web

INTERACTIVITY

FLOW

PLAY

P O S ITIV E A F F E C T

EXPLORATORYB E H A V IOR

Figure 1 - Simplified Version of Hoffman & Novak’s (1996) Conceptual Model

Causal Models. Causal models provide a second approach. Flow research which has used the Activity/Survey methodology has typically used causal models to analyze the results of the survey data. Causalmodels look much like conceptual models, but provide an empirical test of the magnitude and directionalityof hypothesized relationships. We briefly consider three causal models presented in the literature.

Consider the causal model fit by Ghani, Supnick & Rooney (1991). Control and challenges were found topredict flow, which was operationalized as four items for enjoyment and four for concentration). Controland flow predicted exploratory use, which in turn predicted extent of use. In a later study, Ghani andDeshpande (1994) included skill as well as challenge. The resulting causal model is simple, but quiteinteresting, in that skill leads to control which leads to flow. Skill also directly affects flow, as doesperceived challenge. This model provides empirical support for definitions of flow that specify that flowoccurs when challenges and flow are both high, since skill and challenges independently contribute to flow.

A final causal model was fit by Trevino and Webster (1992). A different operational definition of flow isused in this research, consisting of four items measuring control, attention focus, curiosity, and intrinsicinterest). Skill was measured, but not challenges. Ease of use was identified as an intermediate variablebetween skill and flow.

One difficulty with the above research is the operational definition of flow. Constructs of enjoyment,concentration, control, attention focus, curiosity and intrinsic interest are used to define flow, rather thanbeing considered as potential antecedents or consequences of flow.

Our intention in this current paper is not to fit structural equation models to constructs relating to flow,although we have work in progress (Novak, Hoffman & Yung 1997) which aims to constructcomprehensive structural equation models which test and refine the model shown in Figure 1. In thiscurrent paper, we focus upon the relationship of skills and challenges with flow. This relationship is anintegral component of Hoffman and Novak’s (1996) conceptual model, has been tested in the causalmodels discussed above, and is the key defining characteristic of the flow channel segmentation models we

9

discuss next.

Flow Channel Segmentation Models. The third and final category of models of flow are "flow channelsegmentation models." While causal models have been used in studies employing the Activity/Surveymethod, channel segmentation models have been exclusively used in studies employing the ExperienceSampling Methodology. This distinction is arbitrary, however, in that it stems from differentmethodologies chosen by different sets of researchers. In this paper, we will collect data using theActivity/Survey method, but analyze it from the framework of flow channel segmentation models.

Flow channel segmentation models are based upon Csikszentmihalyi's definition of flow in terms of skillsand challenges. However, the segmentation models attempt to account for all possible combinations(channels) of high/low skills and challenges. Here are two simple models. The early three channel modelshown in Figure 2 identified flow as congruent skills and challenges, both high and low. Anxiety isidentified as high challenges and low skills, and boredom as high skills and low challenges.

ANXIE T Y

F L O W

B O R E D O M

Low SKILL High

Low

C

HA

LLE

NG

E

Hig

h

Figure 2 - Three Channel Flow Model

Greater empirical support has been found for the reformulated four channel model shown in Figure 3,where flow is defined as high skills and high challenges, and apathy as low skills and low challenges.Numerous researchers (e.g., Ellis, Voelkl & Morris 1994; LeFevre 1988; Nakamura 1988; and Wells 1988)have found clear patterns of differences among the four “flow segments,” where the differences can becharacterized as:

1) Flow is distinctly different from other states. Respondents who rated the skills and challenges of anactivity as higher than average for the week rated those activities as higher on virtually all of thepositive indicator variable collected in these studies: enjoyment, positive affect, activation,concentration, creativity, self-esteem, etc.

2) Flow is in complete opposition to apathy. The empirical evidence suggests that low challenge/lowskill activities are opposite to flow activities, and the four channel model is more appropriate than thethree channel model.

10

ANXIE T Y F L O W

B O R E D O M

Low SKILL High

Low

C

HA

LLE

NG

E

Hig

h

A P A T H Y

Figure 3 - Four Channel Flow Model

A natural extension of the four channel model is the eight channel model (Massimini & Carli 1988; Ellis,Voelkl & Morris 1994), which also allows for intermediate (moderate) levels of skills and challenges, andidentifies four additional channels: arousal, control, relaxation, and worry. Figure 4 shows ourconceptualization of the eight-channel model, superimposing dimensions of Skills and Challenges. Figure4 represents a 45 degree rotation of Figure 3, so that the horizontal direction in Figure 4 now corresponds tothe sum of skills plus challenges (apathy vs. flow), while the vertical direction corresponds to the differenceof skills minus challenges (boredom vs. anxiety). The southwest to northeast direction corresponds to skills(control vs. worry), while the northwest to southeast direction corresponds to challenges (arousal vs.relaxation). The four channel model thus proposes four bipolar constructs which lie in a two dimensionalspace, and where the space is spanned by orthogonal vectors for skills and challenges.

11

Skills

S kills + Challenges

S kills - Challenges

Challenges

BOREDOM

RELAXATION

APATHY

WORRY

ANXIETY

CONTROL

FLOW

A R O U S A L

Figure 4 - Eight Channel Flow Model

Massimini and Carli (1988) reported mean ratings on 23 scales, by the eight segments shown in Figure 4,for a sample of 47 Italian students between 16 and 19 years of age. By reanalyzing Massimini & Carli'sdata, we can demonstrate clear support for the 8 channel model. A principal components analysis of themeans of the 23 items on the 8 flow channels identifies two factors which explain 89% of the variance. InFigure 5 we plot means of the 8 flow channels on the two factor scores, and represent (most of) the 23items in the plot according to their correlations with the factors.

Figure 5 - Reanalysis of Massimini & Carli (1988)

The plot is extremely interesting. First, we notice that the pattern of means of the eight flow channels is

12

virtually identical to the conceptual diagram of the eight flow channel model presented in Figure 4.Second, we can visually see that all items point to the right (flow) side of the map. There is also a cleardifferentiation between items that point toward the top right (control) and the bottom right (arousal).

2) Description of Empirical Research

Questionnaire Development and Data Collection

Our objective in our empirical research was to develop a comprehensive instrument for measuring theconstructs discussed in the conceptual model from Hoffman and Novak (1996). The ultimate objective ofthis instrument was twofold; first, to provide data on skills and challenges of Web users that could be usedto validate the eight-channel flow channel model and relate flow to consumer behavior on the Web (ourcurrent paper), and second, to empirically test the conceptual model in Hoffman and Novak (1996)(subsequent research). The process of questionnaire development involved a) construction of a preliminaryquestionnaire based upon a literature review; b) a series of four pretests; and c) the final field survey. Wedescribe each in turn.

Preliminary Questionnaire. Based upon the 16 studies in Tables 1 and 2, a comprehensive list of 100items were compiled that have been used to measure the following constructs: flow, skill, challenge,control, attention focus, concentration, telepresence, curiosity, intrinsic interest, extrinsic interest, play/fun,affect, activation, ease of use, self-reinforcement, interactivity, self esteem, outer structure of task, andperformance.

Pretesting. A series of pretests were conducted over a four month period, which drew from and refined theitems from the preliminary questionnaire. The major objective of the pretests was to refine items used tomeasure skills and challenges for using the Web, plus constructs measuring flow, play, and the eightchannel flow model. The four pretests are briefly described below:

Pretest 1. (n=49, in-person 22 item written questionnaire administered in person to MBA students at atop 25 business school, December 1996). Subjects rated seven-point semantic differentialscales describing how they usually feel when using the Web for “in flow vs. apathetic,”“anxiety vs. bored,” “in control vs. worried,” and “excited vs. relaxed.” Subject ratings forskills and challenges for using the Web on 9 point rating scales were also obtained, bothfrom an absolute perspective (none to very high) and from a relative perspective (how muchmore or less compared with other activities?). From Figure 4, we can predict that skillshould positively correlate with flow and control, and negatively correlate with anxiety,while challenge should positive correlated with flow, arousal, and anxiety. The standardizedregression coefficients summarized in Table 3 show that the predictions were empiricallysupported for skills, but not for challenges.

13

Table 3 - Pretest One Results

Hypothesis: Empirical (Absolute): Empirical (Relative):Flow Channel: Skill Challenge Skill Challenge Skill Challenge

Flow + + .22 -.17 .42* -.04Control + .44* .01 .43* -.23Arousal + -.02 .33 .07 .18Anxiety - + -.20 -.05 -.01 .12

*p value for standardized beta < .05

To explain the poor performance of the challenge scales, we examined two open endedquestions: “What makes you skilled at using the Web?” and “What makes the Webchallenging.” We found that the skill item was being interpreted as we desired (frequency ofuse, understanding search engines, experiences, curiosity, computer at home, frequency,experimentation, trial and error, practice, prior knowledge of computers, etc.). However, thechallenge item was interpreted in a very negative manner. Rather than being interpreted as astimulating cognitive task, the challenge of using the Web was described by phrases such as:loading time too slow, browser crashes, too much information, Web is saturated, wait timefor connection, computer freezes, dial up busy signals, knowing the correct key words, etc.In subsequent pretests, we were careful to develop items for challenge that specificallycaptured the construct we were seeking to measure. It was clear from the first pretest, thatunlike typical flow studies employing the ESM that have measured challenge with a simplesingle item rating scale asking for a direct assessment of the challenge of the activity, thiswould not be possible when measuring the challenge of the Web.

Pretest 2. (n=108, 59 item Web fillout form1 administered to Project 2000 Pretest Panel, January 1997).Based upon the results from Pretest 1, 15 items dealing with skills and 15 items dealing withchallenges of using the Web were constructed, plus additional items for barriers to using theWeb, dimensions underlying the eight channel flow model, and correlates of flow.

Pretest 3. (n=86, in-person 59 item written questionnaire administered in person to MBA students at atop 25 business school, February 1997). Based upon item analyses of Pretest 2, skill andchallenge items were further modified, and a list of 14 each were included in thequestionnaire. In addition, items were included for playfulness, dimensions underlying theeight channel flow model, and correlates of flow.

Pretest 4. (n=146, 78 item Web fillout form2 Project 2000 Pretest Panel, March 1997). Based uponitem analyses of Pretest 3, skill and challenge items were further modified, and reduced setsof 7 items for each were included in the questionnaire. In addition, items were included forplayfulness, dimensions underlying the eight channel flow model, and correlates of flow.

1 http://www2000.ogsm.vanderbilt.edu/cgi-bin/SurveyArchive/pretest.jan97.pl2 http://www2000.ogsm.vanderbilt.edu/cgi-bin/SurveyArchive/pretest.march97.pl

14

Final Survey.

The final survey consisted of 77 items, and was administered as a Web fillout form3 which was posted fromApril 10 to May 10, 1997 in conjunction with the 7th WWW User Survey fielded by the Graphic,Visualization, and Usability Center (GVU) at the Georgia Institute of Technology4. Respondents whoregistered to participate in the 7th WWW User Survey were given a unique identifying code, and werepresented with an online list of 13 different surveys, including our survey, which they could potentially fillout. With the exception of our survey, which was posted on Project 2000 Web server(www2000.ogsm.vanderbilt.edu), all WWW User Surveys were posted on the GVU server. However, arespondent’s unique identifying code was passed back and forth between the GVU and Project 2000servers, as a hidden field in the fillout form. Thus, depending upon the overlap in respondents who filledout each of the 13 surveys, the Project 2000 survey can be linked to the 12 other surveys fielded by GVU.In this paper, we combine our data with items from one of these 12 surveys, the “Information Gatheringand Purchasing Questionnaire5” developed by Sunil Gupta.

The GVU WWW User Survey employs non-probabilistic sampling and self-selection (GVU 1997), and isnot representative of the general population of Web users. Comparison with population projectable surveysof Web Usage (e.g. Hoffman, Kalsbeek and Novak 1997) shows the GVU User Survey sample to containmore long-term, sophisticated Web users than the general population. Participants were solicited byannouncements placed on Internet-related newsgroups, banner ads placed on specific pages on highexposure sites (e.g. Yahoo, Netscape, etc.), banner ads randomly rotated through high exposure sites (e.g.Webcrawler, etc.), announcements made to the www-surveying mailing list maintained by GVU, andannouncements made in the popular media.

19,970 respondents filled out at least one of the 13 surveys that comprised the 7th WWW User Survey, and4,550 filled out the Project 2000 survey. Of the 4,550 respondents, we retained 4,232 for analysispurposes, eliminating 318 respondents who:

• had more than 5 of 77 items missing (135 respondents)• had constant responses throughout (i.e. all 1's or all 7's, only 2 respondents)• did not have a valid GVU identifying code (17 respondents)• had strong evidence of responding randomly to survey items by either a) responding with all 5's to one

of two sets of semantic differential items, or b) by having large variability (top 5%) on summed ratingskills for flow, play, and challenge (164 respondents).

3 http://www2000.ogsm.vanderbilt.edu/gvusurvey/project2000.gvu.html4 http://www.gvu.gatech.edu/user_surveys/survey-1997-04/5 http://www.gvu.gatech.edu/user_surveys/survey-1997-04/questions/purchase.html

15

3) Scale Construction and Reliability

Tables 4, 5, and 6 provide coefficient alphas and item-total correlations for scales for skill, challenge andplay. All items in these Tables were measured on 9 point scales, and items whose scale values werereversed for the purpose of combining summed rating scales are noted. The items for the skill andchallenge scales were developed from the series of pretests, while the play items are from Webster andMartocchio’s (1992) scale of computer playfulness. Coefficient alphas are above .8 for all scales reportedin these Tables.

For skill and challenge, coefficient alpha was improved by reducing the seven item scales from 7 items to 5and 4 items, respectively. Although coefficient alpha was reduced slightly by eliminating two items fromthe 7 item play scale, a principal components analysis of the 7 play items produced two factors witheigenvalues greater than one. Playful and spontaneous were the two items which correlated the most withthe second rotated factor, and the least with the first rotated factor. When these two were eliminated, onlyone eigenvalue was greater than one for the remaining five items. For all full 7-item and reduced-item skilland challenge scales in Tables 4 and 5, only one eigenvalue greater than one was found. In subsequentanalyses reported in this paper, we use five item scales for skill and play, and a four item scale forchallenge.

Table 4: Scale construction for Skill

item-total correlations:

Skill item:7 itemscale

6 itemscale

5 itemscale

I am very skilled at using the Web. .708 .742 .736I consider myself knowledgeable about good search techniques on the Web. .786 .794 .802I know less about using the Web than most users. (reversed) .668 .694 .686I find the Web easy to use. .680 .650 .633I know how to find what I want with a search engine. .680 .645 .664Downloading software is easy for me to do on the Web. .567 .587It is hard for me to find information on the Web. (reversed) .428

coefficient alpha .865 .875 .875

Table 5: Scale construction for Challenge


Challenge item:7 itemscale

6 itemscale

5 itemscale

4 itemscale

Using the Web challenges me. .745 .760 .759 .772Using the Web challenges me to perform to the best of my ability. .728 .726 .708 .682Using the Web provides a good test of my skills. .742 .758 .768 .789I find that using the Web stretches my capabilities to the limits. .659 .672 .692 .695The Web provides many opportunities for action .564 .528 .506Using the Web makes me think. .546 .519The Web provides many possible things for me to do. .514

coefficient alpha .868 .865 .865 .876

16

Table 6: Scale construction for Play


Play item:7 itemscale

6 itemscale

5 itemscale

I feel unimaginative when I use the Web. (reversed) .549 .585 .611I feel flexible when I use the Web. .572 .550 .507I feel creative when I use the Web. .686 .652 .602I feel unoriginal when I use the Web. (reversed) .622 .645 .660I feel uninventive when I use the Web. (reversed) .604 .640 .670I feel spontaneous when I use the Web. .548 .495I feel playful when I use the Web. .454

coefficient alpha .829 .825 .818

Table 7 provides item-total correlations and coefficient alphas for the four constructs underlying the eight-channel flow model in Figure 4. These items were measured as nine-point semantic differential scales, andwere developed over the course of the four pretests. There are four scales, one for each of the followingbipolar constructs: flow/apathy; control/worry, arousal/relaxation, and anxiety/boredom. One item in eachof these pairs is simply a rephrasing of these adjective pairs (excited was used instead of “aroused” becauseof the possible sexual connotation of “aroused,” given the media publicity about the extent of pornographyon the Internet (Hoffman and Novak, 1995)).

With the exception of the arousal scale, and the three-item anxiety scale, coefficient alphas are above .6 andacceptable for all scales. In subsequent analyses, we use the two-item anxiety scale. For simplicity, in thefollowing discussion, we refer to bipolar scales (e.g. “Flow/Apathy”) by the first name of the pair (e.g.“Flow”).

17

Table 7: Scale construction for Flow, Control, Arousal, and Anxiety

Flow/Apathy items (alpha=.722):item-total

correlations:“in flow” vs. apathetic .606active vs. passive .524the Web challenges my capabilities to their limits vs. I don’t use the Web much and don’t care to

.523

Control/Worry items (alpha=.664):worried vs. in control (reversed) .482clearly know the right things to do vs. feel confused about what to do .442frustrated vs. not frustrated (reversed) .527

Arousal/Relaxation items (alpha=.541):calm vs. excited (reversed) .316stimulated vs. relaxed .422alert vs. soothed .307

Anxiety/Boredom items (alpha=.564):using the Web is boring vs. using the Web makes me anxious (reversed) .174I’ve lost interest in using the Web lately because I’m too skilled vs. I’d enjoy using the Web more if I were more skilled at it (reversed)

.546

when I encounter a problem using the Web, I get stuck because I don’t know what to do next, vs. the Web isn’t as challenging to me as it used to be

.472

Anxiety/Boredom items, reduced scale (alpha = .689)I’ve lost interest in using the Web lately because I’m too skilled vs. I’d enjoy using the Web more if I were more skilled at it (reversed)

.533

when I encounter a problem using the Web, I get stuck because I don’t know what to do next, vs. the Web isn’t as challenging to me as it used to be

.533

4) Congruence of Constructs With Eight-Channel Flow Structure

Figure 4 identifies the sum and difference of skills and challenges as central to the distinction betweenflow/apathy and the orthogonal construct of anxiety/boredom. Table 8 presents correlations of constructsunderlying the eight channel model in Figure 4. Sample sizes for the correlations range from 4088 to 4164,depending upon the pattern of missing data. Due to the large sample sizes, all correlations in Figure 4 aresignificantly different from zero at p<.05, so that the magnitude of the correlation provides a more useablebasis of comparison than statistical significance. Based upon Figure 4, and assuming that Play should be adirect outcome of Flow, we predict that Skill and Challenge should not be correlated with each other, andthat:

• Skill + Challenge should have a strong positive correlation with Play and Flow, a positive correlationwith Control and Arousal, and not be correlated with Anxiety.

• Skill - Challenge should have a strong negative correlation with Anxiety, a negative correlation with

Arousal, a positive correlation with Control, and not be correlated with Play and Flow. • Skill should have a strong positive correlation with Control, a positive correlation with Flow and Play,

a negative correlation with Anxiety, and not be correlated with Arousal.• Challenge should have a strong positive correlation with Arousal, a positive correlation with Play,

Flow, and Anxiety, and not be correlated with Control.

18

With some exceptions, the correlations in Table 8 correspond to these predictions. Most problematic arecorrelations with Arousal; however Arousal is the least reliably measured construct. The relationship ofChallenge to Play and Flow is also stronger than that of Skill to Play and Flow contrary to our expectations.

Table 8: Correlations Among Summed Rating Scales

Skill + Challenge Skill - Challenge Skill ChallengeSummed Rating Scale: r: predicted: r: predicted: r: predicted: r: predicted:

Skill + Challenge 1.000 -.228 .586 .766Skill - Challenge -.228 1.000 .655 -.801Skill .586 .655 1.000 -.072 0Challenge .766 -.801 -.072 0 1.000

Play .657 ++ -.190 0 .355 + .528 +Flow .607 ++ -.384 0 .155 + .626 +Control .508 + .271 + .620 ++ .135 0Arousal .271 + -.292 - -.028 0 .361 ++Anxiety -.126 0 -.623 -- -.656 - .269 +

A Principal Components Analysis of the constructs in Table 8 yielded two factors with eigenvalues greaterthan one (72.9% variance explained). The loadings of constructs with these two factors are graphicallyrepresented in Figure 6. For the most part, the vectors for the constructs correspond very closely to thepositions predicted by the eight-channel flow model in Figure 4. We note from Figure 6 (and can observein Table 8) that Play appears closer to the sum of Skills and Challenges than does Flow.

challengearousal

anxiety

skill + challenge

flow

play

control

skill

skill - challenge

Figure 6 - PCA of Constructs from the Eight Channel Flow Model

5) Two-Dimensional Model of Flow

19

The distinction between the four-channel model of flow in Figure 3 and the eight-channel model of flow inFigure 4 is somewhat arbitrary, in the sense that it is largely a decision of how finely one chooses tosegment respondents based upon the sum and difference of skills and challenges. The PCA solution inFigure 6 supports a two-dimensional model of flow that underlies both the four and eight channel models offlow. Depending upon one’s perspective, one could select Skill and Challenge as orthogonal axes (fourchannel model) or Skill+Challenge and Skill-Challenge as orthogonal axes (eight channel model).

Graphical Descriptive Analysis of Skill and Challenge

In this section, we perform a series of graphical descriptive analyses to further investigate the relationshipbetween skills and challenges of using the Web. Figure 7 plots Skill vs. Challenge. This plot, and theremaining plots in this section, were produced with XGobi6 (e.g. Buja, Cook and Swayne 1996; Swayne,Cook & Buja 1997), an interactive data visualization system for multivariate data. The summedcomposites for Skill and Challenge are divided by the sum of items in the scale, so that the compositescores range from 1 to 9. While Challenge is distributed across the range of the scale, Skill is skewedtoward the upper end of the scale.

Figure 7 represents over 4200 observations. Because there will be considerable overprinting with so manyobservations, the jittering facility of XGobi was used to interactively add small amounts of uniform randomerror to the points so that the density of points in the plot can be better observed. The smooth facility wasalso used, and Figure 7 shows smoothed trend lines for each of five segments defined by how long therespondent has used the Web. We can see there is no correlation between skill and challenge, as shown inTable 8, but that as we might expect, skill is greater for those who have used the Web for longer periods oftime.

Figure 7- Skill vs. Challenge by Time Used Web

Figure 8 plots the sum of skills and challenges against the difference. Again, the plot is jittered andsmoothed trend lines are shown for the length of time the person has used the Web. The trend lines becomeincreasingly U-shaped as the person’s experience with the Web increases. This implies that for long termusers (e.g. 3+ years), individuals with intermediate levels of skills plus challenges tend to have higher skills

6 http://www.research.att.com/~andreas/xgobi/index.html

20

than challenges (the “boredom” flow channel). For recent users (e.g. <6 months), those with intermediatelevels of skills plus challenges will have either higher or lower skills than challenges.

Figure 8 - Skill-Challenge vs. Skill+Challenge by Time Used Web

Figure 9 plots the sample distribution of segments from the four channel flow model. The triangular areasat the extreme corners of the plot show the members of the four segments (flow, anxiety, apathy, andboredom) which are the most clearly defined, and who would be expected to have the sharpest among-group differences. The triangular areas within the square at the middle of the plot show less-well definedmembers of these four segments, whose among-group differences would be expected to be less clearlydefined. Clearly, the majority of respondents in this sample fall into the flow and boredom segments,which is not surprising, given the relatively high level of skill in this sample.

Figure 10 shows side-by-side dot plots for the extreme and middle members of each of the four segments:bored, flow, anxiety, and apathy. The points for the extreme segments are represented as darker points onthe left half of each panel The distribution for each of the four segments (extreme members on the left,middle members on the right side of the plot) is shown for the composite variables play, anxiety, control,and arousal. Results agree with patterns that would be predicted by Figure 4. In the first panel, we see theflow segment (as defined by skill plus challenge) is clearly highest on play, and the apathy segment thelowest. (We use play rather than the flow/apathy composite because of the superior representation of playin Figure 6). In the second panel, the anxiety segment (as defined by skill less than challenge) is higheston the anxiety composite, while the bored segment is lowest. The third panel shows the bored and flowsegments (both with high skill) as highest on the control composite. And, the fourth panel shows the flowand anxiety segments (both high on challenge) as highest on the arousal composite. In all cases, patternsare clearer for the extreme members of a segment than for those less well-defined.

21

Boredom (S>C )

Anxiety (S<C )

Flowhigh (S+C )

Apathy(low S +C )

Figure 9 - Sample Distribution of Four Channel Model Segments

Figure 10 - Side-by-Side Dot Plots

22

Demographic and Web Usage Differences for Skill and Challenge

Figure 11 represents the distribution shown in Figure 9 separately for women and men, for eachcombination of two variables: when the respondent started using the Web (vertical) and how many hoursper week the respondent uses the Web. This conditional plot was generated in XGobi by manually rotatingthe horizontal axis in Figure 9 (skill plus challenge) into a single dimensional space along with projectionsof discrete variables for gender and hours/week use the Web, and rotating the vertical axis into the singledimensional space along with the projection of the discrete variable of when the respondent started usingthe Web (Buja, Cook, and Swayne 1996). The rotations result in combined projections of continuous anddiscrete variables and serve to break the plot in Figure 9 into a matrix of separate plots for eachcombination of the three discrete variables.

Figure 11 - Flow/Apathy vs. Anxiety/Boredom by Segments

Using the distribution in Figure 9 as a reference template, we can see how the distribution of apathy/flow(left-right direction) vs. anxiety/boredom (down-up direction) differs across combinations of gender,hours/day, and when the respondent started using the Web. A few general observations:

• men tend to be more likely to be at the “top-left-edge” of the distributions (in the flow and boredomsegments) than women;

• long term (two year or more) Web users who use the Web 20 hours/week or more are almost

exclusively in either the flow or boredom segments;

23

• there is considerable heterogeneity in skill and challenge level for recent (one year or less) Web users,regardless of how many hours/week they use the Web.

For a more formal analysis of the relationship of skill and challenge to these three variables plus additionaldemographics, we performed a MANOVA, predicting the sum of skill and challenge and the difference ofskill and challenge from a series of demographic variables. Tests of two-way interactions identified aMANOVA model with the following significant main effects and interactions:

COMPJOB 2 categories of employment in the computer field (yes/no)AGE 7 age categoriesSTARTUSE 5 categories of when started to use the WebTIMEUSE 5 categories of hours/day use the WebGENDER 2 categories of men/women

GENDER*TIMEUSE interactionCOMPJOB*AGE interaction

To obtain a map of the main effects and interactions (Novak 1995), we obtained the two uncorrelatedprincipal components of the sum and the difference of skill and challenge, and group means on theseprincipal components for main effects and two-way interactions were plotted in Figure 12. If a main effectwas involved in an interaction (i.e. GENDER, TIMEUSE, COMPJOB, AGE) group centroids were plottedonly for the two-way combinations of factors (i.e. GENDER*TIMEUSE and COMPJOB*AGE). Inaddition, correlations of skill, challenge, and the sum and difference of skill and challenge with the twoprincipal components were obtained and used as direction cosines to plot vectors for these four constructsin Figure 12.

Time Use (women)

Time Use (men)

FLOW (S+C)

BORED (S-C)

SKILL

CHALLENGE

Age (Computer Job)

Age (Non-Computer Job)

Started Using Web

< 6 months

3+ years

>40 hours

1-5 hours

1-5 hours

>40 hours

<18 years

<18 years

55-64 years

55-64 years

65+ years

Figure 12 - Centroids for Demographic and Usage VariablesWe can briefly interpret the pattern of group centroids as follows:

24

• Skill and the sum of skill and challenge (flow) increases with the number of hours the respondent usesthe Web per day. This increase is greater for women than it is for men (interaction of gender with timeuse the Web).

• Women find using the Web to be more challenging than men, regardless of how many hours/day they

use the Web.

• Skill and the difference between skill and challenge (boredom) increases with the length of time therespondent has used the Web.

• Skill and the difference between skill and challenge (boredom) increases for younger respondents.

While there is a significant interaction of age with whether the respondent is in a computer-relatedoccupation, there is no clear interpretation of the different pattern of age centroids for responding withor without a computer-related occupation. (Note that due to small sample size, the centroid forrespondents 65+ in a computer-related occupation is not plotted).

• Skill and the sum of skill and challenge (flow) is higher for respondents in a computer-related

occupation.

6) Flow and Consumer Search and Purchase Behavior

We have established that the sum and difference between a respondent’s skills and challenges for using theWeb are consistent with constructs underlying the four and eight channel flow models, and that there aredemographic and Web usage differences along these dimensions. In this section, we investigate therelationship of flow/apathy, operationalized as the sum of skills and challenges, and anxiety/boredom,operationalized as the difference of skills and challenges, on consumer behavior in online environments andtraditional marketing channels. We hypothesize that flow should relate to increased likelihood ofconsumer search and purchase in online environments, over and above the length of time the respondenthas been a user of the Web.

We now investigate the subsample of 2364 of our 4232 respondents who answered both the Project 2000survey and GVU’s “Information Gathering and Purchasing” (IGP) questionnaire. The IGP consisted of aseries of checklist items for consumer search and purchase activity for 23 product categories7 in differentdistribution channels: online search/purchase; newspaper/magazine search and retail store purchase; directmail search and catalog purchase. The product categories fall into three broad groups, and are shown inTable 9 - A) computer products; B) entertainment products; C) other products. For search, respondentswere asked to check whether the consulted a particular information search for each of the list of productcategories. For purchase, respondents were asked to check whether they had purchased from the productcategory within the past six months.

Hypotheses

We specify a series of hypotheses regarding the relationship of flow/apathy and anxiety/boredom to searchand purchase behavior:

7 Due to a programming error, GVU was not able to capture the purchase responses for the last six productcategories for each of the three distribution channels; thus, for each distribution channel there are 23product categories for search, but only 17 product categories for purchase.

25

H1: Flow, defined as Skill + Challenge, (a) will positively relate to Online Search and Purchase for allproduct categories; (b) will not relate to Newspaper/Magazine Search, Direct Mail Search, RetailStore Purchase, or Catalog Purchase for non-computer categories; and (c) will positively relate tocomputer-related categories for non-online media.

H2: The length of time the respondent has used the Web (Startuse) (a) will positively relate to OnlineSearch and Purchase for all product categories; and (b) will not relate to Newspaper/MagazineSearch, Direct Mail Search, Retail Store Purchase, or Catalog Purchase for all product categories.

H3: Boredom /Anxiety, defined as Skill - Challenge, (a) will not relate to Online Search and Purchase;(b) but will negatively relate to Newspaper/Magazine Search, Direct Mail Search, Retail StorePurchase, and Catalog Purchase for all product categories.

Results

Tables 9 through 11 present the results of a series of logistic regressions predicting binary variables forsearch or purchase for each product category, for different marketing channels. The Tables presentstandardized parameter estimates for each logistic regression, from a model which includes main effects ofa) skill plus challenge; b) skill minus challenge; and c) length of time the respondent has used the Web.

Interactions between the length of time the respondent has used the Web and both a) skill plus challengeand b) skill minus challenge were tested in a separate series of logistic regressions by including threedummy variables specifying the skill+challenge parameter for those using the Web 1-2 years, 2-3 years,and 3 or more years, and by including three dummy variables specifying the skill-challenge parameter forthose using the Web 1-2 years, 2-3 years, and 3 or more years. For the 40 models fit in Table 9, 19 of 240(40 models times 6 dummy variables) interaction parameters were significant at alpha = .05 (7.9%); forTable 10, 12 of 240 interaction parameters were significant (5%); and for Table 11, 5 of 240 (2.1%) weresignificant. Thus, except for Table 9, the number of significant interaction parameters was less than orequal to the number expected by chance. As there was no consistent clear pattern to the significantinteraction parameters in Table 9, only main effect models were fit for all three Tables.

26

Table 9 - Logistic

Table 9X - Logistic Regressions for Online Search and Online Purchase

Odds Ratios for...Online Search: Online Purchase:

Product: Skill Chall. Startuse Skill Chall. Startuse

Group A:computer hardware < $50 1.303*** 1.095** 1.368*** 1.347*** 1.075** 1.406***computer hardware > $50 1.368*** 1.051 1.607*** 1.289*** 1.073* 1.495***computer software < $50 1.268*** 1.095** 1.283*** 1.230*** 1.065* 1.260***computer software > $50 1.344*** 1.085* 1.417*** 1.251*** 1.077* 1.374***other Web services 1.202*** 1.180*** 1.072 1.203*** 1.181*** 1.108*

Group B:home electronics < $50 1.141** 1.090** 1.291*** 1.259* 1.167** 1.428**home electronics > $50 1.173*** 1.078** 1.356*** 1.244* 1.220*** 1.454***videos/movies 1.151*** 1.085** 1.184*** 1.145* 1.219*** 1.257*CDs/tapes/albums 1.221*** 1.058* 1.182*** 1.268*** 1.096** 1.205**books/magazines 1.279*** 1.090** 1.154* 1.198*** 1.107*** 1.301***concerts/plays 1.183*** 1.079** 1.187** 1.189* 1.093* 1.157

Group C:legal services 1.085 1.169*** 1.088 .975 1.256* 1.052food/condiments/beverages 1.102* 1.131*** .977 .977 1.228** 1.331*investment choices 1.104* 1.088** 1.264*** 1.066 1.076* 1.353***travel arrangements 1.117** 1.110*** 1.228*** 1.115* 1.113*** 1.339***apparel/clothing/shoes 1.069 1.147*** 1.145* 1.196* 1.096* 1.043sunglasses/personal items 1.144* 1.218*** 1.103 .903 1.212* 1.311*real estate 1.084 1.115*** 1.143* .insurance services 1.064 1.157*** 1.209*automobiles/motorcycles 1.112* 1.049* 1.135* NOT ASKEDjewelry/watches 1.085 1.211*** 1.125household appliances 1.003 1.119*** 1.365***banking/financial services 1.139** 1.084** 1.237***

*** p<.0001; **p<.001; *p<.05

27

Table 10X - Logistic Regressions for Newspaper/Magazine Search and Retail Store Purchase

Odds Ratios for:Newspaper/Magazine Search: Retail Store Purchase:


Group A:computer hardware < $50 .987 1.025 1.056 1.181*** 1.030 1.269***computer hardware > $50 1.060 .942* 1.142* 1.115* 1.023 1.295***computer software < $50 1.035 1.006 .946 1.097* 1.080** 1.115*computer software > $50 1.028 .989 1.069 1.099* 1.035 1.257***other Web services 1.046 1.153*** .946 1.014 1.235*** 1.010

Group B:home electronics < $50 1.024 1.062* .987 1.092 1.127*** 1.128*home electronics > $50 1.041 1.002 1.046 1.112* 1.065* 1.111videos/movies 1.041 1.108 .967 .970 1.133*** 1.017CDs/tapes/albums 1.013 1.106 1.005 1.046 1.106*** .940books/magazines .989 1.021 1.035 .933 1.126*** .990concerts/plays 1.010 .975 1.058 1.035 1.081 .978

Group C:legal services 1.058 1.099** .977 .974 1.348** .778food/condiments/beverages .935* 1.087** .956 .988 1.143** .967investment choices 1.015 1.016 1.101* 1.012 1.202* 1.115travel arrangements .960 1.045* 1.047 .905 1.251*** 1.059apparel/clothing/shoes .895* 1.105*** .995 .937 1.110*** .994sunglasses/personal items .923* 1.148*** 1.001 .938 1.202*** .985real estate .936* 1.093*** 1.080insurance services .935 1.133*** 1.066automobiles/motorcycles .959 1.030 1.012 NOT ASKEDjewelry/watches .944 1.158*** .914household appliances .906* 1.095*** 1.022banking/financial services .968 1.071* 1.123*

*** p<.0001; **p<.001; *p<.05

28

Table 11X - Logistic Regressions for Direct Mail Search and Catalog Purchase

Odds Ratios for:Direct Mail Search: Catalog Purchase:


Group A:computer hardware < $50 1.104* 1.054* 1.022 1.048 1.011 1.088computer hardware > $50 1.056 1.045 1.037 1.014 1.042 .974computer software < $50 1.005 1.102*** 1.034 1.053 1.052* .918*computer software > $50 1.029 1.086** 1.069 1.042 1.077* .997other Web services .933 1.248*** 1.051 1.051 1.212*** .892

Group B:home electronics < $50 .980 1.095** .954 1.067 1.030 1.096*home electronics > $50 .995 1.069* .971 1.062 1.055* 1.032videos/movies .965 1.159*** .885* 1.033 1.003 .967CDs/tapes/albums 1.004 1.154*** .908* 1.167* .939* .971books/magazines .930 1.164*** .989 1.149* .955 1.005concerts/plays .964 1.119* .992 1.068 .956* 1.125*

Group C:legal services .884* 1.206** .898 1.134* 1.111* .990food/condiments/beverages .920 1.089* .915 .914 .942* .985investment choices .942 1.139*** .934 .961 1.047 1.084travel arrangements .907* 1.209*** 1.003 1.020 .990 1.136*apparel/clothing/shoes .911* 1.122*** .999 1.009 1.009 .873*sunglasses/personal items .898* 1.257*** .972 .972 1.036 .886*real estate .877* 1.254*** .929insurance services .875* 1.185*** .965automobiles/motorcycles .912 1.176*** .946 NOT ASKEDjewelry/watches .925 1.247*** .837*household appliances .898* 1.212*** .959banking/financial services .905* 1.157*** 1.010

*** p<.0001; **p<.001; * p<.05

29

We summarize the results of Tables 9-11 in the context of our six Hypotheses:

H1: Flow, defined as Skill + Challenge, (a) will positively relate to Online Search and Purchase for allproduct categories; (b) will not relate to Newspaper/Magazine Search, Direct Mail Search, RetailStore Purchase, or Catalog Purchase for non-computer categories; and (c) will positively relateto computer-related categories for non-online media.

The diagram below summarizes the significance tests in Tables 9 through 11 for the Skill + Challengeparameter in the logistic regressions. The shaded area indicates parameter estimates which are predicted tobe positive by Hypothesis 1. The non-shaded area should contain non-significant parameter estimates.Positive and negative parameters where p<.05 are indicated by + or minus signs. Thus, seven of theparameter estimates for Online Search for the Computer category are positive and significant.

Hypothesis One:

Channel: (A)

Computer (B)

Entertainment (C)

Other Online Search + + + + + + + + + + + + + + + + + + + + + + Online Purchase + + + + + + + + + + + + + Newspaper/Magazine Search + + Retail Purchase + + + + + + + - Direct Mail Search + + + + + + + + Catalog Purchase + + + + + +

Hypotheses 1a and 1b are supported. Hypothesis 1c is supported, with the exception ofNewspaper/Magazine Search for computer products and services. One explanation is that online marketingefforts may be more integrated with retail and catalog sales and with direct mail search than withnewspaper and magazine search.

H2: The length of time the respondent has used the Web (Startuse) (a) will positively relate to OnlineSearch and Purchase for all product categories; and (b) will not relate to Newspaper/MagazineSearch, Direct Mail Search, Retail Store Purchase, or Catalog Purchase for all productcategories.

In the diagram below, the shaded area corresponds to the parameter estimates which are predicted to bepositive by Hypothesis 2, while the non-shaded area should contain nonsignificant parameter estimates.

Hypothesis Two:

Channel: (A)

Computer (B)

Entertainment (C)

Other Online Search + + + + + + + + + + + + + + + + + + + Online Purchase + + + + + + + + + + + + + + Newspaper/Magazine Search + + + Retail Purchase + - + + - - Direct Mail Search - - - Catalog Purchase + + + +

Hypothesis 2a is supported. Hypothesis 2b is supported, with the exception of catalog purchase ofcomputer-related products. It appears that long-term Web users are more likely to buy computer-relatedproducts and services from catalogs.

30

H3: Boredom vs. Anxiety, defined as Skill - Challenge, (a) will not relate to Online Search andPurchase; (b) but will negatively relate to Newspaper/Magazine Search, Direct Mail Search,Retail Store Purchase, and Catalog Purchase for all product categories.

In the diagram below, the shaded area corresponds to the parameter estimates which are predicted to benegative by Hypothesis 3, while the non-shaded area should contain nonsignificant parameter estimates.

Hypothesis 3:

Channel: (A)

Computer (B)

Entertainment (C)

Other Online Search + + + + + + - Online Purchase + + + + + - - Newspaper/Magazine Search + - - - - - - - - - - Retail Purchase + + + Direct Mail Search - - - - - - - - - - - - - - - - - - - Catalog Purchase + - - - - - -

Hypothesis 3a is largely supported, with the exception of Online Search and Online Purchase of computer-related products and services. Individuals whose skills exceed challenges (the boredom condition) aremore likely to have searched and purchased online for computer goods. This is intuitive, if the peoplewhose skills for using the Web are greater than the perceived challenges are technophiles who are likely touse the Web for utilitarian purposes to efficiently search and purchase computer hardware and softwareonline. Hypotheses 3b is supported for some channels but not for others. Respondents who find the Webto be more challenging than their skill level are more likely to search for all three groups of productsthrough direct mail, and are more likely to purchase entertainment and other products through catalogs. Inaddition, they are more likely to search for other products through newspapers and magazines. We alsonote that there are more significant negative relationships of skill - challenge with search in traditionalmedia (23 of 32 coefficients) than in purchase (5 of 20 coefficients). That is, respondents who find theWeb to be too challenging are more likely to be those who will search for product information in traditionalmedia, than they are to buy from traditional media.

To obtain a broader overview of the patterns in Tables 9-11, we performed a principal components analysisof the standardized regression coefficients for S+C, S-C, and Startuse (three variables) for each of thelogistic regression models run (120 observations). Figure 13 presents the plot of average scores for 18 cellsdefined by 1) search vs. purchase, 2) three levels of media type, and 3) three groups of product categories,on the first two principal components (which explain 86.8% of the variance), Vectors for S+C, S-C, andStartuse are drawn using the component loadings as direction cosines. We recognize that this is purely adescriptive and ad-hoc analysis, but it provides a useful organization of the results.

31

Skill - Challenge

Skill + Challenge (Flow)

Time Used Web

Online Search A

Online Search B

Online Search C

Online Buy A

Online Buy B

Online Buy C

Print Search APrint Search A

Print Search CPrint Search A

Print Search BPrint Search A

Retail Buy CPrint Search A

Retail Buy BPrint Search A

Retail Buy A

Mail Search A

Mail Search BMail Search C

Catalog Buy B

Catalog Buy CCatalog Buy A

A: computers, electronicsB: books, CD's, videos, concertsC: other products

S kill < Challenge relates to

Flow and long-term usage relate to

no strong relationships

S kill > Challengerelates to

Figure 13 - PCA of Logistic Regression Parameters, Subgroup Centroids

Three patterns of observed relationships are indicated in Figure 13. First, flow and long term usage relatesto online search and purchase, for each of the three groups of product categories. Respondents who areskilled at the using the Web and find it challenging, as well as longer-term users, are more likely to searchfor and buy products online. Second, there is a positive relationship of the difference between skill andchallenge with online search and purchase of computer-related categories. In addition, those whose skillsexceed challenge are more likely to search for entertainment online and buy it in retail stores. Thussuggests a practical, skilled and technologically-oriented person who sees the Web as a tool to buy andsearch for computer-related goods and services, and also to search for entertainment information online butthen buy in traditional retail outlets. The third pattern is a negative relationship of the difference betweenskill and challenge with search and purchase of non-computer product categories in traditional media.These are individuals who find the Web more challenging than their skill level, and they turn to traditionalmedia for search and purchase.

7) Discussion and Conclusion

In this paper, we have described an empirical study in which constructs were developed and validated thatcorrespond to the underlying structure of flow channel segmentation models: skill, challenge, flow, play,control, arousal, and anxiety. The correlational structure for these constructs that we found in ourempirical data was consistent with a theoretically predicted structure. Adopting an operational definition offlow as the sum of skills and challenges, and an operational definition of anxiety/boredom as the differenceof skills and challenges, we combined these two constructs to form a simple two-dimensional model offlow, and investigated demographic and Web usage differences on the sum and difference of skills andchallenges for using the Web. Considerable attention was paid to examining the degree to which consumersearch and purchase behavior in online and traditional marketing environments could be predicted by a

32

respondent’s skill and challenge at using the Web. Our hypotheses regarding the relationships of skill andchallenge to search and purchase behavior were largely supported.

The results suggest a number of issues for discussion and subsequent research. While we have focusedupon skills and challenges, our present research largely ignores the precise relationship of these constructsto other constructs shown in Figure 1. Instead, we have focused upon flow channel segmentation models,as in Figures 2 through 4, which are defined by an two-dimensional basis spanned by skill and challenge.Work in progress (Novak, Hoffman and Yung 1997) seeks to use structural equation models to test theconceptual model proposed by Hoffman and Novak (1996), fitting skill, challenge, flow, and play into abroader set of constructs, including interactivity, focused attention, arousal, control, telepresence,exploratory behavior, and positive affect.

By focusing upon a continuous, rather than discrete, formulation of skill, challenge, and their sum anddifference, we have moved away from the idea of developing market segments based upon the flowconstruct, while at the same time using constructs which can be used to form such market segments. Withthe exception of the analysis in Figure 10, we chose to perform continuous-level graphical analyses,MANOVA, and logistic regressions, rather than categorize respondents into market segments for a numberof reasons. First, inspection of Figure 9 reveals there is no clear clustering tendency. We can formsegments based upon cutoffs of skill and challenge, but discretizing the data into segments would attenuatethe relationships we observed in our logistic regressions.

Second, we did attempt to form market segments corresponding to the four-channel flow model in Figure 3,both by dividing respondents into four quadrants at the median of skill and challenge, and at the absolutemidpoint of the skill and challenge composite scales. In both cases, we found that skill level in each of thefour segments increased with the length of time the respondent used the Web. This made a segment-levelanalysis difficult, because segments were confounded with length of time the respondent used a Web.While this could have been handled by including time using the Web as a covariate, we felt that it wasproblematic that the definition of an individual who was “in flow” differed according to time using theWeb, in that the long-term users in the flow segment were significantly more skilled than the recent users.

Third, the results in Tables 9 through 11 suggest that individuals who find the Web to be too challengingare likely to turn to traditional media for their search activity, and somewhat less so for their purchaseactivity. This suggests that it would be very helpful to develop a conceptualization of ways in which searchand purchase behavior on the Web is similar to and different from search and purchase behavior intraditional media, so that predictions could be made of which combinations of product categories andmedia types will be most likely to be seen as substitutes for the Web by individuals who find their skillsgreater than the challenges offered by the Web, and by individuals who find the challenges of the Web tobe greater than their skills.

Fourth, our analysis is based upon a very general respondent evaluation of their skill at using the Web andtheir perception of the challenge offered by the Web. We would expect there to be considerable differencesamong Web sites, in terms of a respondent’s perception of the skill/challenge for that specific site.Additionally, there may be different perceptions of skill/challenge for using the Web for different usageoccasions (e.g., experiential vs. goal-directed use). It is also not clear whether our measurement of flow asthe sum of skill and challenging is capturing a respondent trait or a respondent state. Hoffman and Novak(1996) discussed the autotelic personality (Csikszentmihalyi 1977, pp. 21-22) as an individual trait whichaccounted for consumer heterogeneity in their ability to experience flow in a computer-mediatedenvironment.

We would also like to mention some caveats and limitations to this research. The sample we used is non-representative, and is biased toward long-term sophisticated Web users, as compared to the generalpopulation of Web users. We can see from Figure 11 that there is greater heterogeneity in skill andchallenge levels among recent Web users than there is for long-term Web users. In subsequent research,such as research which will fit structural equation models to validate and refine conceptual models such asFigure 1, it will be important to determine whether the structural relationships among model constructsdiffer between recent and long-term Web users. Another limitation was the technical difficulty affecting

33

Tables 9 through11, where a programming error in the 7th WWW Survey resulted in data for six productcategories being unavailable for questions on purchase. Also, since the Information Gathering andPurchase Questionnaire was not written by us, we were not able to phrase the online search and purchasequestions as “Web search and purchase,” which might have yielded somewhat stronger relationships toskill and challenge of using the Web. Finally, certain constructs, specifically Arousal/Relaxation, needrefinement to increase their reliability. And, our three item bipolar scale for Flow/Apathy does not relate toskill and challenge as strongly as our Play scale; further development of this scale is also recommended.

In conclusion, we have provided evidence that skill and challenge for using the Web can be reliablymeasured, and that these measures correlated as anticipated with previous theoretical frameworks of flow.We have further demonstrated a significant predictive ability of flow, and its orthogonal construct,anxiety/boredom, for consumer search and purchase behavior in online and traditional media environments.

34

References

Buja, Andreas, Dianne Cook, and Deborah F. Swayne (1996), “Interactive High-Dimensional DataVisualization,” Journal of computational and Graphical Statistics, 5(1), 78-99.

Clarke, Sharon G. and John T. Haworth (1994), "'Flow' Experience in the Daily Lives of Sixth-FormCollect Students," British Journal of Psychology, 85, 511-523.

Csikszentmihalyi, Mihaly (1977), Beyond Boredom and Anxiety, second printing. San Francisco:Jossey-Bass.

Csikszentmihalyi, Mihaly (1990), Flow: The Psychology of Optimal Experience,New York: Harper andRow.

Csikszentmihalyi, Mihaly and Isabella Csikszentmihalyi (1988), "Introduction to Part IV" in OptimalExperience: Psychological Studies of Flow in Consciousness, Mihaly Csikszentmihalyi andIsabella Selega Csikszentmihalyi, eds., Cambridge, Cambridge University Press.

Csikszentmihalyi, Mihaly and Judith LeFevre (1989), "Optimal Experience in Work and Leisure," Journalof Personality and Social Psychology,56 (5), 815-822.

Ellis, Gary D., Judith E. Voelkl, and Catherine Morris (1994), "Measurement and Analysis Issues withExplanation of Variance in Daily Experience Using the Flow Model," Journal of LeisureResearch,26(4), 337-356.

Ghani, Jawaid A. and Satish P. Deshpande (1994), "Task Characteristics and the Experience of OptimalFlow in Human-Computer Interaction," The Journal of Psychology, 128(4), 381-391.

Ghani, Jawaid A., Roberta Supnick, & Pamela Rooney (1991), "The Experience of Flow in Computer-Mediated and in Face-to-Face Groups," Proceedings of the Twelfth International Conference onInformation Systems, DeGross, J.I, I. Benbasat, G. DeSanctis, and C. M. Beath, Eds., New York,New York, December 16-18.

GVU (1997), “GVU's 7th WWW User Survey,” http://www.gvu.gatech.edu/user_surveys/survey-1997-04/.

Hoffman, D.L. and T.P. Novak (1995), “A Detailed Analysis of the Conceptual, Logical andMethodological Flaws in the Article: "Marketing Pornography on the Information Superhighway,"July 2. [www2000.ogsm.vanderbilt.edu/novak/rimm.review.html]

Hoffman, Donna L. and Thomas P. Novak (1996), "Marketing in Hypermedia Computer-MediatedEnvironments: Conceptual Foundations," Journal of Marketing, 60 (July), 50-68.

Hoffman, D.L., W.D. Kalsbeek and T.P. Novak (1996), "Internet and Web Use in the United States:Baselines for Commercial Development," Special Section on "Internet in the Home,"Communications of the ACM, 39 (December), 36-46.

LeFevre. Judith (1988), "Flow and the Quality of Experience During Work and Leisure." In M.Csikszentmihalyi and I. Csikszentmihalyi (Eds.), Optimal Experience: Psychological Studies ofFlow in Consciousness, New York: Cambridge University Press.

Lutz, Richard J. and Michael Guiry (1994), "Intense Consumption Experiences: Peaks, Performances, andFlows," presented at the Winter Marketing Educators' Conference, St. Petersburg, FL, February.

Mannel l, Roger C., Jiri Zuzanek, and Reed Larson (1988), "Leisure States and 'Flow' Experiences: TestingPerceived Freedom and Intrinsic Motivation Hypotheses," Journal of Leisure Research, 20 (4),289- 304.

Massimini, Fausto and Massimo Carli (1988), "The Systematic Assessment of Flow in Daily Experience."

35

In M. Csikszentmihalyi and I. Csikszentmihalyi (Eds.), Optimal Experience: PsychologicalStudies of Flow in Consciousness, (pp. 288-306), New York: Cambridge University Press.

Novak, T.P. (1995), "MANOVAMAP: Graphical Representation of MANOVA in Marketing Research,"Journal of Marketing Research, 32(3), 357-374.

Novak, Hoffman and Yung (1997), “The Flow Experience Among Web Users: Measurement and StructuralModels,” work in progress.

Privette, Gayle and Charles M. Bundrick (1987), "Measurement of Experience: Construct and ContentValidity of the Experience Questionnaire," Perceptual and Motor Skills, 65, 315-332.

Swayne, Deborah F., Dianne Cook, and Andreas Buja (1997), “XGobi: Interactive Dynamic DataVisualization in the X Window System,” working paper, AT&T Labs.[http://www.research.att.com/~andreas/xgobi/]

Trevino, Linda Klebe and Jane Webster (1992), "Flow in Computer-Mediated Communication,"Communication Research, 19(5), 539-573.

Webster, Jane and Joseph J. Martocchio (1992), "Microcomputer Playfulness: Development of a MeasureWith Workplace Implications," MIS Quarterly, 16 (June), 201-226.

Webster, J., L.K. Trevino, L. Ryan (1993), "The Dimensionality and Correlates of Flow in HumanComputer Interactions," Computers in Human Behavior, 9(4), Winter, 411-426.

Measuring the Flow Experience Among Web Users · Measuring the Flow Experience Among Web Users...

Documents

Transcript of Measuring the Flow Experience Among Web Users · Measuring the Flow Experience Among Web Users...