Inducing compensatory information processing through decision aids that facilitate effort reduction:...

Inducing Compensatory InformationProcessing through Decision Aidsthat Facilitate E�ort Reduction:An Experimental Assessment

PETER TODD1* and IZAK BENBASAT2

1University of Houston, USA2University of British Columbia, Canada

ABSTRACT

This paper examines the role of computer-based decision aids in reducing cogni-tive e�ort and therefore in¯uencing strategy selection. It extends and complementsthe work reported in the behavioral decision theory literature on the role of e�ortand accuracy in choice tasks. The central proposition of the research is that if adecision aid makes a strategy that should lead to a more accurate outcome at leastas easy to employ as a simpler, but less accurate, heuristic, then the use of adecision aid should induce that more accurate strategy and as a consequenceimprove decision quality. Otherwise, a decision aid may only in¯uence decision-making e�ciency. This occurs because decision makers use a decision aid in sucha way as to minimize their overall level of e�ort expenditure. Results from alaboratory experiment support this proposition. When a more accurate normativestrategy is made less e�ortful to use, it is used. This result is consistent with the®ndings of our prior studies, but more clearly demonstrates that decision aids caninduce the use of normatively oriented strategies. The key to inducing thesestrategies is to make the normative strategy easier to execute than competingalternative strategies. Copyright # 2000 John Wiley & Sons, Ltd.

KEY WORDS decision support; decision strategy; e�ort±accuracy trade-o�s

Keen (1979) argued that the relationship of decision support systems to decision quality was largelyadvocated as an article of faith. He also asserted that it is important to understand the role e�ort playsin determining the net bene®t of decision support system use. This paper examines the moderating role

CCC 0894±3257/2000/010091±16$17.50Copyright # 2000 John Wiley & Sons, Ltd.

Journal of Behavioral Decision MakingJ. Behav. Dec. Making, 13: 91±106 (2000)

* Correspondence to: Peter Todd, College of Business Administration, University of Houston, Houston, TX 77204-6283, USA.E-mail: [email protected]

Contract grant sponsor: Social Sciences and Humanities Research Council of Canada.Contract grant sponsor: Natural Sciences and Engineering Research Council of Canada.Contract grant sponsor: Research Program at the Queen's School of Business.Contract grant sponsor: Information Systems Research Center, University of Houston.

of e�ort and strategy selection in the relationship between decision aid usage and decision quality.Speci®cally, we assert that.

In order to induce the use of a superior (normative) decision strategy and, as a consequence, improvedecision quality, a decision aid must make that superior strategy at least as easy to employ as anysimpler but less accurate heuristic. Otherwise, the decision aid may only improve decision makinge�ciency. This will occur because decision makers use decision aids in such a way as to minimizetheir overall level of e�ort expenditure.

In general, our prior research has supported this proposition (Todd and Benbasat, 1991, 1992, 1993,1994a,b). However, in these studies the decision aids supported only some of the steps involved in usingnormative strategies. Thus, normative strategies were only used when support for simpli®ed heuristicswas not provided at all. Only in these situations was the normative choice strategy relatively easier toutilize than the simpler heuristics. It may be that by fully automating the processing steps required for anormative strategy, it would be used regardless of the level of support provided for alternative strategies.This would occur to the extent that the strategy is made at least as easy to employ as competingheuristics. Thus, by manipulating e�ort requirements, the enhanced e�ectiveness of a normativestrategy may be induced. This study provides a test of this e�ciency±e�ectiveness trade-o� in thecontext of full support that completely automates the processing steps required for a normative strategy.Decision makers are provided with tools that automate all the key processing steps involved in theimplementation of a normative choice strategy. We expect the use of the normative strategy to bedominant over simplifying heuristics that are also supported only when the e�ort needed to execute thenormative strategy is not greater than that required for the simple choice heuristics.

The paper proceeds as follows. In the second section the literature on cognitive e�ort in decisionmaking is brie¯y reviewed. In the third section the preferential choice task to be employed in theexperiments is examined and the strategies of interest in these studies are discussed. The speci®cdecision aids employed and their relationship to the choice strategies are described in the fourthsection. The ®fth section presents the hypotheses to be tested. The sixth section describes the experi-ment, and the seventh section presents the results and discussion. The ®nal section provides someconclusions.

THE ROLE OF COGNITIVE EFFORT IN CHOICE

Payne (1982) proposed a cost±bene®t framework of cognition suggesting that decision makers trade o�accuracy and e�ort in various decision-making tasks. A subsequent series of both empirical andsimulation work undertaken by Payne and his colleagues, among others, is largely supportive of thisnotion (see, for example, Johnson and Payne, 1985; Bettman, Johnson and Payne, 1990; Johnson,Payne and Bettman, 1988; Payne, Bettman and Johnson, 1988; Jarvenpaa, 1989; Stone and Schkade,1991; Creyer, Bettman and Payne, 1990; Payne, Bettman and Johnson, 1993).

According to the cost±bene®t framework, the joint objectives of a decision maker are to maximizeaccuracy (or decision quality) and minimize e�ort. As these objectives often con¯ict, trade-o�s aremade. Overall, the empirical (Russo and Dosher, 1983; Christensen-Szalanski, 1980; Bettman et al.,1990; Bettman and Kakkar, 1977; Johnson et al., 1988), simulation (Thorngate, 1980; Johnson andPayne, 1985; Payne et al., 1988, 1990) and conceptual (Beach andMitchell, 1978; Shugan, 1979; March,1978; Jungerman, 1985; Einhorn and Hogarth, 1978; Kleinmuntz and Schkade, 1993) literaturesindicate that e�ort is an important factor in¯uencing strategy selection. By extension, we arguethat e�ort requirements will also in¯uence the manner in which a decision support system is used,i.e. whether it will lead to improved e�ectiveness or e�ciency.

Copyright # 2000 John Wiley & Sons, Ltd. Journal of Behavioral Decision Making, Vol. 13, 91±106 (2000)

92 Journal of Behavioral Decision Making Vol. 13, Iss. No. 1

This interpretation is supported by our earlier work (Todd and Benbasat, 1991, 1992, 1993, 1994a,b).In general, the ®ndings of these studies indicate that decision makers tend to adapt their strategyselection to the type of decision aids available in such a way as to maintain a low level of e�ortexpenditure. In two experiments comparing the behavior of aided and unaided decision makers(Todd and Benbasat, 1991, 1992), a decision aid which reduced the relative e�ort of using elimination byaspects (EBA) as compared to using a conjunctive strategy led to the use of EBA. This occurredeven though without the aid, subjects were more likely to employ a conjunctive strategy. A thirdexperiment (Todd and Benbasat, 1992) provided limited evidence that when support for amore accurateadditive strategy was provided subjects were more inclined to use that strategy. A fourth experimentlooked at this phenomenon while focusing more clearly on support for speci®c choice strategies (in thiscase additive di�erence and EBA) and using a more re®ned form of data analysis (Todd and Benbasat,1994a). Consistent with the previous studies, strategy selection was in¯uenced by the availability ofdi�erent decision aids. More importantly, it reinforced the ®nding that decision aids can induce additivestrategy formulation when they reduce the e�ort required to implement such strategies. The robustnessof this ®nding was con®rmed in a ®fth study (Todd and Benbasat, 1994b) which replicated the fourthexperiment under conditions where the decision maker faced a higher cognitive load (a larger choice set).

Thus, overall the results appear to be consistent across several experiments. However, the priorstudies provided only limited support for normative strategies. In other words, while the use ofnormative strategies was made easier, it was still relatively e�ortful when compared to simplerelimination strategies. Thus, it could be that the relatively low level of support for normative processingdid not provide enough e�ort reduction, keeping the balance tipped in favor of elimination-orientedstrategies whenever support for them was provided. In this study we examine the e�ect of morecomplete support for normative strategies, presented to decision makers in the context of availablelevels of support for simpler, elimination-based, choice heuristics.

CHOICE STRATEGIES AND EFFORT

Description of preferential choice strategiesMulti-alternative, multi-attribute preferential choice problems deal with tasks where a decision makerchooses one of a number of alternatives, each of which is described by a common set of attributes(Keeney and Rai�a, 1976). Svenson (1979) describes twelve strategies applicable to choice problems.Two of the more commonly studied strategies are: additive compensatory (AC) and elimination byaspects (EBA) (see, for example, Payne, 1976; Olshavsky, 1979; Biggs et al., 1985; Sundstrom, 1987;Jarvenpaa, 1989).

In order to better understand each choice strategy and how it can be supported, the elementaryinformation processes (EIPs) associated with each are identi®ed. An EIP is a low-level cognitiveoperation such as reading value, comparing two values or storing a result in long-term memory. EIPshave been used to model a variety of decision processes (see, for example, Chase, 1978; Johnson andPayne, 1985; Payne et al., 1988). Breaking down the EIPs included in a strategy helps to understand thelevel of e�ort involved. Exhibit 1 breaks down the e�ort requirements of each strategy into the threecategories of processing, attribute recall, and tracking e�ort in the context of our experimentalconditions that provide varying levels of support (high and low) for a normative additive compensatorystrategy (AC) and a simpler elimination by aspects strategy (EBA). The detailed processing demands ofeach strategy are outlined below and re¯ected in Exhibit 1.

It should be noted that the numbers provided in Exhibit 1 are estimates of EIP load and are meant toillustrate the relationships between the strategies. We will discuss the EIPs required for variousstrategies in the context of an r*c matrix, where r represents the number of rows/attributes and c the


P. Todd and I. Benbasat Inducing Compensatory Information Processing 93

Exhibit 1. Impact of the decision aid treatments on the e�ort required for EBA and AC

Elimination by aspects Additive compensatory

ComponentFormula

Attribute recall1*rab

Tracking1*(rÿ 1)*c

Processing4*r*c

Total Attribute recall1*r*c

Tracking3*(cÿ 1)

Processing7*r*c

Total

Unaided 8 8 70 320 398Low AC/low EBA 8 0c 320 328 80 27 560 667High AC/low EBA 8 0c 320 328 8 0d 0d 8Low AC/high EBA 8 0e 0e 8 80 27 560 667High AC/high EBA 8 0e 0e 8 8 0d 0d 8

ar is the number of rows (attributes) and c is the number of columns (alternatives).bFor the numerical examples we assume r � 8 and c � 10. This represents the 8 attribute by 10 alternative problem used in the experiment.cThe DROP function eliminates the need to track alternatives remaining in the feasible set.dThe AC support tools eliminate the processing and tracking e�ort associated with AC.eCONDITIONAL DROP eliminates all processing and tracking e�ort associated with EBA.

Copyrig

ht#

2000JohnWiley

&Sons,Ltd.

JournalofBehavio

ralDecisio

nMaking,Vol.13,91±106(2000)

94

JournalofBehavio

ralDecisio

nMaking

Vol.13,Iss.

No.1

number of columns/alternatives (as in Exhibit 6). Thus the term r*c would represent the number ofcells or units of information to which operators, or EIPs, are applied.

The additive-compensatory model (AC) is roughly equivalent to the normative process forpreferential choice decisions as speci®ed by Keeney and Rai�a (1976). Using the AC strategy, adecisionmaker evaluates one alternative at a time along all relevant attributes. Each attribute is assigneda weight, with larger weights indicating higher value or importance. For a particular alternative, thedecision maker reads the ®rst attribute value. The attribute value is then combined with its weight. Thisprocess is repeated for each attribute of the alternative. A score for each alternative is determined bysumming the products of the attribute values and weights. Once these computations are completed foreach alternative, the one with the highest weighted score is chosen.

This process can be speci®ed as a series of elementary information processes. The sequence of EIPsneeded to examine a single attribute would involve: move to the speci®c attribute value, read theattribute value, retrieve attribute weight from long-term memory, multiply attribute value and weight,yielding a weighted attribute score, retrieve the current alternative score, addweighted attribute score tocurrent alternative score and store the updated alternative score. To completely evaluate one alternativethis process of seven operators is repeated r times for each attribute in the choice set. These 7*rprocesses must be completed c times, where c � the number of alternatives in the choice set. The resultof this set of operations leads to a score for each speci®c alternative, and this score is then compared tothe score of the current best alternative, which requires the decision maker to: retrieve the score of thecurrent best alternative, compare the current best alternative score to the score of the new alternative,store (a pointer to) the alternative with the higher score, and store the new best alternative score. Thisseries of four processes is repeated (cÿ 1) times, as each alternative is considered. Thus the totalprocessing for AC is 7*�r*c� � 4*�c ÿ 1�.

The elimination by aspects (EBA) strategy is based on a comparison of attribute values to somethreshold level with the elimination of any alternative that does not meet the threshold level for any oneof its attributes. The attributes are selected in hierarchical order of importance, with the most importantbeing selected ®rst. The sequence of four EIPs required to evaluate a single attribute is: move to thespeci®c attribute value, read attribute value, compare attribute to threshold, and eliminate if attributefails to meet threshold. Since processing is by attribute, each threshold is retrieved only once; thus theattribute recall load of EBA is relatively low. However, EBA requires considerable additional e�ort totrack the status of alternatives in order to avoid reprocessing those that had been previously eliminated.

The total number of EIPs required to follow an EBA strategy for an r*c matrix is �1*r��1*�r ÿ 1�*c� � �4*�r*c��. The ®rst value, 1*r, represents the number of memory retrieval operations,one for each attribute, needed to access attribute threshold values. The constant 4 represents the fourprocessing operations referred to above for each comparison of a cell value to a threshold. The factor1*(rÿ 1)*c represents the tracking e�ort required to maintain the status of alternatives in the matrix.This is required since the attribute-wise evaluation must take into account whether or not an alternativehas been eliminated based on the value of a previous attribute in order to avoid unacceptablealternatives being reconsidered as potential choices. When the very ®rst attribute (row) is examined, notracking is necessary since all alternatives are still in the candidate choice set. For subsequentattributes, the tracking operation must occur for each attribute examined since some alternatives mayhave already been eliminated.

In general, AC requires more e�ort than EBA, regardless of factors such as information load,similarity among alternatives, or other task characteristics. This occurs for a number of reasons. First,AC is more process intensive, requiring seven operations to process a single attribute value, comparedto four for EBA. Furthermore, the types of operations required by AC are more e�ortful in that theytend to involve computation rather than simple comparison (Payne et al., 1988). Finally, using ACrequires an exhaustive examination of all available information, whereas with EBA several alternatives



are often eliminated prior to all attribute values being considered. Though it requires more e�ort, ACshould lead to a better choice being made than does EBA since it considers all available information,weights attributes according to their importance to the decision maker, and is compensatory, allowinghigh values on some attributes to o�set low values on others (Keeney and Rai�a, 1976). Thus, ingeneral, we would expect an unaided decision maker who is concerned with accuracy to utilize AC, butone who places a signi®cant weight on e�ort to employ the EBA process. However, in practiceindividuals typically employ hybrid strategies as opposed to pure approaches (Bettman and Park,1980). One such process is to prescreen alternatives using an elimination strategy and then apply anadditive evaluation to the remaining alternatives (Todd and Benbasat, 1991).

Identi®cation of strategies and operatorsThe dependent variables of interest in this study are the operators employed by the decision maker(e.g. Todd and Benbasat, 1994a). These operators represent small routines of one or more EIPs (Stoneand Schkade, 1991) and are re¯ective of decision strategies. The particular operators of interest for thisstudy are: (1) independent evaluations which are characteristic of EBA processing; (2) eliminationswhich are characteristic of EBA processing, and (3) compensatory processes characteristic of ACprocessing. For each operator, we examine the proportion of statements in the protocol devoted to thatoperator. The protocol contains a verbal description of the decision process generated by the decisionmaker concurrent with the decision process. The proportional measures for the individual operatorsindicate how the mix of operations changes, and is particularly important if the total number ofoperations is di�erent between groups.

Independent evaluations are statements which compare the value of an attribute for a givenalternative to some externally identi®ed standard or reference point. For example, statements such asÀpartment F is expensive' or Àpartment G is too far from campus' are independent evaluations.Independent evaluations are characteristic of the EBA strategy. In performing an independentevaluation a decision maker is, in e�ect, executing a series of four EIPs which include moving to anattribute value, reading an attribute value, retrieving a threshold value from memory, and comparingthe threshold to the attribute value.

Elimination statements indicate that an alternative has been explicitly dropped from considerationprior to a complete evaluation of an alternative. This occurs when an alternative is eliminated after it hasbeen found de®cient on some attribute. An example: `The rent is too high so I am going to dropApartment C.' Elimination statements require the decision maker to store in memory, or track, the factthat the alternative has been eliminated. This operation requires only one EIP. This operation ischaracteristic of EBA processing.

Compensatory statements involve the aggregation and/or trade-o� of two or more attributes for asingle alternative. For example, statements such as `Rent per square foot is good for Apartment A' or Ìam willing to accept the noise level of Apartment M because of its cleanliness' are considered compen-satory statements. They are indicative of additive strategies, where a high value on some attribute mayo�set low values on others. These statements can take on a variety of forms but typically require, at aminimum, the move to and the reading of two values, and an operation (e.g. arithmetic or comparison)involving those two values. Thus, a compensatory statement requires at least ®ve EIPs.

THE DECISION AIDS

The decision aids, their function, and syntaxes are described in Exhibit 2. They were designed toreplace one or more of the operations associated with the AC and EBA strategies described above.Information is presented to subjects in a matrix form (see Exhibit 6).



All subjects were provided with the OPEN, CLOSE, and DROP commands. OPEN reveals thevalues of speci®ed cells, rows, or columns of the choice matrix (see Exhibit 6). CLOSE or DROPcommands eliminate undesirable alternatives from the choice set. These commands could also be usedto discard rows (e.g. attributes that carry zero weight), thereby reducing information load. Theexperimental treatments were set up by manipulating the availability of commands which supportedeither the AC strategy (CREATE, GLOBAL, and ROW TOTAL commands) or the EBA strategy(CONDITIONAL DROP command).

The CREATE, GLOBAL, and ROW TOTAL commands together provide almost complete supportfor the AC strategy. Executing these three commands in sequence automates all the operationsassociated with AC, with the exception of the ®nal alternative selection. CREATE generates a columnof weights indicating the relative importance of each attribute. GLOBAL multiplies a vector of valuesby the elements in the choice matrix. For example, the GLOBAL command can be used to multiply thematrix of attribute values by a vector of weights developed using the CREATE command to yield amatrix of weighted scores. Finally, the ROW TOTAL command can be used to sum the weightedscores for each alternative. Thus, these three commands reduce the AC strategy to three systemprocessing steps: CREATE weights, multiply attribute values by weights using GLOBAL, and sum theweighted scores using ROW TOTAL.

The CONDITIONAL DROP command automates the operations associated with EBA, auto-matically comparing attribute values to a threshold value and eliminating any alternatives whichviolate that threshold. The user enters a speci®c threshold value into the CONDITIONAL DROPcommand and all subsequent processing steps are automated.

By manipulating the availability of the commands, subjects in the experiment were assigned todi�erent treatment groups. The decision aids used in each of the four treatment groups are illustrated inExhibit 7.

Exhibit 2. DSS command descriptions

Commands Description and syntax

General purposeOPEN Uncovers a speci®ed cell, row (attribute) or column (alternative)

Syntax (Open row, or column, or cell), e.g. OPEN 1 ACLOSE Covers a speci®ed cell, row, or column that has been previously opened

Syntax (Close row, or column, or cell), e.g. CL 1 AUNDO Reverses the e�ect of the previous command. Successive undo commands

progressively undo prior operations for up to six stepsSyntax UNDO

EBA supportDROP Causes a speci®ed row or column to be deleted from the matrix

Syntax (Drop row or column), e.g. DR ACONDITIONAL DROP Drops columns contingent upon the value of an attribute

Syntax (Drop row {operator} threshold), e.g. DR 34 500AC supportCREATE Creates a new column into which user-speci®ed values can be entered

Syntax (Create column label), e.g. CR WeightsCALCULATE Performs a speci®ed arithmetic operation on any pair of rows or columns

Syntax (Calculate Column 1 (�, ÿ, *, /) Column 2), e.g. CALC A*BROW TOTAL Sums all the values in each column and places the results in a new row

Syntax RTOTALGLOBAL Performs arithmetic operations to combine the values in one column with all

other columns. Current values in those columns are overwrittenSyntax (Global (�, ÿ, *, /) column label, e.g. GLOBAL*Weight



HYPOTHESIS DEVELOPMENT

Our basic proposition is that, all other things being equal, if a decision aid reduces the e�ort associatedwith employing a particular strategy relative to other strategies, a decision maker will be more inclined toemploy that strategy. Furthermore, given two equally e�ortful strategies, decision makers are more likelyto employ the one they perceive will provide a better solution. Exhibit 1 shows how the level of e�ortrequired to use the AC and EBA strategies changes with the di�erent levels of support for eachexperimental condition. The total e�ort is decomposed into three components: attribute recall e�ort,alternative tracking e�ort, and processing e�ort. The entries in the Exhibit 1 provide estimates of thenumber of EIPs that would be required under each strategy assuming that the decision aid wasemployed in an appropriate fashion to support that strategy.

It is important in examining Exhibit 1 to note the changes in e�ort between strategies that occurwhen the di�erent decision aids are provided. These di�erences are summarized in Exhibit 3.1 Whenboth AC support and EBA support are low, using the AC strategy requires 339 more units of e�ortthan using EBA (667 versus 328 operations); therefore EBA should be more likely to be followed.When AC support is low and EBA support is high, there is an even more signi®cant e�ort advantage tofollow EBA of 659 units (8 versus 667). On the other hand, with high AC support, the e�ort required tofollow the AC strategy is virtually eliminated Ð being reduced to the eight attribute recall operationsneeded to provide attribute weights. Thus, when support for AC is high and support for EBA is low,AC is signi®cantly easier to employ, requiring 320 fewer operations (8 versus 328). Finally, whensupport for both AC and EBA is high, the two are equivalent in terms of e�ort. Together, thesedi�erences lead to speci®c main e�ect hypotheses for the AC and EBA treatment e�ects as describedbelow. The numbers in Exhibit 3 also suggest that no AC*EBA interaction is anticipated.

AC support hypothesesReferring to the ®gures in Exhibit 3, when support for AC is high, AC will, on average, be 160 units lesse�ortful to employ than EBA, whereas when support for AC is low, it will be signi®cantly (499 units)more e�ortful than EBA. Therefore, we expect that:

H1: Subjects with high AC support will be more likely to follow an AC strategy than those with low ACsupport.

Thus, subjects with high AC support will make

H1a: proportionally fewer independent evaluations than those with low AC support.H1b: proportionally fewer elimination statements than those with low AC support.H1c: proportionally more compensatory statements than those with low AC support.

Exhibit 3. E�ort di�erential (e�ort required for AC minus e�ort required forEBA) by treatment

Support for AC EBA main e�ecte�ort di�erentialHigh Low

Support for EBAHigh 0 659 329Low ÿ320 339 9

AC main e�ect e�ort di�erential ÿ160 499

1We are grateful to an anonymous reviewer for suggesting this approach for comparing the e�ort e�ects.



Since independent evaluations and elimination statements are not key operations associated withAC, we expect that these operations will be used less to the extent that high AC support induces ACprocessing. By contrast, compensatory statements are the key to the AC strategy. Thus, their incidenceshould be higher for groups with high AC support.

EBA support hypothesesReferring to Exhibit 3, when support for EBA is high, the EBA strategy is, on average, 329 units lesse�ortful to employ than AC. In the low EBA support condition, on average, there is only a very smalle�ort di�erential (9 units) between the use of AC and EBA, indicating that based on e�ort considera-tions alone individuals may be largely indi�erent between EBA and AC. Therefore overall we expect

H2: Subjects with high EBA support will be more likely to follow an EBA strategy than those with lowEBA support.

Thus, subjects with high EBA support will make

H2a: proportionally more independent evaluations than those with low EBA support.H2b: proportionally more elimination statements than those with low EBA support.H2c: proportionally fewer compensatory statements than those with low EBA support.

Based on the e�ort di�erentials shown in Exhibit 3, we would expect these EBA e�ects to be somewhatsmaller than the AC e�ects predicted under H1. This would occur to the extent that the e�ortdi�erential incentives for AC are, on average, twice as strong as those for EBA.

EXPERIMENTAL SETTING AND PROCEDURES

Fifty-two undergraduate university students enrolled in business, arts, or engineering programsparticipated on a strictly voluntary basis. Each subject was paid $15 for participating. They wereassigned at random to each of the four combinations of decision aids.

Subjects performed an apartment selection task similar to that employed by Payne (1976) andemployed in our previous studies (Todd and Benbasat, 1991, 1992, 1993, 1994a,b). The task requiredthe subjects to choose one from a set of ten apartments that are each described by a set of commonattributes. The attributes used are shown in Exhibit 6. Participants in the study were requested to makethe selections as if they were making a choice for themselves based on their own personal situation,likes, and dislikes. Subjects were run through the study one at a time. Each subject completed threesteps: (1) a tutorial, (2) a practice session, and (3) an experimental session, which involved completingtwo trials of the apartment selection task with a di�erent set of data for each trial.

First, a tutorial was provided to explain each of the commands and enable subjects to performvarious tasks that required the use of all the commands available to that subject. The tutorial was open-ended and allowed the subjects to spend as much time as they wished to acquaint themselves with thesystem. The tutorial did not make any direct linkages between the commands and choice strategies.Further, the tutorial was based upon a `dummy' problem of selecting alternative widgets, eachdescribed by a set of meaningless attribute names. Thus, subjects did not develop any decision strategyduring the tutorials but rather worked through the mechanics of interacting with the decision aid. Eachtreatment group had a tutorial that only included the commands available to them. During thetutorial, a lab assistant was present to answer any questions.

Following the tutorial, subjects went through a practice session and then the experimental session.In each case the choice problem consisted of eight attributes and ten alternatives. Both dealt withapartment selection problems; the attributes of the choice set were the same, but di�erent data were



used. A standard `spreadsheet' format similar to Exhibit 6 was employed to present the information. Allattribute values were presented on a standardized 10-point rating scale, where 1 represents the lowestscore on that attribute and 10 the highest score. Presentation of information on a standard scale wasemployed to facilitate the AC strategy that requires the integration of values across attributes.2 Initially,all cells in the presentation are blank. To access information, the user must explicitly open a speci®edrow, column, or individual cell. Commands were entered into the system at a command line. Allcommands could be abbreviated as shown in Exhibit 2.

While making a choice, the subjects were asked to think out loud. The practice problem was intendedto give the subjects a chance to familiarize themselves with the task setting, gain further experience withcommand use, and become comfortable with verbalizing. Instructions were provided to the subjects onhow to verbalize, following the recommendations of Russo, Johnson and Stephens (1986).

Two experimental trials were performed with di�erent data sets used for each trial. The data setswere presented to each subject in a random order. While completing the experimental trials, subjectswere told to use as much or as little information as they wished and to use whatever commands, fromthose available, that they thought were appropriate. Subjects were to treat the problem as if they weremaking a selection for themselves. A typical session involving completion of the tutorial, practicesession, and the two experimental trials took approximately 60 minutes. However, no time constraintswere imposed or speci®ed to the subjects.

During the entire experiment a lab assistant was present, though out of sight of the subjects. The labassistant would prompt subjects to `please say what you are thinking' if they were silent for more than10 seconds while completing the experimental trials. While engaged in the choice process, an audio-tape was made of each subject's verbalization. Subjects were aware that they were being recorded. Thedata used in the following analyses were the ones collected during the two experimental trials.

PRESENTATION OF FINDINGS

A MANOVA was used to analyze the e�ect of AC and EBA support conditions for a 2� 2 between-subjects factorial design with repeated measures. The proportions of independent evaluations,elimination statements, and compensatory statements were the dependent variables entered into theMANOVA. This in e�ect provides an overall test of strategy change as hypothesized in H1 and H2.Univariate tests were then used to investigate the sub-hypotheses, H1a, b, and c and H2a, b, and c.

Prior to analysis, the reliability of the protocol coding scheme was tested. All protocols wereanalyzed by two independent coders using the coding categories outlined above. The coders were givena written description of the coding rules and these were reviewed with the authors to ensure that theywere properly understood. The same coding scheme was applied for each of the four treatmentconditions, and the coders were blind to the treatment conditions and the hypotheses being tested inthe experiment. A total of approximately 6480 statements were coded; an average length protocol forone trial contained 60 coded statements.

The raw proportion of agreement between the two coders was 0.86 and the value for Kappa, whichrepresents raw agreement adjusted for chance (Cohen, 1960), was 0.79. In addition, this coding schemehas been employed and shown to be reliable in previous research (Biggs et al., 1985; Jarvenpaa, 1989;Todd and Benbasat, 1991, 1994a,b). The ®ndings reported are based upon the coding of one of theresearch assistant coders, selected at random.

2 It should be noted that this presentation format facilitates AC in a way that a traditional format with naturally scaled attributesdoes not and that the processing of information may be in¯uenced by this formatting choice. While such formats are common insome presentations, such as those used by Consumer Reports for product comparisons, they are not universal.



Exhibit 4 shows the results of the MANOVA. Exhibits 5(a), (b), and (c) show the results of theunivariate tests for the proportions of independent evaluations, elimination statements, compensatorystatements in the protocol, respectively.3

AC support

OverallThe MANOVA results (Exhibit 4) indicate a signi®cant AC main e�ect. H1 is supported. Univariatetests indicated that the level of support for AC signi®cantly in¯uenced the proportion of independentevaluations [F(3,100) � 6.26, p � 0.01], elimination statements [F(3,100) � 12.47, p � 0.001], andcompensatory statements [F(3,100) � 17.61, p � 0.0001]. H1a, b, and c were supported. The data inExhibit 5 show that those with high AC support made proportionally fewer independent evaluations(36% for high AC and 47% for low AC) and proportionally fewer elimination statements (7% versus16% for low AC) but proportionally more compensatory statements (30% versus 7.5% for low AC).These results are all consistent with the notion of subjects using the AC strategy when support for AC ishigh and EBA when support for AC is low.

EBA support

OverallThe MANOVA results (Exhibit 4) indicate a non-signi®cant EBA e�ect. H2 is not supported. The levelof support for EBA did not signi®cantly in¯uence the proportion of independent evaluations,elimination statements, or compensatory statements [see Exhibits 5(a), (b), and (c)]. The MANOVAresults also indicate an AC*EBA interaction e�ect that was not hypothesized. While signi®cant at theaggregate level, it does not show individual interaction e�ects at the univariate level. We have nocompelling explanation for this e�ect.

DISCUSSION AND CONCLUSIONS

There were, as expected, major di�erences between the treatment groups in terms of support for AC.However, we did not observe the hypothesized di�erences for EBA support.

The anticipated EBA e�ect was based on the notion (provided above and summarized in Exhibit 3)that the magnitude of e�ort di�erences summed across treatment conditions would explain theoutcomes e�ected by the manipulation of the EBA and AC support conditions. The lack of anticipated

Exhibit 4. MANOVA results

Wilkes' L DF F-value P-value

AC support 0.7488 5,44 2.95 0.022EBA support 0.9173 5,44 0.79 0.560AC*EBA 0.7575 5,44 2.81 0.027

3 The total number of statements di�ered across the AC treatment groups, with more total statements in the high AC conditionthan in the low (F � 6.7; p5 0.01). There was no signi®cant di�erence in the number of statements for the high and low EBAtreatment conditions.



Exhibit 5. Univariate results Ð number of operators

Exhibit 5(a). ANOVA results Ð proportion of independent evaluationsSource DF F-value P-value

AC support 1 6.26 0.01EBA support 1 1.02 0.31AC*EBA 1 0.91 0.34

Additive compensatoryHigh Low

High Elimination by aspects 36 (23)a 52 (23) 44Low Elimination by aspects 35 (30) 43 (18) 39

36 47

Exhibit 5(b). ANOVA results Ð proportion of eliminationsSource DF F-value P-value




7 16

Exhibit 5(c). ANOVA results Ð proportion of compensatory statementsSource DF F-value P-value




30 7.5

aMean percentage of compensatory statements in protocol statements (standard deviation).

Exhibit 6. The attribute by alternative matrix used in the apartment selection task

Apartments

A B C D E F G H I J

NoiseBrightnessRentRoomsSizeLaundryDistanceCleanliness



results suggests the possibility of an alternative explanation. It may be that strategy selection isin¯uenced by di�erential levels of e�ort that must meet certain threshold points. Thus, once thethreshold di�erence is met, the fact that a strategy may require half or a quarter of the e�ort of analternative approach becomes immaterial. So, for example, if EBA is 20% easier than AC in somespeci®c instance, that may be enough to induce its use. Making it 50% easier may have no additionale�ect. Thus, the absolute size of the di�erence does not provide a continuously increasing impetus toutilize a strategy to a greater or lesser extent, but rather a lexicographic process triggers the selection ofa particular strategy. In essence, if the threshold level for a strategy is low in an absolute sense, makingit lower may have no real e�ect on behavior. Empirical study of actual strategy selection mechanismsand the e�ect of relative e�ort di�erentials could help to shed further light on this process and theaptness of the threshold explanation.

On the surface, the EBA results appear at odds with our prior studies. These studies have shown thatindividuals provided with high levels of EBA support utilized more EBA operators (elimination andindependent evaluations) than those in low EBA support conditions (e.g. Todd and Benbasat,1994a,b). However, in our previous studies, we provided a high level of support for EBA but only amoderate level of support for an additive strategy (Todd and Benbasat, 1991, 1994a,b). Under theseconditions, the support for the additive strategy was equal to that for EBA only when support for EBAwas low and support for the additive strategy was high. In all other conditions, the EBA strategy waspreferable from an e�ort-minimization perspective. Thus, the requisite threshold e�ect to induce theadditive strategy was only met in one case. For this study, complete support for AC processing isprovided. Thus, EBA is only preferred when support for AC is low; otherwise, AC is preferredregardless of the level of EBA support. Even when the e�ort needed to implement them is equally low(the high AC, high EBA condition), AC is preferred because it is expected to lead to better outcomes.

The AC support results are consistent under both an e�ort di�erence and threshold model. Whensupport for AC is high, the AC strategy should be preferred regardless of the level of EBA support.This occurs because the AC strategy is more comprehensive, should lead to better choices than theEBA strategy, and when the level of AC support is high, requires no more e�ort than using the EBAstrategy (see Exhibit 1). As expected, there were main e�ects for AC in terms of the proportions ofindependent evaluations, elimination statements, and additive statements. Thus, when the moreaccurate AC strategy is made less e�ortful to use, it is used. This result is consistent with our priorstudies, but more clearly demonstrates that decision aids can induce the use of normatively orientedstrategies. The key to inducing these strategies is to make the normative strategy easier to execute thancompeting alternative strategies.

This study is a continuation of our on-going exploration of the mediating role of e�ort in therelationship between decision aid usage and decision strategy. Overall, these studies have demonstratedthat e�ort is an important mediator between decision aid use and decision strategy selection and by

Exhibit 7. Decision aids available in each treatment group

Support for AC

Low High

Support for EBALow Open, Close, Drop Open, Close, Drop, plus

Create, Global, Row Total

High Open, Close, Drop, plusConditional Drop

Open, Close Drop, plusCreate, Global, Row Total,and Conditional Drop



extension should a�ect decision-making outcomes. Thus, it appears that the potential in¯uence ofdecision aids on decision quality cannot be understood without taking into account theway the decisionaid in¯uences the e�ort required to use alternative strategies. All our studies suggest that decisionmakers use the decision aids in such away as tomaintain a low overall level of e�ort expenditure and willemploy a particular strategy if the decision aid makes it easier relative to competing alternativestrategies. Thus, by attending to the relationship between decision aids and decision-making e�ort, itbecomes possible for a designer of decision aids to alter the way in which information is processed.

To understand this relationship, we have developed an analytical approach to study the impact ofdecision aids on decision processes. This approach involves decomposing the decision strategies intotheir sub-components, as described above, in order to estimate the relative cognitive e�ort associatedwith each strategy. Based on these decompositions, we design decision aids that support one or moresub-components of the strategies. By considering the e�ort components of the strategies and how theyare alleviated by the decision aids, it is possible to predict which strategies will most likely be selectedwhen di�erent decision aids are made available. An example of the calculations made for suchpredictions is shown in Exhibit 1.

This analytical approach is relatively easy to employ and has been quite useful in predicting decision-maker behavior. Across a variety of studies and settings, the e�ects of the decision aids on behaviorwere predictable based on the amount of e�ort reduction the decision aids provided. These results havebeen consistent for di�erent types and levels of decision aids, for di�erent ways of identifying strategies,for problems of di�ering complexity, and under conditions where decision maker motivation was atdi�erent levels (Todd and Benbasat, forthcoming). That e�ort plays such a key role is consistent withthe ®ndings of research in behavioral decision theory (e.g. Payne et al., 1993). The results of our worksuggest a way to direct decision makers towards normatively oriented strategies by focusing on e�ortconsiderations. Using our decomposition approach, designers should be better able to understand theimpact of decision aid support on user behavior and to design systems that would lead to desireddecision strategies. To re®ne our understanding of this, it may be useful to examine individual di�er-ences in the use of the decision aids considering the in¯uence of factors such as the need for cognition.

Finally, in order to extend this approach more fully, it will be necessary to explicitly include the costof using a decision aid. The cognitive e�ort associated with learning and using a decision aid couldincrease the e�ort of executing the strategies it supports, thus reducing its overall in¯uence. This issueneeds to be investigated in future work to formulate a more comprehensive theory which describes therelationship between decision aids, e�ort, decision strategy and, ultimately, decision quality.

ACKNOWLEDGMENTS

This work has been supported by operating grants from the Social Sciences and Humanities ResearchCouncil of Canada and the Natural Sciences and Engineering Research Council of Canada as well asthe Research Program at the Queen's School of Business and the Information Systems Research Centerat the University of Houston.

REFERENCES

Beach, L. R. and Mitchell, T. R. `A contingency model for the selection of decision strategies', Academy ofManagement Review, 3 (1978), 439±449.

Bettman, J. R. and Kakkar, P. `E�ects of information presentation format on consumer information acquisition',in Anderson, B. (ed.), Advances in Consumer Research, 3 (1977), 316±320.



Bettman, J. R., Johnson, E. J. and Payne, J. W. À componential analysis of cognitive e�ort in choice',Organizational Behavior and Human Decision Processes, 45 (1990), 111±139.

Bettman, J. R. and Park, C. W. È�ects of prior knowledge and experience and phase of choice process onconsumer decision processes: A protocol analysis', Journal of Consumer Research, 7 (1980), 234±248.

Biggs, S. F., Bedard, J., Gaber, B. G. and Linsmeier, T. J. `The e�ects of task size and similarity on the decisionbehavior of bank loan o�cers', Management Science, 31 (1985), 970±987.

Chase, W. G. Èlementary information processes', in Estes, W. K. (ed.), Handbook of Learning and Cognition,Vol. 5, Hillsdale, NJ: Erlbaum, 1978, 1990.

Christensen-Szalanski, J. J. J. `Problem solving strategies: A selection mechanism, some implications and somedata', Organizational Behavior and Human Performance, 25 (1980), 307±323.

Cohen, J. `Coe�cient of agreement for nominal scales', Educational and Psychological Measurement, 20 (1960),37±46.

Creyer, E. H., Bettman, J. R. and Payne, J. W. `The impact of accuracy and e�ort feedback and goals on adaptivedecision behavior', Journal of Behavioral Decision Making, 3 (1990), 1±16.

Einhorn, H. and Hogarth, R. `Con®dence in judgement: Persistence of the illusion of validity', PsychologicalReview, 85 (1978), 395±416.

Einhorn, H. and Hogarth, R. `Behavioral decision theory: Processes of judgement and choice', Annual Review ofPsychology, 32 (1981), 53±88.

Jarvenpaa, S. `The e�ect of task and graphical format congruence on information processing strategies anddecision making performance', Management Science, 35 (1989), 285±303.

Johnson, E. and Payne, J. È�ort and accuracy in choice', Management Science, 31 (1985), 395±415.Johnson, E., Payne, J. and Bettman, J. Ìnformation displays and preference reversals', Organizational Behaviorand Human Decision Processes, 42 (1988), 1±21.

Jungerman, H. `The two camps on rationality', in Scholtz, R. W. (ed.), Decision Making under Uncertainty,Amsterdam: Elsevier Science, 1985.

Keen, P. J. `The marginal economics of e�ort', Working paper, Sloan School of Management, MIT (1979).Keeney, R. and Rai�a, H. Decisions with Multiple Objectives: Preferences and value trade-o�s, New York: Wiley,1976.

Kleinmuntz, D. N. and Schkade, D. A. Ìnformation displays and decision processes', Psychological Science, 4(1993), 221±227.

March, J. G. `Bounded rationality, ambiguity and the engineering of choice', The Bell Journal of Economics, 19(1978), 587±608.

Olshavsky, R. `Task complexity and contingent processing in decision making: A replication and extension',Organizational Behavior and Human Performance, 24 (1979), 300±316.

Payne, J. W. `Task complexity and contingent processing in decision making: An information search and protocolanalysis', Organizational Behavior and Human Performance, 16 (1976), 366.

Payne, J. W. `Contingent decision behavior', Psychological Bulletin, 92 (1982), 382±402.Payne, J. W., Bettman, J. and Johnson, E. J. Àdaptive strategy selection in decision making', Journal ofExperimental Psychology: Human Learning, Memory and Cognition, 14 (1988), 534±552.

Payne, J. W., Bettman, J. and Johnson, E. J. The Adaptive Decision Maker, New York: Cambridge UniversityPress, 1993.

Payne, J. W., Bettman, J., Johnson, E. J. and Coupey, E. Ùnderstanding contingent choice: A computersimulation approach', IEEE Transactions on Systems, Man and Cybernetics, 20 (1990), 296±309.

Russo, J. and Dosher, B. `Strategies for multiattribute binary choice', Journal of Experimental Psychology:Learning, Memory and Cognition, 9 (1983), 676±696.

Russo, J., Johnson, E. and Stephens, D. `When are verbal protocols valid?' Working paper, Cornell University(1986).

Shugan, S. `The cost of thinking', Journal of Consumer Research, 7 (1979), 99±111.Stone, D. N. and Schkade, D. `Numeric and linguistic information representation in multiattribute choice',Organizational Behavior and Human Decision Process, 49 (1991), 42±59.

Sundstrom, G. Ìnformation search and decision making: The e�ects of information display', Acta Psycologica,65 (1987), 165±179.

Svenson, O. `Process descriptions of decision making', Organizational Behavior and Human Decision Processes, 23(1979), 86±112.

Thorngate, W. È�cient decision heuristics', Behavioral Science, 25 (1980), 219±225.Todd, P. and Benbasat, I. Àn experimental investigation of the impact of computer based decision aids on thedecision making process', Information Systems Research, 2 (1991), 87±115.



Todd, P. and Benbasat, I. `An experimental investigation of the impact of computer based decision aids onprocessing e�ort', MIS Quarterly, 16 (1992), 373±393.

Todd, P. and Benbasat, I. `Decision makers, decision aids and decision making e�ort: An experimentalinvestigation', INFOR, 31 (1993), 80±100.

Todd, P. and Benbasat, I. `The in¯uence of decision aids on choice strategies: An experimental analysis of the roleof cognitive e�ort', Organizational Behavior and Human Decision Processes, 60 (1994a), 36±74.

Todd, P. and Benbasat, I. `The in¯uence of decision aids on choice strategies under conditions of high cognitiveload', IEEE Systems, Man and Cybernetics, 24 (1994b), 537±547.

Todd, P. and Benbasat, I. `Exploring the impact of cognitive e�ort and commitment on decision aid usage',Information Systems Research, forthcoming.

Authors' biographies:Peter Todd is a Professor of Management Information Systems and Director of the Information Systems ResearchCenter in the College of Business Administration at the University of Houston. Before joining the University ofHouston he was a Professor of MIS at Queen's University in Kingston, Ontario, where he also served as Directorof Research and PhD Programs. He has a PhD in information systems from the University of British Columbia.His principal research interests are in the areas of information technology adoption and management, decisionsupport, and human±computer interaction.

Isak Benbasat is CANFOR Professor of Management Information Systems and Associate Dean, Faculty ofDevelopment, at the Faculty of Commerce and Business Administration, The University of British Columbia(UBC), Vancouver, Canada. He received his PhD (1974) inManagement Information Systems from the Universityof Minnesota. His current research interests are in the areas of designing intelligent systems, measuring informa-tion systems competence, evaluating human±computer interfaces, and investigating the in¯uence of decisionsupport systems on problem solving.

Authors' addresses:Peter Todd, College of Business Administration, University of Houston, Houston, TX 77204-6283, USA.

Izak Benbasat, Faculty of Commerce, University of British Columbia, Vancouver, BC V6T 1Z2, Canada.



Inducing compensatory information processing through decision aids that facilitate effort reduction:...

Documents

Transcript of Inducing compensatory information processing through decision aids that facilitate effort reduction:...