Why Do Auditors Over-rely on Weak Analytical Procedures? The Role of

Outcome Bias and Insensitivity to Precision

Steve GloverDoug Prawitt

Jeff Wilks

Brigham Young University

Importance of Analytical Procedures

Frequent and increasing use of analytical procedures (Ameen & Stawser 1994; McDaniel & Kinney 1995; Hirst & Koonce 1996; Kinney & McDaniel 1996; Bell et al. 1997)

Findings of the POB (2000)

– Analytical procedures used for substantive testing provided insufficient assurance 20% of the time.

– Insufficient procedures typically related to income statement accounts.

Reliance on Audit Evidence

Auditors sensitive to– Source competence (Bamber 1983)

– Objectivity (Hirst 1994, Caster & Pincus 1996)

– Amount, composition, directness, and deviations from expectations (Caster & Pincus 1996)

What might explain over-reliance on audit evidence, and analytical procedures in particular?

Favorable Outcome Effect

When analytical procedures produce expected or even desired results, auditors are less critical of the procedure.– Motivated reasoning (Kunda 1990)

– Inherited hypotheses (Swann & Giuliano 1987)

– Biased hypothesis testing (Pyszczynski & Greenberg 1987)

Sensitivity to Precision Precision

– The expected closeness of an observed statistic to the “correct” amount

– A critical determinant of the quality of evidence produced by substantive analytical procedures

– SAS No. 56 suggests that• Disaggregated data produces more precise expectations than

aggregated data

• Precision also affected by reliability of source data, predictability of account, and type of procedure

Sensitivity (continued)

Contingent auditor sensitivity to precision– Auditors sensitive to precision when testing a

hard account, but not estimates (McDaniel & Simmons 2003)

– We argue “no-significant-difference” outcomes diminish auditor sensitivity to precision

Experiment 1

No Significant Difference

Significant Difference

Aggregate Procedure

Assess Evidential Strength

H1: Outcome Bias

The assessed strength of audit evidence obtained from an aggregate analytical procedure will be greater for auditors who observe no significant difference (a favorable outcome) than for auditors who observe a significant difference.

Experiment 1

No Significant Difference

Significant Difference

Aggregate Procedure

Assess Evidential Strength

Significant Difference

Disaggregate Procedure

Assess Evidential Strength

Reassess Strength of

Part A

After evaluating the disaggregated procedure, auditors who initially observe no significant difference will lower their assessments of the aggregate procedure’s evidential strength more than will auditors who initially observed a significant difference.

H2: Sensitivity to Precision

Expected Results

Evi

dent

ial S

tren

gth

No

Yes

Initial Assessment of

Aggregate Procedure

Reassessment of Aggregate

Procedure

Originally Observed

Significant Difference

Experiment 1: Method Task

– Determine appropriateness of interest income– Compute expectation for current year based on

annual loan balances and rates– Compute new expectation based on quarterly and

categorized loan balances and rates

Participants– 67 supervising seniors from one Big Four firm

participated during national training sessions– 34.5 months of audit experience (standard deviation

12.8)

Experiment 1: Method

Design (2x2 mixed design)– Favorable outcome: Yes or no (between

subjects)– Precision of analytic: Aggregate or

disaggregate (within-subject)

Primary dependent measure– Evidential strength: 1 (extremely weak) to 7

(extremely strong)

Results (Table 1)

Estimates of Evidential Strength for

Outcome of Aggregate Analytical Procedure

Part A-Aggregate Procedure (Initial

Assessment)

Part B-Disaggregated Procedure – Suggests Material Misstatement

Aggregate Procedure

(Reassessment)

No Significant Difference

3.9 (0.81) [29]

3.7 (0.99) [29]

2.7 (1.16) [29]

Significant Difference 2.9

(1.02) [36]

3.8 (1.03) [36]

2.8 (1.21) [36]

Combined 3.4 3.8 2.8

Control Conditions

Evi

dent

ial S

tren

gth

No

Yes

Initial Assessment of

Aggregate Procedure

Reassessment of Aggregate

Procedure

Originally Observed

Significant Difference

Cueing Sensitivity to Precision

Ex ante warning to consider weaknesses can lead to more careful scrutiny (Petty & Cacioppo 1986)

– Evidenced by lower assessments of strength– Possible demand effect

Include two types of participants– Auditors with relevant task knowledge– Students without such knowledge

H3: Cueing Sensitivity to Precision

Part 1: The perceived strength of evidence obtained from a weak, high-level analytical procedure will be lower for auditors who are cued ex ante to critically evaluate the quality of the analytical procedure than for auditors who are not cued to critically evaluate the procedure.

Part 2: An ex ante cue to critically evaluate the quality of a procedure will make no difference in students’ assessments of evidential strength.

Expected Results

Evi

dent

ial S

tren

gth

Students

Auditors

No Cue Cued to Critically Evaluate

Experiment 2: Method Participants

– 84 supervising seniors from one Big Four firm participate during national training sessions (34.5 months of audit experience, standard deviation 12.8 months)

– 90 graduate students in a second auditing course Design (2x2 between subjects) Task and materials same as Part A, Exp. 1 (ND) Dependent measures same as Part A, Exp. 1 (ND)

Experiment 2: Method The text of the cue read as follows:

Before computing an expectation of interest income for the current year, please consider the strength (quality and sufficiency) of the evidence provided by the interest income analytical procedure used by the audit team last year. Please list in the space provided below one or more weaknesses of this analytical procedure.

Results

Participant

Explicit Guidance to Consider Weaknesses

Professional

Participants

Student

Participants Combined

No 4.1

(0.64) [42]

2.9 (1.27) [45]

3.5 (1.18) [87]

Yes 3.3

(1.05) [42]

2.8 (1.13) [45]

3.0 (1.11) [87]

Combined 3.7 2.9 3.3

Conclusions & Implications

An outcome effect can explain in part why auditors may over rely on weak analytical procedures.– Hinders consideration of precision– May also hinder consideration of…?

• Source competence (Bamber 1983)

• Objectivity (Hirst 1994, Caster & Pincus 1996)

• Amount, composition, directness, and deviations from expectations (Caster & Pincus 1996)

Conclusions & Implications

Ex ante prompt to consider weaknesses of an analytical procedure appears to– Produce no demand effect – Lead to more careful scrutiny by those who

have relevant task knowledge

Limitations & Future Research

All auditors from one Big Four firm The impact of teamwork, the review

process, and accountability not examined