APIPA 20101 INTRODUCTION TO NONSTATISTICAL SAMPLING FOR AUDITORS Jeanne H. Yamamura CPA, MIM, PhD.

APIPA 2010 1

INTRODUCTION TO NONSTATISTICAL SAMPLING FOR AUDITORS

Jeanne H. YamamuraCPA, MIM, PhD

APIPA 2010 2

SITUATION

You are auditing the Dept. of Admissions & Records for Micronesia College.

One of your objectives is to verify that student records are being updated correctly and timely.

You decide to select a sample of grades posted from the most recent semester completed.

APIPA 2010 3

SITUATION

What would you normally document about this sample?Sample sizeSelection methodPopulationProcedures to be performedPurpose of test

What kind of test is this?

APIPA 2010 4

OBJECTIVES

Review of sampling conceptsTypes of sampling - overviewNonstatistical attribute sampling

Steps in applyingAdditional coverage of:

Sampling methods

Compliance auditing

APIPA 2010 5

Applicable Professional Standards

SAS 39 Audit SamplingSAS 111 Amendment to SAS 39 Audit

SamplingISA 530 Audit Sampling

APIPA 2010 6

AUDIT SAMPLING

Application of an audit procedure to less than 100% of the items in a population Account balance Class of transactions

Examination “on a test basis” Key: Sample is intended to be representative

of the population. Objective: To reach a conclusion about the

population based on the sample items tested.

APIPA 2010 7

SAMPLING RISK

Possibility that the sample is NOT representative of the population

As a result, auditor will reach WRONG conclusion

Decision errorsType I – Risk of incorrect rejectionType II – Risk of incorrect acceptance

APIPA 2010 8

TYPE I – RISK OF INCORRECT REJECTION

Internal control: Risk that sample supports conclusion that control is NOT operating effectively when it really isAKA – Risk of underreliance, risk of

assessing control risk too highSubstantive testing: Risk that sample

supports conclusion that balance is NOT properly stated when it really is

APIPA 2010 9

TYPE II – RISK OF INCORRECT ACCEPTANCE

Internal control: Risk that sample supports conclusion that control is operating effectively when it really isn’tAKA – Risk of overreliance, risk of

assessing control risk too lowSubstantive testing: Risk that sample

supports conclusion that balance is properly stated when it really isn’t

APIPA 2010 10

WHICH RISK POSES THE GREATER DANGER TO AN AUDITOR?

Type I - Risk of incorrect rejectionEfficiency

Type II - Risk of incorrect acceptanceEffectiveness

Auditor focus on Type IIAlso provides coverage for Type I

APIPA 2010 11

NONSAMPLING RISK

Risk of auditor errorSample wrong populationFail to detect a misstatement when applying

audit procedureMisinterpret audit result

Controlled through Adequate trainingProper planningEffective supervision

APIPA 2010 12

SAMPLE SIZE FACTORS

Desired level of assurance (confidence level)

Acceptable defect rate (tolerable error)

Historical defect rate (expected error)

APIPA 2010 13

CONFIDENCE LEVEL

Complement of sampling risk 5% sampling risk, 95% confidence level

How much reliance will be placed on test results

The greater the reliance and the more severe the consequences of Type II error, the higher the confidence level needed

Sample size increases with confidence level (decreases with sampling risk)

APIPA 2010 14

TOLERABLE ERROR AND EXPECTED ERROR

“Precision” – the gap between tolerable error and expected errorExpected population error rate = 1%Auditor’s tolerable error rate = 3%

AKA Allowance for sampling riskSample size increases as precision

decreases

APIPA 2010 15

WHEN DO YOU SAMPLE?

Inspection of tangible assets, e.g., inventory observation

Inspection of records or documents, e.g., internal control testing

Reperformance, e.g., internal control testing

Confirmation, e.g., verification of AR balances

APIPA 2010 16

WHEN IS SAMPLING INAPPROPRIATE?

Selection of all items with a particular characteristic, e.g., all disbursements > $100,000

Testing only one or a few items, e.g., automated IT controls, walk throughs

Analytical procedures Scanning Inquiry Observation

APIPA 2010 17

WALKTHROUGHS

Designed to provide evidence regarding the design and implementation of controls

Can provide some assurance of operating effectiveness BUT Depends on nature of control (automated or

manual) Depends on nature of auditor’s procedures to test

control (also includes inquiry and observation combined with strong control environment and adequate monitoring)

Walkthough = sample of 1

APIPA 2010 18

STATISTICAL VS NONSTATISTICAL SAMPLING

Statistical sampling Statistical computation of sample size Statistical evaluation of results

Nonstatistical sampling Sample sizes should be approximately the

same (AU 350.22) Sample sizes must be sufficient to support

reliance on controls and assertions being tested

APIPA 2010 19

WHEN IS SAMPLING NONSTATISTICAL?

If sample size determined judgmentallyIf sample selected haphazardlyIf sample results evaluated judgmentally

APIPA 2010 20

TYPES OF SAMPLING

Attribute samplingMonetary unit samplingClassical variables sampling

APIPA 2010 21

ATTRIBUTE SAMPLING

Used to estimate proportion of a population that possesses a specific characteristic

Most commonly used for T of CCan also be used for dual purpose

testing (T of C and Substantive T of T)

APIPA 2010 22

MONETARY-UNIT SAMPLING

AKA probability proportional to size (PPS) sampling, cumulative monetary unit sampling

Used to estimate dollar amount of misstatement

APIPA 2010 23

CLASSICAL VARIABLES SAMPLING

Uses normal distribution theory to identify amount of misstatement

Useful when large number of differences expectedSmaller sample size than MUS

Effective for both overstatements and understatements

Can easily incorporate zero balances

APIPA 2010 24

STEPS IN NONSTATISTICAL ATTRIBUTE SAMPLING APPLICATION

Planning1. Determine the test objectives2. Define the population characteristics3. Determine the sample size

Performance4. Select sample items5. Perform the auditing procedures

Evaluation6. Calculate the results7. Draw conclusions

APIPA 2010 25

STEP 1: DETERMINE THE TEST OBJECTIVES

Objective for T of C: To determine the operating effectiveness of the internal controlSupport control risk assessment below

maximum (FS audit)Identify controls to be tested and

understand why they are to be tested

APIPA 2010 26

TESTS OF CONTROLS

Concerned primarily withWere the necessary controls performed?How were they performed?By whom were they performed?

Appropriate when documentary evidence of performance exists

APIPA 2010 27

SUBSTANTIVE TEST OF TRANSACTIONS

Objective for S T of T: To determine whether the transactions contain monetary misstatementsAlternatively, to determine whether the

system is operating as designedIdentify transactions to be tested and

understand why they are to be tested

APIPA 2010 28

STEP 2: DEFINE THE POPULATION CHARACTERISTICS

Define the sampling population Can be defined however desired BUT must include

entire population as defined Test population for completeness

Define the sampling unit Determined by available records Based on definition of population and audit

objective

Define the control deviation conditions

APIPA 2010 29

STEP 3: DETERMINE THE SAMPLE SIZE

Consider desired confidence level, tolerable deviation rate, and expected population deviation rate

Judgmentally determine sample sizeNOTE: Check against statistical sample

size tables to verify adequacy

APIPA 2010 30

TOLERABLE RATE GUIDELINES

Significance of the transactions and related account balances that the IC are intended to affect

Highly significant balances Tolerable Rate of 4%

Significant balances Tolerable Rate of 5%

Less significant balances Tolerable Rate of 6%

Preliminary Assessment of CR

Tolerable Rate

Low < = 5%

Moderate < = 10%

High Do not test controls

APIPA 2010 31

TOLERABLE RATE GUIDELINES

Assessed importance of the control

Tolerable Rate

Highly important 3 - 5%

Moderately important 6 – 10%

APIPA 2010 32

ESTIMATE OF POPULATION ERROR RATE

Prior year resultsPreliminary sampleShould be low – 0, 1%Higher rates increase sample size

APIPA 2010 33


Guidelines for nonstatistical sample sizes for tests of controls

If any errors found, increase sample size or increase control risk (Probably not applicable to Public Auditor)

Desired level of controls reliance (how important is the control/process)

Sample size

Low 15-20

Moderate 25-35

High 40-60

APIPA 2010 34

SMALL POPULATIONS AND INFREQUENTLY OPERATING CONTROLS

Small Population Sample Size Table

Control Frequency and Population Size

Sample Size

Quarterly (4) 2

Monthly (12) 2-4

Semimonthly (24) 3-8

Weekly (52) 5-9

APIPA 2010 35

STEP 4: SELECT SAMPLE ITEMS

Random sampleSystematic sample (with random start)Haphazard selection

APIPA 2010 36

RANDOM SELECTION

Every possible combination of population items has an equal chance of being included in the sample

Random number tablesComputer generation of random

numbers

APIPA 2010 37

SYSTEMATIC SELECTION

Interval calculated and items selected based on size of interval Interval = Population / Desired Sample Size

Starting point is random number within interval

Need to consider if bias present due to patterns in data

APIPA 2010 38

HAPHAZARD SELECTION

Selection by auditor without any conscious bias If you select large, risky, or unusual items, it is NOT

haphazard selection and it is NOT audit sampling. Instead – targeted or directed selection

Still desire representative sample Avoid unusual, large, first or last Useful for certain situations

Example: Tracing credits from AR to CR/other sources looking for fictitious credits

Less costly and simpler

APIPA 2010 39

STEP 5: PERFORM THE AUDITING PROCEDURES

Conduct planned audit proceduresWhat if?

Voided documents - if properly voided, not a deviation; replace with new sample item

Unused or inapplicable documents – replace with new sample item

Inability to examine sample item – deviationStopping test before completion – large

number of deviations detected

APIPA 2010 40


Deviations observed Investigate nature, cause, and consequence

of every exceptionUnintentional error? Or fraud?Monetary misstatement resulted?Cause – misunderstanding of instructions?

Carelessness?Effect on other areas?

APIPA 2010 41

STEP 6: CALCULATE THE RESULTS

No computed upper deviation rate (per table in statistical sampling)

Compute Calculated Sampling Error = Tolerable Error Rate – Sample Error Rate.

APIPA 2010 42

STEP 7: DRAW CONCLUSIONS

Control not effective (system not working as designed) ifCalculated Sampling Error too small

Depends on sample size usedSample Error Rate > Tolerable Error RateSample Error Rate > Expected Population

Error Rate

APIPA 2010 43

COMPLIANCE AUDITING

Performance of auditing procedures to determine whether an entity is complying with specific requirements of laws, regulations, or agreements

Governmental entities and other recipients of governmental financial assistance Compliance with laws and regulations that

materially affect each major federal assistance program

APIPA 2010 44

COMPLIANCE AUDITING OF FEDERAL ASSISTANCE PROGRAMS

Definition of population for testing of an internal control procedure that applies to more than one programDefine items from each major program as a

separate population, ORDefine all items to which control is

applicable as a single populationSecond choice usually more efficient

APIPA 2010 45

COMPLIANCE AUDITING - EXAMPLE

Federal financial assistance for Island CityThree major federal financial assistance

programsFour nonmajor programs

Control: Transaction review to ensure that only legally allowable costs are charged to each program

APIPA 2010 46

COMPLIANCE AUDITING - EXAMPLE

More efficient to select one sample from population of all transactions (major and nonmajor programs)

Confidence level = 95%Tolerable deviation rate = 9%Expected population deviation rate = 1%Sample size: 511 allowable deviation

APIPA 2010 47

T of C versus S T of T

Test of ControlVerifies that a control is operating effectively

Substantive Test of TransactionsVerified that a transaction does not contain

a misstatement

APIPA 2010 48

ASSERTIONS FOR CLASSES OF TRANSACTIONS

Occurrence: Transaction actually occurred and pertains to the entity (existence/validity)

Completeness: All transactions have been recorded

Accuracy: Amounts and other data have been recorded correctly

APIPA 2010 49

ASSERTIONS FOR CLASSES OF TRANSACTIONS

Cutoff: Transactions have been recorded in the correct accounting period

Classification: Transactions have been recorded in the proper accounts

APIPA 2010 50

CALCULATED SAMPLING ERROR

Tolerable error rate – Sample error rate = Calculated sampling error

Sample error rate = Population error rate

due to sampling errorAuditor must evaluate calculated

sampling error to see if it is big enough (sufficiently large to allow for sampling error in population)

APIPA 2010 51

CALCULATED SAMPLING ERROR

If Sample error rate > Tolerable error rate = REJECT – CONTROL NOT WORKING or PROCEDURE NOT BEING FOLLOWED

If Sample error rate > Expected population error rate, REJECT – CONTROL NOT WORKING OR PROCEDURE NOT BEING FOLLOWED

APIPA 2010 52

STEPS IN NONSTATISTICAL SUBSTANTIVE SAMPLING APPLICATION

Planning1. Determine the test objectives2. Define the population characteristics3. Determine the sample size

Performance4. Select sample items5. Perform the auditing procedures

Evaluation6. Calculate the results7. Draw conclusions

APIPA 2010 53


Identify individually significant items Some items too risky, must be audited, OR Easier to pull out and test large items

Stratify population Divide population into homogeneous units For example, all items > $10,000 Items tested 100% are not part of the sample

APIPA 2010 54


Define the sampling population Consists of an account balance or class of

transactions Will project sample results to population Must be sure to adequately identify population For example: Accounts Receivable could be

defined as All accounts Accounts with zero balances Accounts with debit balances Accounts with credit balances

APIPA 2010 55


Define the sampling unit Any item in the defined population Could be an account or a transaction

APIPA 2010 56


Subjective determination OK Factors to consider:

Amounts of individual items Accounting populations usually include a few

very large items, a number of moderately large amounts, and a large number of small amounts

If not stratified, will need larger sample Variability and size of population

The greater the variability, the larger the sample size needed

Population size – little effect on sample size so usually ignored

APIPA 2010 57


Factors to consider: Risk of incorrect acceptance (RIA)

As RIA increased, sample size decreases If controls good, can accept larger RIA for

substantive testing

Tolerable misstatement and expected misstatement

Larger tolerable misstatement, smaller sample size

Larger expected misstatement, larger sample size

APIPA 2010 58

STEP 4: SELECT SAMPLE ITEMS

Any method that will result in representative sampleRandom sampleSystematic sample (with random start)Haphazard selection

APIPA 2010 59


Deviations observed Investigate nature, cause, and consequence

of every exceptionUnintentional error? Or fraud?Monetary misstatement resulted?Cause – misunderstanding? Carelessness?Effect on other areas?

APIPA 2010 60

STEP 6: CALCULATE THE RESULTS

Compute sample error amount or sample error rate

Project to population Projected misstatement

Error * number of items in population Error rate * dollar population value

APIPA 2010 61


Compare projected misstatement to tolerable misstatement

If projected misstatement < tolerable misstatement, population OK

If projected misstatement > tolerable misstatement, population misstated

APIPA 2010 62


Consider sampling risk If projected misstatement < expected misstatement,

probably safe to conclude that population is OK (i.e., there is an acceptably LOW risk that the true misstatement exceeds thetolerable misstatement)

If projected misstatement > expected misstatement, greater risk present (i.e., there is an UNACCEPTABLY HIGH risk that the true misstatement exceeds the tolerablemisstatement).

APIPA 2010 63


If recorded amount believed to be misstated, need more work!

Investigate misstatementsAdjust recorded amounts

APIPA 2010 64

RESOURCES

Audit Sampling: An Introduction, 3rd Edition, Guy, Carmichael & Whittington

Audit Guide: Audit Sampling, New Edition as of May 1, 2008, AICPA

Auditing & Assurance Services, 6th Edition, Messier, Glover, & Prawitt

Auditing & Assurance Services, 12th Edition, Arens, Elder & Beasley

APIPA 2010 65

QUESTIONS?

APIPA 20101 INTRODUCTION TO NONSTATISTICAL SAMPLING FOR AUDITORS Jeanne H. Yamamura CPA, MIM, PhD.

Documents

Transcript of APIPA 20101 INTRODUCTION TO NONSTATISTICAL SAMPLING FOR AUDITORS Jeanne H. Yamamura CPA, MIM, PhD.