Fraud Detection Using A Database Platform

173
Fraud Detetcion using a database platform EZ-R Stats, LLC Fraud Detection Using a Database Platform February 23, 2009 Mike Blakley Mike Blakley Central Carolina Chapter of the Association of Certified Fraud Examiners

description

Presentation made at the CCCFE conference of 2-23-2009 in Raleigh, NC

Transcript of Fraud Detection Using A Database Platform

Page 1: Fraud Detection Using A Database Platform

Fraud Detetcion using a database platform EZ-R Stats, LLC

Fraud Detection Using a Database Platform

February 23, 2009

Mike BlakleyMike BlakleyCentral Carolina Chapter of the Association of Certified Fraud Examiners

Page 2: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Session objectives

1. Understand why and how2. Understand statistical basis for

quantifying differences3. Identify ten general tools and

techniques4. Understand how pattern

detection fits in

Page 3: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Session agenda and timings

Managing the business risk of fraud (30 minutes) Overview of statistical approach (10 min) Discussion of databases (10 min) Break (10 min) Details of the approach (40 min) Brief demo (5 min) Open discussion and question and answer (15 min)

Page 4: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Handout (CD)

CD with articles and software PowerPoint presentation More info at www.ezrstats.com

Page 5: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Optional quiz

Test your understanding Entirely optional On home page under “events” – quiz Results can be e-mailed

Page 6: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

“Cockroach” theory of auditing

If you spot one roach….

Page 7: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

“Cockroach” theory of auditing

There are probably 30 more that you don’t see…

Page 8: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Statistics based “roach” hunting

Many frauds coulda/woulda/shoulda been detected with analytics

Page 9: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Overview Fraud patterns detectable with

digital analysis Basis for digital analysis

approach Usage examples Continuous monitoring Business analytics

Page 10: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

The Why and How

Three brief examples ACFE/IIA/AICPA Guidance Paper Practice Advisory 2320-1 Auditors “Top 10” Process Overview Who, What, Why, When & Where

Objective 1

Page 11: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Example 1Wake County Transportation Fraud

Supplier Kickback – School Bus parts

$5 million Jail sentences Period of years

Objective 1a

Page 12: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Too little too late

Understaffed internal audit Software not used Data on multiple platforms Transaction volumes large

Objective 1a

Page 13: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Preventable

Need structured, objective approach

Let the data “talk to you” Need efficient and effective

approach

Objective 1a

Page 14: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Regression Analysis

Stepwise to find relationships

– Forwards– Backwards

Intervals– Confidence– Prediction

Objective 1

Page 15: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Data outliers

Objective 1

Sometimes an “out and out Liar”

But how do you detect it?

Page 16: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Data Outliers

Plot transportation costs vs. number of buses

“Drill down” on costs– Preventive maintenance– Fuel– Inspection

Objective 1

Page 17: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Scatter plot with prediction and confidence intervals

Page 18: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Cost of six types of AIDS drugs

Total Cost of AIDS Drugs

0

50

100

150

200

NDC1 NDC2 NDC3 NDC4 NDC5 NDC6

Drug Type

Dol

lar A

mou

nt NDC1

NDC2

NDC3

NDC4

NDC5NDC6

Example 2 Objective 1a

Page 19: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Medicare HIV Infusion Costs

Objective 1

CMS Report for 2005 South Florida - $2.2 Billion Rest of the country combined -

$.1 Billion

Page 20: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Pareto ChartObjective 1

Medicare HIV Infusion Costs - 2005 ($Billions)data source: HHS CMS

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

1 3 5 7 9 11 13 15

County

Annu

al M

edic

are

Cost

s

PctCum Pct

Page 21: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Typical Prescription Patterns

AIDS Drugs Prescription Patterns

0.0

10.0

20.0

30.0

40.0

50.0

60.0

Prov 1 Prov 2 Prov 3 Prov 4 Prov 5 Prov 6

Prescriber

Dolla

r Val

ue

NDC1NDC2NDC3NDC4NDC5NDC6

Example 2 Objective 1a

Page 22: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Prescriptions by Dr. X

Dr. X compared with Total Population

050

100150200250300350

NDC1 NDC2 NDC3 NDC4 NDC5 NDC6

Drug Type

Dolla

r Am

ount

PopulationDr. X

Example 2 Objective 1a

Page 23: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Off-label use

Serostim– Treat wasting syndrome, side effect of

AIDS, OR– Used by body builders for recreational

purposes– One physician prescribed $11.5 million

worth (12% of the entire state)

Example 2 Objective 1a

Page 24: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Revenue trends

Overall Revenue Trend

0.90.95

11.05

1.11.15

1.2

2001 2002 2003

Calendar Year

Annu

al B

illing

s

OverallLinear (Overall)

Example 3 Objective 1a

Page 25: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Dental Billings

Rapid Increase in Revenues

0

1

2

3

4

5

2001 2002 2003

Calendar Year

Annu

al B

illing

s ($

milli

ons) Billings A

Billings BLinear (Billings A)

Example 3 Objective 1a

Page 26: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Guidance Paper

A proposed implementation approach “Managing the Business Risk of Fraud: A

Practical Guide” http://tinyurl.com/3ldfza Five Principles Fraud Detection Coordinated Investigation Approach

Objective 1b

Page 27: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Managing the Business Risk of Fraud: A Practical Guide

ACFE, IIA and AICPA Exposure draft issued 11/2007, final 5/2008

Section 4 – Fraud Detection

Objective 1b

Page 28: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Guidance Paper

Five Sections– Fraud Risk Governance– Fraud Risk Assessment– Fraud Prevention– Fraud Detection– Fraud Investigation and

corrective action

Page 29: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Risk Governance

Fraud risk management program Written policy – management’s expectations

regarding managing fraud risk

Page 30: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Risk Assessment

Periodic review and assessment of potential schemes and events

Need to mitigate risk

Page 31: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Fraud Prevention

Establish prevention techniques Mitigate possible impact on the organization

Page 32: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Fraud Detection

Establish detection techniques for fraud “Back stop” where preventive measures fail,

or Unmitigated risks are realized

Page 33: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Fraud Investigation and Corrective Action

Reporting process to solicit input on fraud Coordinated approach to investigation Use of corrective action

Page 34: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

“60 Minutes” – “World of Trouble”

2/15/09 – Scott Pelley– Fraud Risk Governance – “one grand wink-wink,

nod-nod “– Fraud Risk Assessment - categorically false – Fraud Prevention – “my husband passed away”– Fraud Detection - We didn't know? Never saw one.– Fraud Investigation and corrective action - Pick-A-

Payment losses $36 billion

Page 35: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Section 4 – Fraud Detection Detective Controls Process Controls Anonymous Reporting Internal Auditing Proactive Fraud Detection

Objective 1b

Page 36: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Proactive Fraud Detection

Data Analysis to identify:– Anomalies– Trends– Risk indicators

Objective 1b

Page 37: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Fraud Detective Controls

Operate in the background Not evident in everyday business

environment These techniques usually –

– Occur in ordinary course of business– Corroboration using external information– Automatically communicate deficiencies– Use results to enhance other controls

Page 38: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Examples of detective controls

Whistleblower hot-lines (DHHS and OSA have them)

Process controls (Medicaid audits and edits) Proactive fraud detection procedures

– Data analysis– Continuous monitoring– Benford’s Law

Page 39: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Specific Examples Cited

Journal entries – suspicious transactions

Identification of relationships Benford’s Law Continuous monitoring

Objective 1b

Page 40: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Data Analysis enhances ability to detect fraud

Identify hidden relationships Identify suspicious transactions Assess effectiveness of internal

controls Monitor fraud threats Analyze millions of transactions

Objective 1b

Page 41: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Continuous Monitoring of Fraud Detection

Organization should develop ongoing monitoring and measurements

Establish measurement criteria (and communicate to Board)

Measurable criteria include:

Page 42: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Measurable Criteria – number of

fraud allegations fraud investigations resolved Employees attending annual ethics course Whistle blower allegations Messages supporting ethical behavior

delivered by executives Vendors signing ethical behavior standards

Page 43: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Management ownership of each technique implemented

Each process owner should:– Evaluate effectiveness of technique regularly– Adjust technique as required– Document adjustments– Report modifications needed for techniques which

become less effective

Page 44: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Practice Advisory 2320-1Analysis and Evaluation

International standards for the professional practice of Internal Auditing

Analytical audit procedures– Efficient and effective– Useful in detecting

Differences that are not expected Potential errors Potential irregularities

Page 45: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Analytical Audit Procedures

May include– Study of relationships– Comparison of amounts with

similar information in the organization

– Comparison of amounts with similar information in the industry

Page 46: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Analytical audit procedures

Performed using monetary amounts, physical quantities, ratios or percentages

Ratio, trend and regression analysis Period to period comparisons Auditors should use analytical audit

procedures in planning the engagement

Page 47: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Factors to consider

Significance of the area being audited Assessment of risk Adequacy of system of internal control Availability and reliability of information Extent to which procedures provide support

for engagement results

Page 48: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Peeling the Onion

Population as Whole

Possible Error Conditions

Fraud Items

Objective 1c

Page 49: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Fraud Pattern Detection

Market Basket

Stratification

Trend Line

HolidayDay of Week

Duplicates

Univariate

Gaps

Benford’s LawRound Numbers

Target Group

Objective 1d

Page 50: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Digital Analysis (5W) Who What Why Where When

Objective 1e

Page 51: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Who Uses Digital Analysis

Traditionally, IT specialists With appropriate tools, audit

generalists (CAATs) Growing trend of business

analytics Essential component of

continuous monitoring

Objective 1e

Page 52: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

What - Digital Analysis

Using software to:– Classify– Quantify– Compare

Both numeric and non-numeric data

Objective 1e

Page 53: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

How - Assessing fraud risk

Basis is quantification Software can do the “leg work” Statistical measures of difference

– Chi square– Kolmogorov-Smirnov– D-statistic

Specific approaches

Objective 1e

Page 54: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Why - Advantages

Automated process Handle large data populations Objective, quantifiable metrics Can be part of continuous monitoring Can produce useful business analytics 100% testing is possible Quantify risk Repeatable process

Objective 1e

Page 55: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Why - Disadvantages

Costly (time and software costs) Learning curve Requires specialized knowledge

Objective 1e

Page 56: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

When to Use Digital Analysis

Traditional – intermittent (one off) Trend is to use it as often as possible Continuous monitoring Scheduled processing

Objective 1e

Page 57: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Where Is It Applicable?

Any organization with data in digital format, and especially if:– Volumes are large– Data structures are complex– Potential for fraud exists

Objective 1e

Page 58: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Objective 1 Summarized

Three brief examples CFE Guidance Paper “Top 10” Metrics Process Overview Who, What, Why, When & Where

Objective 1

Page 59: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Objective 1 - Summarized

1. Understand why and how 2. Understand statistical basis for quantifying

differences3. Identify ten general tools and techniques4. Understand use of Excel5. How pattern detection fits in

Next is the basis …

Page 60: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Basis for Pattern Detection

Analytical review Isolate the “significant few” Detection of errors Quantified approach

Objective 2

Page 61: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Understanding the Basis

Quantified Approach Population vs. Groups Measuring the Difference Stat 101 – Counts, Totals, Chi

Square and K-S The metrics used

Objective 2

Page 62: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Quantified Approach

Based on measureable differences

Population vs. Group “Shotgun” technique

Objective 2a

Page 63: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Detection of Fraud Characteristics

Something is different than expected

Objective 2a

Page 64: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Fraud patterns

Common theme – “something is different”

Groups Group pattern is different than

overall population

Objective 2b

Page 65: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Measurement Basis

Transaction counts

Transaction amounts

Objective 2c

Page 66: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

A few words about statistics (the “s” word)

Detailed knowledge of statistics not necessary

Software packages do the “number-crunching”

Statistics used only to highlight potential errors/frauds

Not used for quantification

Objective 2d

Page 67: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

How is digital analysis done?

Comparison of group with population as a whole

Can be based on either counts or amounts Difference is measured Groups can then be ranked using a selected

measure High difference = possible error/fraud

Objective 2d

Page 68: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Demo in Excel of the process

Based roughly on the Wake County Transportation fraud

Illustrates how the process works, using Excel

Page 69: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Histograms

Attributes tallied and categorized into “bins” Counts or sums of amounts

Objective 2d

Page 70: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Two histograms obtained

Population and groupPopulation

0100200300400500600700

Jan-07

Feb-07

Mar-07

Apr-07

May-07

Jun-07

Jul-07

Aug-07

Sep-07

Oct-07

Nov-07

Dec-07

Group

01020304050607080

Jan-07

Feb-07

Mar-07

Apr-07

May-07

Jun-07

Jul-07

Aug-07

Sep-07

Oct-07

Nov-07

Dec-07

Objective 2d

Page 71: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Histograms

Attributes tallied and categorized into “bins” Counts or sums of amounts

Objective 2d

Page 72: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Compute Cumulative Amount for each

Count by Month

0

10

20

30

40

50

60

70

80

Jan-0

7Feb

-07Mar-

07Apr-

07May-

07Ju

n-07Jul

-07

Aug-07

Sep-07

Oct-07Nov-

07Dec

-07

M onth

Coun

t

Cum Pct

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

Jan-0

7

Mar-07

May-07

Jul-0

7

Sep-07

Nov-07

Objective 2d

Page 73: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Are the histograms different?

Two statistical measures of difference

Chi Squared (counts) K-S (distribution) Both yield a difference metric

Objective 2d

Page 74: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Chi Squared

Classic test on data in a table Answers the question – are the

rows/columns different Some limitations on when it can be

applied

Objective 2d

Page 75: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Chi Squared

Table of Counts Degrees of Freedom Chi Squared Value P-statistic Computationally intensive

Objective 2d

Page 76: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Kolmogorov-Smirnov

Two Russian mathematicians

Comparison of distributions Metric is the “d-statistic”

Objective 2d

Page 77: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

How is K-S test done?

Four step process1. For each cluster element

determine percentage2. Then calculate cumulative

percentage3. Compare the differences in

cumulative percentages4. Identify the largest difference

Objective 2d

Page 78: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Kolmogorov-Smirnov

Objective 2d - KS

Page 79: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Classification by metrics

Stratification Day of week Happens on holiday Round numbers Variability Benford’s Law Trend lines Relationships (market basket) Gaps Duplicates

Objective 2e

Page 80: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Auditor’s “Top 10” Metrics

1. Outliers / Variability2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Objective e

Page 81: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Understanding the Basis

Quantified Approach Population vs. Groups Measuring the Difference Stat 101 – Counts, Totals, Chi Square

and K-S The metrics used

Objective 2

Page 82: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Objective 2 - Summarized

1. Understand why and how 2. Understand statistical basis for quantifying

differences3. Identify ten general tools and techniques4. Understand examples done using Excel5. How pattern detection fits in

Next are the metrics …

Page 83: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

It’s that time!

Session Break!

Page 84: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

The “Top 10” Metrics

Overview Explain Each Metric Examples of what it can detect How to assess results

Objective 3

Page 85: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Trapping anomalies

Objective 3

Page 86: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Fraud Pattern Detection

Market Basket

Stratification

Trend Line

HolidayDay of Week

Duplicates

Univariate

Gaps

Benford’s LawRound Numbers

Target Group

Objective 3

Page 87: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Outliers / Variability

Outliers are amounts which are significantly different from the rest of the population

1 - Outliers

Page 88: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Outliers / Variability

Charting (visual) Software to analyze “z-scores” Top and Bottom 10, 20 etc. High and low variability (coefficient

of variation)

1 - Outliers

Page 89: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Drill down to the group level

Basic statistics– Minimum, maximum

and average– Variability

Sort by statistic of interest– Variability (coefficient

of variation)– Maximum, etc.

1 - Outliers

Page 90: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Example Results

Provider N Coeff Var

3478421 3,243 342.232356721 4,536 87.233546789 3,421 23.255463122 2,311 18.54

Two providers (3478421 and 2356721) had significantly more variability in the amounts of their claims than all the rest.

1 - Outliers

Page 91: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Next Metric

1. Outliers2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Page 92: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Unusual stratification patterns

Do you know how your data looks?

2 - Stratification

Page 93: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Stratification - How

Charting (visual) Chi Squared Kolmogorov-Smirnov By groups

2 - Stratification

Page 94: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Purpose / types of errors

Transactions out of the ordinary “Up-coding” insurance claims “Skewed” groupings Based on either count or amount

2 – Stratification

Page 95: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

The process?

1. Stratify the entire population into “bins” specified by auditor

2. Same stratification on each group (e.g. vendor)

3. Compare the group tested to the population

4. Obtain measure of difference for each group

5. Sort descending on difference measure

2 – Stratification

Page 96: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Units of Service Stratified - Example Results

Two providers (2735211 and 4562134) are shown to be much different from the overall population (as measured by Chi Square).

Provider N Chi Sq D-stat2735211 6,011 7,453 0.8453

4562134 8,913 5,234 0.7453

4321089 3,410 342 0.5231

4237869 2,503 298 0.4632

2 – Stratification

Page 97: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Next Metric

1. Outliers2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Page 98: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Day of Week

Activity on weekdays Activity on weekends Peak activity mid to late week

3 – Day of Week

Page 99: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Purpose / Type of Errors

Identify unusually high/low activity on one or more days of week

Dentist who only handled Medicaid on Tuesday

Office is empty on Friday

3 – Day of Week

Page 100: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

How it is done?

Programmatically check entire population Obtain counts and sums by day of week

(1-7) Prepare histogram For each group do the same procedure Compare the two histograms Sort descending by metric (chi square/d-

stat)

Page 101: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Day of Week - Example Results

Provider 2735211 only provided service for Medicaid on Tuesdays. Provider 4562134 was closed on Thursdays and Fridays.

Provider N Chi Sq D-stat

2735211 5,404 12,435 0.9802

4562134 5,182 7,746 0.8472

4321089 5,162 87 0.321

4237869 7,905 56 0.2189

3 – Day of Week

Page 102: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Next Metric

1. Outliers2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Page 103: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Round Numbers

It’s about….

Estimates!

4 – Round Numbers

Page 104: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Purpose / Type of Errors

Isolate estimates Highlight account numbers in

journal entries with round numbers

Split purchases (“under the radar”) Which groups have the most

estimates

4 – Round Numbers

Page 105: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Round numbers

Classify population amounts– $1,375.23 is not round– $5,000 is a round number – type 3 (3

zeros)– $10,200 is a round number type 2 (2

zeros) Quantify expected vs. actual (d-statistic) Generally represents an estimate Journal entries

4 – Round Numbers

Page 106: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Round Numbers in Journal Entries - Example Results

Two accounts, 2735211 and 4562134 have significantly more round number postings than any other posting account in the journal entries.

Account N Chi Sq D-stat

2735211 4,136 54,637 0.9802

4562134 833 35,324 0.97023

4321089 8,318 768 0.321

4237869 9,549 546 0.2189

4 – Round Numbers

Page 107: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Next Metric

1. Outliers2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Page 108: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Made up Numbers

Curb stoning Imaginary numbers

Benford’s Law

5 – Made up numbers

Page 109: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

What can be detected Made up numbers –

e.g. falsified inventory counts, tax return schedules

5 – Made Up Numbers

Page 110: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Benford’s Law using Excel

Basic formula is “=log(1+(1/N))” Workbook with formulae available at

http://tinyurl.com/4vmcfs Obtain leading digits using “Left”

function, e.g. left(Cell,1)

5 – Made Up Numbers

Page 111: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Made up numbers

Benford’s Law Check Chi Square and d-statistic First 1,2,3 digits Last 1,2 digits Second digit Sources for more info

5 – Made Up Numbers

Page 112: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

How is it done?

Decide type of test – (first 1-3 digits, last 1-2 digit etc)

For each group, count number of observations for each digit pattern

Prepare histogram Based on total count, compute expected

values For the group, compute Chi Square and

d-stat Sort descending by metric (chi square/d-

stat)

5 – Made Up Numbers

Page 113: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Invoice Amounts tested with Benford’s law - Example Results

During tests of invoices by store, two stores, 324 and 563 have significantly more differences than any other store as measured by Benford’s Law.

Store Hi Digit Chi Sq D-stat

324 79 5,234 0.9802

563 89 4,735 0.97023

432 23 476 0.321

217 74 312 0.2189

5 – Made Up Numbers

Page 114: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Next Metric

1. Outliers2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Page 115: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Market Basket

Medical “Ping ponging” Pattern associations Apriori program References at end of slides Apriori – Latin a (from) priori

(former) Deduction from the known

6 – Market Basket

Page 116: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Purpose / Type of Errors

Unexpected patterns and associations

Based on “market basket” concept Unusual combinations of diagnosis

code on medical insurance claim

6 – Market basket

Page 117: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Market Basket

JE Accounts JE Approvals Credit card fraud in Japan –

taxi and ATM

6 – Market basket

Page 118: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

How is it done?

First, identify groups, e.g. all medical providers for a patient

Next, for each provider, assign a unique integer value

Create a text file containing the values

Run “apriori” analysis

6 – Market basket

Page 119: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Apriori outputs

For each unique value, probability of other values

If you see Dr. Jones, you will also see Dr. Smith (80% probability)

If you see a JE to account ABC, there will also an entry to account XYZ (30%)

6 – Market basket

Page 120: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Next Metric

1. Outliers2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Page 121: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Trend BustersDoes the pattern make sense?

ACME Technology

05,000

10,00015,00020,00025,00030,000

Jan-0

7Mar-0

7

May-07

Jul-0

7

Sep-07

Nov-07

Jan-0

8Mar-0

8

May-08

Date

Amou

nt Sales

Em ployee Count

7 - Trends

Page 122: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Trend Busters

Linear regression Sales are up, but cost of goods sold is

down “Spikes”

7 – Trends

Page 123: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Purpose / Type of Errors

Identify trend lines, slopes, etc.

Correlate trends Identify anomalies Key punch errors where

amount is order of magnitude

7 – Trends

Page 124: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Linear Regression

Test relationships (e.g. invoice amount and sales tax)

Perform multi-variable analysis

7 – Trends

Page 125: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

How is it done?

Estimate linear trends using “best fit”

Measure variability (standard errors)

Measure slope Sort descending by slope,

variability, etc.

7 – Trends

Page 126: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Trend Lines by Account - Example Results

Generally the trend is gently sloping up, but two accounts (43870 and 54630) are different.

Account N Slope Std Err

32451 18 1.230 0.87

43517 17 1.070 4.3

32451 27 1.023 0.85

43517 32 1.010 0.36

43870 23 0.340 2.36

54630 56 -0.560 1.89

7 – Trends

Page 127: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Scatter plot with prediction and confidence intervals

Page 128: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Next Metric

1. Outliers2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Page 129: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Numeric Sequence Gaps

What’s there is interesting, what’s not there is critical …

8 - Gaps

Page 130: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Purpose / Type of Errors

Missing documents (sales, cash, etc.)

Inventory losses (missing receiving reports)

Items that “walked off”

8 – Gaps

Page 131: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

How is it done?

Check any sequence of numbers supposed to be complete, e.g.

Cash receipts Sales slips Purchase orders

8 – Gaps

Page 132: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Gaps Using Excel

Excel – sort and check Excel formula Sequential numbers and dates

8 – Gaps

Page 133: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Gap Testing - Example Results

Four check numbers are missing.

Start End Missing

10789 10791 1

12523 12526 2

17546 17548 1

8 – Gaps

Page 134: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Next Metric

1. Outliers2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Page 135: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Duplicates

Why is there more than one?

Same, Same, Same, and

Same, Same, Different

9 - Duplicates

Page 136: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Two types of (related) tests

Same items – same vendor, same invoice number, same invoice date, same amount

Different items – same employee name, same city, different social security number

9 – Duplicates

Page 137: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Duplicate Payments

High payback area “Fuzzy” logic Overriding software

controls

9 - Duplicates

Page 138: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Fuzzy matching with software

Levenshtein distance Soundex “Like” clause in SQL Regular expression

testing in SQL Vendor/employee

situations

Russian physicist

9 - Duplicates

Page 139: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

How is it done?

First, sort file in sequence for testing

Compare items in consecutive rows

Extract exceptions for follow-up

9 - Duplicates

Page 140: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Possible Duplicates - Example Results

Five invoices may be duplicates.

Vendor Invoice Date Invoice Amount

Count

10245 6/15/2007 3,544.78 4

10245 8/31/2007 2,010.37 2

17546 2/12/2007 1,500.00 2

9 - Duplicates

Page 141: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Next Metric

1. Outliers2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Page 142: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Date Checking

If we’re closed, why is there …

Adjusting journal entry?Receiving report?

Payment issued?

10 - Dates

Page 143: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Holiday Date Testing

Red Flag indicator

10 – Dates

Page 144: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Date Testing challenges

Difficult to determine Floating holidays –

Friday, Saturday, Sunday, Monday

10 – Dates

Page 145: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Typical audit areas

Journal entries Employee expense

reports Business telephone calls Invoices Receiving reports Purchase orders

10 – Dates

Page 146: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Determination of Dates

Transactions when business is closed

Federal Office of Budget Management

An excellent fraud indicator in some cases

10 – Dates

Page 147: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Holiday Date Testing

Identifying holiday dates:– Error prone– Tedious

U.S. only

10 – Dates

Page 148: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Federal Holidays

Established by Law Ten dates Specific date (unless

weekend), OR Floating holiday

10 – Dates

Page 149: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Federal Holiday Schedule

Office of Personnel Management Example of specific date – Independence

Day, July 4th (unless weekend) Example of floating date – Martin Luther

King’s birthday (3rd Monday in January) Floating – Thanksgiving – 4th Thursday in

November

10 – Dates

Page 150: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

How it is done?

Programmatically count holidays for entire population

For each group, count holidays Compare the two histograms (group

and population) Sort descending by metric (chi

square/d-stat)

10 – Dates

Page 151: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Holiday Counts - Example Results

Two employees (10245 and 32325) were “off the chart” in terms of expense amounts incurred on a Federal Holiday.

Employee Number

N Chi Sq D-stat

10245 37 5,234 0.980232325 23 4,735 0.9702317546 18 476 0.32124135 34 312 0.2189

10 – Dates

Page 152: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

The “Top 10” Metrics

Overview Explain Each Metric Examples of what it can detect How to assess results

Objective 3

Page 153: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Objective 3 - Summarized

1. Understand why and how 2. Understand statistical basis for quantifying

differences3. Identify ten general tools and techniques4. Understand examples done using Excel5. How pattern detection fits in

Next – using Excel …

Page 154: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Use of Excel

Built-in functions Add-ins Macros Database access

Objective 4

Page 155: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Excel templates

Variety of tests– Round numbers– Benford’s Law– Outliers– Etc.

Objective 4

Page 156: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Excel – Univariate statistics

Work with Ranges =sum, =average, =stdevp =largest(Range,1),

=smallest(Range,1) =min, =max, =count Tools | Data Analysis | Descriptive

Statistics

Objective 4

Page 157: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Excel Histograms

Tools | Data Analysis | Histogram Bin Range Data Range

Objective 4

Page 158: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Excel Gaps testing

Sort by sequential value =if(thiscell-lastcell <>

1,thiscell-lastcell,0) Copy/paste special Sort

Objective 4

Page 159: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Detecting duplicates with Excel

Sort by sort values =if testing =if(=and(thiscell=lastcell, etc.))

Objective 4

Page 160: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Performing audit tests with macros

Repeatable process Audit standardization Learning curve Streamlining of tests More efficient and effective Examples -

http://ezrstats.com/Macros/home.html

Objective 4

Page 161: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Using database audit software

Many “built-in” functions right off the shelf with SQL

Control totals Exception identification “Drill down” Quantification June 2008 article in the EDP Audit &

Control Journal (EDPACS) “SQL as an audit tool”

http://ezrstats.com/doc/SQL_As_An_Audit_Tool.pdf

Objective 4

Page 162: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Use of Excel

Built-in functions Add-ins Macros Database access

Objective 4

Page 163: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Objective 4 - Summarized

1. Understand why and how 2. Understand statistical basis for quantifying

differences3. Identify ten general tools and techniques4. Understand examples done using Excel5. How Pattern Detection fits in

Next – Fit …

Page 164: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

How Pattern Detection Fits In

Business Analytics Fraud Pattern Detection Continuous monitoring

Objective 5

Page 165: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Where does Fraud Pattern Detection fit in?

Business Analytics Fraud Pattern Detection Continuous fraud pattern

detection Continuous Monitoring

Right in the middle

Objective 5

Page 166: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Business Analytics

Fraud analytics -> business analytics

Business analytics -> fraud analytics

Objective 5

Page 167: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Role in Continuous Monitoring (CM)

Fraud analytics can feed (CM) Continuous fraud pattern detection Use output from CM to tune fraud

pattern detection

Objective 5

Page 168: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Objective 5 - Summarized

1. Understand why and how 2. Understand statistical basis for quantifying

differences3. Identify ten general tools and techniques4. Understand use of Excel5. How pattern detection fits in

Next: Links …

Page 169: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Links for more information

Kolmogorov-Smirnov http://tinyurl.com/y49sec Benford’s Law http://tinyurl.com/3qapzu Chi Square tests http://tinyurl.com/43nkdh Continuous monitoring

http://tinyurl.com/3pltdl

Page 170: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Market Basket

Apriori testing for “ping ponging” Temple University

http://tinyurl.com/5vax7r Apriori program (“open source”)

http://tinyurl.com/5qehd5 Article – “Medical ping ponging”

http://tinyurl.com/5pzbh4

Page 171: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Excel macros used in auditing

Excel as an audit software http://tinyurl.com/6h3ye7

Selected macros - http://ezrstats.com/Macros/home.html

Spreadsheets forever - http://tinyurl.com/5ppl7t

Page 172: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Questions?

Page 173: Fraud Detection Using A Database Platform

Fraud detection using a database platform EZ-R Stats, LLC

Contact info

Phone: (919)-219-1622 E-mail:

[email protected] Blog: http://blog.ezrstats.com