T107: Proactive Use of Data Analytics in Audits, Fraud Prevention and Detection

Pleased to introduce our panel members – Carrie A. Hug, Director of Accountability,

Recovery Accountability and Transparency Board

– Tina Kim, Deputy Comptroller for State Government Accountability, New York State Office of the Comptroller

– Vijay Chheda, Senior Director –Audits Amtrak Office of Inspector General

POLLING QUESTION 1

Which is a famous Lord Acton quote?

a) Never buy a car you can't push.

b) When everything's coming your way, you're in the wrong lane.

c) Nobody cares if you can't dance well. Just get up and dance.

d) The early worm gets eaten by the bird, so sleep late.

e) Power corrupts, and absolute power corrupts absolutely.

Reasons We Miss Fraud

Not adequately verifying—drive bys Tend to avoid conflict with people Education—fraud detection not taught in

school Pressure to finish audits Auditor vs. investigator—auditors have bias

toward documents while investigators have bias toward witnesses

Don’t understand business operations and impact of control weaknesses

Not talking to lower level personnel Warning signs not recognized

Reasons We Miss Fraud When We Get Data Poorly defined scope Data acquisition Manually maintained data False positives Lack of familiarity Data storage systems Software systems Organizational processes Lack of support from Sr. Leadership

Proactive Use of Data Analytics in Audits, Fraud Prevention and Detection

Carrie A. Hug

Director of Accountability

Recovery Accountability and Transparency Board

September 15, 2015

Overview

• Establishing a Data Intelligence Center

• Obtaining Data• Ingesting Data• Using Data• Sharing Data

Establishing a Data Intelligence Center

• Mission/Scope• Resources• Challenges• Partnerships/

Customers

Obtaining Data

• What? Why? How?• Costs• Governance• Documentation

Ingesting Data

• Accessibility• Normalization• Correlation• Storage/Space

Using Data

• Legal Authority• Systems/Tools• Process/Policies• Analytics

Sharing Data

• Legal Barriers• Other Obstacles• Agreements• Cost/Benefit

Benefits of Data Analytics

• Resource Allocation

• ROI/Savings• Detection of

Fraud

Lessons Learned

• Plan, Test, Re-align• Explore Multiple Tools • Communicate• Collaborate

Polling Question 2• Which analytics tool(s) does your organization

use to help identify high risk transactions or activities (pick as many as used)?a) Data mining and predictive analytics

b) Data interrogation – e.g., ACL, IDEA, MS Access, Excel

c) Statistical analysis – e.g., SPSS and SAS

d) Link analysis – I2

e) Lexis-Nexis

f) Data conversion utilities (Monarch)

g) Internet, open-source research

h) Access to system query tools

i) Sampling

Outline

• Background, Tools Used• Eye Care Audit – Use of Data Models• Pharmacy Audit –Creative Data Analytics• Pre-School Special Education – Risk Analysis • New Initiative –Data Scraping

New York State Office of the State Comptroller (OSC)

The NYS Comptroller is the State's chief fiscal officer. Responsibilities

include:• Acting as sole trustee of the $176.8 billion (as of March 31, 2014) NYS

Common Retirement Fund; administering the State & Local Retirement System for public employees

• Maintaining the State's accounting system; administering the State’s approximately $15 billion payroll

• Reviewing State contracts and payments before they are issued• Conducting audits of State agencies and public benefit corporations• Overseeing the fiscal affairs of local governments, including New York City• Acting as custodian of more than $13 billion in unclaimed funds and restoring

lost accounts to their rightful owners• http://osc.state.ny.us

© New York State Office of the State Comptroller

NYS OSC SGA May 2015

OSC’s Medical Claims UnitNew York State Medicaid Program• Government program that provides medical services to low income

residents• Over 6 million enrollees• About $50 billion in annual expenditures

New York State Health Insurance Program (NYSHIP)• Provides health insurance coverage to active and retired State,

participating local government, and school district employees and their dependents

• Over 1.2 million enrollees • About $6 billion in annual expenditures

NYS OSC SGA May 2015

High Rate Of Return On InvestmentHave you hired the right individuals to

advance your organization’s use of

data analytics??

MCU cost savings since Jan 2011

* Approximately $900 million *

Audit team comprised of

individuals with various

backgrounds

Tools?? Medicaid Data Warehouse (Oracle database); NYSHIP Decision Support System

eMedNY – State’s Medicaid claims processing and payment system

• IBM SPSS Modeler; SAS JMP – data analysis and data mining software

• R – Free software used for data analytics

• ESRI ArcGIS – Geographical Information System

• LexisNexis Accurint – software used to detect relationships between people and/or businesses

• Internet – research people, businesses, and locations

provided by auditee

NYS OSC SGA May 2015

Brooklyn Eye Care Providers

What We Found:

Ten eye care providers in Brooklyn, NY exploited a control weakness in the Medicaid claims processing system to repeatedly bill for excessive services

• It appeared services were not provided and providers were colluding in their inappropriate billing practices

• We recommended the review and recovery of $3.2 million in Medicaid payments to the ten providers

• Project Roadmap• Data Preparation• Identify Relevant Data:*Feature Selection*• K-Means Cluster Model• QA the Results

How We Did It

Plan, Collaborate, Brainstorm??

Do you and your teams take time to• Plan out the Roadmap of the Project…

what,how,who,when• Team brainstorming to create “OSC Variables”

(summary billing statistics by provider that capture billing behavior, represent potential billing risks)

“OSC Variables”Eye Care Providers

Avg amount paid/day

Max amount paid/day

Avg amount paid/patient visit

Max amount paid/patient visit

Avg # of procedures performed / patient visit

Max # of procedures performed / patient visit

Avg # of patients seen/day

Max # of patients seen per day

(40+ other Variables)

Provider A $417 $650 $20 $60 2 3 20 26 …

Provider B $550 $601 $31 $120 3 5 21 25  

Provider C $500 $543 $25 $100 2 4 20 26  

Provider D $1,650 $1,900 $44 $190 5 6 40 49  

Provider 1,001 …                

Identify Data:• Do you have a sound Business Knowledge of the data you are working

with??

• If not, understand the program and data you are working with (opticians, optometrists, ophthalmologists…do we want all claims?)

• Coordinate with the auditee or program experts• Know what data is available• Know what the data fields mean (data dictionary)

Obtain and Test Data Integrity:• Format• Reasonability• Completeness

Features Selection• Identifies the data that is relevant to your analysis• In our case, the importance of each OSC Variable

You only want to use

‘data fields’ (variables)

that have an impact on

your target (key variable)

in the analysis

*such as amount paid

or number of patients*

K-Mean Cluster AnalysisUncovers patterns in the data, clusters the data into different groups• The OSC Variables were run through the K-Means model• Providers in each group are similar to one another• **Look further into groups and remove “noise.” Refine the analysis by

removing certain providers. Re-run it!**

• Interested in outliers

Identify Potential CollusionUse data such as names, addresses and phone numbers to reveal

potential associations

• Some tools:– WizSame– SoundEx – phonetic similarities (sounds the same, but spelled differently).

The following have the same SoundEx encoding:• Smith (ex: SoundEx converts it to: S530)• Schmidt (ex: SoundEx converts it to: S530)• Smythe (ex: SoundEx converts it to: S530)

• 1234 15th street (works best if split address into separate fields)• 12-34 15 str

• 5558675309, 555-867-5309

Number of Patients in Common

1257

423

6239

454281

92

201

1

2

1

2

121

350 350

Test/QA Data Results

Throughout the analysis, do you test your results back to source data??

• Helps ensure high quality, accurate results• Helps ensure you are getting what you expect• Ultimately, it saves time - address problems

upfront, refine analysis

Polling Question 3

• How many of those participating use automated techniques to follow-up on high risk transactions or activities?

• Yes, we use automated techniques• No, we do not use automated techniques

Davis Ethical Pharmacy

What We Found:

An analysis of the pharmacy’s Medicaid & NYSHIP claims identified irregular billing patterns

• Billed for medications for individuals who, at the time the medications were supposedly prescribed, were not patients of the prescribing physicians

• Billed for medications that were not authorized by prescribing physicians; billed for excessive quantities

• Prescriptions were filled outside of pharmacy’s normal business hours

• Prescriptions had no indication they were received by patients

What worked for us:

• Enriching our Business Knowledge of Pharmacy Data– Learned key information about adjudication of

pharmacy claims

• Brought in outside data to analyze pharmacy claims– Performed data analytics on the outside data and the

pharmacy claims - identified patterns

• Obtained an image of the pharmacy’s computer server– Merged pharmacist’s login data with pharmacy

claims; determined which pharmacist was logged in when suspicious claims were adjudicated

– Merged pharmacy’s patient signature data and delivery data with pharmacy claims; identified drugs that were not received by patients

Pre-School Special Education

• New York spends about $1.4 billion annually to provide special services and classroom instruction to approximately 81,000 preschool children with physical, developmental and emotional disabilities. We have identified fraud and improper use of taxpayer funds in a recent series of audits and investigations of special education providers, resulting in multiple criminal convictions [10 arrests and five criminal convictions] and the recovery of more than $5 million.

• OSC has completed 46 audits of preschool special education providers, finding over $43 million in unsupported or inappropriate charges.

New Venture - Data Scraping

• (aka Web Scraping) – Extract unstructured data from websites and return it in a structured data format that can be used for data analytics

PROACTIVE USE OF DATA ANALYTICS

AMTRAK OFFICE OF INSPECTOR GENERAL

WHO Background and Introduction

WHAT Data Analytics work at Amtrak OIG

HOW Data Analytics strategy at Amtrak OIG

Examples

Reasons to use Data Analytics

Challenges of Data Analytics Work

How to leverage Data Analytics for your work

Agenda

AmtrakOIG: - 50 Auditors, 30 Investigators,

17 Support Staff - Data Analytics Team - 8 (5

employees, 3 contractors)

DA Team: - Performs own audits

- Provides support to other audits

- Supports Investigations group

Background and Introduction

Data Analytics work at Amtrak OIG

Access to 15+ systems (80% of Amtrak’s financial data)

Audit in 10 different business areas including- Accounts Payable Procurement Payroll Human Capital Health care Operations

150+ tests 18 Reports

Opportunities to reduce cost

Opportunities to increase revenue

Opportunities to improve control effectiveness or increase program efficiency

Recalculations/Compliance testing

Identify potential fraud

Data Analytics for different Audit Objectives

Polling Question 4

• How many of those attending use automated techniques perform analytics testing on the results provided from follow-up on high risk transactions or activities – to identify anomalous behavior by individuals, business units, components, or the organization?

• Yes, we use automated techniques• No, we do not use automated techniques

Support – OIG and Amtrak leadership

Strategy – Shared services model

Sourcing – Hired technical expertise

Environment – Centralized

Value – Pilot quick win

Tools – Limit to ACL and Excel

Security – Encrypt all data

Data Analytics program at Amtrak OIG

E X A M P L E S

From October 2005 - June 2013, $14.1 billion paid to vendors

Data Analysis identified potential duplicate invoices paid of about $7.5 million

Finance staff recovered about $3.5 million

Four major causes:

Clerks processed known duplicate payments despite system warnings

Duplicate vendors not detected by the automated controls

Clerks did not ensure correct invoice numbers are entered

Same Invoices received by different departments were paid

Background: Duplicate Payments

Duplicate Payments

Vendor NameKeyed

Invoice NoInvoice Date

Invoice Amount

ERICO PRODUCTS INC 130587 06/07/11 8,062.32

VOSSLOH TRACK MATERIAL INC 0000130587 06/07/11 8,062.32

ADT SECURITY SERVICES INC 75277679 07/07/12 5,224.02

DO NOT USE 75277679 07/07/12 5,224.02

W FRANKLIN LP 51017 10/21/11 3,600.00

Lorraine K Koc 51017A 10/21/11 3,600.00

FEDEX 790264830 05/28/12 1,901.04

FEDEX EXPRESS 790264830 05/28/12 1,901.04

Background: Material Price Variance Company buys materials form different vendors for

different plants across the country

Analyzed $35 million worth of material POs

If lowest price vendor was selected for all materials bought in CY 2013, company could have saved $3.4 million

Causes:

Weaknesses in material requirement forecasting

Limited number of approved vendors

Material Price Variance

Material NumberVendor Name

Nbr Of POs PO Amount PO Qty

Avg Unit Price

% Variance

000000003710500004 EXXONMOBIL 6 $29,921 5,376 $6 16.52

000000003710500004 VALDES ENTERPRISES INC 1 $14,953 2,304 $6 16.52

000000000299900382 GE TRANSPORTATION SYSTEMS 11 $36,895 235 $157 12.74

000000000299900382 GE TRANSPORTATION SYSTEMS 1 $885 5 $177 12.74

000000000104500004 KOPPERS INDUSTRIES 11 $723,013 364 $1,986 11.34

000000000104500004 LB FOSTER 17 $705,459 319 $2,211 11.34

Material Number PO Amount PO QuantityLowest Avg

Unit PriceAmt At

Lowest PriceVariance To

Lowest

000000000104500004 $1,428,472 683.00 $1,986 $1,356,643 $71,829

000000003628500557 $161,299 4,940.28 $21 $104,092 $57,208

000000003733300001 $2,480,196 371,450.80 $7 $2,425,574 $54,622

000000000256404084 $66,486 60.00 $952 $57,120 $9,366

Background: Profile of Timesheet Data Amtrak’s major expenses is labor – $1.2

billion paid to union employees in CY 2014

Amtrak has 14 unions and 23 bargaining agreements representing different crafts

6 timekeeping systems

Data revealed trends and patterns that raise questions about whether overtime and regular time is appropriately reported

Summary of Weekly Overtime as Percent of Regular

Time

Summary of Regular and Overtime Hours Reported in Daily Timesheets

Summary of Consecutive Days Worked

Top Occurrences – Consecutive Days Worked

SAP ID Job Title UnionStart Date

End Date

Days Worke

d

MAINTAINER SD BRS-SW 12/19/2013 5/14/2014

147

AGENT TICKET CLK FC

TCU-OFF 4/30/2014 9/8/2014 132

MAINTAINER SD BRS-SW 12/26/2013 5/1/2014 127

MAINTAINER SD BRS-SW 2/20/2014 6/18/2014

119

COACH CLEANER JCC 3/2/2014 6/17/2014

108

C XXXXXXXXX XXXXXX XXXXXX XXXXXX XXX

TICKET/ACCTNG CLERK

TCU-OFF 6/4/2014 9/15/2014

104

ENG WORK EQUIP SD B

BMWE-NEC 6/24/2014 10/1/2014

100

Identify risk areas with high degree of assurance in finding results

Mine through 100% of transactions

Advantage over business

Read data from any system, no size limitations

Bring disparate sets of data in one view – hard for business to do

Helps break down complex business processes

Reasons to use Data Analytics

Individual Level

New skill to acquire – lack of commitment to learn

Lack of vision and support from management

Overwhelmed with the data - not knowing where to start

Unclear objectives – a fishing exercise

Understanding the data and the business process

Uncooperative auditee makes the process difficult to get meaningful results

Challenges of Data Analytics work

Organization/Agency Level

Obtaining access to the data

Storing and securing sensitive data

Recruiting, training, and retaining

Building sustainable processes and infrastructure

Challenges of Data Analytics work

Build the right team

Pick right projects - low hanging fruits first

Identify the need for data analysis at the beginning of your project

Understand the data and the business process

Validate your results

How to Leverage DA for Audits

Sample Tests Duplicate Payments

Compare two transactions with same Invoice Number, Invoice Date, Invoice Amount

Check for duplicate entries in vendor master – same name, tax ID/SSN, bank account, address, phone number

Identify vendors who repeatedly submit duplicate invoices

Procurement Compare contract price (PO or BPO) against invoice price to verify if

vendor is honoring agreed upon pricing terms Check if vendor is honoring discount terms – compare PO vs Invoice Check if Accounts Payable is losing early discounts because of late

payments – compare discount allowed per PO/Invoice vs discount taken

Check if there is opportunity to negotiate longer payment terms with the vendors – most companies are asking for 45 to 60 days

Material Price variance (use following steps)1. Aggregate PO quantity and amount by material no and vendor no.

2. Calculate average price per material unit per vendor (Sum PO Amt / Sum PO Qty).

3. Identify the lowest price per material unit per vendor and segregate vendors who charged 10% more than the lowest priced vendor.

4. Calculate the higher amount paid for each material by multiplying the average lowest price paid for that material with the total quantity bought from all vendors.

Timekeeping Filter timecards with more than 24 regular and overtime hrs in a day. Identify employees who submit excessive regular hours in one day to

hide overtime (16 hrs or more). Filter employees who reported regular and overtime hrs on 30 or

more consecutive calendar days. Filter employees whose weekly overtime hrs were at least as many

or more than their regular hrs.

Sample Tests

Health Care Fraud Indicators Practitioners with high average payments Practitioners charging 3 times more than the average

amount per Procedure Code Practitioners using a Procedure code 3 times more

frequently than other practitioners with similar patient volume

Practitioners with 3 times more than average units per Procedure Code

Practitioners charging 6 times more than the average amount per Diagnosis Code

Practitioners with high number of new patients Practitioners with high transaction volume Practitioners serving patients with high medical visits Practitioners not charging copay in high number of visits

Sample Tests

Polling Question 5

• How many of those attending use automated techniques to record results from follow-up on high risk transactions or activities – like in the form of a questionnaire or survey?

• Yes, we use automated techniques• No, we do not use automated techniques

QUESTIONS OR COMMENTS

Vijay Chheda202-906-4661

vijay.chheda@amtrakoig.gov