Continuous audit: today and tomorrow
-
Upload
patrick-boyer -
Category
Documents
-
view
25 -
download
0
description
Transcript of Continuous audit: today and tomorrow
Continuous audit:today and tomorrow
Miklos A. VasarhelyiKPMG Professor – Rutgers UniversitySenior Consultant- AT&T Laboratories
2
Continuous Audit and Reporting Laboratory
Outline
• An evolving framework• Some Key issues / the state of the art• Some CARLAB experiences• Six Steps in Implementing CA• Organizational Context• Opportunities and Challenges• Conclusions
An evolving audit framework
4
Continuous Audit and Reporting Laboratory
An evolving audit framework
Assurance ofData elements
Data level
Assurance
Assurance ofKey Processes
Process level
Assurance
Assurance ofReports
Report level
Assurance
•XML/ XBRL datum•Generated and modified by different processes•Balkanization of data•Control / Assurance tags
•Process reviews a la Systrust•Internal or outsourced•Third party processes are to become the norm•Intra and Inter process controls an issue
•Compliance reports becoming commonplace•Traditional audit is an instance of RLA•Generated and modified by different processes
5
Continuous Audit and Reporting Laboratory
An evolving continuous auditframework
•Automation
•Sensoring
•ERP
•E-Commerce
Continuous
Audit
ContinuousControl
Monitoring
Continuous
Audit
Data
CA = CCM+ C(D)ACA -> Continuous AuditCCM -> Continuous Control MonitoringC(D)A -> Continuous Data Assurance
6
Continuous Audit and Reporting Laboratory
Some Key Issues
• Two recent surveys (ACL and PWC) show that a large number of key companies are attempting to perform continuous audit like functions
• An industry of software is evolving with ACL, IDEA, APPROVA, and others growing rapidly
• Control Monitoring and Continuous Data Assurance are the main approaches
• The first recorded application was AT&T Bell Laboratories CPAS effort in the 1986-1991 period
• The Rutgers CarLab is working in leading applications
7
Continuous Audit and Reporting Laboratory
Continuous Auditing Value Proposition•
– Improved business performance
• Innovations in information technology & analytical modelling enable:
– More frequent, timely, accurate & relevant business performance information
– Lower compliance risk
– Cost reduction
8
Continuous Audit and Reporting Laboratory
CAR-Lab Experiences• Control monitoring at Siemens
• Transaction monitoring at Unibanco
• Continuous (data) assurance at HCA
• Other
– Conceptual developments
– Simulating Liberty
– EBR work
– KPMG projects
Overview of CaR-Lab examples
10
Continuous Audit and Reporting Laboratory
Siemens' – Project Value Proposition
Operational Audit Why CA at Siemens?
•Improve Governance (Fraud Detection,
SOX Compliance, Monitoring, etc)
•Reduce Compliance Costs
•Improve skill level and quality of work life
for auditing and compliance Associates
•Move closer to real time reporting
capabilities
•ETC….
Operational Audit
Value Proposition“Value = Quality + Cost”
COST:
• Consider a large multinational corporation with 400 auditors (internal & external), each with a fully absorbed (sal./fee, benefits, travel, etc.) $200,000/yr cost for a total annual compliance cost of $80 million dollars. Assume further that the proposed continuous auditing model cost $1 million dollars to develop and implement and only reduced manual compliance effort by 25% in the firm. The annual net estimated savings or cost avoidance of this project for the firm defined above would be:
$19 Million dollars (Or nearly $100 million dollars over 5 years)!
Note: Leverage the model further by increasing the percentage of impact or in support of other assurance or monitoring functions and the value proposition grows.
Expanded Audit Coverage
Significant Cost Savings
Automated Business Process Controls Monitoring Project
11
Continuous Audit and Reporting Laboratory
Siemens' – Project Features
• Formalize & automate internal audit procedures used for business process controls monitoring
• Conduct “man vs. model” assessments
• Calibrate “exception rules” to optimize model performance
• Scale up to all SAP instances
• Increase frequency of model application, where feasible
• Transition to Approva application and extend the model where optimal
12
Continuous Audit and Reporting Laboratory
A 3 pronged approach to audit automation• Automate audit plan using delivered Rule
Sets: Est 25% of a typical manual audit plan
• Automate using external data sets (Static & Variable): Est an additional 25% a typical manual audit plan
• Re-enginer manual controls into automated controls with improved control precision: Est an additional 25% a typical manual audit plan• Total = Automation Opportunity ~75%!!
13
Continuous Audit and Reporting Laboratory
MCP
A.A.S(audit ActionItems)
From SiemensApprovaand otherliterature
Class of
Auditable
Actions
----
of Audit
Processes
AuditEvidence
Receptacle
MasterAudit
Program AuditParameterization
Tool
OtherStatic
Parameters
EvergreenOpinion
InferenceEngine
AuditorManagementOperating Alarm Flows
Operating Alarm Flows
CAControl
Dashboard
Deter-ministic
Stocha-stic
ExternalTable
comparisons
Snapshotcomparisons Other
RemoteAudit
Communic.Tool
DataExtraction
InteractiveMail
ManagementTool
SustainableObject
VerificationTool
Other
14
Continuous Audit and Reporting Laboratory
IT / IA Continuous Auditing Program at Unibanco
Continuous Audit at our Bank Mission
Automatically evaluate risks and controls on a continuous basis in order to identify exceptions and anomalies, trends and risk indicators.
I ssue opinions about controls, risk assessment for top management, audit committee and other interested parties. Contribute to corporate Governance of the Conglomerate.
Scope
All products, processes and services in the conglomerate that allow for the systemic extraction and analysis of data generated by Information Technology.
Approach
Extraction and analysis of information of the products, processes and services that exist in the Conglomerate using IT to improve timeliness and scope of Internal audit work..
Inform events of non-compliance in these being effective on the generation of products needed for the prevention and correction of risks and events.
PRODUTIVITY WITH QUALITY AND EFFICIENCY
15
Continuous Audit and Reporting Laboratory
Unibanco – Some CA Program Features
• Automated monitoring of over 5 million customer accounts on a daily basis using 25 automated procedures to:– Detect errors– Deter inappropriate events & behaviors– Reduce or avoid financial losses– Help assure compliance with existing laws, policies, norms
and procedures
• Examples of “low hanging fruit:”– Customer advances– Excess over credit limit– Returned checks– Federal tax payment cancellations– TED emissions (should this be omissions?)
16
Continuous Audit and Reporting Laboratory
Unibanco – Advances to Clients Monitoring
17
Continuous Audit and Reporting Laboratory
Continuous Data Assurance (CDA) at a Major Health Services Provides (HSP)• HSP is a large national provider of healthcare services,
composed of locally managed facilities that include numerous hospitals and outpatient surgery centers.
• IT internal audit provided access to unfiltered extracts from their transactional databases, comprising all procurement cycle daily transactions from October 1st, 2003 through June 30th, 2004: Over 500,000 data points.
• Dataset mimics what a CDA system has to deal with: highly disaggregate data flowing through CA system in real time.
• Audit procedures have to be developed for this environment.
18
Continuous Audit and Reporting Laboratory
Analytical Procedures in CA• Analytical procedures used in the planning, substantive testing, and
reviewing stages of an audit. We focus on substantive testing.
• In conventional auditing first apply analytical procedures to identify potential problems, Then, focus detailed transaction testing on the identified problem areas.
• In CDA the sequence is reversed:
1. Use automated general transaction tests to all the transactions and filter out identified exceptions for resolution.
2. Apply automated analytical procedures to the filtered transaction stream to identify unforeseen problems.
3. Alarm humans to investigate anomalies.
19
Continuous Audit and Reporting Laboratory
Continuous Data Assurance• Automation of Transaction Testing:
– Formalization of business process rules as transaction integrity and validity constraints.
– Verification of transaction integrity and validity detection of exceptions generation of alarms.
• Automation of Analytical Procedures:
– Selection of critical business process metrics and development of stable business flow (continuity) equations.
– Monitoring of continuity equation residuals detection of anomalies generation of alarms.
20
Continuous Audit and Reporting Laboratory
Enterprise System Landscape
Ordering
Accounts Payable
Materials Management
Sales
Accounts Receivable Human Resources
Business Data Warehouse
Automatic Transaction Verification
Exception Alarms
Automatic Analytical Monitoring: Continuity Equations
Anomaly Alarms
Continuous Data Assurance System
Responsible Enterprise Personnel
21
Continuous Audit and Reporting Laboratory
Establishing Data Integrity: A Procurement Example
• Referential integrity along the business cycle and identification of completed cycles:P.O. Shipment receipt voucher payment.
• Identification of data consistency issues and automatic alarms to resolve exceptions:– Changes in purchase order vendor numbers;– Discrepancies between the totals and the sums of line
items;– Discrepancies between matched voucher amounts.
22
Continuous Audit and Reporting Laboratory
Detection of Exceptions• Referential integrity violations
– PO without matching requisition– Received item without matching PO– Payments without matching received items
• Data integrity violations– PO has zero order quantity– Received item has negative quantity– Invalid payment check numbers (e.g. All 0s)– Gross payment amount is smaller than net payment
amount
23
Continuous Audit and Reporting Laboratory
Continuity Equation Based CDA• Continuity Equations:
– Stable probabilistic models of highly disaggregated business processes, uses as the expectation models for process based analytical procedures.
– Originated in physical sciences (various conservation laws: e.g. mass, momentum, charge).
• Continuity equations are developed using statistical methodologies of:1. Linear regression modeling (LRM);2. Simultaneous equation modeling (SEM);3. Multivariate time series modeling (MTSM) using various
Vector Autoregressive Models (VAR).
24
Continuous Audit and Reporting Laboratory
Basic Procurement Cycle
P.O.(t1)
Receive(t2)
Voucher(t3)
t2-t1
t3-t2
25
Continuous Audit and Reporting Laboratory
Ideal Continuity Equations of Basic Procurement Cycle
Receive(t2)= P.O.(t1)
Voucher(t3)= Receive(t2)
• Aren’t partial deliveries allowed?
• Are all orders delivered after exactly the same time lag?
• Are there any feedback loops?
26
Continuous Audit and Reporting Laboratory
P.O.(t)= 0.24*P.O.(t-4) + 0.25*P.O.(t-14)+ 0.56*Receive(t-15) + εPO
Receive(t)= 0.26*P.O.(t-4) + 0.21*P.O.(t-6)+ 0.60*Voucher(t-10) + εR
Voucher(t)=0.54*Receive(t-1) - 0.17*P.O.(t-9) + 0.22*P.O.(t-17) + 0.24*Receive(t-17) + εV
Estimated Continuity Equations of Procurement Using VAR Model
27
Continuous Audit and Reporting Laboratory
Detection of Anomalies
• Anomalies are detected if:– Observed P.O.(t) < Predicted P.O.(t) - Var
or– Observed P.O.(t) > Predicted P.O.(t) + Var
• Similarly for:– Receive(t)– Voucher(t)
• Var = acceptable threshold of variance.• If there is anomaly generate alarm!
28
Continuous Audit and Reporting Laboratory
Measuring Anomaly Detection• False positive error (false alarm, Type I error): A non-
anomaly mistakenly detected by the model as an anomaly. Decreases efficiency.
• False negative error (Type II error): An anomaly failed to be detected by the model. Decreases effectiveness.
• Detection rate is used for clear presentation purpose: The rate of successful detection of seeded errors.
• A good analytical model is expected to have good anomaly detection capability: low false negative error rate (i.e. high detection rate) and low false positive error rate.
29
Continuous Audit and Reporting Laboratory
Simulated Error Correction• Access to highly disaggregate data in real time makes it
possible for CA system to detect, investigate and correct anomalies also in (nearly) real-time.
• Real-time error correction enables utilizing the corrected rather than the erroneous data in revised continuity equation benchmarks.
• Real-time error correction is likely to benefit future anomaly detection. We investigate the magnitude of this benefit using simulation.
• Error correction raises important issues about auditor independence, and the line between auditing and monitoring of business processes.
30
Continuous Audit and Reporting Laboratory
Benefit of Real-time Error Correction: MTSM
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
120.00%
10%E 50%E 100%E 200%E 400%E
MTSM_Error_Correction MTSM_No_Error_Correction
32
Continuous Audit and Reporting Laboratory
Takeaways from HSP Study• Various statistical methods can be used to derive expectation
models of acceptable quality.
• But key is access to highly disaggregate data, not which benchmark is used. With such data, most reasonable continuity equation models give usable results.
• Real-time error correction significantly improves error detection.
• More disaggregated models are not always better: weekly data can be more stable than the daily one.
• Alarms have to be managed – trade-off between Type I and Type II errors.
Implementation Issues in CA
34
Continuous Audit and Reporting Laboratory
• Background – While technologies of continuous audit
have been extensively discussed and are progressively emerging the more mundane issues of their implementation in a socio-technical environment have been neglected
– http://www.theiia.org/itaudit/features/in-depth-features-2-10-08/feature-2/
35
Continuous Audit and Reporting Laboratory
2. Rule
5. Follow-up
1. PriorityAreas
6. Action andReaction
4. Parameterization
3. FrequencyAudit Control Panel
Six steps of process implementation
36
Continuous Audit and Reporting Laboratory
– 1. Identification of Priority Areas • Modularize risk areas, rate these risks and
evaluate the cost x benefits• Identify the basic audit objects• Choose critical business processes that will
be the focus of continuous audit (low hanging fruit)
• Identify key data in for the implementation of Continuous Audit in the mapped processes
• Political Considerations
37
Continuous Audit and Reporting Laboratory
• Key Objective of Audit Procedure – Detective– Deterrent– Financial– Compliance
38
Continuous Audit and Reporting Laboratory
• 2. Rules of Monitoring and Auditing – Once an area of CA is chosen the “rules” of
monitoring, alarming, and assurance must be established
– These must take into consideration the legal and environmental issues as well as the objectives of the particular process
– The CA process is established adopting certain rules, frequencies, and parameters.
– e.g. we will monitor bank accounts in overdrafts or in excess limits
39
Continuous Audit and Reporting Laboratory
•3. Frequency – The natural rhythm of the process
• Timing of computer processes• Timing of business processes
– Cost benefit considerations– Nature of procedure objectives
• Deterrence• Prevention
40
Continuous Audit and Reporting Laboratory
– 4. Parameterization•Define parameter to analyze in
accordance with the risk•eg.: Monitoring all accounts in
overdrafts in daily basis , that have a balance of debt 20% larger than its limit of loan and bigger than 1000 USD
41
Continuous Audit and Reporting Laboratory
• 5. Follow-up– Who will receive the alarm?
• Management?• Audit leadership?• Immediate superior of the responsible for the data• The timing of the follow up
– Pass the alarm along immediately– Reconcile the alarm prior to follow up– Wait for 3 sequential days of similar alarms to follow
up• Escalation guidelines
– E.g. after three days send to the immediate superior’s superior or wait for 3 days prior to the re-escalation
42
Continuous Audit and Reporting Laboratory
• 6. Action and Reaction– Guidelines for dealing with
auditees•Lack of bias•Consistency of response•Guidelines for individual factor
considerations•Concern with collusion
Organizational Issues
44
Continuous Audit and Reporting Laboratory
• Organizational Structure for CA– Is CA a part of the audit function or of
management?– Its part of the audit function– Should there be a separate continuous
audit group?– Yes, to facilitate its implementation
progressively in the many areas of continuous audit
45
Continuous Audit and Reporting Laboratory
• Workforce Effects – Progressively labor requirements
for the traditional audits supported by CA will reduce and deeper audit will become possible
– Rebalancing of workforces– High technological competencies
needed
Opportunities and Challenges
47
Continuous Audit and Reporting Laboratory
Opportunities for business and research (1)• Control system measurement
– We are in a pre-paradigmatic stage of control documentation and measurement
– We do not know how to monitor controls in large ERPs– We do not know how to provide a really supportable opinion on
controls – We do not know how to rate combinations of controls
48
Continuous Audit and Reporting Laboratory
Opportunities (2)
• Business Process Monitoring and Alarming– Auditors have to carve a position on the new monitoring and
control environment– Auditors can collect exception “alarms” as trusted parties and
incorporate these into evidentiary matter– Auditors can be “trusted”
49
Continuous Audit and Reporting Laboratory
Opportunities (3)
• Automatic Confirmation Tools– Confirmations will have an increased evidentiary role with
eventual elimination of population and integrity worries– Intelligent confirmatory tags can do much– Database to database hand-shaking will be medium– Business opportunity for auditors
50
Continuous Audit and Reporting Laboratory
Opportunities (4)
• Audit bots (agents)– Many of the basic audit functions can be emulated by software– These must be eventually developed by the profession to work
hand-in-hand with human auditors in the new audit world– These agents will work on all areas including: 1) audit
planning, 2) analytical reviews, 4) confirmations, and )5 evergreen opinions
51
Continuous Audit and Reporting Laboratory
Opportunities (5)
• Collecting forensic trails– Auditor “black” box
• Publishing real-time authenticated reports for different compliance masters
• Publishing FD independent compliance reports
52
Continuous Audit and Reporting Laboratory
Challenges
• Standards are needed for CA– Audit monitoring needs to be defined– Types of evidence are to change and must be reconsidered– Independence needs to be re-defined
• The billing model has to be restructured to bill on function not hours
53
Continuous Audit and Reporting Laboratory
Challenges
• Audit firms must put improved knowledge collection and management processes to feed their audit analytic toolkit
• Audit firms have to engage in auditor automation and pro-actively promote corporate data collection during-the-process
• Value added must be justified in terms of data quality
54
Continuous Audit and Reporting Laboratory
• Conclusions – Attention must be paid to the
organizational processes that implement continuous audit
– There are 6 key steps to progressively implement a CA program module by module
– The CA process is dynamic and CA management will change schedule and parameters of each process
– The organization of the audit process must be evolved progressively