Application Auditing Scope, Approach, & Execution January 2009 Michael Kirk, CIA, CISA Risk-based.
37
Application Auditing Scope, Approach, & Execution January 2009 Michael Kirk, CIA, CISA Risk-based
-
Upload
damon-biddulph -
Category
Documents
-
view
214 -
download
0
Transcript of Application Auditing Scope, Approach, & Execution January 2009 Michael Kirk, CIA, CISA Risk-based.
Application AuditingScope, Approach, & Execution
January 2009Michael Kirk, CIA, CISA
Risk-based
2
Mike Kirk, CIA, CISA– Application experience includes: Oracle,
PeopleSoft, JD Edwards, and industry-specific and customized applications for healthcare, insurance, energy/utility, manufacturing, construction, and environmental industries.
– Board member of the Central Ohio Chapter of ISACA
Introduction
3
Introduction
JD Edwards
4
Risk Considerations for Application Auditing
• Defining Your Scope
• Developing Your Approach
• Execution
• Q & A
Overview
5
Where do you start?
1. Understand the environment
2. Understand business & technical processes
3. Assess risks & controls
4. Develop improvements
ScopeAudit Methodology
6
1. Understand the business environment the application supports
Develop a complete understanding of the business process flow (inputs, processing, & outputs),
and, the data flow, including interfaces
Defining Your Scope
7
2. Understand the application’s technical environment
– IT control environment– “Off-the-shelf” or customized – Legacy or web-based – Housed internally or external provider– Developed in-house or 3PP – User population– Dispersion of users
Defining Your Scope
8
Understand the business process and technical environment
Assess business information risks
Defining Your Scope
Start
Notify
shipping
Confirm customer
order
Item
out-of-stock
?
Log revenue
A
Information System
EndConfirm receipt
Create
invoice
Send info to
Acct
A Close books
1
2 3
# Critical Risk / Control Points
Financial Accounting Systems
EDI Hub
Treasury ManualCheck Printing
System
Invoice Imaging A/P
A/P OpenItems
A/P ClosedItems
A/R
A/R OpenItems
DD & A
Depr. AssetMaster
AFE
Actual DetailCosts
BudgetedCosts
CorporateG/L
Hyperion
Invoicing Systems
OperationalSystems
C
Payroll System
1 1
Risks
Data integrity and completeness are notmaintained in the transfer between applications.
Data security is not maintained in the transferbetween applications.
2
Standard password protection required in transfer protocol
Transfers are automated using a job scheduler.
Application logs into Windows NT using a password-protectedaccount and uses an interface directory protected by NT FileSystem (NTFS) security.
Application logs into UNIX environment with password protectedaccount to make the transfer.
Interface uses integrity checking tools to detect data corruption.
2
Controls
3
4
1 12 2 41 12 2 3
1 12 2 3
5
1 2 5
Invoices
ReconciliationFigures
Manual Check Information
EDI Communications
General Ledger Entries
Corp Tax1 2 3Acct. Info. for Tax Return (manual)
International G/L
Time andAttendance
Employee Hours
Intranet
Manual reconciliation of data is performed after transfer.
Only one user and the System admins have access to theinterface directory on the UNIX server.
Error reports are generated after the data is transferred.
Record Counts are compared to ensure completeness andaccuracy.
6
7
8
7
7
9
8 9
81 2
1 2
Payroll Entries
Critical Process Maps Data-Flow/Interface Diagrams
9
Business & Information Risks:– Regulatory compliance – F/S integrity– PCI-related– Integration of an acquisition– Data privacy– Integrity of the application– Process control validation
Defining Your Scope
10
Lessons Learned on Scope
• Critical dependencies: personnel, technical, external providers [BCP]
• “Canned” still seems to get modified
• Don’t forget spreadsheets & report-writers!
• Leverage existing flow diagrams
• Invest the time to understand the process flow...– Adds value to your internal
client– You may find that you are
now the expert!
11
Top-down Approach:
Auditing around the application…
Information Technology General Controls (ITGCs)
Bottom-up Approach:
Auditing the insides…
Application Controls Testing
Developing Your Approach
12
Relationship between ITGCs and application controls
1. Development
2. Change Management
3. Security Administration
4. Operations
Top-down Approach
13
Top-down Approach
Source: COBIT 4.1 Framework
Boundaries of Business, General, and Application Controls
14
DevelopmentAI2 Acquire and Maintain Application Software
AI2.7 Development of Application Software• Ensure that automated functionality is developed in accordance
with design specifications, development and documentation standards, QA requirements, and approval standards.
Top-down Approach
Source: COBIT 4.1 Framework
15
Change ManagementAI6 Manage Changes
AI6.1 Change Standards and Procedures• Set up formal change management procedures to handle all
requests (including maintenance and patches) for changes to applications, procedures, processes, system and service parameters, and the underlying platforms in a standardised manner.
Top-down Approach
Source: COBIT 4.1 Framework
16
Security AdministrationAI2 Acquire and Maintain Application Software
AI2.4 Application Security and Availability• Address application security and availability requirements in
response to identified risks and in line with the organisation’s data classification, information architecture, information security architecture and risk tolerance.
Top-down Approach
Source: COBIT 4.1 Framework
17
Top-down Approach
ChangeMgt
NetworkSecurity
Development
ApplicationSecurity
DatabaseSecurity
Process Controls
ApplicationControls
18
• Security roles and functionality
• Software Development Life Cycle– Test plans & supporting documentation– Testing to break vs. testing to pass
Lessons Learned on Top-down
19
• Dispersion and diversity of the business, processes, and technology compounds the effort
• Don’t underestimate the value of a thorough general controls review!
Lessons Learned on Top-down
20
Auditing the functionality and control effectiveness of the application can only be determined based on the maturity level and effectiveness of the ITGCs
Auditing the insides… Application Controls Testing
Bottom-up Approach
21
Bottom-up Approach
BusinessManagement
Reporting Controls
ApplicationData Processing
Controls
Access andData Source
Origination SetupControls
Input Processing Output
22
Understand the business process and technical environment
Assess business information risks
Bottom-up Approach
Start
Notify
shipping
Confirm customer
order
Item
out-of-stock
?
Log revenue
A
Information System
EndConfirm receipt
Create
invoice
Send info to
Acct
A Close books
1
2 3
# Critical Risk / Control Points
Financial Accounting Systems
EDI Hub
Treasury ManualCheck Printing
System
Invoice Imaging A/P
A/P OpenItems
A/P ClosedItems
A/R
A/R OpenItems
DD & A
Depr. AssetMaster
AFE
Actual DetailCosts
BudgetedCosts
CorporateG/L
Hyperion
Invoicing Systems
OperationalSystems
C
Payroll System
1 1
Risks
Data integrity and completeness are notmaintained in the transfer between applications.
Data security is not maintained in the transferbetween applications.
2
Standard password protection required in transfer protocol
Transfers are automated using a job scheduler.
Application logs into Windows NT using a password-protectedaccount and uses an interface directory protected by NT FileSystem (NTFS) security.
Application logs into UNIX environment with password protectedaccount to make the transfer.
Interface uses integrity checking tools to detect data corruption.
2
Controls
3
4
1 12 2 41 12 2 3
1 12 2 3
5
1 2 5
Invoices
ReconciliationFigures
Manual Check Information
EDI Communications
General Ledger Entries
Corp Tax1 2 3Acct. Info. for Tax Return (manual)
International G/L
Time andAttendance
Employee Hours
Intranet
Manual reconciliation of data is performed after transfer.
Only one user and the System admins have access to theinterface directory on the UNIX server.
Error reports are generated after the data is transferred.
Record Counts are compared to ensure completeness andaccuracy.
6
7
8
7
7
9
8 9
81 2
1 2
Payroll Entries
Critical Process Maps Data-Flow/Interface Diagrams
23
• Understand the transaction process related to the application… identify key transactions
• Assess Application Controls
Bottom-up Approach
24
Bottom-up ApproachTransaction and Data Flow Detail
Screens to Test
25
Bottom-up Approach
Screens to Test
Application Walk Through: Key Screens
26
Bottom-up Approach
Source: COBIT 4.1 Framework
Boundaries of Business, General, and Application Controls
27
Bottom-up ApproachApplication Controls• AC1 Source Data Preparation and Authorisation
– Ensure that source documents are prepared by authorised and qualified personnel following established procedures…
• AC2 Source Data Collection and Entry– Establish that data input is performed timely
• AC3 Accuracy, Completeness and Authenticity Checks– Ensure that transactions are accurate, complete and valid. Validate data that were input, and edit or send back for
correction as close to the point of origination as possible.
Source: COBIT 4.1 Framework
28
Bottom-up Approach
Application Controls• AC4 Processing Integrity and Validity
– Maintain the integrity and validity of data throughout the processing cycle.
• AC5 Output Review, Reconciliation and Error Handling– Establish procedures and associated responsibilities to ensure that… verification, detection and correction of the
accuracy of output occurs.
• AC6 Transaction Authentication and Integrity– When sharing data between internal applications and business/operational functions… maintain integrity.
Source: COBIT 4.1 Framework
29
Bottom-up Approach
• Testing Documentation– Application Function (screen mapping)– Function Description– Testing Procedures– Control Observations (testing results)
& Recommendations– Key Risks & Impact
• Lots of screen shots!
30
Bottom-up ApproachApplication
FunctionFunction
DescriptionTesting
ProceduresControl Observations &
RecommendationsKey Risks & Impact
Order Entry – Override Pricing
Screen #:OE480
Screen allows user to override pricing information for the order.
Examined updateable fields for data validation edits. 1. Gross Price2. Sales, Inventory, and Expense G/L Codes
Noted existence of adequate validation checks of the following:1. Table Look-up (Sales Inventory, and Expense G/L Codes)2. Existence Check (Sales Inventory, and Expense G/L Codes)3. Completeness Check (Sales Inventory, and Expense G/L Codes)Noted the lack of the following validation checks:4. Limit Check (Gross Price)5. Reasonableness Check (Gross Price)
Noted that limit and reasonableness checks are not in place for the gross price field to restrict the price of items ordered. This screen will also allow the gross price of the ordered item to be zero. We recommend that the appropriate limit and reasonableness checks be implemented for the gross pricing field to prevent blank or unreasonable data entry.Noted the following control weakness:Access to Override G/L Accounts
Noted that the sales, expense, and inventory general ledger code fields could be edited through the override pricing screen. User access to edit critical fields should be minimal. We recommend that only appropriate and essential users be granted the permission to edit the G/L codes.
Reports that rely on the gross price field or perform calculations based on this field have the potential to contain incorrect information based upon the observed control weakness.
Modifying the G/L codes in the order screens could result in the posting of orders to inappropriate G/L accounts and subsequently impact updates to the Sales General Ledger and the Sales by Branch/Product Group (SLS042) report.
31
Bottom-up Approach
Testing Procedures– Good code vs. not-so-good code
• Sequence checks• Limit checks• Range checks• Validity checks• Reasonableness checks
• Table lookups• Existence checks• Completeness checks• Duplicate checks• Logical relationships
32
System-Based Data Entry Integrity ControlsEdits Description
Sequence Checks
The control number follows sequentially and any control numbers out of sequence or duplicated are rejected or noted on an exception report for follow-up purposes. For example, invoice numbers are systematically and generated sequentially generated and cannot be overridden.
Limit Checks Data should not exceed a predetermined amount. For example, it may be determined that quantities ordered should not exceed a maximum of 500 each, or that a sales commission cannot exceed 12% without an override. If a quantity exceeds the limit, the data would be rejected for further verification or authorization.
Range Checks Data should be within a predetermined range of values. For example, date ranges. Once books are closed for the current month, entries are not permitted for the previous month’s date range.
Validity Checks Programmed checking of the data validity in accordance with predetermined criteria. For example, department & expenses relationships, account codes, company numbers, vendor numbers, customer numbers, invoice types, etc.
Reasonableness Checks
Input data are matched to predetermined reasonable limits or occurrence rates. For example, the tolerances exist within purchasing application to allow for quantity variations or unit price variations based on preset tolerable limits.
Table Lookups Input data are agreed to predetermined criteria maintained in a data table of possible values. Essentially the same type of controls as validity – any pull-down menu, pre-defined field… an employee must exist within the EMPMSTR table for payroll processing.
Existence Checks Data are entered correctly and agree to valid predetermined criteria. For example, a picking pack slip can only be generated for an existing order number, or in order to receive against a P.O., it must exist.
Completeness Checks
Data fields should always contain data and not zeros or blanks. A company number, accounting unit, and account code for a general ledger entry are required fields prior to the record being added to the system. A “hard stop” is received by the user if the required fields are not completed. Completeness check controls can be utilized by individual field or by data entry screen.
Duplicate Checks New transactions are matched to those previously input to ensure that they have not already been entered. For example, accounts payable invoices cannot be entered twice for the same vendor invoice.
Logical Relationship Checks
If a particular condition is true, then one or more additional conditions or data input relationships may be required to be true and consider the input valid. Company, accounting unit, & account relationships exist. For example the marketing department cannot charge an expense to lease expenses, only to accounts logically tied to that department.
33
Bottom-up Approach
Testing Procedures contd.
– Field formats are defined & locked– “Grayed” fields… better yet, linked to user and
displayed only if applicable to functionality– On screen, visual feedback
– Not-so-good… better have monitoring reports as mitigating controls!
34
• Applications pre-dating the current climate of control
• Baselining• Value of an implementation project audit• Application functionality… best if conducted
throughout testing stage • End user training & desktop procedures
Lessons Learned on Bottom-up
35
• Monitoring controls… issues typically arise from the one $1,000,000 transaction, not from the million $1 transactions
• Please remember to conduct audit work in a Test environment
• Don’t underestimate the time commitment!• Emerging technology trends… web-based,
PDAs, now smartphones
Lessons Learned on Bottom-up
36
• Risk considerations for application auditing
• Risk-based approach drives increased efficiency + decreased costs = organizational value
Summary
