OIA Tools & Technology Evaluation Methodology

Prepared for State of Ohio, Office of Budget & Management

Deloitte Consulting

April 18, 2008

2

Content Overview

Objectives

OIA Technology Needs & Requirements

Current Vendor & Tools Landscape

Software Selection Methodology

Next Steps

3

Objectives

Identify high level Office of Internal Audit (OIA) business needs, and technical and functional requirements for software tools

Perform high level review of existing software tools and functionality related to the OIA’s needs and requirements

Develop a software evaluation methodology, to include a criteria evaluation approach that could be used in drafting a Request for Information (RFI) to software vendors

OIA Technology Needs & Requirements

5

Current Needs & RequirementsPer discussion with key stakeholders within OBM, OIT, and the HB166 Advisory Committee, the following high level business and technology needs and requirements were identified for OIA:Needs:

– Tools to support the management of OIA, to include time reporting, accountability, reporting and billing

– Tools to support the management, maintenance, and retention of audit workpapers– Tools to support the risk assessment and audit universe maintenance function– Tools to organize and drive governance, risk, and compliance initiatives (e.g. CSA’s by IT

Departments)– Tools to support the auditing of key financial and operational controls for critical application

systems– Process mapping capabilities for documentation and education purposes– Continuous and automated monitoring of controls

Requirements:– Solutions must be able to support a large and distributed workforce (approx 125 auditors in

various locations w/ remote capability (web-based), custom reporting, & centralized QA monitoring needs)

– Solutions must be cost effective– Solutions would benefit by incorporating existing OIA tools and technology (e.g. Sharepoint, web

architecture, etc.)– Solutions must have adequate security measures to safeguard audit evidence– Web-based for customer usage (e.g. comment tracking) and improved transparency for the public

(e.g. final reports)– Facilitates record retention requirements & timely public record requests– Software vendor must have stability & provide adequate training/support

Current Vendor & Tools Landscape

7

Current Situation Analysis Internal audit departments have growing pressure for increased

oversight and assurance With mounting pressure and increased workloads, internal audits are

overwhelmed with manual and decentralized processes Organizations are now looking to reduce the cost and improve efficiency

associated with their internal audit departments – Focus has shifted from reliance on “error-prone” manual controls to automating and

monitoring the execution of those controls– Interim solutions provide adequate controls repository functionality but offer little in

improving the approach– Tools are able to assist with enterprise wide risk assessment, compliance, planning,

scheduling, control automation, control monitoring, review, report generation, trend analysis, and storage.

– Flexibility of the software allows OIA to customize and scale the tool according to their business or changes in IT infrastructure and platforms

– Tools help assess various risks (financial, environmental, health & safety, IS) under a consistent risk methodology

8

Organizations are assessing the value/cost of implementing a tools solution to help with internal audit processes

– Focus on a broader Governance, Risk and Compliance (GRC) program– Understand your current process & issues and focus on improvement– Timing, cost and level of effort to implement, to include initial and ongoing training

needs, software licensing, and maintenance

How best to enhance the existing audit program into a more sustainable, repeatable process

– Improve documentation version control– Consider self-assessment procedures– Improve efficiency of the process (i.e. workflow)– Enhance reporting features to improve effectiveness

Understanding the vendor landscape for the next generation of audit technologies

– Find the tool that best fits your needs– GRC is the “hot” label today – Many boutique vendors in this space today to address cost concerns and unique

needs

Decisions to Consider

9

Software VendorsIn the past, there was a clear definition between Internal Audit software and GRC software for controls auditing. Presently, these competencies are merging in new releases of software products from numerous vendors

Internal Audit Software Focus on audit procedures and

maintenance – Workpaper creation and

maintenance, management signoff, audit planning and scheduling, audit budget

– Centralized data repository, online checkout functionality, best practice/knowledgebase repository

– Compliance with SOX

GRC Software Focus on governance, risk, and

compliance initiatives– Policy management, incident

management, asset management

– Risk assessment, threat management, risk dashboards

– Internal audit components incorporated

10

Market Trend – Internal Auditor Software Survey• Survey respondents:

• 21% Government industry

• Majority of respondents from small audit shops (reason for Excel’s stronghold)

• Large departments rely on specialty products (TeamMate, AutoAudit)

• Software related concerns noted:

•#1 – Ability to find software that meets the department’s specific needs

•#2 – Cost

Source: Grey, Glen. “An Array of Technology Tools.” Internal Auditor August 2006: 56-62.

11

Market Trend – Internal Auditor Software Survey

70% of Government agencies use audit management and risk management software tools

Source: Grey, Glen. “An Array of Technology Tools.” Internal Auditor August 2006: 56-62.

Among companies who use a risk management analysis tool (beyond Excel), TeamMate (6%) and AutoAudit (3%) showed largest market share.

Software Selection Methodology

13

Selection Methodology

A three phased approach is recommended for effectively selecting a software solution:

Phase I

Planning and RequirementsDefinition

Phase II

Request for Proposal Development and Execution

Phase III

Final Analysis and Recommendation

During this phase of the selection process, the project team will take the information learned during phase I and trim the vendor list to only the most viable candidates. A request for information (RFI) will be sent to each vendor, responses will be compiled and analyzed and a demo list of 2-3 vendors will be created. Phase II will be completed by facilitating the vendor demonstration process, scoring and compiling of results.

During the final phase of the selection process, the project team finalizes the selection process, presents the compliance system recommendation to executive management and facilitates next steps toward solution implementation.

Phase I will begin by defining the scope for OBM’s tool selection process, to include business needs and requirements and setting a timeline for the process. Next, an extensive list of requirements will be reviewed and weighted according to the specific needs of OIA. A preliminary list of potential vendors will be gathered.

14

Sample Timeline / Milestones

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6

Phase 1Initiate Project, Establish Team Roles/ResponsibilitiesIdentify Unique Functional & Technical RequirementsProvide overview of an Enterprise Risk Management strategyEstablish Critical Success Factors

Develop Short List of Vendor CandidatesDevelop RFI/Scorecard and Solicit Vendor BidsReceive RFI and Coduct Vendor Q&AFacilitate Comparative Analysis of the RFI ResponsesFacilitate Vendor Demonstration Sessions

Facilitate Vendor Reference ChecksPrepare Final Selection & Recommendation ReportSelect the Best Compliance System VendorFinalize Selection & Recommendation ReportPresent Selection & Recommendation Report to Executive ManagementFacilitate Vendor Contract Negotiation

Denotes Milestone

Phase 3

Project Timeline & Milestones

Phase 2

15

Selection Process – Phase ITasks Initiate Project and Establish Team Roles and

Responsibilities– Establish project objectives, scope, priorities,

and determine key milestones.– Assist OIA in identifying project team

members, formalize team structure and reporting responsibilities, and develop detailed project work plan.

Identify Unique Functional and Technical Requirements

– Develop the unique business and system requirements for the software tools. These requirements will be used in Phase II as a basis for determining which tool provides OIA the best fit to their requirements.

Establish Critical Success Factors– Meetings focused on communicating and

affirming issues and critical success factors, understanding specific project expectations, and identifying how these will impact OIA’s organization and the selection project.

– Functional and technical requirements coupled with the critical success factors will serve as a detailed checklist to guide and facilitate vendor demonstrations.

Tool Selection ProcessTool Selection Process

Phase III

Final Analysis and Recommendation

Phase II

Request for Proposal Development and Execution

Phase I

Planning and Requirements Definition

Key Deliverables Detailed Project Work Plan Defined and Weighted Functional and

Technical Requirements List

16

Requirements Weight business and technical requirements according to OIA’s needs

17

Selection Process – Phase II

Phase III

Final Analysis and Recommendation

Phase I

Planning and Requirements Definition

Tool Selection ProcessTool Selection Process

Phase II

Request for Proposal Development and Execution

Tasks Develop Short List of Vendor Candidates

– Gather knowledge within the marketplace to identify a short list of 4-5 potential candidates.

Develop RFI/Scorecard and Solicit Vendor Bids– The Request for Information (RFI) will require each vendor

to provide a sample implementation schedule, pricing, warranty, references and other pertinent guidelines for the bidders to follow. OIA to populate the vendor scorecard.

– Vendors will be contacted and bids solicited from them. Facilitate Comparative Analysis of RFI Responses

– Review requirements and institute a ranking system to evaluate the vendor proposals.

– Collect vendor RFI responses and prepare a comparative analysis report.

– Utilize comparative analysis report to further condense candidate list to 2-3 vendors for demonstration.

Facilitate Vendor Demonstration Sessions– Invite top candidates to OIA to present their system and to

answer/clarify specific questions related to their RFI response.

– Develop demo scripts to evaluate vendors

Key Deliverables Short List of Vendor Candidates Request for Information (RFI) Vendor Scorecard Comparative Analysis Report

18

Sections: Company (OIA) background and overview, description of desired solution, benefits sought Response directions - OIA contact information, response due date, target demo dates Vendor information – company profile and tool, including financial solvency & market share Requirements – customized questions according to OIA requirements, response from vendor limited to

500 characters. Customer references – list of customers of similar size/requirements Disclaimer – RFI solely for informational and planning purposes

Request for Information

19

Selection Process – Phase III

Phase II

Request for Proposal Development and Execution

Phase I

Planning and Requirements Definition

Tool Selection ProcessTool Selection Process

Phase III

Final Analysis and Recommendation

Key Deliverables Reference Check Summary Final Business Case

Tasks Facilitate Vendor Reference Checks

– Utilize a customized questionnaire to evaluate each vendor’s performance at comparable clients. The questionnaire will be customized to OIA’s specific requirements and interests.

Prepare Final Selection & Recommendation Report

– The project team will work together to put the finishing touches on the selection report.

– Provide guidance for executive-level presentation.

Select the Best Internal Audit Tool (or Tools)– Conduct a detailed review session to reach

agreement that OIA’s requirements are met by the chosen vendors and solutions.

Finalize Selection & Recommendation Report– Provide recommendation report template for OIA

to complete and present.

Support OIA in Vendor Contract Negotiations – Present OIA with applicable rate and licensing

information based on existing vendor relationships.

Sample System RequirementsSample System Requirements