Internal Controls_overall Effectiveness

AS S U R A N C E A N D ADV I S O RY

BU S I N E S S SE RV I C E S

Evaluating InternalControlsEvaluating Overall Effectiveness, Identifying Matters forImprovement, and Ongoing Assessment of Controls

!@#Fourth in the Series

Establishing and maintaining effective

internal control is an important

management responsibility. Internal

control is fundamental to the accurate recording

of transactions and the preparation of reliable

financial statements. Many business activities

involve a high volume of transactions and

numerous judgments regarding the related

accounting for transactions, estimates, and

other events that require recognition in the

accounting records and financial statements.

Without adequate controls to ensure the proper

recording of such items, the resulting financial

data may be unreliable and undermine

management’s ability to make well-informed

decisions, as well as damaging management’s

credibility with shareholders, regulators,

business partners, and the public.

While companies have long recognized the importance ofstrong internal control, the Sarbanes-Oxley Act of 2002(the Act) now makes executive management responsible

not just for establishing, evaluating, and assessing overtime the effectiveness of internal control over financialreporting and disclosure, but also periodically asserting toits effectiveness. The Act changes the regulatory landscapefor public companies and presents serious consequencesfor those companies that fail to comply with the newrequirement. However, this new challenge presentsopportunities for companies that not only recognize andaccept the new regulatory landscape, but rather view theevaluation of internal control over financial reporting asmore than a compliance process and recognize it as anopportunity to reach a higher level of financial reportingintegrity and corporate performance.

Our publication, Preparing for Internal Control Reporting—A Guide for Management’s Assessment under Section 404 ofthe Sarbanes-Oxley Act (The Guide, Ernst & Young SCORERetrieval File No. EE0677), provides a methodology andframework for completing the evaluation. The methodologyoutlined in the Guide includes five phases:

1 Understand the Definition of Internal Control

1 Organize a Project Team to Conduct the Evaluation

1 Evaluate Internal Control at the Entity Level

1 Understand and Evaluate Internal Control at theProcess, Transaction, or Application Level

1 Evaluate Overall Effectiveness, Identify Matters forImprovement, and Establish Monitoring System

Guidance on the first two phases of the methodology isprovided in the Guide. Detailed guidance on the third andfourth phases is provided in two other Ernst & Youngpublications, Evaluating Internal Controls—Considerationsfor Evaluating Internal Control at the Entity Level (Ernst &Young SCORE Retrieval File No. EE0687), and Evaluating

To Our Clients and Other Friends

EVA L UAT I N G IN T E R NA L CO N T RO L S

TO OU R CL I E N T S A N D OT H E R FR I E N D S

Internal Controls—Considerations for DocumentingControls at the Process, Transaction, or Application Level(Ernst & Young SCORE Retrieval File No. EE0692). Thispublication addresses the fifth and final phase.1

Management’s report on internal control will contain itsassertion that the Company maintained, as of the end of thefiscal year, effective internal control over financial reporting,based on an established set of control criteria (e.g., COSO2).“Effective” implies that management was able to concludethat internal control over financial reporting providesreasonable assurance that misstatements (either individuallyor collectively) caused by error or fraud in amounts thatwould be material in relation to the financial statementswould be prevented or detected and corrected in a timelyperiod by employees in the normal course of performingtheir assigned functions. In making its assessment,management will need to consider whether the significantcontrols that have been identified would likely preventand/or detect a material misstatement relating to each of the relevant financial statement assertions. The approachdescribed herein for making this determination draws uponknowledge and evidence gained from evaluating internalcontrol at the entity level and at the significant process,transaction, or application level, and uses the results of thoseprocedures to make an overall evaluation of the effectivenessof internal control.

A company’s certifying officers must certify each quarterthat they have disclosed to the company’s independentauditors and the audit committee of the company’s board ofdirectors all significant deficiencies and material weaknessesin the design or operation of internal control over financialreporting which are reasonably likely to adversely affect thecompany’s ability to record, process, summarize, and reportfinancial information. In this publication we discuss theconcepts of significant deficiency and material weaknessand describe the matters management should consider inevaluating control exceptions or deficiencies to determinewhether they rise to the level of a significant deficiency ormaterial weakness.

The evaluation of the overall effectiveness of internalcontrol is both the end and the beginning of the process. In a dynamic business environment, controls will requiremodification from time to time. Certain systems mayrequire control enhancements to respond to new productsor emerging risks. In other areas, the evaluation may pointout redundant controls or other procedures that are nolonger necessary. The system of internal control should be self-monitoring and self-correcting. This means acompany should establish mechanisms to continuallyevaluate and maintain the system of internal control and,when necessary, take corrective action in a timely manner.The discussion of the evaluation process and ongoingassessment included in this publication will be useful inestablishing such mechanisms.

As always, we would be pleased to discuss the evaluationof internal control over financial reporting with you, andprovide our advice and assistance where appropriate.

Understand the Definition of Internal Control

Organize a Project Team to Conduct

the Evaluation

Evaluate Internal Control

at the Entity Level

Understand and Evaluate

Internal Control at the Process,Transaction, or

Application Level

Evaluate Overall Effectiveness,

Identify Matters for Improvement,

and Establish Monitoring System

Phase 1 Phase 2 Phase 3 Phase 4 Phase 5

1 At the time of publication, the Public Company Accounting OversightBoard had announced plans to issue revised standards and guidancefor independent auditors in performing an audit of internal controlover financial reporting. The revised standards, when issued, may alterthe conditions under which an auditor can perform an audit of internalcontrol over financial reporting and indirectly impact the requirementsfor management.

2 In its report, Internal Control-Integrated Framework, the Committee of Sponsoring Organizations (COSO) of the Treadway Commissionprovided suitable criteria against which management may evaluate andreport on internal control.

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

SEC Publishes Final Rule to Implement Section 404 . . . . . . . . . . . . . . . . . . . . . . 2

Perform Procedures to Evaluate the Operating Effectiveness of Controls. . . . . . . . . . . . . . . . 4

Designing Procedures to Evaluate the Operating Effectiveness of Controls . . . . . . . . . . . . 5

Types of Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Prevent Controls. . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Detect Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Controls Related to IT Processes . . . . . . . . . . . . . . . 7

Testing Controls for Each Component of Internal Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Determining the Extent of Tests of Controls . . . . . . . . 9

Timing of Tests of Controls . . . . . . . . . . . . . . . . . . . . 12

Other Considerations . . . . . . . . . . . . . . . . . . . . . . . . . 13

Coordinating Management’s and the Independent Auditors’ Procedures to Evaluate Operating Effectiveness . . . . . . . . . . . 13

Consider Adequacy of Tests Performed by Others . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Fraud Prevention and Detection Programs. . . . . . . 14

Self-Assessment Programs. . . . . . . . . . . . . . . . . . . 14

Consider Other Sources of Information. . . . . . . . . 15

Evaluate the Results of Procedures and Identify Matters for Improvement. . . . . . . . . . . . . . . . . 16

Determining Whether Control Exceptions AreDeficiencies in Internal Control. . . . . . . . . . . . . . . 17

Significant Deficiencies. . . . . . . . . . . . . . . . . . . . . 17

Material Weaknesses . . . . . . . . . . . . . . . . . . . . . . . 18

Updating the Evaluation of Controls to the Date of the Assertion . . . . . . . . . . . . . . . . . . . . . . . 19

Documentation Considerations . . . . . . . . . . . . . . . . . 19

Summarize and Evaluate Control Deficiencies . . . . . 19

Identify Matters for Improvement . . . . . . . . . . . . . 20

Establish Processes to Evaluate the System of Internal Control Over Time. . . . . . . . . . . . . . . . . . . . . 21

Ongoing Monitoring Activities. . . . . . . . . . . . . . . . . . 21

Separate Evaluations . . . . . . . . . . . . . . . . . . . . . . . . . 22

Appendix A—Review of Methodology and Steps toEvaluate the Design Effectiveness of Controls . . . . . 23

Appendix B—Evaluate the Effectiveness of Programmed Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Tests of Programmed Controls . . . . . . . . . . . . . . . . . . 25

Benchmarking Programmed Controls . . . . . . . . . . . . 26

Appendix C—Summary of Control Deficiencies . . . . . . . 28

Table of Contents

EVA L UAT I N G IN T E R NA L CO N T RO L S

1

An effective system of internal

control is comprehensive and

involves people throughout the

organization, including many who do not

think of themselves as having any accounting

or control responsibilities. Of course, it also

involves those who keep accounting records,

prepare and disseminate policies, and monitor

systems as well as members of audit

committees and the board of directors, who

have ultimate responsibility for oversight of

the financial reporting process.

The system of internal control should be under ongoingsupervision and monitoring by management to ensure that itis operating as intended and that it is modified appropriatelyfor changes in conditions. The initial evaluation effortnecessitated by the Act will establish a baseline from whichmanagement can measure the continued effectiveness ofinternal control in a dynamic environment.

The steps to complete the overall evaluation of theeffectiveness of internal control in the final phase of themethodology include:

(1) Performing procedures (including testing) to evaluatethe operating effectiveness of controls.

(2) Evaluating the results of the procedures, takingcorrective action in instances where controls do notachieve their control objectives, or otherwiseidentifying matters for improvement.

(3) Establishing processes to evaluate the system ofinternal control over time.

Procedures to evaluate the operating effectiveness ofcontrols include assessing the results of ongoing monitoringactivities that are ingrained in the company’s operations andperformed on a real-time basis, and other procedures thatfocus on evaluating operating effectiveness such as controlself-assessments, special reviews by internal audit or thirdparties, and procedures and tests performed by members of management. Procedures to evaluate the operatingeffectiveness of controls also might include obtaining andevaluating reports on the operating effectiveness of internalcontrols at service organizations. These various proceduresand tests will be needed to provide an appropriate basis formanagement to conclude on the operating effectiveness ofcontrols the project team has identified and managementrelies on to prevent or detect and correct the types of errorsor fraud that could occur.

While management is required to make an assertion as ofthe end of the company’s fiscal year, it ordinarily will benecessary for practical reasons to perform procedures toevaluate the operating effectiveness of controls at one ormore points earlier in the year. In addition to addressinglogistical issues, this can allow management time toimplement corrective actions, if necessary, and re-evaluate

Overview

2 EVA L UAT I N G IN T E R NA L CO N T RO L S

OV E RV I E W

any remediated controls before the assertion date. Further,the Company’s external auditors will need sufficient timeto consider the results of management’s procedures and toperform procedures to support their opinion on theCompany’s internal control.

Management will need to update its evaluation of controlsfrom the time of the interim procedures through the end ofthe fiscal year by identifying significant changes in internalcontrol at both the entity level and at the activity/processlevel. The extent of the procedures management will needto perform likely will increase as the period of timebetween the interim testing date and year-end increases.When management plans to make significant systemschanges prior to the end of the year, it should carefullyconsider the implications of these planned changes on itsassessment of internal control, including the need toincorporate sufficient time to evaluate the operatingeffectiveness of the revised systems (i.e., those that will be in effect at the reporting date). It is at this juncture inthe process that management also should evaluate whetherit has performed sufficient procedures to support itsoverall conclusion as to the effectiveness of internalcontrol over financial reporting.

The final step is for management to make an assertion aboutthe effectiveness of internal control over financial reportingas required by the Act. To accomplish this, management will need to evaluate any control deficiencies or exceptionsand determine whether they, either individually or in theaggregate, are deficiencies requiring remediation and/ormight need to be disclosed to the company’s independentauditors and the audit committee of the company’s board ofdirectors. Timely evaluation of control deficiencies will allowsufficient time for follow-up remediation and re-evaluationof controls before management must make its assertion. In addition to evaluating control deficiencies or exceptionsobtained directly through procedures it performed to evaluatethe operating effectiveness of controls, management alsoshould consider information obtained from other relevantsources, including the results from work performed byexternal and internal auditors, and regulators.

As part of performing the comprehensive evaluation ofinternal control over financial reporting, the project teamshould evaluate whether management has implementedappropriate self-monitoring and self-correcting mechanisms,or, where appropriate, it should make reasonably specificrecommendations for establishing such mechanisms.Investing in these mechanisms may serve to reduce theongoing time and cost to comply with the requirements of Section 404 of the Act, and also might provide timelyassistance to management in its quarterly certificationrequired by SEC rules implementing Section 302 of the Act.

SEC Publishes Final Rule to Implement Section 404On June 6, 2003 the SEC published its final rule toimplement Section 404 of the Sarbanes-Oxley Act of 2002(the “Act”). The final rule applies to all public companies,except registered investment companies and issuers ofasset-backed securities, but including small businessissuers and foreign private issuers. For “accelerated filers”(i.e., seasoned U.S. companies with public float exceeding$75 million), management and auditor reporting oninternal control over financial reporting is effective forfiscal years ending on or after June 15, 2004. For all otherissuers, including smaller companies, foreign privateissuers and companies with only registered debt andpreferred securities, the rule is effective for fiscal yearsending on or after April 15, 2005.

In subsequent fiscal quarters, the rule requiresmanagement to evaluate any change that has “materiallyaffected, or is reasonably likely to materially affect, theregistrant’s internal control over financial reporting.”However, the rule does not require management to fullyevaluate the issuer’s internal control over financialreporting as of the end of each interim reporting period.Instead, the SEC retained the requirement for managementto evaluate on a quarterly basis the effectiveness of theissuer’s disclosure controls and procedures (as defined),which would include some, but not necessarily all,elements of internal control over financial reporting. Theformal requirement to evaluate material changes in internalcontrol over financial reporting is effective in the firstperiodic report following the company’s first Section 404

report. However, for periodic reports due on or afterAugust 14, 2003, issuers must disclose any such materialchanges in internal control over financial reporting.

The final rule clarifies that to the extent a “materialweakness” exists at the end of the fiscal year, it must bedisclosed in management’s internal control report, and amaterial weakness would preclude management fromconcluding that internal control over financial reporting is effective. Further, the SEC points out in the adoptingrelease that an aggregation of significant deficiencies ininternal control over financial reporting could also amountto a material weakness that would preclude managementfrom determining that a company’s internal control overfinancial reporting is effective.

The SEC’s decision to extend the transition period was, atleast in part, based on concern about the cost and time itwill take to properly implement the new rules. Cost andtime factors also largely contributed to the SEC’s decisionto not require a full quarterly evaluation of internal control,in the end believing the cost would exceed the benefits.

Another contributing factor in the decision to defer theeffective dates was the perceived confusion as to the levelof work required of management and auditors to complywith the rules. The SEC’s adopting release clarifies thatboth the scope of management’s assessment of acompany’s internal control over financial reporting and thelevel of required documentation as to the design andoperating effectiveness of internal control must beextensive. Given that reporting on internal control overfinancial reporting is a very significant task, the SECclearly concluded that additional time beyond theoriginally proposed implementation date will be requiredto complete it in a high quality manner. The extendedtransition period demonstrates that the SEC listened to theconcerns expressed by commenters as to the shortimplementation period provided in the SEC’s proposedrule. However, management should not take this as anopportunity to delay their efforts to document and evaluatetheir internal control or they run the risk of not benefitingfrom the added time the SEC is allowing them to get ready.

3


Perform Procedures to Evaluate theOperating Effectiveness of Controls

In previous steps of the evaluation

methodology, the project team

determined whether one or more controls

address the significant risks identified as

“what can go wrong” questions. They also

determined whether the controls that address

the identified risks are adequately designed to

prevent or detect errors of importance or

fraud. Appendix A provides a description of

the steps the project team should follow to

document the design of controls identified

over specific processes. The project team

should be satisfied that it has addressed these

steps for each significant process before

designing procedures to evaluate the operating

effectiveness of controls, as it would be

inefficient to perform procedures to evaluate

the operating effectiveness of controls that are

not adequately designed or have not been

placed in operation.

At this point, the project team may ask itself, “Why do weneed to perform procedures to evaluate the operatingeffectiveness of controls?” The answer is quite simply thatmanagement’s assertion as to the effectiveness of internalcontrol over financial reporting needs to address the operatingeffectiveness of the controls as well as their effective design.3

While identifying controls that address each of the significantrisks and walking through them will confirm their design andconfirm whether they have been placed in operation, otherprocedures are needed to provide management with assurancethat the controls operate effectively to reduce the risk ofmaterial misstatements in the financial statements.

Management will need to perform procedures to evaluate the operating effectiveness of controls for each significantaccount balance, class of transactions, and disclosure eachyear so as to have a basis for its annual assertion. Thisrequirement suggests that management should, to the extentfeasible, seek to implement methods to evaluate operatingeffectiveness that are part of the normal recurringoperations of the entity.

The extent of testing and verification management needs to support its assertion is a judgmental determination. Indetermining the extent of necessary procedures to verifyoperating effectiveness, it may be practical to focus on theobjectives of the procedures. The objectives of the proceduresgenerally include determining whether the controls:

1 Operated according to the way the project teamunderstood (and documented) they would be operated.

3 In its adopting release implementing rules addressing the requirementsof Section 404 of the Act, the SEC specified, “The assessment of acompany’s internal control over financial reporting must be based onprocedures sufficient to both evaluate its design and to test itsoperating effectiveness.”

5

1 Were applied on a timely basis by the person(s)specified in the design of the control.

1 Encompassed all applicable transactions.

1 Were based on reliable information (i.e., informationused in the performance of the control is complete and accurate).

1 Resulted in the timely correction of any errors identifiedby the controls being relied upon by management.

Management may be able to determine that controls areoperating effectively through direct and on-going monitoringof the functioning of controls. This might be accomplishedthrough regular management and supervisory activities,monitoring adherence to policies and procedures, and otherroutine actions such as comparisons and reconciliations,supplemented by internal audit or other compliance functionsthat test, monitor, and evaluate the functioning of controls,or by various self-assessment programs.

As a technique, management could ask, from the top-down,“How do we know a particular control is operatingeffectively?” This question may help identify currentprocedures in place that management can potentially rely onas a basis for its assertion that a particular control operateseffectively. One example is a self-assessment programdesigned to monitor the operating effectiveness of one ormore controls. Self-assessment programs are discussed laterin this section. If management cannot identify currentprocedures that provide a basis for concluding the controloperates effectively, it should develop procedures to test andevaluate the operating effectiveness of the controls.

Designing Procedures to Evaluate the OperatingEffectiveness of ControlsFor certain controls or groups of controls, management maydetermine the most effective approach for evaluatingoperating effectiveness is for management, internal audit orothers, under the direction of management, to design andexecute procedures to test the functioning of the controls.Management is not constrained by statistical sampling ortypical sample sizes used by independent auditors. Thetesting strategy should be sufficient to provide managementwith adequate information to conclude that a particularcontrol or group of controls operates effectively.

The project team’s evaluation of the entity’s risk assessmentand monitoring processes may affect its selection of thereporting units or business locations included in theassessment, the procedures performed, the controls tested,and the timing of the procedures. However, all significantreporting units or business locations and all significantcontrols must be evaluated in connection with each annualassessment of the effectiveness of internal control.

Procedures to evaluate the operating effectiveness ofcontrols ordinarily include inquiry of appropriate personnel,inspection of relevant documentation, observation of thespecific operations, and testing the operation of the controlin the processing of selected transactions (e.g., reapplication,reperformance). The usefulness of these procedures and anyother procedures that help the project team arrive at aconclusion on the effectiveness of controls depends onwhether such procedures help satisfy the project team thatthe control in question operates as intended.

Factors management should consider in designing a testing strategy include:

1 The nature of the control and its significance in achieving thecontrol criteria and whether more than one control achieves a particular objective.

1 Whether significant changes in the volume or nature oftransactions might adversely affect control design or operating effectiveness.

1 Whether there have been changes in the design of the control.

1 The degree to which the control relies on the effectiveness of other controls (for example, the control environment orinformation technology (IT) general controls).

1 Whether there have been changes in key personnel who perform the control or monitor its performance.

1 Whether the control relies on performance by an individual or is automated.

1 The complexity of the control.


PERFORM PROCEDURES TO EVALUATE THE

OPERATING EFFECTIVENESS OF CONTROLS

Ordinarily, some combination of procedures to evaluatecontrols will be needed to provide assurance that thecontrol operated as intended. For example, the project teammay observe the procedures for opening the mail andprocessing cash receipts to test the operating effectivenessof controls over cash receipts. Because an observation ispertinent only at the point in time at which it is made, theproject team may supplement the observation with inquiriesof appropriate personnel and inspection of documentationabout the operation of such controls at other times. Theinquiries might address: (1) whether deviations from theprocedures observed ever occur and with what frequency,(2) whether errors or exceptions are ever identified, and (3)how errors or exceptions are handled.

Another example might be a project team that is testing thereconciliation of cash. The procedures might include thefollowing techniques:

1 Inquiry – The project team may ask the employee whoprepares the cash reconciliation how reconciling itemsare identified, the reasons for them, and the procedures inplace to ensure that the accounting records are correctedon a timely basis when there are reconciling items thatrequire correction. The project team may also askmanagement how it assures itself that the reconciliationsare prepared correctly and on a timely basis.

1 Observation – The project team may observe thepreparation of the cash reconciliation. However, projectteam members should be aware that employees mightperform procedures more diligently when they knowthey are being observed.

1 Reperformance and Inspection of Physical Evidence(e.g., the cash reconciliation itself) – The project teammight trace the amounts on one or more of the cashreconciliations to the related records or to otherdocuments (e.g., a bank statement) to gain assurancethat the procedures were properly performed(reperformance). The project team may read or reviewsome or all of the cash reconciliations for other periodsand examine the reconciling items to determine whetherthe reconciliation detected errors, and whether thoseerrors were appropriately addressed (inspection).

In designing a strategy to test the operating effectiveness ofcontrols, the project team might determine that it need nottest all controls that have been identified. For example, if theproject team believes that a particular assertion is likely to besupported by more than one control, it does not need to testall of the controls; rather the project team might decide to testonly the key controls on which management intends to baseits assertion. The choice of the controls to test depends on theteam’s evaluation of (1) whether the controls to be relied onare likely to be effective in supporting the related financialstatement assertion, and (2) whether the controls can betested more effectively and efficiently than the other controls.If the project team believes that one control supports morethan one assertion, the project team may test that controlonce to support achievement of all of the related assertions.

Inquiry is a procedure that typically is used extensivelythroughout the evaluation process and usually iscomplementary to performing other procedures. Inquiryconsists of seeking information of knowledgeable personsand may range from formal written inquiries to informalinterviews. Often the information obtained through inquiryis corroborated by other inquiries (both written and oral).Evaluating responses to inquiries is an integral part of theprocess. Inquiry alone generally will not provide sufficientevidence to support the operating effectiveness of controls.

Types of ControlsControls can include any procedures used and relied on bymanagement to: (1) prevent material misstatements,whether caused by error or fraud, from occurring duringtransaction processing or (2) detect and correct on a timelybasis material misstatements that may occur in processingtransactions. Two broad types of controls and theirdescriptions are listed below.

Control Types Description

Prevent Controls Policies and procedures designed to preventan error or fraud. Prevent controls are normallyapplied to individual transactions.

Detect Controls Policies and procedures that are designed todetect and correct errors or fraud that mightpreclude the achievement of the relevantprocess objectives. Detect controls generallyare applied to groups of transactions.

7

Prevent ControlsTo be effective, systems of internal control should includestrong prevent controls (either programmed or manual) inaddition to detect controls. For example, where there is ahigh volume of transactions, a lack of prevent controls, suchas edit or validation checks, on the input of transactionsbefore they are recorded significantly increases the risk oferrors and increases the need for particularly sensitive detectcontrols. In the absence of prevent controls, a high numberof errors can render detect controls ineffective in detectingand correcting errors in a timely manner. Prevent controlsoften are programmed controls and can take a myriad offorms. In addition to edit or validation checks, preventcontrols may include approvals, segregation of duties, file or transaction transmission controls, automated postings toledgers, and automated functionality (e.g., translation,mapping, calculations).

The type of prevent control also often affects the type ofevidence available to support the functioning of the control.For example, an effective manual prevent control (e.g., theshipping staff’s comparison of picked merchandise to thebills of lading) may not result in any physical evidenceindicating whether the control was performed, by whom, or how well.

In other cases, there may be evidence that indicatesperformance of the control, but evidence regarding theeffectiveness of the controls may not be available. Forexample, a signature on a voucher package to indicate thatthe signer approved it does not necessarily mean that theperson carefully reviewed it before signing; the packagemay have been signed based on only a cursory review (orwithout any review). As a result, the quality of the evidenceregarding the effective operation of a prevent controlevidenced by a signature or initials may not be sufficientlypersuasive to become satisfied that the control operatedeffectively without testing of the validity of data. Theproject team may need to look to other available evidence,such as interviews with supervisors or other evidence ofsupervisory oversight, to corroborate that the signatures areevidence that the prevent control is functioning. Thus,when the control is a checking routine (e.g., checkingprices, extensions, additions) evidenced by a signature orinitials, the project team should reperform the checkingroutine as part of the test of the control.

Tests of controls normally should not take the form of testsof individual transactions to infer the effectiveness ofcontrols, unless it is not otherwise practical to design directtests of the controls to support their proper functioning.For example, it generally is not appropriate for the projectteam to simply test a sample of cash disbursementtransactions to determine that transactions are properlyrecorded in the accounting records and then infer that allcontrols associated with initiating, recording, processing,and reporting those transactions function as intended.

Detect ControlsIn contrast to manual prevent controls that are oftenaccompanied only by evidence suggesting that the controloperated (e.g., initials, signatures), detect controls, such asa monthly reconciliation, are often accompanied byphysical evidence of their performance (e.g., a completedreconciliation). The project team normally should examineevidence that the reconciliation was properly completed(including tracing certain amounts for agreement) and thatthe appropriate reviews and follow-ups were carried out.

Evaluating detect controls also can be more efficient thanmanual prevent controls because detect controls often areapplied to groups of transactions rather than on atransaction-by-transaction basis, and they are performedless frequently than prevent controls. Management mayplace more emphasis on prevent controls on the premise itis more efficient to prevent errors or misstatements,however, no system of controls can be expected to betotally effective and should incorporate an appropriatemixture of prevent and detect controls.

Controls Related to IT ProcessesWe anticipate that, except in certain rare instances, theproject team will determine that management is relying onIT programmed controls (i.e., controls performed by acomputer) or that identified controls are dependent on IT-generated data (i.e., electronic evidence). The project teamwill need to evaluate the effectiveness of controls thatmanagement relies upon to ensure that the programmedcontrols or the electronic evidence relied upon in theperformance of other controls continue to operateeffectively over time. Ordinarily companies that rely heavilyon IT applications have implemented IT general controlsthat are applied above the computer application (i.e.,




program) level. IT general controls include controls that aredesigned to address a number of important aspects of theoverall IT operations, including business continuity andsecurity. However two aspects of IT general controls thatdirectly impact financial reporting objectives are controls:

1 Ensuring that changes to programs are properly authorized,tested, and approved before they are implemented.

1 Ensuring that only authorized persons and programs haveaccess to data, and then only to perform specificallydesigned functions (e.g., inquire, execute, update).

In the event the project team determines that managementhas not implemented IT general controls or the IT generalcontrols are not effective, it will need to identify othercontrols that ensure key IT applications continue tofunction as intended. For example, in the absence ofeffective IT general controls over program changes,company personnel may have adopted procedures toroutinely recheck calculations performed by variouscomputer applications or reconcile output from thecomputer applications to source data to ensure that keycomputer applications continue to function as designed.

When a computer-generated report is important tomanagement’s performance of a detect control and is aprimary source of information for a control managementrelies upon, the project team must assure itself that the datain the report used in the performance of the control iscomplete and accurate. For example, if a report (e.g., cashbalances report from the cash ledger system) is importantto the performance of a detect control (e.g., cashreconciliation) and that detect control providesmanagement with its primary evidence related to one ormore assertions embodied in the financial statements, theproject team will need to test controls that ensure theaccuracy of the underlying data provided in the report.

In computerized environments, prevent controls often canbe tested more effectively and efficiently than detectcontrols. This is because of the inherent consistency ofcomputer processing absent changes to the program andthe direct evidence of their operating effectiveness that canbe obtained (e.g., review and follow-up of edit reports).Certain aspects of automated processes to initiate, record,process, and report transactions may represent control

activities in that they consistently apply predefinedbusiness rules and perform complex calculations inprocessing large volumes of transactions or data, andotherwise enhance the timeliness, availability, andaccuracy of information.

Certain aspects of the company’s electronic information mayexist only at a certain point in time, or such information maynot be retrievable after a specified period of time. Therefore,the project team will need to consider the time during whichinformation exists or is available in determining the nature,timing, and extent of their tests of controls. For example, theproject team might determine that it will need to perform aportion of its planned procedures to test controls overtransaction processing at various times throughout the year.

Appendix B—Evaluate the Effectiveness of ProgrammedControls provides additional examples of procedures theproject team might perform to evaluate the operatingeffectiveness of programmed controls.

Testing Controls for Each Component of Internal ControlThe project team should evaluate the operating effectivenessof significant controls for each of the components4 ofinternal control in each annual assessment. Project teamsmay have difficulty identifying the type (or nature) of testsof controls they should design and execute to evaluate theoperating effectiveness of certain components of internalcontrol. It generally will be easier to design and execute testsof controls within control activities and information andcommunication components since many of these controlsoperate at the significant account, class of transactions, ordisclosure level. The project team may find it more difficultto design and execute tests to evaluate controls for the

4 The components that constitute a company's internal control are afunction of the definition and description of internal control used bymanagement for the purpose of assessing its effectiveness. For example,a majority of companies will likely select the definition and descriptionof internal control based on the internal control framework set forth inInternal Control-Integrated Framework, published by the Committee ofSponsoring Organizations (COSO) of the Treadway Commission.Internal Control-Integrated Framework describes an entity's internalcontrol as consisting of five components: control environment, riskassessment, control activities, information and communication, andmonitoring. If management selects another definition and description of internal control, these components may not be relevant.

control environment, risk assessment, and monitoringcomponents since many of the controls comprising thesecomponents operate as entity-wide controls and thereforeevidence gathered about their operating effectivenessgenerally will be less conclusive and require more subjectivejudgment on the part of the project team.

Entity-wide controls are controls the company hasimplemented to monitor the operations at various locationsor business units and therefore operate over controls at thevarious locations or business units and may encompass anumber of aspects of the control environment, riskassessment, and monitoring components of the entity’soverall internal control. These entity-wide controls areimportant because they might have a pervasive effect oncontrols over individual processes at individual locationsor business units, and increase or decrease the likelihoodthat such controls will continue to operate as intended.Entity-wide controls ordinarily include a combination ofthe following:

1 Management’s assignment of authority andresponsibility, use of consistent policies and procedures,and implementation of entity-wide programs such ascodes of conduct and fraud prevention that apply to alllocations and business units.

1 Centralized processing and controls, including sharedservice environments.

1 Monitoring results of operations.

1 Monitoring of controls, including activities of theinternal audit function and self-assessment programs.

Entity-wide controls relating to the risk assessmentcomponent of internal control might include management’sidentification of the risks of material misstatement in thesignificant accounts, classes of transactions, and disclosuresand related assertions in the financial statements and itsimplementation of controls to prevent or detect errors orfraud. For example, the risk assessment process may addresshow management considers the possibility of unrecordedtransactions or identifies and analyzes significant estimatesrecorded in the financial statements. Management’sassessment may result in it assigning levels of authority fortransactions based on perceived risk. Risks relevant to

reliable financial reporting also relate to specific events ortransactions. The project team should consider whetherprevious failure to identify such risks or to control them is adeficiency that should be addressed.

Entity-wide controls relating to the information andcommunication component generally are the same systemsand processes covered by the audit of financial statements(e.g., accurate and timely financial information, controls oversystems development, access to IT resources and data, and business continuity/disaster recovery planning). Thiscomponent also includes controls that ensure the safeguardingof assets and processes for authorization of transactions andthe maintenance of records.

Entity-wide controls relating to the monitoring componentextends to management’s monitoring of the functioning ofthe significant controls, including control activities, thatmanagement has designed to prevent or detect errors ofimportance in significant accounts, classes of transactions,and disclosures and related assertions in the financialstatements. Controls relating to the monitoring componentalso include the role and functioning of internal audit andmanagement’s processes for responding to internal controlrecommendations and correcting known deficiencies.

Because of their potential pervasive impact on theeffectiveness of other controls, the project team shouldevaluate entity-wide controls closely. The relative strengthof entity-wide controls is one of several factors the projectteam should consider in determining the extent of testingof individual controls at the process, transaction, orapplication level.

Determining the Extent of Tests of ControlsManagement can evaluate the operating effectiveness ofcontrols by designing and executing tests of the controls orthrough regular management or supervisory activities,robust control self-assessment programs, or other routineactions. The project team may vary from year to year thenature, timing, and extent of its procedures to evaluatecontrols however, management will need to gatherevidence as to the operating effectiveness of controls foreach significant account balance, class of transactions, anddisclosure in each annual assessment. For example, the

9




project team may test the controls at a different interimperiod; increase or reduce the number of tests performed;or change the combination of procedures used. The tests ofcontrols should be designed to provide evidence to enablemanagement to conclude whether the company’s system ofinternal control over financial reporting (i.e., including thecontrols in all five components) is operating effectively.The project team also should be mindful that varying theextent of testing in a particular year potentially could affectthe external auditor’s determination of the nature, timingand extent of its procedures to evaluate the operatingeffectiveness of controls.

Because the internal control components are interrelated,the evidence produced from the combination of tests ofeach component provides better evidence that thecompany’s internal control is operating effectively thanevidence from tests of only a single component. Similarly,tests of more than one control related to a significantaccount provide better evidence than a test of a singlecontrol for a significant account. Accordingly, in additionto testing the entity-wide controls related to significantaccounts (that is, the control environment, risk assessment,and monitoring controls), the project team also shouldperform some tests of control activities for each significantaccount balance, class of transactions, and disclosure ineach annual assessment.

The project team will need to use judgment to determinethe extent of procedures (i.e., testing an individual control)necessary to support management’s assertion as tooperating effectiveness. There are a number of factors theproject team should consider which are listed below:

1 How often the control is performed. The less frequentlythe manual control is performed (e.g., a manual controlapplied monthly compared to a manual control appliedon a transaction-by-transaction basis) the fewer numberof times the test needs to be performed in order for theproject team to be satisfied regarding the effectiveoperation of the control. Likewise, if the control is aprogrammed control the testing may be less extensivesince the control will always function in the same manneruntil the program is changed.

1 The degree to which management plans to rely onthe control as a basis for its assertion. The higher thedegree of reliance the more extensively the project teamshould test the control to provide the desired assurance.When other controls, related to one of the assertions, are also in place and tested, the level of assurance thatmight be needed from any one control need not be ashigh as would be the case where management is placingreliance solely on the one control.

1 The persuasiveness of the evidence produced by thecontrol. If the performance of a control activity resultsin little or no direct evidence that the control or controlactivity operated effectively, then testing it—no matterhow extensively —may not provide the necessaryassurance. Conversely, if direct evidence regarding theeffective operation of a control is available, the projectteam might decide that they only need to examine alimited amount of the evidence to be satisfied regardingits effective operation.

1 The relative importance of the possible errors thatcould result if the control is not functioning. Mattersto consider in assessing the amount of assurance neededfrom the test include:

—The materiality of the transactions being processed.

—The complexity of the transactions.

—The volume of transactions.

1 Other factors that relate to the likelihood that thecontrol operated as intended. In determining the extentof tests, the project team also should consider severalother factors that affect the likelihood that the control isoperating as intended. Such factors may include:

—The competency of the person who performs thecontrol. The apparent competence and integrity of theemployees performing the control, their objectivityabout the related processing procedures, the degree towhich the employees are supervised, and the extent ofemployee turnover all contribute to the likelihood thatthe control operates as intended.

—The effectiveness of internal control at the entity level.The effectiveness of internal control at the entity levelmay influence (either positively or negatively) the

operating effectiveness of a specific control, and it alsomay influence the project team’s determination of theextent of testing. For example, in deciding on theextent of testing, the project team should consider:

•• The likelihood that a control is bypassed duringpeak processing periods.

•• The potential for management override of a control.

•• The potential risk of fraud.

•• The extent to which the controls have been subjectedto on-going monitoring activities throughout the year.

•• The likelihood that a control will continue tooperate as intended until year-end (i.e., the date ofmanagement’s assertion). Effective internal controlat the entity level provides a basis for expectingthat controls tested during the year will continue tooperate as intended until year-end.

—Changes in the related processes. When there arechanges in the related processes during the year, theproject team might consider whether a controlcontinues to be effective after changes to the processhave been implemented. This can be especially truewhen implementing a new IT application.

Even though one or more of the factors just described mayreduce the expectation of errors, the procedures to evaluatethe operating effectiveness of controls need to besufficiently extensive to provide appropriate evidence thatthe controls are operating effectively. The project team willneed to exercise judgment in determining the appropriatesample size, and, regardless of the size of the test, it shouldplan to investigate all control exceptions (i.e., control did notfunction as designed). Generally, in situations where theproject team is performing tests of more than one controlrelated to a significant account, class of transactions, ordisclosure, and where deviations from designed controls arenot expected and the likelihood of material errors isconsidered low, the samples need not be large. In situationswhere deviations from designed controls are expected, thelikelihood of errors or override is considered other than low,or the control is the primary or only control related to asignificant account, class of transactions, or disclosure,larger sample sizes may be appropriate. There may be other

circumstances, for example when a control is applied by anumber of different personnel at various reporting units,where larger sample sizes also may be appropriate.

The project team may need to exercise additional judgmentin determining the extent of its testing of a control whichfunctions over a common process that is applied throughoutnumerous locations or reporting units of a company. Forexample, company policy may require that operationspersonnel obtain a credit check from an external sourcebefore extending terms to a customer. On one hand, thecontrol is designed to function similarly across all locationsor reporting units. However, different personnel execute thecontrol at each of these locations or reporting units, andtherefore the control may not function with the sameoperating effectiveness, or the control may be overridden.In addition to considering the factors discussed previously(e.g., importance of the control to management’sassessment, control is one of a combination of controls,competence of those who perform the control), the projectteam also should consider whether management hasimplemented procedures to monitor the terms extended tocustomers at the various locations or reporting units, andmonitor the functioning of the controls. In this example,routine monitoring of the terms extended to customers bymanagement, or internal audit procedures to determine thatthe control is functioning as part of ongoing monitoringactivities, may allow the project team to evaluate the

11

Sample Sizes

While the extent of testing is a judgment, statistically based samplesizes can be useful in promoting a consistent approach to testingindividual controls across an organization. Generally, a sample of 25,without the occurrence of exceptions, provides a statistical reliabilityof 90% that the error rate does not exceed 10%. In situations wherethe project team is testing a number of controls to evaluate theeffectiveness of internal controls that are responsive to one ormore assertions, sample sizes of 25 or more generally would providea sufficient basis for concluding a control operated effectively. Theproject team should consider larger sample sizes when the control itplans to test is the only control identified for one or more significantassertions. Sample sizes of 60 or more provide a statistical reliabilityof 95% that the error rate does not exceed 5% when no exceptionsare found.




effectiveness of these higher level controls and limit thenumber of times it plans to test the functioning of theindividual control (i.e., obtaining a credit report). In theabsence of these monitoring activities, the project teamlikely will determine that it needs to expand the number oftimes it plans to test the functioning of the control acrossthe various locations or reporting units to supportmanagement’s evaluation.

When a project team is determining the extent of its testsof a control, it should consider the work it undertook aspart of its earlier walk-through of the process to verify its understanding of the related controls. In certain cases, a walk-through might provide sufficient evidence ofoperating effectiveness. For instance, a walk-through of acomputer process and the related programmed control may,in some cases, allow the project team to conclude that thecontrol is operating effectively. This conclusion might bebased on the premise that a programmed control issystematic and if successfully tested once, and IT generalcontrols are effective, it will work the same way repeatedlyunless the program is changed.

If the project team observes an exception, it should becareful not to dismiss the exception as a random (non-systematic) occurrence. Accordingly, if a sample wasselected for testing based on the expectation of not findingany exceptions, the detection of one control exceptionshould result in the project team:

1 Performing a qualitative analysis to determine the causefor the control exception.

1 Extending the test (in anticipation of determining that,while control exceptions exist, the rate of occurrence isacceptable to management).

1 Amending its decision to rely on that control.

1 Considering whether another control is available thatcan be substituted for the one being tested.

If the project team extends its testing and additional controlexceptions are observed, they should either address how tocorrect the control so the error does not continue to occur,or they should identify another control that addresses theassertion. The other control is a different control or set ofcontrols that achieve the same control objective as theoriginal control. Timely evaluation of control exceptions

and follow up remediation of deficiencies or identificationof other controls will enable management to take advantageof the point-in-time nature of its assertion on internalcontrol. If addressed early, management can allowsufficient time to re-evaluate the control and concludewhether it operates effectively.

Timing of Tests of ControlsThe period of time over which controls should be evaluatedis a matter of judgment, however, it should vary with thenature of the controls being evaluated, the frequency withwhich specific controls operate, and the specific policiesthat are applied. Some controls operate continuously (e.g.,controls over sales), while others operate only at certaintimes (e.g., controls over the preparation of financialstatements, controls over physical inventory counts). Theproject team should evaluate controls over a period of timethat is adequate to determine whether the controls necessaryfor achieving the relevant assertion are operating effectively.

As a matter of necessity, the project team will need toevaluate controls before the end of the fiscal year and,therefore, the project team will need to update its evaluationof controls from the time of the original procedures throughfiscal year end. The project team will need to determinewhat additional evidence should be obtained for theremaining period. In making that determination, the teamshould consider the specific controls that were originallytested, the degree to which evidence about the operatingeffectiveness of those controls was obtained, and the lengthof time from the original date of testing to the end of theyear. Generally, the extent of the procedures the project teamperforms to update its evaluation should increase as theperiod of time between the original testing date and year-endincreases. For example, the performance of control testing inJune would require more extensive procedures to update theevaluation than the performance of testing in October for acompany with a December year end. The project team alsoshould obtain evidence about the nature and extent of anysignificant changes in internal control at the entity level andat the activity/process level that occurred subsequent to theoriginal evaluation procedures.

Some controls over financial reporting may only operate afterthe date of management’s assertion (i.e., end of the company’sfiscal year). For example, many aspects of the financial

statement close process only function after year end. Theproject team will need to ensure that its procedures toevaluate the operating effectiveness of controls includesthose significant controls that only operate after year end.

Other ConsiderationsCoordinating Management’s and the Independent Auditors’ Proceduresto Evaluate Operating Effectiveness The entity’s independent auditors may consider the resultsof the entity’s procedures to evaluate the operatingeffectiveness of controls in determining the nature, timingand extent of their procedures. However, the independentauditors will need to obtain the principal assurance as to theoverall effectiveness of the entity’s internal control fromtests they have performed. Similarly, the independentauditors should not rely solely on the results of proceduresperformed by others (i.e., internal audit) as evidence of theoperating effectiveness of controls over significant accountbalances, classes of transactions, and disclosures. Generally,this means the independent auditors will need to reperformor perform independent tests of controls for each significantaccount balance, class of transactions, and disclosure tocorroborate the results of any tests performed by others thatit plans to rely on. The independent auditors may not use theresults of tests performed by others in evaluating evidenceabout the control environment, including fraud relatedprograms and controls, and additionally must limit the useof the work of others in evaluating:

1 Controls that have a pervasive effect on the financialstatements (such as IT general controls on which theoperating effectiveness of other controls depend).

1 Controls over significant non-routine and non-systematic transactions (such as accounts involvingjudgments and estimates).

1 Controls over the period-end financial reportingprocess, including controls over procedures used toenter transaction totals into the general ledger; toinitiate, record, and process journal entries in thegeneral ledger; and to record recurring and nonrecurringadjustments to the financial statements (for example,consolidating adjustments, report combinations, andreclassifications).

These requirements may limit the extent to which testingprocedures performed by internal audit and other groupsmay be used by the independent auditors to reduce therequired extent of their work. While management cannotrely on the results of the independent auditors’ testing informing its conclusions, there may be complementarytesting strategies that result in an appropriate and efficienttesting of the operating effectiveness of controls.

Consider Adequacy of Tests Performed by Others Procedures to evaluate the operating effectiveness ofcontrols might include testing performed directly bymanagement personnel (including control self assessmentprocesses) or by internal audit or third parties under thedirection of management. Procedures to evaluate theoperating effectiveness of controls also might includeservice organization reports that include procedures todetermine if controls are operating effectively. Regardlessof who performs the tests to evaluate operatingeffectiveness of controls, the project team must satisfyitself that the nature and scope of the various tests providesufficient evidence to support management’s assertion asto the overall effectiveness of internal control, and thosepersons performing the tests or other evaluative proceduresare objective and competent to do so.

If a service organization is part of the entity’s informationsystem and, accordingly, is part of the entity’s internal control,the project team should consider the activities of the serviceorganization, including obtaining an understanding of thecontrols at the service organization that are relevant to theentity’s internal control and determining that those controlsare operating effectively. Performing procedures at the serviceorganization or obtaining a service auditor’s report mayprovide such evidence. If a service auditor’s report on controlsplaced in operation and tests of operating effectiveness isavailable, the project team may consider whether this reportprovides sufficient evidence to support management’sassertion. The items the project team should consider include:the scope of the examination and applications covered, thecontrols tested and how they relate to the entity’s internalcontrols, the results of those tests of controls, the serviceauditor’s opinion on the operating effectiveness of thecontrols, and the time period covered by the service auditor’stests of controls and its relation to the date of management’sassertion. A service auditor’s report that does not include tests

13




of controls and the service auditor’s opinion on operatingeffectiveness (i.e., “report on controls placed in operation”)does not provide evidence of operating effectiveness.

When the period of time covered by the tests of controls inthe service auditor’s report significantly precedes the date ofmanagement’s assertion (e.g., service auditor’s testingperformed through June 30 and management’s assertion is asof December 31), the project team should perform additionalprocedures, including making inquiring of appropriatepersonnel within the company who are responsible for the company’s relationship with the service organization.The inquiries should focus on whether management hasidentified any changes in the service organization’s controlssubsequent to the period covered by the service auditor’sreport (e.g., changes communicated to the companythrough communications from the service organization,changes in personnel at the service organization withwhom company personnel interact, changes in reports orother data received from the service organization, changesin contracts or service level agreements with the serviceorganization, errors identified in the service organization’sprocessing). If such changes are identified, the project teamshould perform procedures to evaluate the effect of suchchanges on the assessment of internal control.

Fraud Prevention and Detection ProgramsManagement also should consider evidence about theoperating effectiveness of its fraud prevention and detectionprograms in evaluating the overall effectiveness of theentity’s internal controls over financial reporting. Generally,the steps that management implements to prevent, deter,and detect fraud include:

1 Creating and maintaining a culture of honesty and high ethics.

1 Evaluating the risks of fraud and implementingprocesses, procedures, and controls needed to mitigatethe risks and reduce the opportunities of fraud.

1 Developing an appropriate oversight process.

The steps management undertakes to create and maintain aculture of honesty and high ethics most likely are rooted ina strong set of core values that provides the foundation foremployees as to how the organization conducts itsbusiness. These steps should include:

(1) Setting the tone at the top.

(2) Creating a positive workplace environment.

(3) Hiring and promoting appropriate employees.

(4) Training about the entity’s values.

(5) Periodic confirmation of accountability.

(6) Disciplining incidents of fraud or wrongdoing.

Management generally can evaluate the operatingeffectiveness of these steps through its assessment of thecontrol environment component of internal control overfinancial reporting. Specific activities that the project teammight evaluate include oversight processes of the auditcommittee, the board of directors, and management: (1) reinforcing the entity’s commitment to creating theappropriate tone and high ethical values, (2) providingmechanisms for employees to report concerns about unethicalbehavior, actual or suspected fraud, or violations of the entity’scode of conduct, (3) demonstrating proactive involvement andoversight of financial reporting activities, and (4) proactivelyinvestigating indicators of fraudulent activities.

Self-Assessment ProgramsManagement may be able to use the results of robust controlself-assessment programs to, at least in part, evaluate theoperating effectiveness of various controls. Self-assessmentprograms are becoming a popular method of incorporatingthe real-time nature and efficiency of ongoing monitoringactivities into separate evaluations that provide an explicit,documented assessment and reporting to management.

Companies that have implemented one or more control self-assessment processes will want to determine the extentthese processes can be leveraged to evaluate the operatingeffectiveness of controls over financial reporting. The degreeto which these programs can be effectively leveraged willdepend on the focus of the self-assessment processes and thenature and extent of the assessment procedures. If the controlself-assessment processes focus on business processes,

efficiency and effectiveness of operations, or compliancewith various laws and regulations, and only indirectly onassessing the effectiveness of controls over financialreporting objectives, they likely will not provide a basis forevaluating internal control over financial reporting. In suchcases, management should consider expanding the focus ofthese processes to include assessment of key controls thatassure achievement of financial reporting objectives.

Similarly, some self-assessment programs involve onlysurvey or inquiry of those who operate the control(s). Inquiryalone does not adequately evaluate whether the controls areoperating effectively and will need to be supplemented withother procedures to provide an appropriate basis to supportmanagement’s assessment. In the absence of objectiveverification procedures to evaluate operating effectiveness,self-assessment processes generally function as part ofmanagement’s risk assessment or monitoring processes bycreating awareness of poorly designed or missing controls.

Consider Other Sources of Information The project team should consider other relevant sources of information in addition to its own documentation andtesting to provide management with all of the informationit will need to make its assertion about the effectiveness ofinternal control. The following sources of informationmight be relevant to management’s assessment:

1 Work Performed by External Auditors—The externalauditors may identify errors and weaknesses in performingthe financial statement audit. Among the items that couldprovide information with respect to errors or weaknessesare legal letters and other substantive audit procedures(e.g., confirmation requests returned noting differences).

1 Internal Audit Reports—Internal audit reports issuedduring the year may include discussions of weaknessesin internal control and related matters.

1 Service Organization Reports—Situations may arisewhere a service organization report indicates weaknessesin the controls, either at an outside service organizationthat is part of the entity’s internal control, or at the entitywhen the subject of the service organization report is oneor more operations of the entity (e.g., trust operations of a bank holding company, investment operations of aninsurance company). These weaknesses in the controls ofthe service organization also may affect the controls ofthe company.

1 Regulatory Reports—Recent regulatory reports ofexamination, supervisory memorandums, or reports of regulatory actions to which the company has beensubject may provide information with respect to errorsor weaknesses.

15

Elements of an effective control self-assessment process for evaluating the operating effectiveness of controls might be found in the following examples:

1 A company has a program to assess whether its reporting unitsaround the world reconcile key general ledger accounts on atimely basis, and pursue and resolve reconciling items. Theprogram involves each site controller periodically assessing andreporting on the reconciliation activities by the financialaccounting personnel. Corporate management supplements thisprocess by independently verifying the accuracy of the reportedinformation and the quality of the reconciliation activitiesthrough inquiry and observation during routine site visits.

1 A company has a program where teams of accounting andoperations personnel from various reporting units around theworld visit selected reporting units and perform procedures toevaluate each reporting unit’s compliance with company policiesand procedures for accounting, financial reporting, andoperations matters.


The project team should document and

accumulate all control exceptions

encountered in performing tests of

operating effectiveness of controls. A control

exception or deviation occurs when procedures

used to evaluate operating effectiveness indicate

that a control did not operate as intended.

For example, the project team might find that keyinformation was not reconciled, reconciling items were notinvestigated in a timely manner, or the error identified bythe control was not corrected in a timely manner. Theproject team also may find that higher-level monitoringcontrols did not result in appropriate follow up, or that the follow up was not effective in achieving the relevantassertions. When the project team identifies a controlexception, it should consider the results in relation tomanagement’s overall evaluation of internal control anddetermine whether the exception potentially indicates thereis a deficiency in the design or operating effectiveness of the control, or both. The project team also should attempt to identify the underlying cause of the error, if possible, todetermine whether the error is due to specific circumstances(e.g., the person that normally applies the control was onvacation and the control did not work). If the project teamdetermines the error and its cause have been corrected, andis satisfied the control is now working as designed, thecontrol is likely to be effective as of the date ofmanagement’s assertion.

Other procedures the project team should perform whencontrol exceptions are found include:

1 Confirm its understanding of the control—Uponfurther investigation, the project team may determinethat its earlier understanding of the process and therelated controls was incorrect. Therefore, the controlspreviously documented by the project team are notfunctioning and they should determine whether othercontrols are in place and, if so, identify, evaluate and testthe other controls.

1 Consider the cause and implications for eachexception—Knowing the cause of a control exceptionaids in determining the potential for errors of a similartype and the need for changes in procedures to evaluatecontrols. For example, the project team might considerwhether the exception is systematic (one that can beexpected to occur in every similar circumstance) orapparently random. The project team also shouldconsider whether the control exception is intentional or unintentional.

1 Determine the accounts and processes affected—The project team should understand all the significantfinancial statement accounts affected by the controlexception. For example, a control exception indicatingthat cash disbursements may not be appropriatelyrecorded in the general ledger has implications for manyfinancial statement accounts.

1 Consider the potential effect on management’sassertions—If the identified controls are not in place orare not effective in preventing and/or detecting materialmisstatements in an account balance, class oftransactions, or disclosure and related assertion,additional or different manual or programmed controlsmay need to be identified and evaluated.

Evaluate the Results of Proceduresand Identify Matters for Improvement

Determining Whether Control Exceptions AreDeficiencies in Internal ControlThe project team will need to analyze and evaluate theerrors and control exceptions identified in testing theoperating effectiveness of controls, and from other sources,to determine whether they represent deficiencies in internalcontrol, and their effect on management’s assertion. Adesign deficiency exists when either a necessary control is missing or an existing control is not properly designed to achieve the desired control objective. An operatingdeficiency exists when a properly designed control either isnot operating as designed or the person performing a controldoes not possess the necessary authority or qualifications toperform the control effectively.

The project team should consider the following factors inevaluating errors, exceptions, and control weaknesses:

1 Has the cause of the error or control exception beenidentified and corrected?

1 How was the error or control exception detected? For example, if detected by management as part of thenormal process, overall internal control may be effectivedue to the existence of compensating controls.

1 Is the error or control weakness confined to a singleapplication, or is it pervasive?

1 Assuming the control weakness is not pervasive, is theerror or control exception attributable to occasionalcarelessness or inadequately trained staff?

1 Provided the error or control weakness is confined toone application, how significant is the application to theoverall entity? For example, for a financial institution, acontrol weakness discovered in the lending area, whichis confined to leasing activities where leasing activitiesrepresent only an immaterial portion of total loanactivity, may not be significant.

1 How significant is the deviation from stated policy? Forexample, if policy states that accounts are reconciledwithin four weeks of month end, an exception wherereconcilements were not performed for six monthsmight be more significant than an exception wherereconciliations were actually done within six weeks.

1 What is the likelihood that other similar errors haveoccurred and remain undetected?

1 How frequent are deviations or control exceptions inrelation to the frequency of performing the control?

This list is not intended to be exhaustive; its purpose is tostimulate thought with respect to other questions that maybe relevant in the circumstances. The project team’sconsideration of these factors and responses to the team’sother inquiries may indicate that the noted errors or controlexceptions are deficiencies that require reporting tomanagement. The project team also may conclude that thedeficiency rises to the level of a significant deficiency ormaterial weakness as defined by the SEC and in auditingliterature, and described in the following sections.

Significant Deficiencies A significant deficiency is an internal control deficiency ina significant control or an aggregation of such deficienciesthat could result in a misstatement of the financialstatements that is more than inconsequential. A significantdeficiency may result from either a design deficiency orlack of operating effectiveness.

In determining whether the control exception potentiallyindicates there is a deficiency in either the design oroperating effectiveness of the control, some factors theproject team should consider include:

1 Whether the control is automated (i.e., in the presenceof effective general controls, an automated applicationcontrol is expected to always perform as designed).

1 The degree of intervention by company personnelcontributing to the deviation.

1 If management was aware of the deviation, the actionstaken by management in response to the issue.

If the reasons for the exception do not indicate a weakness in the general design or operation of the control, then thedeviation may not indicate a significant deficiency. However,regardless of the reasons for the deviation, numerous orrepeated instances may constitute a significant deficiency.A control with an observed non-negligible deviation rate isnot an effective control.

17


EVALUATE THE RESULTS OF PROCEDURES

AND IDENTIFY MATTERS FOR IMPROVEMENT

The project team may identify multiple control exceptionsor deficiencies that are common to a specific account,control component, or location or reporting unit, orotherwise have common features or attributes. A numberof internal control deficiencies that have a common featureor attribute may rise to the level of a significant deficiencyeven though the control exceptions or deficiencies areindividually insignificant. For example, the project teammay identify numerous instances in which management’srisk assessment process operates deficiently with regard toaccounts or locations or both. Similarly, the project teammay determine that a reconciliation of detail to the generalledger has not been performed across a range of accountsor business units. Although such deficiencies, if isolatedoccurrences, may not be individually evaluated as asignificant deficiency, the project team may conclude thatmultiple instances of a deficiency around a common themerise to the level of a significant deficiency.

A significant degree of judgment is required in evaluatingwhether an internal control deficiency is a significantdeficiency. Additional factors the project team shouldconsider include:

1 The frequency of exceptions in the operation of a control.

1 The likelihood that the internal control deficiency couldresult in a misstatement.

1 The magnitude of potential misstatements in the financialstatements resulting from the internal control deficiency.

1 The importance of the control that is deficient,including the degree to which other effective controlsachieve the same control objectives.

1 The nature of the account balances or classes oftransactions affected by the internal control deficiencyand the financial statement assertions involved.

Material WeaknessesA material weakness is a significant deficiency or anaggregation of significant deficiencies that precludes theentity’s internal control from providing reasonable assurancethat material misstatements in the financial statements willbe prevented or detected on a timely basis by employees inthe normal course of performing their assigned functions.

Evaluating whether a significant deficiency is also amaterial weakness is a subjective process that depends on various factors. These may include the nature of theaccounting system and of any financial statement amountsor transactions exposed to the significant deficiency, theoverall control environment, other controls, and the judgmentof those making the evaluation. The absence of identifiedmisstatements is not a criterion for concluding that significantdeficiencies do not constitute material weaknesses.

The project team should focus on both the likelihood andpotential impact of misstatements that could arise, either dueto error or fraud, and remain undetected. The potentialmisstatement could range from zero to more than the grossfinancial statement amounts or gross amounts of transactionsthat are exposed to the significant deficiency. However, thelikelihood of misstatement due to error or fraud generallywill vary for the different possible amounts within that range.For example, the risk of misstatement due to error or fraud inamounts equal to the gross exposure might be very low, butthe risk of smaller amounts might be greater.

There may be situations where the project team will needto consider whether individual significant deficiencies,when aggregated, amount to a material weakness. In thesesituations, the project team should consider the followingin making its evaluation:

1 The range or distribution of the amounts of potentialmisstatement, whether caused by error or fraud, thatmay result during the same accounting period from twoor more individual significant deficiencies.

1 The likelihood that such a combination of misstatementswould be material.

Similar to evaluating whether deficiencies are significant inthe aggregate, the project team generally will considersignificant deficiencies in the aggregate to also be a materialweakness when it determines the impact of a potentialmisstatement that could result from the deficiencies is amaterial amount and the likelihood of a combination ofmisstatements occurring is more than low.

Updating the Evaluation of Controls to the Date ofthe AssertionThe project team will need to update its evaluation ofcontrols from the time of the original evaluation throughthe end of the company’s fiscal year. The project teamshould identify changes in the components of internalcontrol through inquiry of the process and/or controlowners, observation of procedures, and in many cases,walk-through, inspection, or reperformance of the controls.When the project team determines there have not beensignificant changes in the controls from the originalevaluation date to the end of the company’s fiscal year, theteam might limit its procedures to inquiry and observationthat the controls continued to function without the need foradditional tests of controls. For example, the project teammay interview appropriate officers and employees and lookfor evidence of reassigned duties; changes in keypersonnel; the introduction of new equipment, procedures,or programs; and other changes that may affect theirconclusion about the continued effectiveness of specificcontrols. If changes are identified, the project team shouldconsider the effect of such changes on its evaluation andwhether there is a need for additional tests of controls. Forexample, if the project team becomes aware of a reductionin staffing causing understaffing, additional tests ofcontrols may be required to determine that the controlscontinued to function as expected.

Documentation ConsiderationsThe project team’s documentation of controls shouldprovide a basis for the evaluation of controls, includingboth the design of the internal controls and their operatingeffectiveness. As to design, the documentation shouldsubstantiate that management has established appropriatecontrols over initiating, recording, processing, and reportingtransactions that are effectively designed to prevent or detectmaterial misstatements or omissions. We also recommendthe documentation include a description of each significantcontrol, including how the control is performed, whoperforms the control, what data reports, files, or othermaterials are used in performing the control, and whatphysical evidence, if any, is produced as a result ofperforming the control.

As to operating effectiveness, the SEC final rule requiresthat management’s documentation should providereasonable support for the conclusion that the tests wereappropriately planned and performed and that the results ofthe tests were appropriately considered. The documentationshould include descriptions of the nature, timing, andextent of the procedures relied upon in evaluating theeffectiveness of the controls and should include a list ofexceptions (if any), their causes and implications onmanagement’s assertion.

Summarize and Evaluate Control DeficienciesHaving considered the various types of information describedabove, the project team should accumulate and present itsfindings so that executive management may perform anoverall evaluation of the effectiveness of internal control.

The project team might prepare a summary of controldeficiencies to communicate its findings to executivemanagement, the audit committee and the external auditors.The summary should include a description and analysis ofeach control deficiency or weakness in internal control andsummarize all relevant information that was considered.Appendix C provides an illustration of a summary of controldeficiencies that the project might use to accumulate andcommunicate control deficiencies.

19

In assessing the need for additional procedures during the periodfrom the project team’s initial procedures through the end of theyear, the project team might consider the following factors:

1 Size, nature, and subjectivity of the significant account(s)affected by the process.

1 The period of time that has elapsed since the initial procedureswere performed to evaluate the controls to the end of thecompany’s fiscal year. The longer the period, the more likely theproject team will want to perform some additional procedures.

1 Results obtained from inquiries and observations. To the extentthat significant changes have occurred, the project team’sprevious procedures may no longer be relevant. Therefore, theproject team may need to identify and evaluate other controlsor perform additional procedures to evaluate controls.


EVALUATE THE RESULTS OF PROCEDURES

AND IDENTIFY MATTERS FOR IMPROVEMENT

Identify Matters for ImprovementOver the course of the evaluation process it is likely theproject team will identify areas where controls may requiremodification or where the project team determines certainsystems require control enhancements to respond to newproducts or emerging risks. The project team also mightidentify areas where automating manual controls mayimprove both efficiency and compliance with management’spolicies or areas where the team’s evaluation of processesand controls point out redundant controls or other proceduresthat are no longer necessary. Management should encouragethe project team to communicate all suggested enhancementsfor consideration. The project team and managementshould consider the concept of reasonable assurance when evaluating whether suggested improvements shouldbe implemented.

Where the project team’s summary of control deficiencies,internal control weaknesses, and suggestions for improvingcontrols indicate that existing controls do not providereasonable assurance that errors of importance areprevented or detected and corrected in a timely manner,management will need to remediate the current controls oridentify and implement alternate controls that providereasonable assurance. There frequently is more than onecourse of action that will reduce a particular risk, andmanagement will need to weigh the various alternatives.

Public companies and the business

environments in which they operate

are dynamic. As such, the system of

internal control changes over time, the way

controls are applied evolves, and once-effective

procedures can become less effective or perhaps

are no longer performed. New personnel,

inconsistent training and supervision, and time

and resource constraints often contribute to

changes in controls. Also, circumstances for

which aspects of the system of internal control

originally were designed may change, causing

them to be less able to warn of the risks brought

by new conditions. Therefore, to maintain

effective internal control, management should

monitor the quality and performance of the

system of internal control over time to

determine whether it continues to be relevant

and able to address new risks.

Internal control should be structured to some degree to be self-monitoring and self-correcting. Monitoring can be performed in two ways: through ongoing monitoringactivities or separate evaluations.

Ongoing Monitoring ActivitiesOngoing monitoring entails building monitoring proceduresinto the normal recurring operating activities of the entity.Ongoing monitoring activities generally are more effectivethan separate evaluations since they: (1) are ingrained in theentity, (2) are performed on a real-time basis, and (3) reactdynamically to changing conditions. While highly ingrainedand effective ongoing monitoring activities reduce the needfor separate evaluations, most companies nonetheless conductsome separate evaluations of their system of internal controlon a recurring basis.

Ongoing monitoring activities include regular managementand supervisory activities and other routine actions such ascomparisons and reconciliations. Many of these activities are informal and not documented, which normally does notdetract from their effectiveness. However, the documentationlikely will need to be more substantive when these activitiesare used to support management’s assertion as to the overalleffectiveness of controls. Examples of ongoing monitoringactivities might include:

1 Management regularly reviews operating reports thatare integrated with, or reconciled to, informationderived from the financial reporting system, andsignificant inaccuracies or exceptions to anticipatedresults of operations are identified and pursued.

1 Appropriate attention to organizational structure,supervisory activities, and segregation of incompatibleduties provide oversight to control functions andidentification of deficiencies.

1 Data recorded by information systems are compared tophysical assets.

1 Appropriate attention is paid to communications fromexternal parties (e.g., customers, vendors, serviceproviders) that either corroborate internally generatedinformation or indicate problems.

21

Establish Processes to Evaluate theSystem of Internal Control Over Time


ES TA B L I S H PRO C E S S E S TO EVA L UAT E

T H E SY S T E M O F IN T E R NA L CO N T RO L

OV E R TI M E

1 Internal auditors or others performing similar reviewfunctions regularly monitor the entity’s activities.

1 Management provides venues (e.g., employee meetings,planning sessions, surveys, fraud “hotlines”) foremployees to provide feedback regarding problems andconcerns that may indicate control issues.

1 Personnel are asked periodically to state whether theyunderstand and comply with the entity’s code of conduct.

Appropriate mechanisms to continually monitor the systemof internal control and, when necessary, take correctiveaction in a timely manner help to ensure that the system ofinternal control is self-correcting.

Generally, one group should not be assigned exclusiveresponsibility for making the system of internal control self-monitoring and self-correcting. The system of internalcontrol is, in its broadest sense, comprehensive. It involvespeople throughout the organization, including many whomay not think of themselves as having any accounting orcontrol responsibilities. The people involved should includethose who establish, issue, and monitor accounting policiesand procedures: divisional controllers, internal auditors,the corporate controller, the chief financial officer, othermembers of senior management, audit committee members,and members of the board of directors. They all need to beconcerned, with varying degrees of detail, that the systemof internal control is kept “under control.” An importantrequirement of ensuring this is appropriate lines ofcommunication and adequate feedback, both when thesystem of internal control is under control and whenproblems arise.

Separate EvaluationsNot to be confused with the annual overall evaluation ofinternal control over financial reporting required by theAct, separate evaluations of internal control on a periodicbasis that focus on the operating effectiveness of controlsprovide a fresh look and allow management to consider thecontinued effectiveness of the ongoing monitoring activities.Separate evaluations may take the form of: (1) control self-assessments, where persons responsible for a reporting unitor function determine the effectiveness of controls, (2) specialreviews by internal audit or other appropriate groups, and(3) procedures performed by members of management.

Self-assessment programs will likely become more commonin light of the need for recurring annual assessments ofcontrols. Also, the real-time nature and efficiency of ongoingmonitoring achieved through control self-assessments willsupport management’s required quarterly certifications underSection 302 of the Sarbanes-Oxley Act. Management willneed to determine that these assessment programs focus on the controls that assure achievement of key financialreporting objectives. Similarly, in many cases managementwill need to supplement these programs with other proceduresto provide appropriate evidence to support its assessmentof the operating effectiveness of controls.

Separate evaluations of all or portions of the entity’s systemof internal control also might take the form of specialreviews by internal audit, other appropriate groups, ormembers of management. The procedures these groupsemploy to evaluate operating effectiveness should includeprocedures to understand each of the entity’s activities andthe design of the controls. These groups may employ awide variety of methodologies, techniques, and tools toconduct their separate evaluations including checklists,questionnaires, or benchmarking. Each of the methodologies,techniques, or tools can be effective, providing they aresufficiently sensitive to evaluate the operating effectivenessof the controls subject to evaluation.

The project team should not consider its task completeduntil either:

1 It is satisfied that appropriate ongoing monitoringactivities are in place, or

1 It has made reasonably specific recommendations forestablishing them.

Once all the key recommendations have been implemented,a baseline will have been established to enable managementto report on the effectiveness of controls. This does notmean that evaluations will cease. On the contrary, it meansthat evaluations will have become a part of the company’songoing, repetitive processes and thus an important part ofthe company’s internal control. After implementation of anongoing monitoring process, the company can expect thatany weaknesses—and some will inevitably arise—will becorrected within a reasonable period of time.

Our previous publication, Evaluating Internal Controls—Considerations for Documenting Controls at the Process,Transaction, or Application Level (Ernst & Young SCORERetrieval File No. EE0692) describes the methodology andsteps the project team should follow to document thedesign of controls identified over specific processes by:

1 Determining whether significant risks identified as“what can go wrong” questions are addressed by one ormore of the identified controls.

1 Determining whether the controls that address theidentified risks are adequately designed to prevent ordetect material misstatements, whether caused by errorsor fraud.

1 Walking through the significant controls to determinethey have, in fact, been placed in operation.

Walk-throughs are an important preliminary step todesigning procedures to evaluate the operatingeffectiveness of controls. It would be inefficient to performprocedures to test controls that are not adequately designedor have not been placed in operation. Therefore, werecommend the project team perform this important stepbefore investing significant time to design procedures toevaluate the operating effectiveness of controls.

A reviewer in a supervisory capacity (e.g., the divisioncontroller or the subsidiary’s treasurer) or a member of theproject team should make the determination of whethercontrols as designed are effective. In making thisassessment, the reviewer should consider:

1 Account characteristics of related accounts (such assize, susceptibility to error or manipulation).

1 The effectiveness of internal control at the entity level.

1 Conclusions related to the information technology (IT)processes.

1 The design of the control itself.

1 The sensitivity of the control.

1 Policies and procedures regarding authorization, thesafeguarding of assets, asset accountability, andsegregation of duties.

Determining whether the controls are designed to achieve agiven objective (e.g., that errors of importance are preventedor detected and corrected) often requires considerablejudgment. A design deficiency exists when either anecessary control is missing or an existing control is notproperly designed so that even when the control is operatingas designed the control objective is not always met. The keyquestion is whether the essential controls would be likely toprevent and/or detect a material misstatement relating toeach of the relevant financial statement assertions.

To evaluate the design effectiveness of an entity’s internalcontrol, the project team should obtain an understanding ofthe controls within each component of internal control. Theproject team generally obtains an understanding of the designof specific controls by making inquiries of appropriatemanagement, supervisory, and staff personnel; by inspectingentity documents; by observing the application of specificcontrols; by tracing transactions through the informationsystems relevant to financial reporting, or otherwise walkingthrough the processes and related controls.

Procedures to evaluate the effectiveness of the design of aspecific control are concerned with whether the control issuitably designed to prevent or detect and correct errors ormaterial misstatements with respect to specific financialstatement assertions. These procedures will vary depending

23

Appendix A—Review of Methodologyand Steps to Evaluate the DesignEffectiveness of Controls


APPENDIX A—REVIEW OF METHODOLOGY

AND STEPS TO EVALUATE THE DESIGN

EFFECTIVENESS OF CONTROLS

upon the nature of the specific control and relateddocumentation, and the complexity and sophistication of the company’s operations and systems. Controls mayrelate to any of the components of internal control.

Some controls may have a pervasive effect on theeffectiveness of other controls in assuring that assertionsrelevant to numerous significant accounts, classes oftransactions, or disclosures are achieved. For example, ITgeneral controls over program development, programchanges, computer operations, and access to programs anddata help assure that specific controls over the processingof transactions are operating effectively. In contrast, othercontrols are designed to assure that assertions relevant tospecific significant accounts, classes of transactions, ordisclosures are achieved. For example, managementgenerally establishes specific controls, such as accountingfor all shipping documents, to ensure that all valid sales are recorded.

The project team should focus on the significance of acombination of controls in achieving relevant assertionsrather than on specific controls in isolation. The absence or inadequacy of a specific control designed to achieve aspecific assertion may not be a deficiency, if other controlsalso address the assertion.

Performing tests to determine whether controls aredesigned to assure that one or more assertions are achievedmay be accomplished by the reviewer making inquiries ofthe individuals responsible for each control and examiningevidence that the control was performed and was effective(e.g., reviewing bank reconciliations), or by the reviewerretracing a transaction and/or reperforming controls (e.g.,recalculating extensions on a sample of invoices). In otherinstances, assurance that controls are functioning as intendedmay be gained by observing employees as they performtheir work, and through interviews to determine howemployees understand what is required in the event anerror is identified in the performance of their duties.

In instances where transactions are processed by the ITsystem, in addition to following the physical flow ofdocuments and forms, the reviewer also follows the flow of data and file information through the automated processin the application (at a system level, not a detailed logiclevel). This may involve procedures such as inquiry ofindependent and knowledgeable personnel (e.g., IT, processowner), review of user manuals, observation of a userprocessing transactions at a terminal in the case of an on-line application, and review of documentation such asoutput reports.

At the conclusion of this task, the reviewer shoulddocument whether the manual and programmed controlsare effectively designed, and include any other pertinentcomments that might assist the project team.

Tests of Programmed ControlsCompanies often rely on programmed controls to preventor detect errors in automated applications. Programmedcontrols are usually either programmed control procedures(e.g., edits, matching, reconciliation routines) or computerprocesses (e.g., calculations, on-line entries, automatedinterfaces between systems).

For example, in a simple application, controls over thepricing of shipments may include the user’s review of sales invoices to determine whether the correct prices anddiscounts are used and whether extensions were performedcorrectly. In a more complex application, however, a companymay rely on a computerized edit routine to identify pricesand discounts that do not meet established criteria.

When a company relies on programmed controls, theproject team will need to test the functioning of theprogrammed controls just as it would manual controls. For example, testing a programmed edit control mightinclude determining whether the transactions exceedingthe established edit limits are appropriately identified andreported, inquiring of the employee responsible for themanual follow-up procedures that support the programmedcontrol, and reviewing the actual follow-up proceduresperformed. Many times, particularly in more complexenvironments, an individual with experience in auditing IT will need to perform, or assist the project team inperforming, tests of programmed controls.

By recognizing that programmed controls operate in asystematic manner, a project team may be able to limit the extent of its testing of such controls. For example, acomputerized interest calculation may always use the sameformula (principal multiplied by an interest rate from a ratemaster file or table) or a computerized edit check maydisallow further processing of checks over $100,000.

The project team performs tests to obtain evidence that theprogrammed controls operated effectively as documentedby one or more of the following:

1 Testing IT general controls that ensure that any changesto the program performing the calculations areauthorized, tested, and approved before such changes areplaced in operation (i.e., program change controls) andthat access to data files is appropriately controlled (i.e.,access controls). Testing these controls is most effectiveand efficient when they support management’s relianceon several programmed controls.

1 Program change controls:

—Observing and/or reviewing a sample of significantprojects (e.g., those affecting the information thatsupport the processes that are important to management’sassertion on internal control) to determine:

•• Acceptance testing is routinely performed.

•• Environments for development (or modification)and testing of IT solutions are separated fromproduction systems.

•• Development personnel are prohibited frommigrating applications and data from the testenvironment to production.

—Reviewing or conducting a user satisfaction survey todetermine how well users’ information technologyneeds are being met. Alternatively, the project teamcan discuss with end users how well their informationneeds are being fulfilled by the informationtechnology organization.

—Attending or reviewing minutes of the InformationTechnology Steering Committee’s meetings (or theequivalent) where the priorities for informationtechnology projects are discussed.

25

Appendix B—Evaluate theEffectiveness of Programmed Controls


APPENDIX B—EVALUATE THE

EFFECTIVENESS OF PROGRAMMED

CONTROLS

—Attending project design meetings (or holdingdiscussions with users) to understand how users’needs are translated into information technologyproject specifications.

1 Access controls

—Reviewing the process for administering access toinformation systems and obtaining evidence todetermine that appropriate authorizations are obtainedfor users who are granted access to these systems.

—Determining whether the access control softwareoptions are in effect by reviewing appropriatesystems reports, and considering whether the options are appropriate.

—Reviewing the system log reports and accessviolation reports with the process owner, inquiringabout the scope of his/her review, and determiningwhether there is appropriate follow-up.

1 Testing the operation of the programmed controls. Forexample, the project team may choose to select a sampleof invoices and compare the prices to the approved pricesheet. Alternatively, the project team may be able to useaudit software to test the operations of those programmedcontrols. Another example might be transactiontransmission controls over sales orders from individualcustomers via the Internet or another network. Theproject team might determine the relevant controlfeatures through discussions with the network managerand web developers, observe the transmission controlsthrough a demonstration and review the related reports or logs, and evaluate the authentication (security) featuresand procedures.

1 Testing manual controls that support the programmedcontrol. For example, the computer program pricesinvoices using data on the price master file and producesa listing of sales orders, invoices, part numbers, and unitprices. The company has a manual control to review andapprove the list. Note, however, the project team shouldobtain evidence that the control operated effectively.

1 Performing benchmarking procedures (addressed inmore detail in the following section).

Benchmarking Programmed ControlsBenchmarking programmed controls is a testing strategythat may be used to extend the benefits of certain tests ofprogrammed controls into subsequent periods. It may alsoenable a project team to reduce or eliminate certain testingin subsequent years. Benchmarking programmed controlsis based on the premise that a computer will continue toperform a given procedure (e.g., aging of accountsreceivable, edit test) in exactly the same way until theprogram is changed. If a project team verifies that aparticular program executing a process or programmedcontrol has not changed, since the project team last testedit, they may choose to not repeat the specified test(s) insubsequent years.

A project team can establish its programmed controlsbenchmark, as of a point in time by performing a test ofprogrammed controls using traditional testing procedures(e.g., total and extend inventory listings either manually orusing another computer program). Then at an appropriatesubsequent time, rather than repeating the traditionaltesting procedures, the project team’s IT specialist alongwith other appropriate project team members candetermine that the program has not been modified sincethe project team last tested the programmed control.

Obtaining reasonable assurance that the program did notchange could involve discussions with individuals from theIT and end user departments and review of appropriatereliable reports (such as a listing detailing the compilationdates of all programs placed in production), and thenbriefly documenting any conclusions. An IT specialist maybe used to help obtain these assurances. In the case of amore complex system, obtaining reasonable assurance alsocould involve testing IT general controls (as discussed inprior section). The length of time a project team can relyon a benchmark will vary, and will depend on the level ofassurance they can obtain that significant program changeswould be identified. Programmed control benchmarkingprocedures do not extend to the accuracy of the underlyingdata input into the application. The project team will needto test the underlying data or the applicable controls to besatisfied that the data is accurate.

27

Benchmarking programmed controls is usually appropriatewhen:

1 A programmed control can be matched to a definedprogram within an application. For example, the projectteam may be able to benchmark programmed controlsfor the specific program that performs the invoiceextension calculation or aging computation. However,except in the case of simple, pre-packaged software, it isunlikely that a project team could benchmarkprogrammed controls for all processing procedures orprogrammed controls performed by the sales oraccounts receivable application.

1 The application is stable (i.e., few changes from year to year).

1 A report of the compilation dates of all programs placedin production is available and is reliable.

Benchmarking programmed controls can be especiallyeffective when a company uses purchased software wherethe possibility of program changes is remote (e.g., whenthe software does not allow access to the source code orthe client does not have the capability to make changes).When purchased software is integrated with otherapplications, the project team should also consider useraccess to the applications and the adequacy of segregationof duties when determining the opportunity to benchmark.

The project team may need to reestablish (i.e., reperformthe testing) the benchmarks for programmed controls atfuture intervals. Factors the team will need to consider indetermining when to reestablish a benchmark include: theeffectiveness of the information technology environment,including the controls over application and systemsoftware acquisition and maintenance; the project team’sunderstanding of changes, if any, to the specific programs,and the nature and timing of the related tests; and theconsequences of errors associated with the programmedcontrol that was benchmarked.


Company/Business Unit: _____________________ Report Date: _______________________________

Appendix C—Summary of ControlDeficiencies

Application/Process

Working Paper ReferenceDescription of Deficiency

Management’s Response

Is Deficiency a Significant Deficiency?

Summary of Control Deficiencies

Internal Controls_overall Effectiveness

Documents

Transcript of Internal Controls_overall Effectiveness