/' ' -

-'ŽEPORT NO. U.S. DEPARTMENT OF ENERGY RELEASE DATEOFFICE OF INSPECTOR GENERAL

ZR-B-95-02 NOVEMBER 10, 1994

AUDIT OF MANAGEMENT CONTROLS OVER SELECTED

ENERGY RESEARCH MAJOR SYSTEM ACQUISITIONS

~~~IG~~~-IF -1

U.S. DEPARTMENT OF ENERGYOFFICE OF INSPECTOR GENERAL

AUDIT OF MANAGEMENT CONTROLS OVER SELECTED

ENERGY RESEARCH MAJOR SYSTEM ACQUISITIONS

Report Number: CR-B-95-02 Capital Regional Audit OfficeDate of Issue: November 10, 1994 Germantown, Maryland 20585

AUDIT OF MANAGEMENT CONTROLS OVER SELECTEDENERGY RESEARCH MAJOR SYSTEM ACOUISITIONS

TABLE OF CONTENTS

Page

SUMMARY ........................ 1

PART I - APPROACH AND OVERVIEW ........................ 3

Introduction ................................. 3

Scope and Methodology ........................ 3

Background ............. ...................... 4

Observations and Conclusions ......... ...... 5

PART II - FINDINGS AND RECOMMENDATIONS ................. 7

1. Management of Total Project Costs ......... 7

2. Project Management Practices ............. 14

PART III- MANAGEMENT AND AUDITOR COMMENTS .............. 23

U.S. DEPARTMENT OF ENERGYOFFICE OF INSPECTOR GENERALOFFICE OF AUDIT SERVICES

AUDIT OF MANAGEMENT CONTROLS OVER SELECTEDENERGY RESEARCH MAJOR SYSTEM ACOUISITIONS

Audit Report Number: CR-B-95-02

SUMMARY

The Department of Energy (DOE) designates its mostsignificant projects as major system acquisitions based onnational urgency, importance, size, complexity, and dollarvalue. At the time of our review, the Office of Energy Researchwas responsible for managing eight of DOE's 52 major systemacquisitions. The objective of this audit was to determinewhether Energy Research was meeting the intent of Departmentalpolicies in the management of its major system acquisitions, andwhether current project management practices provided DOE withthe controls necessary to ensure that major system acquisitionswere being managed in an economical and efficient manner.

Our audit disclosed that "other project costs" for majorsystem acquisitions were not adequately included in the projectmanagement system and received less management attention thanconstruction costs under this system, The audit also disclosedthat certain project management practices did not ensure thatthe objectives of ensuring accountability, traceability, andvisibility of decisions at all levels were met. Theseconditions occurred, in part, because prior Departmentalguidance did not emphasize management of "other project costs,"and the Office of Energy Research had not fully implementedrevised guidance on managing these costs as part of totalproject costs, The report includes recommendations addressed tothe Office of Field Management and the Office of Energy Researchto improve the management of major system acquisitions.

We had excellent cooperation during the audit from bothField Management and Energy Research staff. We found thatDepartment staff working on these projects were committed totheir successful completion and dedicated to achieving projectgoals. The Office of Field Management concurred with therecommendations directed to their office and indicated thatcorrective actions are underway or planned. Throughout theaudit, however, Energy Research staff expressed their opinionthat the system and procedures utilized were adequate and did

1

not fully concur with all of the report findings andrecommendations. We recognize the difficulties inherent indealing with other project costs on major system acquisitionsand acknowledge Energy Research's efforts to manage thissignificant category of costs. However-, the audit resultssupport the need for enhancements to ensure that greateremphasis is placed on the management of other project costs.Part III of the report contains a detailed discussion ofmanagements' comments and auditor response.

Off Be of In2ector *neral

2

PART I

APPROACH AND OVERVIEW

INTRODUCTION

The Department of Energy (DOE) designates its mostsignificant projects as major system acquisitions based onnational urgency, importance, size, complexity, and dollarvalue. Major system acquisitions include projects with a totalcost or annual funding in excess of $100 million and projectsdesignated by management as warranting special attention.

The objective of our audit was to determine whether theOffice of Energy Research (Energy Research) was meeting theintent of DOE policies in managing its major systemacquisitions and whether current project management practicesprovided the Department with the controls necessary to ensurethat major system acquisitions were being managed in aneconomical and efficient manner.

SCOPE AND METHODOLOGY

At the end of Fiscal Year 1992, Energy Research wasresponsible for eight of the Department's 52 major systemacquisitions. We reviewed three of those eight projects: theAdvanced Photon Source under construction at Argonne NationalLaboratory; the Relativistic Heavy Ion Collider underconstruction at Brookhaven National Laboratory; and theSuperconducting Super Collider. On October 28, 1993, after ouraudit work was completed, the Congress determined that FiscalYear 1994 funds appropriated for the Superconducting SuperCollider would be used for the orderly termination of theproject. Superconducting Super Collider issues are included inthis report because they are cross-cutting in nature and havepotential applicability to future projects. Planned costs forthe three systems reviewed totaled $9.5 billion, with theSuperconducting Super Collider accounting for $8.2 billion.

We performed audit work at Department Headquarters and thelaboratories where the selected major systems were beingconstructed. The audit included reviews of applicable DOEorders and implementing guidance, conceptual design reports,project plans, project management plans, progress reports, fieldwork proposals, and field budget requests. The SuperconductingSuper Collider project was the subject of extensive reviewsduring the course of our audit. Rather than duplicate effort,we relied on the results of other reviews in some instances fordata in support of this audit. We also conducted interviews

3

with over 75 key DOE and contractor officials at Headquartersand field locations.

The audit was conducted in accordance with generallyaccepted Government auditing standards for performance audits.It included tests of internal controls and compliance with lawsand regulations to the extent necessary to satisfy the auditobjective. Since the review was limited, it would notnecessarily disclose all internal control deficiencies that mayexist. We did not rely extensively on computer-processed dataand, therefore, did not fully examine the reliability of thedata.

The firm of Irving Burton Associates, Inc. participatedwith the Office of Inspector General in conducting the audit.The audit began in January 1993 and covered activities betweenJanuary 1989 and May 1993, with primary emphasis on Fiscal Years1990-92. The Office of Energy Research and the Office of FieldManagement both waived the exit conference.

BACKGROUND

The Office of Energy Research carries out activitiesrelated to the Department's energy research and developmentmissions. Energy Research had a Fiscal Year 1992 budget ofapproximately $3 billion; over $775 million (about 26 percent)was for construction projects. The systems included in ouraudit accounted for approximately 85 percent of its Fiscal Year1992 construction project budget, with the Superconducting SuperCollider alone representing 62 percent of the budget.

Since its implementation in 1978, the project managementsystem has evolved as the principal control mechanism to fosterthe concepts of baseline management, accountability, andperformance assessment. Recent changes have emphasized controlof baselines and baseline changes including transition planning.The system is characterized by an extensive review process,including annual validation reviews by the Department,semiannual reviews by Energy Research, and monthly reviews bythe DOE project manager.

The Program/Project Management Division under the AssociateDeputy Secretary for Field Management initiates and updatesproject management system guidance, monitors systemimplementation, and reviews and concurs on reportingrequirements. The Office of Management within the Office ofEnergy Research advises the Director of Energy Research andparticipates in planning and implementation activities

4

associated with major system acquisitions, includingprogram/project organization involvement and businessstrategies.

OBSERVATIONS AND CONCLUSIONS-

The major system acquisitions reviewed during this auditrepresent three of the most significant and challenging projectsundertaken by the Department. We found that DOE and contractormanagement staff working on these projects were committed totheir successful completion and dedicated to achieving projectgoals. The project management system used by DOE management wasgenerally being complied with, and had been refined to make iteven more effective. Recent emphasis on management of totalproject costs rather than just those costs related specificallyto construction was a particularly noteworthy improvement.

While significant and worthwhile accomplishments wereevident in the Department's existing project management system,we concluded that additional improvements could be made tofurther enhance project management and oversight. The Office ofEnergy Research needed to put in place management controls toensure that the intent of project management guidance wasconsistently implemented. The Office of Field Management neededto strengthen project management practices and facilitateadherence to the intent of project management guidance.

We found that almost $439 million of planned other projectcosts were not adequately addressed by the project managementsystems and were given considerably less management attentionthan construction costs. Other project management practices didnot always provide visibility over key decisions nor maintainaccountability and traceability over management actions. Forexample, cost growth occurred without formal recognition by theDOE project manager, and contingency management practicesfurther reduced visibility over the cost status of projectelements. On major system acquisitions, the DOE project managerserves as the focal point between the Department and thecontractor. The project manager is expected to maintain controlover the project and is responsible for exercising appropriatemanagement decisions, as well as informing senior management ofexisting or potential problems.

The Secretary of Energy testified before the Congress thatsignificant steps would be taken to improve management andoversight of contractors in the future. We believe that therecommendations in this report will improve the Department'sfocus on management and oversight of contractor operations.

5

The audit observations concerning the exclusion of costsfrom project controls, as well as the need for visibility andaccountability over key decisions, in our opinion, representweaknesses that should be considered by management whenpreparing the yearend assurance memorandum on managementcontrols.

Part II of this report provides details on our findings andrecommendations. Part III includes detailed management andauditor comments.

6

PART II

FINDINGS AND RECOMMENDATIONS

1. Management of Total Project Costs

FINDING

The Department of Energy's (DOE) project management systemis the principle control system used to manage total projectcosts of major system acquisitions. We found that "otherproject costs" were not adequately included in the projectmanagement system and were given less management attention. Forexample, independent cost estimates for two projects did notassess the reasonableness and completeness of almost$439 million in planned other project costs. In addition,improvements were needed in the content of status reportssubmitted to DOE management and in the level of detail includedin budget documents. These conditions existed because priorDepartmental guidance did not emphasize management of totalproject costs. While current guidance has improved, additionalemphasis was needed. All needed controls were not put in placeby the Office of Energy Research to ensure consistentimplementation of guidance. As a result, there was no assurancethat other project costs planned for the major systemacquisitions reviewed would be adequate to fund all requirementsand would be used in the best interests of the Department.

RECOMMENDATIONS

We recommend that the Director, Program/Project ManagementDivision of the Office of the Associate Deputy Secretary forField Management:

1. Establish controls to ensure that independent costestimates for major system acquisitions include totalproject costs.

2. Revise DOE Order 4700.1 to include specificrequirements that:

a. establish interim milestone dates for otherproject cost activities; and

b. ensure that procedures for development andperiodic updating of project research anddevelopment plans, along with significantmeasurable milestones are set forth in theProject Management Plan.

7

We recommend that the Director, Office of Management,Office of Energy Research:

1. Require that quarterly project. status reportssubmitted by DOE project managers to Headquarterscontain milestone information on other projectcost elements to include research and developmentin support of construction.

2. Require that milestone data prepared for projectcosts financed by operating expense funds beincluded in field work proposals/field budgetrequests.

MANAGEMENT REACTION

The Associate Deputy Secretary for Field Managementconcurred with the intent of the finding and recommendations andadded that initiatives were in process or were planned to ensurethat independent cost estimates encompass total project costsand that other project cost activities be properly managed. TheDirector, Office of Management, Office of Energy Research,concurred in principle with Recommendation 2.a., but did notfully concur with Recommendation 2.b. stating that appropriateinformation on other project cost status and milestones wasincluded in project plans and quarterly status reports. Inaddition, field work proposals are prepared 2 years in advancefor budget requests that include the overall goals andobjectives. Part III of this report includes management andauditor comments.

DETAILS OF FINDING

BACKGROUND

Total project costs are all cost specific to a projectincurred prior to the operation of the facility. They consistof the construction costs of the project, referred to as "totalestimated costs," plus all remaining costs, labeled "otherproject costs." These remaining costs include: conceptualdesign reports; research and development required both prior tostart of construction and subsequently for fabrication, testingand rework of prototype equipment; and one-time costs related totesting, startup, operator training, and commissioning.

8

As shown in the following schedule, total project costsplanned for the three acquisitions we reviewed were estimated at$9.5 billion at the end of Fiscal Year 1992, with totalestimated costs and other project costs about $7.2 billion and$2.3 billion, respectively.

Total Other TotalEstimated Project Project

Project Costs Costs Costs

(in millions)

Advanced Photon Source $ 456 $ 336 $ 792Relativistic HeavyIon Collider 406 103 509SuperconductingSuper Collider 6.347 1.902 8.249

Total $7,210 $2,341 $9,550

PROJECT MANAGEMENT CRITERIA

DOE Order 4700.1, Project Management System, establishesmanagement principles for acquisition of significant projects.DOE Notice 4700.5, Project Control System Guidelines, issuedAugust 1992 clarifies policy for applying control systems to theoverall management of projects. A cornerstone of theDepartment's project management policy is the concept ofaccountability at appropriate levels. An essential element ofaccountability is overall project control of technical scope,cost and schedule baselines, as well as associated research anddevelopment and transition planning.

Funding for other project costs is provided by operatingexpense funds. These costs involve activities that are subjectto change and, therefore, flexibility is needed in managingthem. DOE Order 5700.7C, Work Authorization System, establishesa work authorization and control process for contractor workfinanced by the operating expense fund.

MANAGEMENT OF TOTAL PROJECT COSTS

Other project costs on two projects were not adequatelyincluded in the Department's prescribed project managementsystem, and were given less management attention thanconstruction costs. Independent cost estimates did not assessthe reasonableness and adequacy of almost $439 million inplanned other project costs. Project management control systems

9

needed to be expanded to. include reporting on other projectcosts and increase the level of detail surrounding the relatedactivities in project status reports and budgetary submissionsto DOE Headquarters.

Independent Cost Estimates

Independent cost estimates for two of the projects revieweddid not assess the reasonableness and completeness ofdocumentation for about $439 million in planned other projectcosts. An independent cost estimate serves as an analyticaltool to validate, cross-check, or analyze the cost estimatesthat were developed by proponents of a project. The estimatealso serves as a basis for verifying risk assessments.Independent cost estimates are developed by the Department'sProgram/Project Management Division of the Office of theAssociate Deputy Secretary for Field Management. Cost estimateswere performed as part of the Energy Research Semi-Annualreviews, but, by definition, these program office reviews arenot independent.

Other project costs of about $336 million on the AdvancedPhoton Source project and $103 million on the Relativistic HeavyIon Collider had not been included in the independent costestimates conducted on these projects. Other project costs onthe Advanced Photon Source increased almost 100 percent betweenproject conception and construction start. Managementattributed this increase to a change in the definition ofproject costs which added new cost categories. Other projectcosts on the Collider increased 59 percent after projectconception, due in part to a delay in the project's start and alengthening of the project schedule. The magnitude of the otherproject costs (approximately 42 and 20 percent of total projectcosts, respectively) requires management to ensure thatestimates are valid.

Project Management Control Systems

The overall project management control systems used tomonitor progress on two projects did not adequately includeother project costs. Attempts were made during the early phasesof the projects to monitor costs related to research anddevelopment by using research and development plans. However,the plans were incomplete. While the project management controlsystem on the Superconducting Super Collider project includedother project costs, system implementation difficultiesdecreased its usefulness.

10

Advanced Photon Source. About $75 million of the estimated$336 million in other project costs for this project had beenfunded at the end of Fiscal Year 1992. Since the project'searly phases, concern had been expressed about how these costswere being managed. For example, in 1989 a DOE review teamofficial noted the contractor's lack of accounting over otherproject costs. This review, discussed further on page 18, foundthat the contractor's report did not include cost or schedulestatus of the research and development portion of the project.

At the DOE project manager's request, a revised researchand development plan was published in July 1992. During ourfield visits, we found no evidence that the plan was being usedby either Department or contractor personnel, even though thepotential existed to evaluate progress against milestones.

Relativistic Heavy Ion Collider. Through Fiscal Year 1992,$34.9 million of the estimated $103 million in other projectcosts on this project had been funded. Research and developmentplans were prepared in 1987 and 1988, but a draft update issuedin November 1990 was not finalized. Although the plansaddressed the major tasks to be undertaken before and duringconstruction, only completion dates for major research anddevelopment milestones were shown; no start or interim milestonedates were identified.

Project cost performance reports did not include statusinformation for research and development activities, even thoughthe DOE project manager had, as early as February 1991,requested this information. The DOE manager stated that at thisstage there were no plans to incorporate research anddevelopment activities into cost performance reports, notingthat these activities were subject to baseline control andnarrative status at the subsystem level was included in monthlyreports. The manager stated that emphasis would be placed onincorporating startup and commissioning activities into thesystem. For example, the Department requested that narrativestatus on planning activities for startup and commissioning beincluded in the contractor's monthly progress reports beginningin September 1992 to assure that planning activities wouldproceed well in advance of the work scheduled to begin 2 yearsin the future. We noted, however, that contractor reportsprepared for the last 3 months of calendar year 1992 did notimplement this request.

11

Project Status Reports

Project status reports needed to be expanded to facilitatemore effective management of other project costs. Projectmanagers of major system acquisitions are required to submitquarterly reports on project baselines and present a clearpicture of project status and resource utilization. The reportsare intended to highlight the status and activities of majoracquisitions and keep DOE senior management apprised of existingor potential problems to allow timely followup action.

We found that progress reports did not contain sufficientinformation on other project costs to keep management adequatelyinformed. For example, the report on the Relativistic Heavy IonCollider project for the first quarter of Fiscal Year 1993 didnot include any research and development milestones even thoughabout $35 million had been expended on these activities and thefirst two milestones had slipped. Contractor reportingrequirements established by one project manager did not includeany milestones related to other project cost activities. Thus,neither DOE representatives on site or those at the Headquarterslevel were adequately informed of the status of other projectcost activities through the prescribed reporting system.

Budgetary Submissions for Other Project Costs

The level of detail included in budgetary submissionssupporting other project costs needed expansion if field workproposals/field work requests were to be used effectively bymanagers to monitor project status. The interrelationship ofother project cost activities to overall construction activitiesdictates, in our opinion, close monitorship of such activitiesand the establishment of prerequisite measurement tools to doso.

In reviewing field work proposals, we found that thebudgetary submissions for the three projects we reviewed did notinclude detailed milestone data. Although DOE orders providedfor the completion of a proposed schedule with start andcompletion dates (if requested by the program manager when fundshad been authorized), such requests were not made; therefore,the utility of these budgetary submissions as a tool forprogress measurement after funds were authorized was diminished.

12

REASONS FOR CURRENT MANAGEMENT PRACTICES

The limited inclusion of other project costs in the projectcontrol systems for the Advanced Photon Source and RelativisticHeavy Ion Collider projects was caused, in part, by Departmentalguidance that did not emphasize management of total projectcosts to include other project costs. The Department revisedits guidelines such as DOE Notice 4700.5 to ensure that totalproject costs are accounted for within the project managementsystem. This guidance also requires development of schedulesthat represent all work scope regardless of funding source toensure that all known requirements affecting a project areidentified and considered in developing project baselines.

In our opinion, however, project management practices hadnot been effectively updated for the two projects at existinglaboratories, even though unfunded other project costs atSeptember 30, 1992, totaled $329 million. The managementphilosophy applied by the Office of Energy Research to otherproject costs on its major system acquisitions was essentiallyto separate these costs from construction costs and applydifferent criteria. In our opinion, guidance needs to promotechange in management philosophy, and the Office of EnergyResearch needs to establish additional controls to ensureconsistent implementation of guidance.

The Office of Energy Research also needs to ensure thatprojects are not so fully funded that project management becomesless challenging than it should be. While we recognize that notwo projects warrant identical allocations of other projectcosts, we noted that on two projects other project costs werebetween 25-30 percent of construction costs, and on a thirdproject other project costs represented almost 75 percent ofconstruction costs.

IMPACT OF CURRENT MANAGEMENT PRACTICES

The project management practices we noted raise concern asto whether estimates of other project costs yet to be funded forthese projects are adequate and would be used properly.Comprehensive independent cost estimates should improveidentification of all relevant costs at the start of a project.In our opinion, independent cost estimates that exclude otherproject costs from their scope increase the likelihood of futurecost increases.

13

2. Project Management Practices

FINDING

The objectives of the Department's project managementsystem include ensuring accountability, traceability, andvisibility of management decisions at all levels as well asreporting accurate, valid, and traceable performance and trenddata to management. Certain project management practices wereviewed did not ensure that these objectives were consistentlyachieved. Cost growth occurred without formal recognition bythe Department; some other changes that impacted baselines werenot formally processed; and contingency practices limitedvisibility over the cost status of individual project elements.Contractors also did not implement progress reporting thatprovided management with accurate, comprehensive information.These conditions occurred because the policy establishingthresholds for cost changes did not ensure that incrementalgrowth would be appropriately accounted for, replan actions wereconsidered to be approved by the Department outside of formalchange control procedures, and the Department did notconsistently take action to ensure that contractor progressreporting was timely and meaningful. As a result, theDepartment did not always assume adequate responsibility forproject actions, and senior management did not have assurancethat it consistently received the information needed to provideearly indicators of potential problems; validate correctiveactions; and make informed decisions.

RECOMMENDATIONS

We recommend that the Director, Program/Project ManagementDivision of the Office of the Associate Deputy Secretary forField Management:

1. Revise the provisions of DOE Order 4700.1 regardingcost thresholds to require that accumulated cost growthexceeding any fixed threshold value established when aproject is started be recognized in a formal changeaction and approved by DOE's project manager.

2. Identify cost-effective incentives that can beincorporated into the project management control systemto encourage timely and accurate reporting of projectstatus.

14

We recommend that the Director, Office of Management,Office of Energy Research:

1. Ensure that project management plans for future majorsystem acquisitions require that any use of contingencyfunds be approved by DOE's project manager (i.e. is aLevel 2 change).

2. Ensure that actions with significant project-wideimpact, such as replans, are processed through thechange control system.

MANAGEMENT REACTION

The Associate Deputy Secretary for Field Managementconcurred with the overall intent of the finding andrecommendations and stated that initiatives were in process orplanned to make implementation and policing of establishedcontrols more visible. The Director, Office of Management,Office of Energy Research, stated that they concurred inprinciple with Recommendation 2.a. and Recommendation 2.b. andstated that projects already control the disbursement of fundsto the contractor. In addition, project management was incompliance with the change control system. Part III of thisreport includes management and auditor comments.

DETAILS OF FINDING

PROJECT MANAGEMENT CRITERIA

The Department's project management criteria discussed inFinding 1 provides the means to ensure that fundamentally soundproject control systems are in place and provide valid, usefuldata to management in time to aid in decision making processes.The establishment and maintenance of project baselines are themost important aspects of project control, and changes tobaselines must be controlled to avoid distortions in performancereporting. Two objectives of the Department's formal changecontrol process are to ensure that decisions are made at theappropriate management level and to enhance accountability andtraceability in decision making. Baseline changes approved bythe DOE project manager are called "Level 2" changes. Toincrease visibility over changes, quarterly progress reportsshould include all DOE-approved changes.

The objective of project control system reporting iscommunication of timely, accurate information to the appropriatemanagement level enabling analysis, evaluation, and correctiveaction of project performance against approved baselines.

15

Reporting by the project manager to higher management shouldpresent a clear picture of project status and resource use.Using cost performance reports, managers can compare workplanned against work performed and determine cost and schedulevariances. Managers can also use cost and schedule trends toforecast an "estimate at completion" and assess thereasonableness of contractor plans to recover any cost overrunsor schedule slippage. The discipline required to produce validreports also helps ensure that key elements of the contractor'soverall management system (such as accounting and budgeting) arefunctioning properly.

PROJECT MANAGEMENT PRACTICES

The.DOE project manager is responsible for tracking,reporting, and managing baselines and related changes. It isthe project manager's responsibility to clearly lay out and haveapproved procedures used for management of funds and baselinechanges and to clearly support project actions in formal reportsto establish traceability and an audit trail. Projectmanagement practices, however, did not consistently providevisibility over key decisions nor maintain accountability andtraceability over management actions.

Change Control Practices

Incremental Cost Growth

Several Advanced Photon Source project elements hadexperienced significant growth since project inception, but costgrowth trends were not evident from the information reported tosenior management. Level 2 cost changes were not reportedbecause the cost growth was incremental, rather than largeindividual increases. Because baselines were revised after eachchange, however, the reports also showed only minor costvariances. Project management and overhead costs are examplesof this treatment.

The project management budget-at-completion at the end ofFiscal Year 1992 was $9 million over the original baselineestimate, and actual costs incurred exceeded the originalestimate for project management by approximately $314,000.Excluding the impact of directed changes (discussed below), thebudget-at-completion for overhead costs exceeded the originalbaseline estimate of $11.3 million by $10.9 million. However,although no Level 2 cost changes had been approved by theDepartment, cost status was calculated against revised baselinesand reported variances were minor in both cases.

16

Department management provided supporting documentationshowing that some of the cpst growth resulted from DOE-directedchanges imposed on the project regarding accounting practices.But, Field Management officials stated that DOE policy requiresdirected changes be approved as part of the project's baselinechange control approval process. Further, our analysis of thesupporting documentation showed that even without the effect ofthe directed changes, four project areas experienced cumulativecost growth ranging from approximately $5 to $11 milliondollars.

Replans and Other Actions

During the first several years of the Advanced PhotonSource project, semi-annual replans occurred but were notproperly processed through formal change control procedures.Replans assessed all or significant portions of the project,adjustments were made to cost baselines and schedule milestones,and existing variances were frequently eliminated. Contingencyfunds were used as needed to make up differences between therevised estimate and the previous estimate. While replans canbe an excellent informal mechanism for assessing project status,replan actions that result in baseline changes should be subjectto formal change control procedures so that baseline integrityand accurate progress reporting are maintained.

Some later replan actions were processed in the changecontrol system, but cost increases and decreases were groupedtogether. As a result of using the net effect of such actions,changes that should have received DOE project manager approvaldid not. For example, one change action increased theconventional facilities baseline by a net amount of $3.9million, under the $5 million threshold for Departmentalapproval, but included planned reductions of $11.8 million.Because (as confirmed by Field Management) cost thresholds applyto cost decreases as well as increases, this action should havebeen approved by the DOE project manager.

In another case, a replan action was processed for oneproject area that exceeded the established threshold of$5 million by $700,000, but was not formally approved by theDepartment. Management stated that the net effect of severalchanges processed at that time was under the cost threshold.Further, the contractor determined that these changes werewithin their authority and making the "noncontroversial" changeswould simplify discussions with DOE Headquarters. Thecontractor also stated that the change was made to an internalbaseline (below the DOE-approved baseline), and approval was notrequired unless the change exceeded the official baseline. Animportant function of the Department's project control system,

17

however, is to provide early warnings to management of projectareas where prompt action could prevent future cost growth orschedule slippage. We noted that the budget-at-completion forthis project element at January 1993 exceeded the originalbaseline estimate by approximately $8 million.

Contingency Practices

Cost contingency is the amount withheld by the Departmentfrom the contractor's budget authorization that represents thecost uncertainties associated with the project cost estimate.Contingency is a part of the total project costs and must bedefined in the Project Management Plan. Contingency fundmanagement is a key Government function that should not bedelegated to the contractor; DOE orders specify that the (Level2) cost threshold include "all proposed changes for which use ofproject contingency is required." Although differentcontingency practices may be acceptable, they must be integratedwith the change control and reporting process. On the twoCollider projects, DOE project managers appropriately consideredcontingency use to be a Level 2 change, and believed this policyprovided the Department with a strong management control tool.

On the Advanced Photon Source project, however, the DOEproject manager provided contingency funds to the contractorthrough construction directives, as provided for in the approvedProject Management Plan. But the directives did not provideadequate controls over contingency fund applications. TheOffice of Field Management has stated that contingency practicesmust be integrated with the change control and reportingprocesses. While the overall percentage of contingencyavailable to the project was monitored by the Department, theallocation of funds to specific project areas or tasks was notidentified as a Level 2 action. During Fiscal Years 1991 and1992 the contractor processed over 100 change requests involvinga net use of over $22 million in contingency funds applied toproject baselines.

The intent of establishing Departmental control overcontingency funds, in our opinion, is to improve contractordiscipline in achieving project goals within performancemeasurement baselines (excluding contingency), rather thanassuming that contingency funds "belong" to the contractor andwill be available for use. Contingency funds not required tomeet original project goals can be utilized for other purposessuch as accelerating project schedules.

18

Progress Reporting

Although progress reporting was a requirement on the threeprojects reviewed, contractors did not take timely action toensure that they had systems in place producing valid,comprehensive data that could be relied upon by DOE management.

Four years into the Superconducting Super Collider project,a Departmental review team evaluated the project and businessmanagement systems being used and found that these systems wereonly marginally useful because they could not adequately trackproject progress and had only limited ability to provideconsistent and accurate financial data. Another review analyzedproject reporting and identified numerous data quality issues.

A Departmental assessment of the Advanced Photon Sourceproject's initial cost performance reports expressed concernthat the reports did not provide the data needed by managementto make appropriate decisions, and noted that the reports wouldnot likely improve unless the contractor received "strongfeedback early on" in the process. This early review wasrequested by the DOE project manager to determine whether thecontractor's system was making satisfactory progress. ADepartmental review about 2 years later found that thecontractor's monthly reporting still needed to become moreconsistent, more accurate, and "generally contain a moreeffective portrayal of project status." Although correctiveaction was reported as completed in a subsequent review, wefound Fiscal Year 1992 reports that showed negativebudgeted-cost-of-work-scheduled (periodically used to eliminateschedule variances) and data that did not track from month tomonth.

A September 1992 Departmental review of the RelativisticHeavy Ion Collider project's management control system (part ofan appraisal process that began in February 1991) found that theinformation reported by the contractor was insufficient toeffectively document progress and performance. The contractorhad not implemented reporting instructions received from the DOEproject office over 18 months earlier. A follow-up appraisalconducted in March 1993 concluded that corrective actions hadbeen completed.

Senior management was unable to get meaningfulestimate-at-completion data on the Advanced Photon Source in thecost performance reports, and discontinued use of this report.The DOE project manager stated that trend analysis wasaccomplished by other means, including presentations atsemi-annual Headquarters program reviews and contingencyanalyses. External reviews of the Superconducting Super

19

Collider project disclosed that the estimate-at-completionportion of the project management control system was not fullyfunctional and could not generate reliable forecasts. Forexample, the May 1993 computer-calculated estimate-at-completionshowed an overrun of $124 million, while an independentDepartmental assessment of the project reported anestimate-at-completion with an overrun of about $1.6 billion.Generating credible forecasts through project completion is aprimary function of a project control system.

REASONS FOR MANAGEMENT PRACTICES

These conditions occurred because Departmental policy didnot ensure that incremental growth exceeding establishedthresholds would be formally approved by the responsible DOEofficial; replans were considered to be approved by theDepartment outside of formal change control; and the Departmentdid not consistently take action to ensure that contractorprogress reporting was timely and meaningful.

Change Thresholds

Thresholds for determining what changes must be approved bythe Department are established by senior DOE officials when aproject is formally initiated. Current policy regarding costthresholds applies to individual change actions. For example,if the dollar threshold for cost changes requiring Departmentalapproval is $5 million, a single change request affecting adesignated work breakdown structure level would need to exceed$5 million before approval would be required. This policyallows incremental cost growth exceeding $5 million to occur andnot be formally approved by the Department. If variancesreported to management are calculated against increasedbaselines, but significant incremental growth is notperiodically captured as a Level 2 change, senior management isnot ensured accurate visibility over cost growth trends.

The Relativistic Heavy Ion Collider project managerestablished a threshold level of zero (i.e., any cost change tothe baseline) for certain project areas warranting closeattention. Such a policy prevents incremental growth in thesekey areas accumulating without Departmental approval.

Periodic Replans

Office of Energy Research policy allowed replans on aperiodic basis, rather than when necessitated by changedcircumstances. Replans were generally associated with reviewsof the projects conducted by the office's ConstructionManagement Support Division. We found that contractors

20

interpreted approval of project status by the review team asapproval of revised project baselines; but, replan actionsshould be processed as formal change actions to ensure baselineintegrity and accurate progress reporting. We note that replansand other program office reviews may be excellent informalmechanisms for assessing project performance and progress, andannual replans may be of value in adjusting to reduced fundinglevels. However, semi-annual replans tend to obscure long-termtrends in cost growth or schedule slippage that might benefitfrom the development and implementation of corrective actionplans.

Project Reporting

The Department recognized the problems with progressreporting. Departmental guidelines note that on past projects,managers had difficulty assessing and analyzing project statusbecause reports were improper or contained inaccurate data.Contractor management did not always place sufficient emphasison implementing and maintaining satisfactory cost and schedulecontrol systems. Strong encouragement is needed to ensuretimely implementation of project management systems, sincecontractor performance is then evaluated based on the datagenerated. Departmental policy requires compliance reviews becompleted on new major system acquisitions and major projectsshortly after an approved baseline is in place to ensure thatthe data are consistent and valid. Having DOE project managersperiodically assess the data being generated after systemapproval would provide additional assurance as to the validityof such data and the accuracy of overall project status beingreported over the life of the project, and thus on managements'ability to rely on analyses based on the data to assist indecision making.

DEPARTMENTAL PROJECT MANAGEMENT

Departmental management did not always assume adequateresponsibility for contractor actions as anticipated by itsproject management system. Senior DOE management also did nothave assurance that it consistently received the informationneeded to provide early indicators of potential problems,validate corrective actions, and make informed decisions toensure that resources were utilized in the most efficient andeffective manner. Program office practices should enhances butnot replace the Department's established system.

Although project management guidance expects accuratereporting of project status, a Government report on Departmentof Defense projects and prior Office of Inspector General workindicate that incentives are generally lacking for project

21

officials to disclose potential problems and concerns to seniorofficials. The Department of Defense report concluded thatappropriate incentives need to be created to ensure that seniorleaders received realistic perspectives on the cost, schedule,and technical status of projects. Certain project managementpractices cited in our audit report also did not indicate thatDOE management consistently received complete and accurateinformation on project status. Energy Research major systemproject costs have generally increased over original estimates.Thus, Departmental accountability and visibility over costgrowth trends are important to ensure that cost growth isjustified and that limited resources are used effectively.

22

PART I

MANAGEMENT AND AUDITOR COMMENTS

The Associate Deputy Secretary for Field Managementconcurred with the overall intent of the report and recognizedthe need for greater emphasis on management of total projectcosts and strengthened project management practices. TheDirector, Office of Management, Office of Energy Research, didnot fully concur with the findings and recommendations andstated that additional controls were not justified in terms ofneed or possible benefit that could be gained. Managements'comments to each of the recommendations as well as managements'overall comments are included below.

Management of Total Project Costs

Recommendation 1

The Director, Program/Project Management Division of theOffice of the Associate Deputy Secretary for Field Management:

1. Establish controls to ensure that independent costestimates for major system acquisitions include totalproject costs.

2. Revise DOE Order 4700.1 to include specificrequirements that:

a. establish interim milestone dates for otherproject cost activities; and

b. ensure that procedures for development andperiodic updating of project research anddevelopment plans, along with significantmeasurable milestones are set forth in theProject Management Plan.

Management Comments. The Associate Deputy Secretary forField Management agreed with the intent of Recommendation l.a.Several guidance memoranda have-been distributed to program andoperations offices to define, standardize, and emphasize totalestimated costs, other project costs, and total project costs.The Program/Project Management Division has addressed totalproject costs in all recent independent cost estimates andcurrent independent cost estimate procedures require totalproject costs to be addressed. DOE Orders 5700.2D and 4700.1

23

will be updated to include wording to ensure that independentcost estimates for major system acquisitions encompass totalproject costs.

The Associate Deputy Secretary for Field Management alsoagreed with Recommendation l.b. stating that the revision to DOEOrder 4700.1 would include specific requirements for managingother project cost activities. Interim milestone dates will bemandated for all discrete efforts that are measurable. However,interpretation of some results, research and development forexample, must be left to program experts. While the baselineestimate must ensure that funding is defined, justified, and asaccurate as can be determined at the initial stage of theproject, the project manager must be held ultimately responsiblefor management of this effort. The project manager'smethodology for managing these efforts must be approved as partof the Project Management Plan, and funding approvals must bevisible, must be based on measurable results, and must haveperiodic plan updating and revisions as appropriate.

Auditor Comments. Management actions are responsive toRecommendation 1.

Recommendation 2

The Director, Office of Management, Office of EnergyResearch:

1. Require that quarterly project status reports submittedby DOE project managers to Headquarters containmilestone information on other project cost elements toinclude research and development in support ofconstruction.

2. Require that milestone data prepared for project costsfinanced by operating expense funds be included infield work proposals/field budget requests.

Management Comments. The Director, Office of Managementconcurred in principle with Recommendation 2.a. However, EnergyResearch believed that appropriate information on other projectcost status was included in project plans and quarterly statusreports as required. The Office of Management did not fullyconcur with Recommendation 2.b. stating that field workproposals are prepared 2 years in advance for budget requests,and these proposals outline overall goals and objectives. Also,more detailed information on milestone data is included inresearch and development plans and other project documentationfor other project costs, as appropriate.

24

Auditor Comments. Project managers of major systemacquisitions are required by DOE Order 4700.1 to submitquarterly reports to provide the status of projects relative toestablished baselines and present a clear picture of projectstatus and resource utilization. The audit showed thatquarterly progress reports prepared by project managers for thesystems reviewed did not provide adequate visibility for otherproject cost activities.

Actions being taken by the Office of Field Management torequire the establishment of interim milestone dates for alldiscrete efforts that are measurable should improve futurereporting. Our recommendations to the Office of Management aredesigned to promote more consistency in implementing thisrequirement. We recognize that field work proposals presentlyfocus on overall goals and objectives. Including milestone dataon other project costs in these proposals would expand thatfocus and provide a tool for progress measurement after fundsare authorized. While research and development plans shouldinclude more detailed information on milestone data, these plansare maintained locally. On the other hand, field workproposals/field budget requests are provided to Departmentalprogram managers. The interrelationship of other project costactivities to overall construction activities dictates, in ouropinion, close monitorship of such activities and theestablishment of prerequisite management tools to do so.

Prolect Management Practices

Recommendation 1

The Director, Program/Project Management Division of theOffice of the Associate Deputy Secretary for Field Management:

1. Revise the provisions of DOE Order 4700.1 regardingcost thresholds to require that accumulated cost growthexceeding any fixed threshold value established when aproject is started be recognized in a formal changeaction and approved by DOE's project manager.

2. Identify cost-effective incentives that can beincorporated into the project management control systemto encourage timely and accurate reporting of projectstatus.

Management Comments. The Associate Deputy Secretary forField Management concurred with the intent of Recommendationl.a. and stated that guidance to clarify similar policy issuesin the form of a draft Departmental notice was developed inAugust 1993, but issuance was postponed due to a Departmental

25

moratorium on new directives. Guidance from this draft noticewill be incorporated into an upcoming revision to DOE Order4700.1. The position of the Office of Field Management is thata project manager is ultimately responsible for documenting andreporting all changes to the project wi-thin the present monthlyand quarterly reporting requirements. This reporting mustensure traceability so that outside auditors and reviewers caneasily track the actions taken.

The Associate Deputy Secretary for Field Managementconcurred with Recommendation l.b. and stated that the intent ofDOE Notice 4700.5 is to empower DOE field project managers withthe authority to determine the appropriate application ofproject controls, including status reporting, for the managementof their projects. A certification program to train fieldproject managers on the use of project controls has beenestablished. The Office of Field Management has recommended tothe Office of Human Resources that specific language be includedin field project manager performance appraisals to ensureaccountability for promoting accurate reporting of projectstatus at that level of management. The Office of FieldManagement also commented that since the contract award feecriteria may be the single mechanism available and recognizedDOE-wide to incentivize contractors, contract award fee criteriain cost plus award fee contracts will be used as a means toprovide cost-effective incentives to ensure timely and accuratereporting of project status.

Field Management also commented that in February of 1994the DOE's Contract Reform Team released a report recommendingactions that will significantly improve the Department'scontracting practices, and that report was strongly endorsed bythe Secretary of Energy. As a result, Field Managementanticipates that the improvements will result in a moreconsistent and efficient level of contract administration inseveral areas, including a cost incentive for contractor costreduction programs.

Auditor Comments. Management comments are responsive toRecommendation 1.

Recommendation 2

The Director, Office of Management, Office of EnergyResearch:

1. Ensure that project management plans for future majorsystem acquisitions require that any use of contingencyfunds be approved by DOE's project manager (i.e. is aLevel 2 change).

26

2. Ensure that actions with significant project-wideimpact, such as replans, are processed through thechange control system.

Management Comments. The Director, Office of Management,stated that they concurred in principle with Recommendation 2.a.stating that the projects already control the disbursement offunds to the contractor by construction directives and changecontrol, per plans and thresholds approved by senior DOEmanagement. Some flexibility in the use of contingency fundsshould be allowed for efficiency and it is more effective toallow some decisions to be made at the lowest management levels.Any use of contingency funds has to be reported by thecontractors; thus, Energy Research project management becomesaware of any lower level use of contingency. The Director alsostated that they concurred in principle with Recommendation 2.bstating that project management was in compliance. In the earlystages of projects, there may be significant cost optimizationthat result in trade-offs between various work breakdownstructure elements. Certain changes to funding profilesaffecting all elements may be combined and processed as singleactions. These replans were reviewed as part of the Semi-AnnualReview Process, so adequate documentation exists describing thechanges.

Auditor Comments. While flexibility in the use ofcontingency funds may be desirable under certain circumstances,this flexibility must be integrated with the change control andreporting process. We found that this was not always the case.The use of contingency funds was not integrated with the changecontrol process whereby use would be approved and reported as aLevel 2 (Departmentally-approved) change. Project managers"becoming aware" of contractor application of contingency fundsafter the fact does not satisfy the intent of establishingDepartmental control over contingency fund use.

The issues identified in the audit show that significantchanges with project-wide impact were not subject to formalchange control. Establishing thresholds for change control incompliance with Departmental policy is an important front-endcontrol for project management, but does not ensure that theintent of change control will be met as the project matures.Actions with significant project-wide impact should be processedthrough change control as early as possible to allow the intentof the process to occur, which is to ensure visibility overthese actions, explore alternative actions and determine theappropriate level of management involvement necessary before adecision is made. Replans and other actions representing costincreases should not be offset by processing them with othercost decreases to avoid the prescribed threshold. These actions

27

should be processed as separate actions in order thatDepartmental officials may be alerted to potential problems andable to address them up front. Significant changes, whencombined, may not be visible to senior management. Further,combining the actions does not promote traceability of thedecision making process throughout the life of a project. TheSemi-Annual Review should supplement the formal change controlprocess, not replace it.

Overall Management Comments. The Associate DeputySecretary for Field Management agreed with the overall intent ofthe report for greater emphasis on management of total projectcosts and strengthened project management practices. The Officeof Field Management stated that management controls were inplace but implementation and policing of current internalmanagement controls need to be made more visible to ensureadherence. To this extent, the Office of Field Management istaking a number of new initiatives to be more responsive tocustomers in project management including the reorganization anddevelopment of statements of missions and functions, delegatingspecific authority to Departmental Operations Offices, andemphasizing adherence to established processes and managementpractices. An effort to revise DOE Order 4700.1 will beinitiated after January 1, 1994. Moreover, the Office of FieldManagement intends to issue policy that sets the framework forestablishing specific requirements with the project managerresponsible for implementing acceptable procedures.

The Director, Office of Management, Office of EnergyResearch, did not fully concur with all of the findings andrecommendations stating that they are not justified in terms ofneed or possible benefits that could be gained. Energy Researchacknowledged that prior to May 1990, the Department emphasizedtotal estimated costs for construction and, as a result, otherproject costs received less emphasis. It was furtheracknowledged that for some projects, not all of the otherproject costs were included on construction project data sheets,which at times resulted in understated amounts for total projectcosts. However, the Department increased its control of totalproject costs as part of construction project data sheets in May1990. The Office of Management concluded that present policies,orders, and guidelines for managing operating funds associatedwith construction projects were adequate and that the Office ofEnergy Research was in compliance with the Departmental projectmanagement orders referenced in the audit report. In addition,Energy Research did not agree that other project costs should bemanaged in accordance with the provisions of DOE Order 4700.1,and instead emphasized compliance with DOE Order 5700.7C. Field

28

Management agreed with the Inspector General report in thatother project costs are included within the scope of DOE Order4700.1.

The Director, Office of Management, Office of EnergyResearch, considered the management controls in place for theoperating funded portion of projects to be appropriate,recognizing that many tasks need flexibility in controlling themin order to meet mission requirements. The Director furtherstated that other project costs are incurred for complex,one-of-a-kind research tasks where precise estimates aredifficult to pinpoint because of the "state of the art" natureof the projects. Some costs (for example, conceptual design)are expended before construction start is approved and othersare essentially manpower efforts. The Office of Managementconcluded, therefore, that it is not meaningful to use the samemeasurement tools for other project costs that are used forconstruction costs and that the audit report was confusing inthat it heavily considers the project management system processand does not emphasize results.

Overall Auditor Comments. The comments of the Office ofField Management are considered responsive to the auditfindings. Of particular note is the acknowledgment that greateremphasis is needed in the management of total project costs, toinclude other project costs, and the commitment to developingadditional policy setting the framework for specificrequirements applicable to project managers. We view therecognition by the Office of Field Management that change isneeded as a very positive step since that office is theproponent for DOE Order 4700.1, Project Management System.

The comments of the Office of Energy Research tended toemphasize the office's compliance with current policies andprocedures. While we acknowledge that measurement tools forother project costs must recognize the nature of such costs, theinformation set forth in Finding 1 shows that existing practicesand procedures do not adequately ensure that funding for otherproject costs is adequate and would be used properly in the bestinterest of the Department. Additional emphasis on process, aswell as the intent of requirements to manage total projectcosts, is reflective of the specific responsibilities that areassigned to DOE project managers of major system acquisitions tomanage total project costs. The audit report does not mandatethat other project costs be managed in exactly the same manneras construction costs or that Energy Research follow exactly thesame requirements for both total estimated costs and otherproject costs. Rather, the audit results support the need forenhancements to the project control system to ensure thatgreater emphasis is placed on the management of other project

29

costs; the goal of such enhancements is to ensure thatDepartmental project management responsibilities are achievedfor total project costs.

30

Attachment

IG Report No. CR-B-95-02

CUSTOMER RESPONSE FORM

The Office of Inspector General has a continuing interest inimproving the usefulness of its products. We wish to make ourreports as responsive as possible to our customers'requirements, and therefore, ask that you consider sharing yourthoughts with us. On the back of this form, you may suggestimprovements to enhance the effectiveness of future reports.Please include answers to the following questions if they areapplicable to you:

1. What additional background information about the selection,scheduling, scope, or procedures of the audit would havebeen helpful to the reader in understanding this report?

2. What additional information related to findings andrecommendations could have been included in this report toassist management in implementing corrective actions?

3. What format, stylistic, or organizational changes mighthave made this report's overall message more clear to thereader?

4. What additional actions could the Office of InspectorGeneral have taken on the issues discussed in this reportwhich would have been helpful?

Please include your name and telephone number so that we maycontact you should we have any questions about your comments.

Name _ Date_

Telephone Organization

When you have completed this form, you may telefax it to theOffice of Inspector General at (202) 586-0948, or you may mailit to:

Office of Inspector General (IG-1)Department of EnergyWashington, D.C. 20585

ATTN: Customer Relations

If you wish to discuss this report or your comments with a staffmember of the Office of Inspector General, please contact RobJacques at (202) 586-3223.