OFFICE OF
SCIENCE
1
Closeout Report on the
DOE/SC CD-4 Review of the
Utilities Upgrade Project (UUP)
Fermi National Accelerator LaboratoryAugust 22-23, 2018
Raymond Won
Committee Chair
Office of Science, U.S. Department of Energy
http://www.science.doe.gov/opa/
http://www.science.doe.gov/opa/
OFFICE OF
SCIENCEReview Committee
Participants
2
Raymond Won, DOE/SC, Chairperson
Observers
Marc Jones, DOE/SC
David Michlewicz, DOE/SC
Steve Neus, DOE/FSO
Review Committee
Subcommittee 1—Technical
*Pat Hogan, ANL
Dan Nassar, ANL
Subcommittee 2—ES&H
*Michelle Solaroli, TJNAF
Subcommittee 3—Cost and Schedule
*Kelly Krug, TJNAF
Carrie Sauter, BNL
Subcommittee 4—Project Management
*Michelle Solaroli, TJNAF
*Lead
OFFICE OF
SCIENCE
3
Charge Questions
1. Technical: Are all Key Performance Parameters that pertain to the project sufficiently
achieved and documented? Are the modernized utilities ready for operations?
2. Cost, Schedule, Risk and Contingency: Has the project been completed within the
cost and schedule baseline? Was the use of contingency appropriately executed?
3. Management: Is the project properly organized, staffed and managed for successful
completion? Is the Transition to Operations plan updated and approved?
4. Environment, Safety and Health: Are ES&H aspects of the project adequately
addressed? Are prerequisite documents such as the final Hazard Analysis and the
draft Project Closeout Report approved or prepared?
5. Prior Reviews: Has the project responded appropriately to recommendations from
prior reviews?
6. Lessons Learned: Has the project adequately documented lessons learned and best
management practices?
OFFICE OF
SCIENCE
4
2. Technical DRAFT
P. Hogan, ANL, D. Nassar ANL
1. Technical: Are all Key Performance Parameters that pertain to the
project sufficiently achieved and documented? Yes.
Are the modernized utilities ready for operations? Yes.
OFFICE OF
SCIENCE
5
5. Prior Reviews: Has the project responded appropriately to
recommendations from prior reviews? Yes.
6. Lessons Learned: Has the project adequately documented lessons
learned and best management practices? Yes.
2. Technical
P. Hogan, ANL, D. Nassar ANL
OFFICE OF
SCIENCE
6
Findings
• Significant effort was placed on customer interfaces with the impacted utilities.
• Management used regular project meetings including daily interfaces with critical
facilities to coordinate construction activities.
• FESS Facilities Management personnel participated in the Commissioning and
Start-up of systems.
• The project executed seven Beneficial Occupancies on the High Voltage (HV)
portion.
• The project executed 18 sectionalized Beneficial Occupancies on the Industrial
Cooling Water (ICW) portion.
• Commissioning and Transition to Operations is complete for both the HV and ICW
portions.
• All training is complete.
• Operations stakeholders signed-off on Final Acceptance
• The project achieved all Level 1 Milestones.
2. Technical
P. Hogan, ANL, D. Nassar ANL
OFFICE OF
SCIENCE
7
Findings (cont)
• The project implemented a successful work-around in response to a delay caused by
the Master Substation (MSS) vendor: The vendor was late in completing its factory-
based work, including factory assembly and testing. Late delivery of the MSS
would have impacted the work of on-site subcontractors whose work was dependent
on arrival of the MSS. Fermi decided to allow MSS shipment and on-site testing by
the vendor in order to mitigate delays to the on-site subcontractors.
• There were no Technical scope Recommendations in the prior review (July 2016).
• The project has documented a number of lessons learned and best practices.
• The threshold scope was completed, and objective scope was completed as indicated
in the table below:
2. Technical
P. Hogan, ANL, D. Nassar ANL
Description of Scope KPP Threshold Achieved Objective KPP Achieved
High Voltage Electrical (H/V) Upgrade
Replace Master Substation Control Building
Replaced: Building, Switchgear/Equipment,
Protection System, Cable Vault, Yard Wiring and
Portions of Feeder System
Constructed: Blast Wall, Generator and Duct
Bank
Master substation 345kv circuit
breaker replacement
Industrial Cooling Water (ICW) Upgrade
Install new ICW backbone piping network from
Casey's Pond to the Main Injector ICW system
Installed: four miles of new and replacement
ICW pipe and two new pump stations.
Casey’s Pumphouse Electrical Room Addition
Makeup Water Improvements
OFFICE OF
SCIENCE
8
Comments
• The project has achieved all threshold elements of the key performance parameters.
The project has also achieved completion of three objective elements. A list of
elements is included in the Findings Section.
• All of the constructed facilities have been accepted and transitioned to operations as
documented in the Transition to Operations Plan.
• The lessons learned from the project are well documented.
• The project team successfully overcame a significant challenge regarding the vendor-
delayed MSS.
• The project team maintained critical systems operation during construction to ensure
Lab requirements continued to be met.
• The project team included subject matter experts for each discipline that significantly
impacted the success of the project.
• The electrical commissioning reports are well thought out and the testing plans
followed ANSI/NETA test practices
2. Technical P. Hogan, ANL, D. Nassar ANL
OFFICE OF
SCIENCE
• Recommendations
• The project is ready to proceed to CD-4.
9
2. Technical
P. Hogan, ANL, D. Nassar ANL
OFFICE OF
SCIENCE
10
3. Environment, Safety and HealthM. Solaroli, TJNAF / Subcommittee 2
4. Environment, Safety and Health: Are ES&H aspects of the project adequately
addressed? Yes Are prerequisite documents, such as the final Hazard Analysis and the
draft Project Closeout Report, approved or prepared? Yes
5. Prior Reviews: Has the project responded appropriately to recommendations from
prior reviews? Yes
6. Lessons Learned: Has the project adequately documented lessons learned and best
management practices? Yes
OFFICE OF
SCIENCE
11
3. Environment, Safety and HealthM. Solaroli, TJNAF / Subcommittee 2
Findings
• UUP was acknowledged in Lab’s Site Sustainability plan with achieving success
in meeting the Lab’s sustainability goals and improving operational resilience in
FY 2017.
• Project utilized Predictive Solutions, an occupational safety software, for analytics
on workplace injury prevention, which has been integrated into Lab’s ES&H
program.
• Subcontractors have worked at the Lab in the past; therefore, they were aware of
the safety culture and expectations of the Lab.
• Best value selection included ES&H performance as a critical parameter to ensure
subcontractors can meet FNL ISM requirements.
• Subcontractor total hours worked = 64,986 hrs
• Project had a TRC case involving wrist injury and a DART case involving knee
injury. Both the Project incident and DART rates were 3.1.
• Project had three (3) Category 4 ORPS . Two were related to exposure to
hazardous electrical energy and one involved excavation beyond permit limits.
OFFICE OF
SCIENCE
12
3. Environment, Safety and HealthM. Solaroli, TJNAF / Subcommittee 2
Findings (continued)
• Project had (1) ORPS involving lack of proper signage, incorrect PPE and lack of
volt meter verification. The CSO issued safety stand-down until assurance of
subcontractor qualifications, hazard analysis and work controls were in place.
• Project had (1) ORPS during HV Upgrade project involving excavation into
Meson Beam line berm, which was outside approved excavation permit limits.
Lab addressed incident by shutting down Meson beam and having Radiation
group identify boundaries and posting for restricted digging area.
• Project had (1) ORPS involving a LOTO violation. Lab addressed incident with
stand down until an investigation was completed. A second occurrence with same
sub-tier subcontractor resulted in removal from the project.
• Hazard Analysis Report was reviewed from CD-3b and concluded that no updates
were required.
• Security Vulnerability Assessment Report has been updated to include
Cybersecurity Analysis of Master Substation Building. Assessment is pending
DOE approval.
• UUP Quality Assurance Plan has been reviewed and updated.
• Project closeout report is in draft form and contains information on ESH elements,
such as incidents and ORPS.
OFFICE OF
SCIENCE
13
3. Environment, Safety and HealthM. Solaroli, TJNAF / Subcommittee 2
Comments
• ES&H is well integrated into the project team and this directly contributed to the
success of safety on the project.
• Coordination between subcontractors and the Lab successfully mitigated
operational risks.
• The Lab’s proactive management of two ORPS involving electrical violations
including LOTO, improper signage and PPE usage demonstrated the strength of
their safety culture by issuing a stand down until an investigation was completed.
A second occurrence with same sub-tier subcontractor resulted in removal from
the project.
• Project team adequately addressed all recommendations from previous reviews.
• ESH Lessons Learned and Best Management Practices were documented in the
Project Closeout Report.
• The inclusion of the site specific construction safety plan as part of the
subcontract documents for necessary flow down of safety requirements to sub-tier
subcontractors was a notable lessons learned.
OFFICE OF
SCIENCE
14
3. Environment, Safety and HealthM. Solaroli, TJNAF / Subcommittee 2
Recommendations
• The project is ready to proceed to CD-4
OFFICE OF
SCIENCE
15
4. Cost and ScheduleCarrie Sauter, BNL, K. Krug, TJNAF
• Cost, Schedule, Risk and Contingency: Has the project been completed within the
cost and schedule baseline? Yes. Was the use of contingency appropriately
executed? Yes.
• Prior Reviews: Has the project responded appropriately to recommendations from
prior reviews? Yes.
• Lessons Learned: Has the project adequately documented lessons learned and best
management practices? Yes.
OFFICE OF
SCIENCE
▪ Findings
▪ The project TPC is $36.0M with a TEC of $34.9M and OPC of $1.1M
▪ EVMS statistics as of July 31, 2018
▪ BCWS: $35,180k
▪ BCWP: $35,413k
▪ ACWP: $35,484k
▪ BAC: $35,478k
▪ EAC: $35,549k
▪ CPI/SPI: 1.00/1.01
▪ 99.8% complete
▪ Forty-seven (47) baseline changes have been processed/approved through July
2018
▪ $4,750k drawn from TEC contingency since CD-2
▪ TEC Cost variance of $71k
▪ Remaining contingency of $451k
▪ TEC of $96k, OPC of $355k
▪ 4 months remaining schedule contingency to Jan-2019 CD-416
4. Cost and ScheduleCarrie Sauter, BNL, K. Krug, TJNAF
OFFICE OF
SCIENCE
▪ Findings (continued)
▪ All project risks in Risk Registry have been retired. $96k remains in
uncommitted contingency.
▪ The Draft Project Closeout Report identifies 5 Level 1 Milestones and 19 Level
2 Milestones all of which have been completed with the exception of Level 2
Milestones: CD-4 Readiness and CD-4 ESAAB and Level 1 Milestone: CD-4:
Approve Project Completion.
▪ A resource loaded schedule has been used throughout the project which
represents the project scope.
▪ $2.0M of Objective Scope was completed drawing from project contingency,
which equates to 42% of the contingency usage.
▪ All control accounts are 100% complete and closed to charges with the
exception of Project Management which has $65k of work remaining and an
open obligation of $.9k as of July 31, 2018.
▪ When a control account work scope is complete, the CAM notifies Project
Controls, Procurement and Finance. They then follow their procedures to ensure
contracts are closed out, all costs are incurred and the project is closed to new
charges, and activities are complete and closed in the EVMS system.
17
4. Cost and ScheduleCarrie Sauter, BNL, K. Krug, TJNAF
OFFICE OF
SCIENCE
▪ Comments
– The project team has successfully executed the project completing on schedule
and within cost.
– Use of $4,750k in contingency was appropriately executed by the project:
• Threshold Scope used $2.8M, 58%
• Objective Scope used $2.0M, 42%
– 3 of 12 Objective KPPs were accomplished:
18
4. Cost and ScheduleCarrie Sauter, BNL, K. Krug, TJNAF
Objective Value Spend Down Items Cost
Replace the Master Substation 345kV circuit breaker $338,000
Electrical Room Addition for Casey’s Pond Pump House $1,445,000
ICW Makeup Water Improvements $237,000
Total $2,020,000
OFFICE OF
SCIENCE
▪ Comments (continued)
– The project has responded appropriately to previous review recommendations.
– The project has developed a fairly comprehensive list of Lessons Learned and
Best Management Practices (Successes) recognizing areas for improvement
on future projects.
– Project reporting supports the achievement of CD-4 within the approved
baseline cost, however control documentation did not appear to align and
explanations were needed to understand differences. As a consequence,
control data should be aligned prior to issuing the final project closeout report.
• The contingency usage since CD-2 was untraceable in the Change
Control Log.
– The $30.727M PMB at CD-2 is not reflected on the log
– Consider improvement to the Change Control Log documentation to
ensure transparency by adding the running total of the contingency
remaining.
19
4. Cost and ScheduleCarrie Sauter, BNL, K. Krug, TJNAF
OFFICE OF
SCIENCE
▪ Comments (continued)
• The objective scope added to the baseline could not be verified via the
BCRs.
– BCR documentation did not clearly delineate the objective scope
additions when implemented into the baseline. Consider including, in
the BCR form, separate detailed explanations of each change taking
place.
• Discrepancies exist between the milestone dates on PEP and Project
Baseline Schedule
• The actual dates on the Closeout Report should tie with the Project
Working Schedule
20
4. Cost and ScheduleCarrie Sauter, BNL, K. Krug, TJNAF
OFFICE OF
SCIENCE
▪ Recommendations
– Proceed to CD-4
21
4. Cost and ScheduleCarrie Sauter, BNL, K. Krug, TJNAF
OFFICE OF
SCIENCE
22
4. Cost and ScheduleCarrie Sauter, BNL, K. Krug, TJNAF
PROJECT STATUS as of July 2018
Project Type Line Item
CD-1 Planned: Actual: Nov 2010
CD-2/3a Planned: May 2015 Actual: Feb 2015
CD-3b Planned: Dec 2015 Actual: Sep 2015
CD-4 Planned: Jan 2019 Actual:
TPC Percent Complete Planned: 99.2% Actual: 99.8%
TPC Cost to Date $35.5M
TPC Committed to Date $35.6M
TPC $36.0M
TEC $34.9M
Contingency Cost (w/Mgmt Reserve) $ 0.1M 149% to go
Contingency Schedule on CD-4 4 months 67%
CPI Cumulative 1.00
SPI Cumulative 1.01
OFFICE OF
SCIENCE
23
5. Project ManagementM. Solaroli, TJNAF/ Subcommittee 4
3. Management: Is the project properly organized, staffed and managed for successful
completion? Yes Is the Transition to Operations plan updated and approved? Yes
5. Prior Reviews: Has the project responded appropriately to recommendations from
prior reviews? Yes
6. Lessons Learned: Has the project adequately documented lessons learned and best
management practices? Yes
OFFICE OF
SCIENCE
Findings
• All Level 1 milestones were met.
• HV Electrical Upgrade project was completed August 2017 and the project
executed $316K in objective scope.
• ICW Upgrade project was completed July 2018 and the project executed $1,682K
in objective scope.
• Only the WBS Project Management control account remains open.
• Project SPI & CPI stayed within 10% threshold.
• All risks have been retired. $96K remains in contingency.
• Final Commissioning Report for HV 345kV Master Substation had three open
items that were closed at CD-4 review.
• Both HV Electrical Upgrade and ICW Upgrade are complete, commissioned and
operational.
24
5. Project ManagementM. Solaroli, TJNAF/ Subcommittee 4
OFFICE OF
SCIENCE
Findings (continued)
• Multiple Beneficial Occupancies were issued as part of HV and ICW
construction.
• Final Acceptance certificates were issued for the purpose of contractually
establishing subcontracted work has been completed in accordance with the
subcontract documents and constitutes the start of the warranty period.
• The transition to Operations plan is complete and defined FNL basis for
obtaining operability for the systems. Documentation, training and Lab
interfaces have been coordinated and obtained.
• Lessons learned including successes and potential improvements have been
captured.
25
5. Project ManagementM. Solaroli, TJNAF/ Subcommittee 4
OFFICE OF
SCIENCE
Comments
• Threshold KPPs and 3 Objective KPPs were successfully completed within the
project baseline cost and schedule.
• The project team effectively managed cost of additional threshold scope
associated with cost overruns, field conditions, design errors, operational issues
and constraints in the amount of $2,804K which was 13.8% of initial construction
cost ($20,340K).
• The project successfully spent approximately 49% of contingency on objective
scope and various scope enhancements. The cost of executed objective scope
($2.14M) was 10.6% of initial construction cost ($20,340K). Well done!
• Project team successfully managed changes to project staff throughout the project
and is adequately staffed to complete the project.
• The Transition to Operations Plan clearly defined the basis for attaining project
closeout and included requirements for documentation, training and interfaces.
• Subcontract management of ICW with unit pricing is a Best Management
Practice and was applied successfully on this project when field changes
occurred.
26
5. Project ManagementM. Solaroli, TJNAF / Subcommittee 4
OFFICE OF
SCIENCE
Comments
• The MSS Bypass project proactively used Lab funds to install an alternate power
source as a bypass for maintaining Lab operations during UUP HV project. This
interfacing project funded by the Lab was necessary for moving forward on the
HV project is considered a best management practice.
• Project team successfully responded to the late delivery of the substation on the
HV project by implementing additional management tools including engaging
subcontractor management, holding daily meetings, modifying payment
milestones and allowing final testing of some systems onsite. This modification
to the contract’s delivery schedule utilized contingency funds effectively to
mitigate overall project schedule delays.
• The Design Phase Utility Locates was a notable lessons learned using design
funding to locate existing utilities via potholing and hydro excavating. This effort
positively obtained utility location/elevation information and mitigated conflict
risks during construction.
• Project team adequately addressed all recommendations from previous reviews.
• Project team adequately documented lessons learned and best management
practices.
27
5. Project ManagementM. Solaroli, TJNAF / Subcommittee 4
OFFICE OF
SCIENCE
Recommendations
• The project is ready to proceed to CD-4
28
5. Project ManagementM. Solaroli, TJNAF/ Subcommittee 4
Top Related