Generator Weatherization Workshop · prepared for extreme winter weather. ... • Boiler feed pump...

Generator Weatherization

Workshop

September 7, 2017

Alan H. Allgower

Operations Analyst, Senior

[email protected]

512-348-4613 (o)

mailto:[email protected]

PUBLIC

Remember……

2

PUBLIC

2011 Headlines….

3

PUBLIC

Extremely Cold Weather Grips Texas

A Federal Electric Reliability Commission (FERC)’s task force looking into

the causes of rolling blackouts during cold weather in February 2011 said

that “grid operators and power plant operators should have better

prepared for extreme winter weather.”

The task force said “the Electric Reliability Council of Texas (ERCOT)

region experienced a similar event in 1989 when it also shed firm load, but

the lessons learned from that event were not used to prevent outages

during the 2011 event.”

4

PUBLIC

What have we learned since February 2011?

• Identify critical components that will derate, trip or fail to start the

unit if frozen and incorporate into weatherization plan.

• Detailed testing of heat trace is necessary, identifying critical

circuits.

• Verify critical heat trace circuits are still functioning prior to every

extreme cold weather event.

• Heat trace failure rates in Texas on average are 10-15% from

previous season.

• Insulation inspections, focusing on critical components.

• Plant staff training prior to winter.

• Using wind breaks that can sustain long periods of wind and can

last the winter. Avoid off the shelf tarps.

• Continuous improvements to weatherization plan.

• Review scope of contractor work and verify acceptable completion.

5

PUBLIC

Tripped or de-rated capacity due to frozen elements

6

8000MW

3541MW

750MW

2836MW

1775MW

316MW

260MW

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

2/2/2011 1/6/2014 1/8/2015 12/19/2016 1/7/2017

MW

Generation MW tripped Generation MW de-rated

PUBLIC

Comparing Jan 6, 2014 to Jan 8, 2015

January 6, 2014

Dallas: 15° (9MPH wind)

Houston: 27°(16MPH wind)

San Antonio: 27° (15MPH wind)

Austin: 20° (13MPH wind)

Brownsville: 37° (17MPH wind)

Abilene: 11° (5MPH wind)

Midland: 14° (12MPH wind)

January 8, 2015








3541MW of capacity tripped

that was freeze related.

750 MW of capacity tripped

that was freeze related.

ERCOT declared EEA2 Normal Operations in ERCOT

Weather Source: Chris Coleman, ERCOT

7

PUBLIC

Comparing temperatures Feb 2, 2011 to Dec 19, 2016

December 19, 2016









8

February 2, 2011








ERCOT declared EEA3 – 4000MW of

firm load shed.

2836MW of capacity

tripped that was freeze

related. 316MW of

capacity derated.

8000MW of capacity

tripped, failed to start or

derated that was freeze

related.

Normal Operations in ERCOT

PUBLIC

Hours below freezing comparing Feb 2, 2011 to Dec 19, 2016

City 2-Feb-11 19-Dec-16

Dallas 24 12

Houston 12 8

San Antonio 24 10

Austin 24 9

Brownsville 1 0

Abilene 24 12

Midland 24 12

9

PUBLIC

December 18-19, 2016 lessons learned

• Eight generators experienced outages or

derates.

• ERCOT followed up with spot checks.

10

PUBLIC


• Plant 1 – trip; ten critical components froze– GT1 performance heater froze

– GT1 compressor bleed valve air regulators froze

– GT2 performance heater froze

– HRSG1 HP flow transmitter sensing line froze

– HRSG2 HP flow transmitter sensing line froze

– HRSG2 LP flow transmitter froze

– HRSG3 HP attemperator HP bypass froze

– Condensate hotwell level flow transmitter froze

– Instrument air compressor coalescing filter froze

– Instrument air compressor air dryer froze

• Cause – Incomplete list of critical components.

• Corrective action – Critical components list is being revised to include all

critical components and mitigations measures.

• ERCOT recommendation from follow-up spot check

Identify all critical components in weatherization plan. Include associated heat trace

circuit for each component, insulation inspection, wind break installed or transmitter

cabinet and heater inspection.

11

PUBLIC


• Plant 2 – trip; steamer• Steam seal pressure transmitter froze

▬Cause – lack of heated wind break enclosure for transmitter

▬Corrective action – enclose transmitter in heated enclosure


Identify all critical components in weatherization plan. Include associated heat

trace circuit for each component, insulation inspection, transmitter wind break

installed or cabinet and heater inspection.

• Plant 3 – Derate• HP superheated attemperator valve frozen to the valve seat

▬Cause – loose connection on associated heat trace cable

▬Corrective action – improve details of heat trace inspection





12

PUBLIC


• Plant 4 – trip; main turbine throttle pressure

control

• Two of three main turbine throttle pressure transmitters

valve bonnets froze▬Cause – valve bonnets of the root valves were not adequately

insulated

▬Corrective action – replaced insulation on the valves and sensing lines

and the insulation was extended to cover the entire valve and bonnet.

New self-limiting freeze protection wiring was also installed on the

sensing lines and valves.

• ERCOT recommendations from follow-up spot check Identify all critical components in weatherization plan. Include

associated heat trace circuit for each component, insulation inspection,

transmitter wind break installed or cabinet and heater inspection.

13

PUBLIC


• Plant 5 – trip; drum level transmitter• One of four drum level transmitters froze

▬Cause – main breaker tripped in heat trace sub-panel for a ground fault

on one of the circuits, resulting in heat trace not energized to one drum

level transmitter.

▬Corrective action – potential voltage indication lights were added to

sub-panel to assist operator in rounds.

▬ Replaced faulty circuit breaker that failed to trip.

• ERCOT recommendations from follow-up spot check

Identify all critical components in weatherization plan. Include



Verify critical heat trace circuits are still functioning prior to each


Develop simulator training for control room operators for the loss of

drum level transmitters.

14

PUBLIC


• Plant 6 – trip; boiler feed pump “A” and “B”

suction strainer differential pressure.• Boiler feed pump “A”

▬Cause – Damage to the power connection boot for the heat trace cable on the

root valves was cracked and brittle. This damage indicated the cable totally

failed in service.

▬Corrective action – Replaced heat trace cable with Ametek rated at 0DegF.

• Boiler feed pump “B”

▬Cause – Heat stress cracks and brittle heat trace cable around the root valve.

▬Corrective action – Replaced heat trace cable with Ametek rated at 0DegF.



trace circuit audit for each critical component, insulation inspection, transmitter

wind break installed or cabinet and heater inspection.

Verify critical heat trace circuits are still functioning prior to each extreme cold

weather event.

15

PUBLIC


• Plant 7 – temporary derate

• Moisture in instrument air▬Cause – frozen moisture in the instrument air line caused the pressure

transmitter to show a false low pressure reading. Mechanical room

louver vents were left open, vent fan in manual and mechanical room

space heaters were not functioning.

▬Corrective action – Weatherization plan was revised to include closing

louver vents, vent fan in auto and verification of space heaters

functioning in mechanical building prior to winter.

▬Corrective action – install a mechanical building thermometer for

telemetering temperature to the control room.

• ERCOT recommendations from follow-up spot check Identify all critical components in weatherization plan. Include



16

PUBLIC


• Plant 8 – trip

• Deareator level frozen transmitter

▬Cause – deareator heat trace circuit failure due to corrosion on

termination to fuse block.

▬Contributing factor cause – control room operator was not

aware that the deareator level was in single element control.

▬Corrective action – Update heat trace inspection to include

connections to fuse block, verify critical heat trace is functioning

before every extreme cold weather event.

▬Corrective action – Provide additional training to control room

operators.

17

PUBLIC


• Plant 8 – trip (continued) – ERCOT recommendations from follow-up spot check




Update plan to verify critical heat trace circuits are functioning prior to every


Revise cold weather plan to schedule additional staff during extreme cold

weather.

Develop cold weather training for operators focusing on checks during extreme

cold weather.

Revise operator freeze protection checklist to provide clarity for the operators.

Develop refresher training for control room operators focusing on actions due to

frozen transmitters and verifying transmitters are in average and not single

element control as a normal operating mode.

Develop a display for boiler critical transmitters as a “windshield look” for the

control room operators.

Replace all references in weatherization plan from “may perform” to “shall

perform.”

18

PUBLIC

Comparing Feb 2, 2011 to Jan 7, 2017

January 7, 2017









19

February 2, 2011








ERCOT declared EEA3 – 4000MW of

firm load shed.

8000MW of capacity

tripped, failed to start or

derated that was freeze

related.

Normal Operations in ERCOT

1775MW of capacity

tripped that was freeze

related. 260MW derate due

to ice damage to FD fan.

PUBLIC

Hours below freezing comparing Feb 2, 2011 to Jan 7, 2017

City 2-Feb-11 7-Jan-17

Dallas 24 12

Houston 12 11

San Antonio 24 10

Austin 24 11

Brownsville 1 5

Abilene 24 11

Midland 24 12

20

PUBLIC

ERCOT sets new winter peak demand and

January peak demand records

• AUSTIN, TX, Jan. 9, 2017

– ERCOT set another new winter demand record on Jan. 6,

2017. Electricity demand reached 59,650 MW during the 6-7

p.m. hour, exceeding the previous winter record of 57,924

MW set just a couple weeks earlier on Dec. 19, 2016.

• It is ERCOT’s expectation that for every extreme cold weather

event, some generation is going to experience freeze related

derates or trips.

21

PUBLIC

January 6-7, 2017

22

• ERCOT requested information related to the

cold weather event on Jan. 6-7, 2017 from 9

generation units.

• Six generation units were outaged due to

cold weather.

• ERCOT followed-up and performed a spot

check on three of the six units.

PUBLIC

January 6-7, 2017 lessons learned

• Plant 1 – trip; boiler feed pump trip

• Boiler feed pump “B” suction strainer differential

pressure transmitter sensing line froze which resulted in

a pump trip signal and subsequent unit trip.

▬Cause – heat trace failure

▬Corrective action – replace heat trace bundles and install wind

break for additional protection for wind


Identify all critical components in weatherization plan. Include

associated heat trace circuit for each component, insulation

inspection, transmitter wind break installed or cabinet and

heater inspection.

Verify prior to every extreme cold weather event that critical

heat trace circuits are functioning.

Install wind break as additional measure to prevent freezing.

23

PUBLIC


• Plant 2 – derate• Steam seal leak

▬Cause – lower steam coil developed a leak inside the force draft fan intake

resulting in ice forming on the bottom side of steam coil. This ice damaged the

FD fan blades.

▬Corrective action – Inspect steam coil for leaks on a weekly basis during winter

months. When temperatures are forecasted below 36 degrees, additional

operational rounds will be made during the shift.

• ERCOT did not perform a follow-up spot check

• Plant 3 – trip• HRSG HP steam flow transmitters froze

▬Cause – heat trace panel control modules 8 and 9 failed resulting in two heat

trace circuits not being energized

▬Corrective action – replace modules 8 and 9

▬Corrective action – follow-up with vendor to engineer an alarm for module failure


24

PUBLIC



• Deareator (DA) flow transmitter sensing line froze. Unit

has three DA transmitters using two sensing lines.▬Cause – The condensate (feedwater) to DA flow transmitter sensing

line froze creating a false high flow rate, closing the DA level control

valves which allowed the level in the DA to drop resulting in a Master

Fuel Trip (MFT) of the boiler.

▬Heat trace on one of the circuits was found to be working but with one

section that was not. The other heat trace circuit was working but out of

specifications for self regulating het trace.

▬Corrective action – During the spring 2017 unit outage, these

transmitters were relocated to the DA level and provided with new

weatherized transmitter enclosures, new insulation, new heat trace and

sensing lines shortened from 75’ to 6’.

▬Corrective action – OEM specified current and power information for all

critical freeze protections circuits will be added to the critical transmitter

freeze protection preventive maintenance work orders for comparison

to actual operating conditions.

25

PUBLIC


• Plant 4 – trip (continued)

• ERCOT recommendations from follow-up spot check Update plan as follows; prior to every extreme cold weather event,

verify critical components heat trace are functioning.

For winter 2017-2018, verify heat trace circuit alarms are correctly

programmed into system used by operators to monitor performance.


• Loss of instrument air▬Cause – The loss of instrument air was caused by the freezing of the

air dryer pre-filter, which restricted air flow to the instrument air system

resulting in the gas trip valve closure.

▬Corrective action – Install a wind break with heater for additional

protection to prevent freezing.


26

PUBLIC



• HRSG HP Feedwater high range flow transmitter, low

side sensing line froze.▬Cause – The contractor that performed the fall audit on heat trace did

not terminate the associated heat trace circuit in a junction box

correctly.

▬Corrective action – Verify contractor work and only hire a contractor

that has a quality control program in place.

• ERCOT recommendations from follow-up spot checkUpdate plan as follows; prior to every extreme cold weather event,

verify critical components heat trace are functioning.

Hire contractor with QC program for heat trace testing.

Add associated heat trace circuit to critical component list.

27

PUBLIC

Identifying critical components or equipment is required

• §25.53. Electric Service Emergency Operations Plans. (c)(2)(B) A plan that addresses any known critical failure points, including any effects of weather

design limits.

• Nodal Protocol 3.20 (3) No earlier than November 1 and no later than December 1 of each year, each Resource Entity shall

submit the declaration Section 22, Attachment K, Declaration of Completion of Generation Resource

Weatherization Preparations, to ERCOT stating that, at the time of submission, each Generation

Resource under the Resource Entity’s control has completed or will complete all weather preparations

required by the weatherization plan for equipment critical to the reliable operation of the Generation

Resource during the winter time period (December through February).

If the work on the equipment that is critical to the reliable operation of the Generation Resource is

not complete at the time of filing the declaration, the Resource Entity shall provide a list and schedule

of remaining work to be completed.

• Section 22, Attachment K, Declaration of Completion of Generation

Resource Weatherization Preparations “I hereby attest that all weatherization preparations for equipment critical to the reliable operation of

each of the above-listed Generation Resources during the time period stated above are complete or

will be completed, as required by the weatherization plan applicable to each Generation Resource.”

2

8

PUBLIC

Comments

• ERCOT will be focusing on critical components as

required by §25.53, Nodal Protocol 3.20 (3), generator

attestation and measures to mitigate freezing during spot

checks for winter 2017/2018.

Critical components list as part of weatherization plan.

Heat trace audit testing and identifying critical components.

Insulation inspections on critical components.

• Best practice

Prior to every extreme cold weather event, verify critical

components heat trace circuits are functioning.

It is ERCOT’s expectation that for every extreme cold

weather event, some generation is going to

experience freeze related derates or trips.

29

PUBLIC

Food for thought….

• “do or do not; there is no try.” Yoda – Star Wars

• … you say to yourself, “OK, why did it happen?

Why did we make those bad engineering

decisions we made in 1986 with Challenger?”

I'll tell you. It's the human element. I suggest

that there's a complacency there that comes

from success. — Alan Shepard

30

PUBLIC

NERC Reliability Guideline Approved

31

• Generating Unit Winter Weather Readiness Current Industry Practices – Version 2

• Reliability Guideline: Generating Unit Winter Weather Readiness – Current Industry Practices – Version 2

• Copies are available in the back of the room

https://urldefense.proofpoint.com/v2/url?u=http-3A__www.nerc.com_comm_OC_Reliability-2520Guideline-2520DL_Reliability-5FGuideline-5FGenerating-5FUnit-5FWinter-5FWeather-5FReadiness-5Fv2.pdf&d=DwMGaQ&c=trp9rTvIdyEWh1VWB5x8_2JiPaB5oGZOtWPDws2_VoY&r=ttK_USbb1Rc7vANQiz-6kXc8t3JMa6bLU_yX8rC3Al8&m=OvrFPw3usXolXYtDkzgRIGV_Tf2UvWIpZOYesSaiTps&s=XgDbqdENbxRcUecH0mNTCyubMgqrR_6nS08SXCjSmIg&e=

PUBLIC

Resource weatherization objective

“By failing to prepare, you are

preparing to fail.” -- Benjamin

Franklin

Questions?

Generator Weatherization Workshop · prepared for extreme winter weather. ... • Boiler feed pump...

Documents

Transcript of Generator Weatherization Workshop · prepared for extreme winter weather. ... • Boiler feed pump...