Template for Technical Procedure - Gwen...

Maintenance Troubleshooting ActivitiesEffective Date: _____________

Prepared by: Michael C. GanzReliability Programs Specialist

Stakeholder review by: Brad Barksdale _______________________________________________Director, Fossil Engineering and Support Maintenance ServicesDate: _____________

Approved by: Dale Higginbotham _______________________________________________Sr. Director, Fossil Engineering and SupportDate: _____________

LUM-FFA-WPC-0003Maintenance Troubleshooting ActivitiesInitial Draft of 31

Table of Contents1. Purpose............................................................................................................................................................. 52. Applicability.................................................................................................................................................... 53. References....................................................................................................................................................... 5

3.1 Development References..............................................................................................................................53.2 Performance References...............................................................................................................................5

4. Definitions and Acronyms.......................................................................................................................... 65. Responsibilities............................................................................................................................................. 8

5.1 System or Component Engineer.................................................................................................................85.2 Equipment Specialist.......................................................................................................................................85.3 Operations Management...............................................................................................................................95.4 Director, FES Maintenance Services.........................................................................................................95.5 Maintenance Planner/Scheduler...............................................................................................................95.6 Maintenance Technician................................................................................................................................95.7 Configuration Control Manager...............................................................................................................105.8 Plant Engineer.................................................................................................................................................10

6. Instructions.................................................................................................................................................. 106.1 General Requirements.................................................................................................................................116.2 Troubleshooting - Initial Actions............................................................................................................136.3 Complex Troubleshooting..........................................................................................................................146.4 Advanced Troubleshooting.......................................................................................................................156.5 Basic Troubleshooting............................................................................................................................17166.6 Follow-up Actions to Troubleshooting Process........................................................................................................................................176.7 Extended Troubleshooting...................................................................................................................1817

7. Records..................................................................................................................................................... 19188. Attachments............................................................................................................................................ 1918

Attachment A: Troubleshooting Process Flowchart 2019Attachment B: Troubleshooting Plan Contents..................................................................................................2120Attachment C: Troubleshooting Classification Criteria..................................................................................2221Attachment D: Preservation of Evidence and As-Found Conditions for Mechanical Components

2322Attachment E: Preservation of Evidence and As-Found Conditions for Electrical Components. 2423Attachment F: Minor Corrective Actions That May Be Performed During Troubleshooting

Activities .................................................................................................................................................2624


ReviewsInternal Review

Name Position DateDarron Jones Reliability Specialist

Perry Cunningham Reliability Specialist

Duerk Reimer Reliability Manager

Sarah Keough Dev Team

Gwen Holladay Technical Writer

Stakeholder ReviewName Position DateBennett Jones System Engineering

Manager

Bob Wadum Component Engineering Manager

Clint Carter Operations Manager

Tom Mundy Configuration Control Manager

Rick Wehe Support Manager-Big Brown SES

Charles Koudelka Support Manager-Monticello SES

Razen Thomas Support Manager-Martin Lake SES

Todd King Support Manager-Sandow SES

Tom Shaw Support Manager-Oak Grove SES

Chad Donner Production Manager-Big Brown SES


Pete Petrofes Production Manager-Monticello SES

Ben Elliott Production Manager-Martin Lake SES

Jon Rice Production Manager-Sandow SES

Brett Amheiser Production Manager-Oak Grove SES


1. PurposeThe purpose of this Program Procedure is to establish requirements and responsibilities for the performance of maintenance troubleshooting activities across the Lignite fleet.

2. Applicability This procedure applies to troubleshooting activities performed at the following locations:

Big Brown Steam Electric Station Martin Lake Steam Electric Station Monticello Steam Electric Station Oak Grove Steam Electric Station Sandow Steam Electric Station

3. ReferencesDevelopment references are documents used to develop the procedure. Performance references are documents referenced within the procedure.

3.1 Development References[[1]] CPNP MDA-111, Revision 4; Maintenance Department Troubleshooting Activities Rev. 4[[2]] Gulf Publishing Company: Machinery Failure Analysis and Troubleshooting, Third Edition,

Heinz P. Bloch and Fred K. Geitner, Gulf Publishing Company, 1997[[3]] Electric Power Research Institute: System and Equipment Troubleshooting Guideline,

1003093, Electric Power Research Institute, January 2002[[4]] Institute of Nuclear Power Operations: AP-913, Revision 1, Equipment Reliability Process

Description, Rev. 1, Institute of Nuclear Power Operations, November 2001

3.2 Performance References[1] FFA-PRO-AD0007 Processing Condition Reports[2] LUM-FFA-REL-0011 Cause Analysis[3] LUM-LPO-REL-0024 Confirm-Refute Process


DevDocs, 01/27/11,

Revised the ref’s according to dev team’s Style Guide. Should also put #2 after #3 so they’re alphabetical.

4. Definitions and AcronymsThe following definitions and acronyms appear throughout this procedure. The first occurrence of each term and acronym appears in italics.

Advanced Troubleshooting—A troubleshooting activity that does not satisfy the requirements for complex troubleshooting, but which would require more data collection than that required for a basic troubleshooting activity. An advanced troubleshooting activity may require the formation of a formal troubleshooting team at the discretion of plant management.

As-Found Condition—Present condition (e.g., position, physical state, or operating status, etc.) of a system or component at the time of inspection immediately following a problem occurrence.

Basic Troubleshooting—A troubleshooting activity that does not require comprehensive planning, risk assessment, the formation of a cross-functional troubleshooting team, or significant data collection due to its limited scope and impact.

Cause Analysis— The process of investigating, identifying, and eliminating specific causes of events in order to correct the problem and, in some cases, to prevent repetitive or similar human and equipment performance problems; there are three levels of cause analysis: and they are referred to as basic, apparent, and root.

Complex Troubleshooting—A troubleshooting activity that requires comprehensive data collection, rigorous failure analysis, and the formation of a troubleshooting team due to its complexity, repetitive nature, and potential impact on plant operation or on personnel / or environmental safety.

Condition—Existing or potential issue, circumstance, or problem that warrants resolution or trending.

Condition Report (CR)—Name given to theThe electronic record used to identify, classify, evaluate, resolve, trend, report, and manage a condition or potential condition.

Confirm/Refute Matrix (CRM)—Form that is used to ensure that system and equipment failure modes are analyzed in a logical and systematic manner, while ensuring that applicable failure mechanisms have been adequately considered and an accurate cause determination has been obtained. The CRM is mandatory for complex troubleshooting activities, but may be employed for less rigorous troubleshooting activities at the discretion of the troubleshooting team leader.

Corrective Action Program (CAP)—A program implemented to identify, resolve, and trend adverse conditions using the Condition Report module of the ActionWay system.


Extended Troubleshooting—Installation of M&TE for the purpose of monitoring and trending the performance of a system or component for a period of 30 days or less. Extended troubleshooting activities can apply to either a complex, advanced, or basic troubleshooting approach, with the following limitations:

Work order is issued to track/monitor progress. M&TE is installed for more than a single shift; M&TE remains unattended; Impact on plant configuration and operation has been evaluated. Activity is re-evaluated at a specified interval.

Failure Mode—An event that results in the occurrence of a functional failure.

Functional Failure—A failure that results in the inability of an asset to meet a desired standard of performance that is acceptable to the user.

Initial Problem Determination (IPD)—Activity performed prior to formal troubleshooting to assist in identifying the affected equipment and scope of problem. The IPD is typically performed during walkdown activities and is typically limited to visual inspection, verification of operating conditions using in-place instrumentation, and obtaining data relative to equipment status using non-intrusive M&TE.

Lock Out/Tag Out Verify (LOTOV)—The isolation of equipment to ensure safety while testing or performing maintenance tasks on related equipment.

Measuring and Test Equipment (M&TE)—Test equipment used to measure, monitor, record, or trend system or equipment/asset/component performance.

OEM—Original Equipment Manufacturer

ProjectWise—Content management software used to store and develop controlled documents for Luminant Fossil Generation.

SME—Subject Matter Expert

Troubleshooting—Activity of an investigative nature for the purpose of identifying the cause of a problem and defining to define a course of corrective action.

Troubleshooting Team—Formal team consisting of multiple individuals from a cross-functional background assigned to troubleshoot an equipment or system performance problem. Complex problems may also require the assistance of vendor AND and OEM personnel as team members.

Walkdown—Initial activity of the troubleshooting process that is used to gathers as-found information and other plant data, such as the following:

System system line-up Component component indications and positions (e.g., valves, dampers, relays, switches,

etc.) Overall overall system configuration Component component configuration (e.g., open, closed, etc.)


DevDocs, 01/27/11,

“system, equipment, asset, or component”?

In addition to gathering as-found information, the walkdown process can also be used as an opportunity to identify and eliminate obvious issues that would not have any bearing on the cause of the problem under investigation.

5. ResponsibilitiesThe following individuals shall adhere to the key responsibilities listed below. The Director- Fleet Engineering and Support Maintenance Services, is responsible for maintaining the procedure. current.

5.1 System or Component EngineerThe responsibilities of the System or Component Engineer include the following:

Develop troubleshooting plan for Complex complex troubleshooting activities Provide system-level and/or component-level expertise as needed Review and/or approve troubleshooting plan Coordinate activities and /communications with vendors and OEM personnel as needed Design troubleshooting tests, inspections, and measurement techniques for Complex

complex troubleshooting activities Provide acceptance criteria for tests, inspections, and measurements Conduct/lead cause analysis exercises as part of follow-up activities as needed Develop the Confirm-Refute Matrix, as needed, for Complex complex troubleshooting

activities Develop extended troubleshooting plan, as necessary Coordinate/lead Complex complex troubleshooting issues Serve as SME when appropriate Ensure that team members can perform unhindered and free of distractions Appoint qualified off-shift troubleshooting leaders for Complex complex troubleshooting

activities that extend beyond the current shift

5.2 Equipment SpecialistThe responsibilities of the Equipment Specialist include the following:

Perform risk assessment, if applicable Identify and define the problem Monitor system availability Assist in developing the troubleshooting plan as required Provide system operational knowledge and expertise Operate and reposition equipment as required in support of to support the troubleshooting

activities Provide LOTOV assistance as required Assist in development ofdeveloping the troubleshooting plan, as required Participate in follow-up actions (e.g., Cause Analysis) as required


DevDocs, 01/28/11,

or “keeping the procedure current.”

5.3 Operations ManagementThe responsibilities of Operations Management include the following:

Review and approve the troubleshooting plan Manage interface between production equipment and the maintenance organization Grant permission to deviate from the troubleshooting procedure in cases of emergency in

order to increase the level of safety, prevent injury to personnel, or preclude damage to plant equipment

Establish a secure, central location to serve as a meeting area to discuss troubleshooting progress, and provide update status of the troubleshooting plan; the area will also, and serve as an area to carry-out turnover activities if the troubleshooting activity should extend beyond the current shift

Perform risk assessment for any troubleshooting activities that may involve equipment that is functional and operational

5.4 Director, FES Maintenance ServicesThe responsibilities of the Director, FES Maintenance Services include the following:

Maintain this Program Procedure Establish and maintain an effective culture with regard to troubleshooting

5.5 Maintenance Planner/SchedulerThe responsibilities of the Maintenance Planner/Scheduler include the following:

Provide logistical support (e.g., parts/supplier information, drawings/documentation, etc.) to aid the troubleshooting effort

Coordinate the start-up of the troubleshooting team Assist in the development of the troubleshooting plan Assist in the removal, storage, and preservation of physical evidence Assist in the installation and removal of scaffolding and grating in support of the

troubleshooting effort Assist in obtaining any necessary safety permits (e.g., confined space, elevated work, etc.)

5.6 Maintenance TechnicianThe responsibilities of the Maintenance Technician include the following:

Assist in development of the troubleshooting plan, as required Perform troubleshooting activities as directed by team leader Implement troubleshooting plan within defined boundaries Communicate progress and findings to troubleshooting team leader Serve as equipment SME when appropriate Perform Basic basic troubleshooting activities


DevDocs, 01/27/11,

developing

DevDocs, 01/27/11,

installing and removing

DevDocs, 01/27/11,

removing, storing, and preserving

DevDocs, 01/27/11,

developing

DevDocs, 01/27/11,

This abbreviation is not in the list of definitions & acronyms. Should it be?

5.7 Configuration Control ManagerThe responsibilities of the Configuration Control Manager include the following:

Preserve the completed troubleshooting plan and all supporting documentation in ProjectWise

5.8 Plant EngineerThe responsibilities of the Plant Engineer include the following:

Develop troubleshooting plan for Advanced advanced troubleshooting activities Review and/or approve troubleshooting plan Design troubleshooting tests, inspections, and measurement techniques for Advanced

advanced troubleshooting activities Coordinate activities and /communications with vendors and OEM personnel as needed Provide acceptance criteria for tests, inspections, and measurements Conduct/lead cause analysis exercises as part of follow-up activities as needed Develop the Confirm-/Refute Matrix, as needed, for Advanced advanced troubleshooting

activities Develop extended troubleshooting plan, as necessary Coordinate/ and lead Advanced advanced troubleshooting issues Serve as SME when appropriate Ensure that team members can perform unhindered and free of distractions Appoint qualified off-shift troubleshooting leaders for Advanced advanced troubleshooting

activities that extend beyond the current shift

6. InstructionsThe procedures below contain instructions for the following tasks:

General Requirements Troubleshooting - Initial Actions Complex Troubleshooting Advanced Troubleshooting Basic Troubleshooting Extended Troubleshooting Follow-up Actions


6.1 General Requirements

6.1.1 When performing maintenance troubleshooting activities, the following general requirements should be followed at all times:

Perform all troubleshooting activities in accordance with accepted maintenance standards. Use extreme caution in order to avoid inadvertent trips or equipment isolations. Adhere to best maintenance practices when performing troubleshooting activities. Observe all safety precautions to prevent injuries to personnel or damage to plant

equipment. Coordinate troubleshooting activities with the Operations departmentDepartment. The troubleshooting team should attempt to validate or eliminate all possible failure

modes and avoid stopping at the first likely or probable failure mode that can be validated. Establish a secure, central location to serve as a meeting area to discuss troubleshooting

progress and, provide update status of the troubleshooting plan;, and the area will also serve as an area to carry-out turnover activities if the troubleshooting activity should extend beyond the current shift.

Appoint an off-shift troubleshooting team leader if troubleshooting activities should extend beyond the current shift.

Troubleshooting activities may be performed on a system or component that is functional and operational provided a risk assessment has been performed and documented prior to the start of any troubleshooting activity.

Any formal troubleshooting plan must include criteria and provisions for stopping and restoring equipment conditions whenever unexpected results occur.

Troubleshooting activities should be limited to the system or component that is identified in the work order and LOTOV boundary.

Use Procedures procedures that are system or component-specific, and applicable, should be used if available.

In cases of emergency, the on-duty Shift Supervisor may grant permission to deviate from this procedure whenever necessary to:

o Increase increase level of safetyo Prevent prevent injury to personnelo Prevent prevent damage to plant equipment

The preservation ofPreserve physical evidence and the as-found conditions of defective equipment are required forto ensure accurate cause analysis. Collect and store pPhysical evidence shall be collected and stored in a manner that protects the evidence from further damage, degradation, deterioration, loss, or exposure to the elements.


DevDocs, 01/27/11,

The statements at this level are more like text than headings, and the italicized text becomes overwhelming as it becomes larger portions in 6.2-6.5. I suggest removing this heading level.

Troubleshooting activities may include, but are not limited to, the following:

o Installation and removal ofinstalling and removing jumpers (electric, hydraulic, or pneumatic)

o lifting or /landing leadso Application and removal ofapplying and removing temporary powero Rrepositioning switches, valves, or breakerso Removal and replacement ofremoving and replacing fuses, plug-in components, or

printed circuit boardso Ddisconnecting and reconnecting of compression fittings and threaded pipe connectionso Removal and replacement ofremoving and replacing doors, guards, covers, or inspection

plateso Removal and replacement ofremoving and replacing soldered componentso calibration per approved procedureso vibration monitoring, motor current signature monitoring, thermographic inspection,

and acoustic monitoring as applicableo oil samplingo Erection and removal oferecting and removing temporary scaffoldingo Removal and replacement ofremoving and replacing decking or grating

Maintain a chronological record of all troubleshooting activities associated with the problem investigation on the initiating document of record (e.g., Work Order, Condition Report) or in the formal troubleshooting plan, if applicable. The chronological record should include sufficient detail to support further investigations such as engineering evaluations and formal cause analysis. At a minimum, the chronological record should include the following, as applicable:

o Permission permission obtained to commence begin troubleshooting activitieso "As as Found" information and component conditiono Visual visual observation of the affected component(s) or area for obvious defectso Description description of troubleshooting activities performedo Minor minor corrective actions taken within the scope of this procedureo System system restoration completed

Coordinate with the Operations Department whenever Operator assistance is needed to support troubleshooting activities such as:

o Opening opening or closing of valves or circuit breakerso Removing removing or installing fuseso Starting starting or stopping of pumps or motorso Manual manually controlling of a component or loopo Obtaining obtaining oil samples

Revision toRevising a written troubleshooting plan, or to an extended troubleshooting plan, will require the same level of review and approval as the original troubleshooting plan.

Upon completion, the Configuration Control Manager shall preserve the troubleshooting plan and any supporting documentation shall be preserved by the Configuration Control Manager in ProjectWise.


DevDocs, 01/27/11,

Does it make sense to combine all these “removing and replacing” items?

[6.1.2] Minor corrective actions may be performed to correct problems found during the course of troubleshooting activities. Refer toSee Attachment 8F, "Minor Corrective Actions That May Be Performed During Troubleshooting.".

1. Work order revision is not required when minor corrective actions performed within the scope of this procedure correct the problem.

2. Work order revision is required if additional instructions not addressed by this procedure are needed.

6.2 Troubleshooting - Initial Actions

[6.2.1] Ensure that the initial problem report (e.g., Work Order, Condition Report) has adequately described the problem in sufficient detail to facilitate the troubleshooting process. The problem description should include detailed information with regard to plant condition, visual observations, in-progress actions taken, and the location of any physical evidence if different than the original location.

Caution Notify the originator of the initial problem report for clarification of any information or to obtain any missing information that would facilitate the troubleshooting process.

[6.2.2] Prior to commencingBefore beginning formal troubleshooting activities, perform a walkdown to verify the scope and breadth of the problem. Document and include the as-found conditions on the IPD form, TRB-1. If necessary, use additional sheets. may be added as required.

[6.2.3] Record M&TE that is used during the course of the initial investigation must be recorded on Form TRB-4, M&TE Log.

[6.2.4] Preserve any physical evidence in accordance with Attachment 8D or Attachment 8E, as appropriate. Since Because physical evidence may be needed to support any future formal root cause analyses, failure analysis exercises, clearly identify and store each piece of physical evidence must be clearly identified and stored in a manner that would protect it from further damage, deterioration, degradation, loss, and exposure to the elements.

Note 1 Small items may be "bagged and tagged" in a zipper-style plastic bag or other appropriate container and stored on shelving in a secure area, as applicable. Larger items must be clearly tagged and may be stored on a pallet in a similar secure location.

Note 2 Equipment nozzles (e.g., pump suction and discharge ports), if open, should be covered to prevent ingress of contaminants or debris.


DevDocs, 01/27/11,

“will”?

DevDocs, 01/28/11,

This is the first occurrence of this term. According to Definitions section, it should be italicized. Since the surrounding text is italicized, it is in Roman type.

DevDocs, 01/27/11,

its?

[6.2.5] Upon completion of After completing the IPD form and walkdown, determine if further troubleshooting activities will be necessary to resolve the problem. Refer toSee Attachment 8A, Troubleshooting Process Flowchart, for a graphic depiction of the troubleshooting process.

[6.2.6] If further troubleshooting activities are required, determine the classification (e.g., Basic, Advanced, Complex) of the troubleshooting activity. Refer toSee Attachment 8C for guidance in determining the appropriate classification level of the troubleshooting activity.

[6.2.7] Prior to commencement ofBefore beginning any troubleshooting activity, obtain the following as if necessary:

[a.] Ppermission to commence begin troubleshooting

[b.] Nnecessary LOTOV clearances

[c.] Rrequired permits

[d.] Ddocumentation and reference materials (e.g., P&ID, technical manuals, procedures, prints/drawings)

[e.] Nnecessary tools and M&TE as appropriate

6.3 Complex Troubleshooting

[6.3.1] Perform cComplex troubleshooting shall be employed for situations where personnel and environmental safety are at risk, generating capacity may be lost or curtailed, and for most problems that are deemed repetitive in nature. Refer to Attachment C, Troubleshooting Classification Criteria, for additional scenarios that may require Complex complex troubleshooting.

[6.3.2] Complex troubleshooting activities will require the development and approval of a formal troubleshooting plan prior to the start ofbefore beginning any troubleshooting activities beyond the initial IPD and walkdown. Refer to Attachment 8B, Troubleshooting Plan Contents, for criteria that should be included in the troubleshooting plan.

Document Tthe formal troubleshooting plan should be documented on form TRB-2. At a minimum, the The Shift Supervisor on duty should review and approve the written troubleshooting plan should be reviewed and approved by the Shift Supervisor on duty. If necessary, use additional sheets may be added as required.

If M&TE is required and will be installed beyond the end of the current shift, refer to Section 6.7, Extended Troubleshooting, for additional requirements.

6.3.1[6.3.3] The formal troubleshooting plan should be developed by the System or Component Engineer and must identify each team member.

Identify the name of each troubleshooting team member, as well as his or her title and responsible department, on Form TRB-3A, Troubleshooting Team Identification.


DevDocs, 01/27/11,

must?

DevDocs, 01/28/11,

Does this list need to have letters, or could it be a bulleted list?

[6.3.4] Complex troubleshooting must include a Confirm/Refute Matrix to ensure that all possible failure modes that could have resulted in the problem occurrence have been properly identified and resolved. Refer to Reference 3.2.3, Confirm-Refute Process, for guidance in the preparation and completion ofpreparing and completing the CRM.

[6.3.5] Document Troubleshooting troubleshooting steps, their expectations, and results should be documented on Form TRB-3B, Troubleshooting Plan Outline.

[6.3.6] Record M&TE that is used during the course of the troubleshooting activity must be recorded on Form TRB-4, M&TE Log.

[6.3.7] Upon completion of C completing complex troubleshooting activities, the team leader should inform the Shift Supervisor on duty of the problems found and the need for any follow-up actions, if applicable.

6.4 Advanced Troubleshooting

[6.4.1] Perform Advanced advanced troubleshooting may be desired when a problem is intermittent in nature, additional resources are required to participate, or when plant management has determined that additional information beyond that required by a basic troubleshooting activity may be desired. Refer toSee Attachment C, Troubleshooting Classification Criteria, for additional scenarios that may require Advanced advanced troubleshooting.

[6.4.2] The Plant Engineer may developDevelopment of a formal troubleshooting plan, but it is optional. and may be implemented at the discretion of the Plant Engineer.

If a formal troubleshooting plan is developed, it should be documented it on form Form TRB-2. At a minimum, The Shift Supervisor on duty should review and approve the written troubleshooting plan should be reviewed and approved by the Shift Supervisor on duty. If necessary, use additional sheets may be added as required.

If M&TE is required and will be installed beyond the end of the current shift, refer to Section 6.7, Extended Troubleshooting, for additional requirements.

[6.4.3] Any Advanced advanced troubleshooting activity for which a formal troubleshooting plan has been developed must also include the following documents:

Troubleshooting Team Identification, Form TRB-3A Troubleshooting Plan Outline, Form TRB-3B


DevDocs, 01/27/11,

must?

DevDocs, 01/27/11,

“is desired” or “may be necessary.”

DevDocs, 01/27/11,

Additional to what?

6.4.1[6.4.4] Identify the name of each troubleshooting team member, as well as his or her title and responsible department, on the Troubleshooting Team Identification form, TRB-3A.

[6.4.5] If a formal troubleshooting plan has been developed, Advanced advanced troubleshooting should also include a Confirm/Refute Matrix to ensure that all possible failure modes that could have resulted in the problem occurrence have been properly identified and resolved. Refer to Reference 3.2.3, Confirm-Refute Process, for guidance in the preparation and completion of preparing and completing the CRM.

[6.4.6] Document Troubleshooting troubleshooting steps, their expectations, and results should be documented on Form TRB-3B, Troubleshooting Plan Outline.


[6.4.8] Upon completion ofcompleting Advanced advanced troubleshooting activities, the team leader should inform the Shift Supervisor on duty of the problems found and the need for any follow-up actions, if applicable.


6.5 Basic Troubleshooting

[6.5.1] Perform Basic basic troubleshooting shall be employed when the problem investigation does not require comprehensive planning, risk assessment, involvement of a multi-discipline team, or the collection of a significant amount of data. Refer toSee Attachment C, Troubleshooting Classification Criteria, for criteria to be used in order to determine if a troubleshooting activity satisfies the requirements for Basic basic troubleshooting.

[6.5.2] Development of a A formal troubleshooting plan is not required for Basic basic troubleshooting activities. At a minimum, any basic troubleshooting activity should first be planned and discussed with the Operations Department/Shift Supervisor on duty prior to its implementation.

[6.5.3] If M&TE is required and will be installed beyond the end of the current shift, refer tosee Section 6.7, Extended Troubleshooting, for additional requirements.

[6.5.4] If more than a singleone person participates in the basic troubleshooting activity, identify each additional person should be identified on Form TRB-3A, Troubleshooting Team Identification.

[6.5.5] Document Actions actions taken during the Basic basic troubleshooting activity should be documented on the initiating document of record (e.g., Work Order, Condition Report).


[6.5.7] Upon completion of Basic basic troubleshooting activities, inform the Shift Supervisor on duty of the problems found and the need for any follow-up actions, if applicable.

6.6 Follow-up Actions to Troubleshooting ProcessDepending on the type and nature of the problem under investigation, it may be necessary to identify follow-up actions to mitigate future repeat occurrences of the problem or to initiate corrective maintenance activities. Refer toSee Attachment 8A, Troubleshooting Process Flowchart, for additional information.

6.6.1 Non-Repetitive Problems

Problems that were expected to occur or that are not repetitive in nature, should be addressed through the corrective maintenance program; the System or Component Engineer may implement formal troubleshooting activities are optional but may be implemented at the his or her discretion of the System or Component Engineer when deemed necessary.

The System or Component Engineer Formal may implement formal cause analysis is not required, but may be instituted at his or herthe discretion of the System or Component Engineer via the initiationby initiating of a Condition Report. Refer to Reference 3.2.1, Processing Condition Reports.


DevDocs, 01/28/11,

recurrences?

Perform Cause cause analysis should be performed in accordance with the requirements of Reference 3.2.2, Cause Analysis.

6.6.2 Repetitive Problems

Problems that were expected to occur or that are repetitive in nature, as well as those problems that were not expected to occur, will require formal cause analysis as a follow-up action to the troubleshooting process.

Formal cause analysis action should be directed by the System or Component Engineer and documented with a Condition Report. Refer to Reference 3.2.1, Processing Condition Reports.

Perform Cause cause analysis should be performed in accordance with the requirements of Reference 3.2,.2, Cause Analysis.

6.7 Extended Troubleshooting

[6.7.1] Extended troubleshooting may apply to Basicbasic, Advancedadvanced, or Complex complex troubleshooting activities.

[6.7.2] Extended troubleshooting, if employed, should not exceed thirty days from the date of commencement of the troubleshooting activity begins, and is governed by the following limitations:

Work order will be issued to track activity. M&TE will be installed beyond a single shift. M&TE will not be attended continuously. Impact on plant configuration has been evaluated.

[6.7.3] The initiating work order must clearly identify The the scope and expected duration of any extended troubleshooting plan. must be clearly identified on the initiating work order. Any revision to an extended troubleshooting plan would require the same level of review and approval as that of the original plan.

[6.7.4] Record M&TE that is used during the course of an extended troubleshooting activity must be recorded on Form TRB-4, M&TE Log.

6.7.1[6.7.5] Unless stated otherwise, extended troubleshooting plans should be reviewed and approved by:

[1.] Onon-duty Shift Supervisor when troubleshooting systems or equipment that do not have any impact on personnel or environmental safety, or will not result in a decrease or loss of generating capacity.

[2.] Senior senior plant management when troubleshooting systems or equipment that can pose a significant risk to personnel/ or environmental safety or generating capacity.


DevDocs, 01/27/11,

The duration of extended troubleshooting, if employed, should not exceed thirty days, and is governed…

DevDocs, 01/28/11,

The System or Component Engineer shall direct formal cause analysis and document it with a Condition Report. Refer…

7. RecordsNone.

8. AttachmentsThe following attachments are appended to this procedure:

Attachment A: Troubleshooting Process Flowchart

Attachment B: Troubleshooting Plan Contents

Attachment C: Troubleshooting Classification Criteria

Attachment D: Preservation of Evidence and As-Found Conditions for Mechanical Components

Attachment E: Preservation of Evidence and As-Found Conditions for Electrical Components

Attachment F: Minor Corrective Actions That May Be Performed During Troubleshooting Activities


Attachment A: Troubleshooting Process Flowchart


Attachment B: Troubleshooting Plan ContentsA formal troubleshooting plan should include, but not be limited to, the following information:

[[1]] Nname and title of each team member[[2]] Cclearly defined problem statement that is technically correct and accurate[[3]] Pproperly identified component or system, including equipment identification number and

functional description[[4]] Ddetailed assignment(s) for each team member[[5]] Eexpected results of each troubleshooting step[[6]] Pprocess used during troubleshooting[[7]] Iidentification of M&TE equipment used (e.g., Manufacturer manufacturer name, model

number, serial number, etc.)[[8]] Aanticipated alarms and actuations[[9]] Aacceptance criteria for each test, /inspection, and /measurement[[10]] Aawareness of possible consequences of initial intrusion into equipment (e.g., attaching

test equipment, lifting leads, applying power, etc.)[[11]] Ccontingency actions based on the actual readings/measured results[[12]] Ddocumentation requirements[[13]] Ppersonnel safety issues and PPE requirements if beyond basic needs (e.g., safety glasses,

hardhat, safety shoes, gloves)[4][[14]] LOTOV clearances[[15]] Sspecial permits[[16]] Sscope and expected duration of any extended troubleshooting activities[[17]] Ccomponent/system configuration controls[[18]] Cclear definition of work area boundaries and scope of equipment[[19]] Nnecessary reference material (e.g., drawings, procedures, technical manuals, etc.)[[20]] Ddirections for preservation of physical evidence and documentation of as-found

conditions[[21]] Ccommunication and troubleshooting hold points[[22]] Mmethod by which the system/ or component will be restored to design conditions[[23]] Rrecommendations for follow-up actions


Attachment C: Troubleshooting Classification CriteriaUse The the following criteria should be used as a guideline to determine the appropriate level of troubleshooting activities needed to ascertain a problem solution. Even though a solution may require Advanced advanced or Basic basic troubleshooting activities, a Complex complex solution may be utilized at the discretion of Plant plant management.

Complex Troubleshooting troubleshooting is required for any of the following conditions:

Activities activities designated as high-risk due to personnel or environmental safety Risk risk of direct or indirect plant trip or reduction/loss of generating capacity Complex complex activities not frequently performed Activities activities for which vendor or OEM support is required Problem a repetitive problem has been identified as repetitive in nature Problem problem that presents possibility for multiple failures

Advanced Troubleshooting troubleshooting is required for one or more of the following conditions:

Additional additional personnel (e.g., System Engineer, SME) are required to assist in assessing the IPD findings

Problem is intermittent in nature and will require a formal troubleshooting plan Activities activities for which vendor and OEM support may be required Problem problem which will not have any effect on generating capacity Plant problem for which plant management decided that the problem warrants a higher

level of troubleshooting

Basic Troubleshooting troubleshooting is required for one or more of the following conditions:

Problem problem with non-process equipment that does not affect power generation Problem problem with cleared or spare equipment with minimal economic risk Investigation investigation that does not require multi-discipline involvement,

comprehensive planning, risk assessment, or significant data collection Problem problem of limited scope is limited in nature with a clearly understood failure

cause and/ solution


DevDocs, 01/28/11,

intermittent problem which will require a formal troubleshooting plan

Attachment D: Preservation of Evidence and As-Found Conditions for Mechanical Components

1. Perform a visual inspection and record the as-found conditions. The as-found conditions should be documented in the IPD and can include, but not be limited to, any of the following:

Equipment equipment identification number and functional description Evidence evidence of worn areas Evidence evidence of broken items or /parts Amount amount of lubrication present (oil or grease) Abnormal abnormal oil color or smell Missing missing or loose parts Loose loose or missing fasteners/ or bolting Presence presence of leaks Broken broken or defective welds Ease ease of removal or /need for special lifting equipment Unusual unusual noise, vibration, smell, or temperature if equipment is operating

2. Obtain any and all available readings or graphs from vibration analysis, thermography inspections, chart or strip recorders, oil analysis data, and any Operations data up to the time of failure, including any actions that had beenwere taken.

3. Whenever possible, and prior to the removal of any equipment or components, use a still or video camera to monitor/ and record the scene for future forensic analysis and investigation. Images depicting the overall condition, as well as images depicting problem details, should be included in the IPD and made a part of the official record.

4. Clearly identify and store defective items and /components in a manner such thatso they are protected from future damage, deterioration, degradation, loss, or exposure to the elements. Small items may be "bagged and tagged" in a zipper-style plastic bag or an appropriate container and stored on shelving in a secure area. Larger items must be clearly identified and may be stored on a pallet in a secure location with the same protective provisions against further damage, deterioration, degradation, loss, or exposure.

Equipment nozzles (e.g., pump suction and discharge ports), if open, should be covered to prevent ingress of contaminants or debris.

5. Ensure that the System or Component Engineer is aware of the defective item location and the "as-found" conditions.


DevDocs, 01/28/11,

Could the first line read “missing or loose parts (e.g., fasteners, bolting)” then we could delete the second line?

DevDocs, 01/28/11,

are not?

Attachment E: Preservation of Evidence and As-Found Conditions for Electrical Components1. Perform a visual inspection and record the as-found conditions. The as-found conditions should be documented in the IPD and can include, but not be limited to, any of the following:

Equipment equipment identification number and functional description Meter meter readings/calibration/physical condition Position position of any relays (tripped, not tripped) Switch switch positions Fuses fuses installed (including rating information) Position position of breakers (open or closed) Presence presence of any jumpers Condition condition and status of any indicating lamps Status status of any local alarms (actuated, not actuated) Presence presence of any installed M&TE Unusual unusual noise, vibration, smell, or temperature if equipment is operating Missingmissing, loose, or damaged items Note environmental conditions (e.g., hot, cold, steam, wet, etc.) Other other conditions that can be observed visually or by odor

2. Obtain any and all available readings or graphs from voltmeters, ohmmeters, chart or strip recorders, and any Operations data up to the time of failure, including any actions that had beenwere taken. Record in or attach to the IPD Rresults from any all simulations or testing for the problem. should be recorded in, or attached to, the IPD.

3. Whenever possible, and prior to the removal of any equipment or components, use a still or video camera to monitor/ and record the scene for future forensic analysis and investigation. Images depicting the overall condition, as well as images depicting problem details, should be included in the IPD and made a part of the official record.

4. Clearly identify and store defective items/ and components in a manner such thatso they are protected from future damage, deterioration, degradation, loss, or exposure to the elements. Small items may be "bagged and tagged" in a zipper-style plastic bag or other appropriate container and stored on shelving in a secure area. Larger items must be clearly identified and may be stored on a pallet in a secure location with the same protective provisions against further damage, deterioration, degradation, loss, or exposure.

Instrument openings (e.g., input and output legs), if open, should be covered to prevent ingress of contaminants or debris.

5. Ensure that the System or Component Engineer is aware of the defective item location and the "as-found" conditions.


DevDocs, 01/28/11,

are not?

Attachment F: Minor Corrective Actions That May Be Performed During Troubleshooting ActivitiesMinor corrective actions that may be performed during maintenance troubleshooting activities include, but are not limited to, the following:

[[1]] Replacement ofreplacing fuses with proper type and rating[[2]] Realignment ofrealigning fuse holders[[3]] Replacement ofreplacing printed circuit boards[[4]] Replacement ofreplacing plug-in components[[5]] Replacement ofreplacing certain indicator lamps (excluding those lamps and sockets that

could cause an inadvertent plant trip)[[6]] Cleaning ofcleaning components, cabinets, or cubicles[[7]] Ttightening or reconnection reconnecting of electrical connections or hardware[[8]] Ttightening of piping flanges, ductwork, compression fittings, or threaded pipe

connections[[9]] Restoration ofrestoring a component configuration to agree with design/ or vendor

documentation[[10]] calibrating Iinstruments calibration[[11]] adjusting Sstem packing adjustments on air and motor operated valves that do not have

stroke time limitations[[12]] adjusting Sstem packing adjustments on manual valves[[13]] adjusting Ggland packing adjustments on process pumps, if applicable[[14]] replenishing Llubrication levels replenishment[[15]] Retorquing retorquing valve bonnets and covers[[16]] Replacement ofreplacing valve handwheels or chain operators[[17]] Installation ofinstalling safety chains[[18]] Removal and reinstallation ofremoving and reinstalling insulation on piping and

ductwork (does not include fire retardant insulation)[[19]] Ccapping drain and vent lines, as needed, to prevent leakage until valve can be reworked

or replaced


Initial Problem Determination (Form TRB-1)

Work Order # ____________________ Condition Report # _____________________

Identify the Problem/Problem Description

Status and Walkdown Observations


Describe the Problem:

Is the WO or CR problem statement accurate and complete? Yes _____ No _____

Provide any additional information:

Current System or Component Status and Indications:

Observations from Walkdown:

Additional Data (e.g., alarms, first-outs, etc.). Identify and attach reports if applicable.

Reference Documents (e.g., drawings, manuals, procedures, etc.):

Documentation of Physical Evidence (e.g., list of digital pictures, sketches, impounded equipment witness statements, etc.). Attach if applicable.

Troubleshooting Plan (Form TRB-2)


SSSSS


System or Component Status:

Risk Level (e.g.,potential for plant trip or personnel/environmental safety hazard):

Low _____ Medium _____ High _____

Risk Level Basis:

Risk Level Determined by: ______________________________________________ _______________ (Name/Title)

Potential Hazards:

Safety Precautions:

Plan Termination Status:

Post-Maintenance Test Requirements:

Expected Plan Outcome:

Actual Results (attach additional sheets if necessary):

Troubleshooting Team Identification (Form TRB-3A)


Troubleshooting Team Member Name Title Department


Troubleshooting Plan Outline (Form TRB-3B)Must be completed for Advanced troubleshooting activities (if TRB-2 has been developed), and for all Complex troubleshooting activities. Use additional sheets as required.


Step Numbe

r

Action, Expected Result, and Contingency Plan Actual Results Responsible

Organization


M&TE Log (Form TRB-4)List any M&TE that was employed in the troubleshooting activity. Use additional sheets as required.


M&TEDescription

Manufacturer Name/Model No.

SerialNumber

CalibrationDue Date


Revision HistoryRevision Number Approval Date Name ChangesR0 Michael C. Ganz Initial issue


Template for Technical Procedure - Gwen...

Documents

Transcript of Template for Technical Procedure - Gwen...