Evaluation Improvement Initiative A New Evaluation Approach

Evaluation Improvement InitiativeA New Evaluation Approach

0616 CMBG Discussion on Revised Evaluation Approach.PPT

Institute of Nuclear PowerOperations

Suite 1042 700 Galleria ParkwayAtlanta, GA 30339

DAVID M. FARR ManagerConfiguration Management [email protected]

RROP - A “Risk Based Approach”

• NRC’s Mission: To protect the health and safety of the public.– Desired end states:

• Minimize core damage frequency• Avoid a large, early radioactive release

– Manage risk through ensuring the health of these cornerstones:• Initiating events• Mitigating system performance• Containment and fission barrier performance• Emergency Preparedness• Security• Cross cutting issues:

– Human performance– Safety conscience work environment

Shouldn’t INPO be “Risk Informed?”

What is INPO’s “risk model?” (What is it that INPO is charged with avoiding?)

– A significant operational event such that the continued viability of the unit is challenged or one that challenges the ongoing operation of other units in the industry.

– The undetected degradation of a major component or equipment such that safety margins are significantly compromised.

– A unplanned, long-duration shutdown requiring substantial upgrades to station programs, processes and/or people to recover the unit.

– Operation outside of plant design and safety margins.– Withdrawal of accreditation of one or more training programs.– Long-term poor performance of a station, as measured by INPO

assessments and the WANO performance indicators.

INPO’s Mission – Excellence!

• INPO is not about “risk avoidance” or even “risk management.”

• INPO is about plants “striving for operational excellence.”

• ERG input:– INPO should modify the risk model (the avoidance end

states) to focus on desired outcomes.

Operational Excellence Outcomes

Why look at Configuration Management?

Plant safety degraded, long-term shutdowns caused by problems with:

Inadequate operating and design margins Problems with design basis validity Deficiencies in plant status and configuration

control Poor design product quality Inadequate oversight of engineering programs

CM.1

Plant equipment configuration is rigorously

maintained and performance is consistent with design and

licensing requirements

CM.2

Roles and responsibilities for establishing and maintaining licensing and design basis are

understood and applied consistently

OR.5

Station leadership establishes a culture that

thoughtfully manages safety, design, &

operational marginsCM.4

Processes that are used to control plant design bases are

defined and validated

CM.3

Day-to-day activities are conducted in a way that ensures operational and

design margins are maintained

Well-Managed and Understood Safety,

Design, and Operational Margins

Well Managed Margins

An “Enabler”

The “Outcome”

Operating Point Limit

Normal Operating Point

Design or Tech Spec Basis Limit

Functional/Design Failure

Allowed operating range

Analyzed transient range

Greater than design basis range

Operating margin

Design margin

( Design allows operation in the transient range. However operating restrictions consistent with the design, such as the magnitude and frequency of excursions into this range apply)

( The allowed operating range provides the normal margins operators use without violating setpoints )

( This is a prohibited range of operation and represents the additional conservative margin to account for uncertainties in design and construction )

Design and Operating Margins Model

Margin Examples

• Turbine lube oil TCV found almost fully closed during operation.

• Safety Related Service Water throttle valve position changed, reducing EDG starting margin.

• Power uprates.

Reduced Operating Margin Example

ERStation processes and

practices effectively support day-to-day activities and result in high levels of equipment reliability and

availability

Sustainable, High Levels of Plant Performance

ORAn accountable, engaged,

and motivated workforce that achieves station goals

ORStation leadership has

a vision of excellence and acts to keep station on a continuous path of

improvement

ORLeaders emphasize nuclear

safety and establish core values that drive organizational

behaviors

OFIntegrated processes

promote teamwork and align resources to common goals and priorities that support safe and reliable operation

ERPrograms & organizational

behaviors are focused on the prevention of unanticipated or

repeat equipment failures

High Levels of Plant Performance

OF #7Station leadership promotes and uses sound operational decision-

making

OF #8Plant activities are controlled to

manage risks and provide compensatory actions

OF #6Operators recognize and respond

correctly to plant conditions

Sustainable, Event-Free Operations

PIStation personnel are

committed to and accountable for achieving performance improvement

by identifying and correcting problems, applying lessons learned from industry operating

experience, and comparing station standards and performance to industry standards of excellence

PIThe frequency and consequences

of human errors are minimized by the application of individual and organizational error reduction and

mitigation techniques

ERPrograms & organizational

behaviors are focused on the prevention of unanticipated or

repeat equipment failures

ERStation processes and

practices effectively support day-to-day activities and result in high levels

of equipment reliability and availability

Sustainable, Event-Free Operations

OR #14The organization and leadership

team inspires stakeholder confidence in their ability to

address problems

OR #15Station staff and

infrastructure are capable of responding to emerging problems

without adversely affecting routine station activities

Avoidance of Unplanned, Long-

Duration Shutdowns

OR #16Effective change

management plans that consider unintended

consequences

PIStation personnel are

committed to and accountable for achieving performance

improvement by identifying and correcting problems, applying lessons learned from industry

operating experience, and comparing station standards and performance to

industry standards of excellence

ERVulnerabilities to the

long-term reliability and availability of important plant equipment are recognized and mitigation and contingency plans have

been developed

Avoidance of Long-Duration Shutdowns

RPPlant conditions

and processes promote health and safety

RPRadiological and

industrial safety hazards are anticipated, recognized and

managed

High Levels of Plant Worker High Levels of Plant Worker SafetySafety

RPStation leaders establish

and promote a culture that supports worker safety

RPStation personnel

exhibit ownership for industrial and radiation safety

High Levels of Worker Safety

ORManagement executes a

long-term human resource strategy

PIManagement and

leadership skills are systematically developed

and reinforced

Highly-Skilled, Knowledgeable and

Collaborative Workforce

PI Training is used as a strategic

tool to provide highly skilled and knowledgeable personnel for safe, reliable operations and to support

performance improvement

PIThe frequency and consequences of human errors are minimized by the application of individual

an organizational error reduction and mitigation techniques

Highly Skilled Workforce

-7 months -20 weeks -3 months -11 weeks -10 weeks -9 weeks -8 weeks

*TM contacts site counterpart to discuss pre-evaluation observations, involvement with analysis, host-peer(s), OR/CM questionnaire, and utility self-evaluation

*TM develops pre-evaluation observation plan

*Information request letter sent to plant

*3 month letter sent to plant *Information request material received

*OR and CM questionnaire results received

*Preceding evaluation (week 1) *Preceding evaluation (week 2)

*Pre-evaluation analysis complete

*Report Week

-7 weeks -6 weeks -5 weeks -4 weeks -3 weeks -2 weeks -1 week

*Analysis Turnover

*SR Turnover

*Utility Self-evaluation Report Received

*Pre-exit Meeting

*Team Augmentation Decision Point¹

*Exit * Review CM questionnaire *Prep Week *Sequester Week

¹Based on inputs: - Analysis Turnover -Tech DM Input - Utility Self-evaluation - SR Turnover - Pre-evaluation observations²Based on inputs: - Same inputs as team augmentation decision, plus: - Pre-visit - Simulator Observation

INPO/Utility Evaluation Plan Meeting²(T-4 to T-2) Post Pre-visit and Simulator Observation

Simulator Observation(T-6 to T-3)

Pre-visit(T-5 to T-4)

Evaluation Timeline

Analysis Group Functions

• Plant Information Center (PIC) Data– Event history

– Indicators

• Plant Specific CAP database• Plant Specific Information on Engineering

Programs• Self assessments

– “Conduct of”

Analysis Group Output

• Identifies areas of focus– “High value” enablers and attributes

• Targets EN programs for review– Based on performance data from plant– Based on self-assessment information

• Targets systems or components for “vertical slice”– Based on performance data from plant– Consolidate with ER review

• Makes Team Augmentation Recommendation

CM Questionnaire

• Used to gain insight into roles & responsibilities (CM-2)

• Questionnaire is sent in hard copy format as part of the advance information request.

• Target audience is:– No more than 60 people. Distributed equally between:

• Maintenance supervision and craft• Operations supervision and RO/NLO• Engineering supervision and individual contributors

• CM evaluator analyzes results during prep-week

Evaluating Operational Excellence

Six Outcomes:– Sustainable high levels of

performance– Configuration and margin

management– Sustainable event free

operation– Avoidance of extended

shutdowns– High levels of worker

safety– Highly skilled workforce

Five Cross Functional Areas:– Equipment Reliability (ER)

– Configuration and Margin Management (CM)

– Operational Focus (OF)

– Performance Improvement (PI)

– Personnel and Organizational Performance (OR)

Advance information gatheringPED/Analysis Division

(3-4 months)

Data analysis of station CAP

Special evolution observations

Previsit

Station Self assessment in “conduct of”

OR/CC questionnaires

Discriminate areas that are functioning well from those that are questionable or unknown

Analyze data and determine focus areas

TM/ORTM/Analysis(1-2 months)

Cross-FunctionalEvaluation team

(12 total)

Develop evaluation plan in conjunction with station leadership • Functional areas• OEO building blocks• Cross-function focus areasOutput: team focus areas

TM/ORTM/Site Leadership (1 month)

Team Manager

Self-Evaluation and Learning Organization(PI)

Design, Configuration and Engineering Program (CM - EN)

OR and Personnel Performance (ORTM)

Operational Focus Reactor Safety(OF - OP)

1-2 Functional Evaluators

Host Peer

3 Industry Peers (OF, CM, ER)

Industry Advisor

Equipment Performance(ER - MA)

Special INPO reviews

NRC PIs, inspections, reports and public meetings

SARC minutes

Information Request

Previous evaluations

PIs, PIC and events

Analyze data considering:• Operational Excellence Outcomes• Operating Experience (TBD)• Framework Document• Lessons Learned from S/Ds

Department specific indicators

Deciding how to focus INPO resources

Executive Summary

APS1. Operational Focus (Goddard)2. Performance Improvement (Hamlin)3. Configuration Management (Farr)4. Equipment Reliability (P. Wagner)5. Organizational Effectiveness (Link)6. Personnel Safety (Moss)7. Events (Heublein)

AFI’s & StrengthsCross-functional

Functional Based on Significance and Consequence(Would require utility response)

AppendixOne page per functional area (OP,TR,CY,MA,EN,RP,WM)

Items for CAP (previously lower level AFI)Strengths

Additional Perspective (No Utility Response Required)

Discussion on outcomes of interest -Tools to Evaluate Outcome

-Analysis

-Second Week Meeting

-Report Week Discussion

-DM review during report week

Audience: CEOOwner: TMCheck: Pre-exit

Audience: CNOOwner: DMCheck: TM

Audience: SVPOwner: TeamCheck: DMs

Audience: SVP/PMOwner: EvaluatorsCheck: DM/TM

SOER and Operating Experience UseStatus and Any AFI’s

Audience: SVPOwner: ORTMCheck: TM

Revised Report Structure

Evaluation Improvement Initiative A New Evaluation Approach

Documents

Transcript of Evaluation Improvement Initiative A New Evaluation Approach