RTFI Process Assurance

Assuring On Target Performance Through Reliability Metrics

Lloyd A Schaefer - NDTU

Flaw Size

Rel

iabi

lity

IC ACHF

RTFI Process Assurance - Agenda

Radiographic process Goals and Objectives RTFI in the Product Design process Defining and Quantifying Reliability

Human Factors Methods & tools to ensure On Target reliability In process metrics to track variation Conclusions and benefits Questions

Radiographic Process Objectives

Why do we perform Radiographic Inspection? To find all defects? To find the “good” parts? To find the “bad” parts?

When we perform RTFI, have we; Found all of the defects? Found most of the defects? Found all of the rejectable defects according to the code?

Examination of Engineering Design can provide some answers

Engineered Product Design Process

Welds, and other products, examined by RT; Are imperfect structures with variable integrity Are the result of design efforts to;

Application optimize products through knowledge of material, engineering, fabrication & cost, variables and behavior

Design variables which influence product integrity: Material specification based on properties & behavior Fabrication process parameters & capability Radiographic Inspection process capability

RTFI’s role: Describe material & fabrication results

Need a way to integrate the inputs & quantify the result

Integrated Results of Design, Fabrication and Inspection

Flaw/Defect size

Nu

mb

er

of

flaw

s/

Pro

bab

ilit

y o

f D

ete

cti

on

0

Flaw and Detection Distributions

n/1

Accep

tan

ce

Cri

teri

a

Domains & Definitions

Flaw/Defect size

Nu

mb

er

of

flaw

s/

Pro

bab

ilit

y o

f D

ete

cti

on

0

Flaw and Detection Distributions

n/1

Accep

tan

ce

Cri

teri

a

Flaws Detected at [ ] Probability & Rejected

Detected and accepted

Material, Process & Inspection Dynamics

Flaw/Defect size

Nu

mb

er

of

flaw

s/

Pro

bab

ilit

y o

f D

ete

cti

on

0

• Flaw distributions change via Material Properties & Process Capability

n/1

Accep

tan

ce

Cri

teri

a

Material, Process & Inspection Dynamics

Flaw/Defect size

Nu

mb

er

of

flaw

s/

Pro

bab

ilit

y o

f D

ete

cti

on

0

• Detection results change via technology and performance, parameters & variation

n/1

Accep

tan

ce

Cri

teri

a

Material, Process & Inspection Dynamics

Flaw/Defect size

Nu

mb

er

of

flaw

s/

Pro

bab

ilit

y o

f D

ete

cti

on

0

• For a given process/inspection, the Acceptance Criteria determines net distribution of flaws

n/1

Accep

tan

ce

Cri

teri

a

Goal for RTFI Process Reliability:Stay on Target!

Flaw/Defect size

Pro

bab

ilit

y o

f D

ete

cti

on

(P

OD

)

0

• Limit POD variability to ensure design estimate of part imperfection distribution is accurate

1

Historical design assumption for RT/RTFI process

What is RTFI, or any NDE, Reliability?

NDE processes can be defined as: A physical process with intrinsic limits (IC) Limited and varied by a specific sets of application

parameters, (AC) Further varied by human factors during the performance

of the inspection (HF)

Conceptual Model for NDE Reliability:R=f [AC,HF] < ICand,AC = f (AP,HF) < IC, where HF=0

Graphical Representations of NDE Reliability

R

IC

AC

HFPredicted Reliability

Flaw size, a, or condition measured

R

… or in terms of probability

Phy. limit

100%------IC AC

HF

Measuring RTFI Process Reliability

RTFI is a Human Factors (HF) intensive process Specific measurements of reliability, are not routinely

made for RTFI Reliance upon a controlled process and trained, certified

inspectors to meet legacy performance assumptions

Possible outcomes from an RTFI activity:

NDT TP FN

TN FP

TP + FN = 100%

TN + FP = 100 %

truth inspection report

TP: true positive indication (hit) FN: false negative indication

TN: true negative indication FP: false positive indication (false alarm)

or

or

system

Measuring RTFI Process Reliability

Receiver Operating Characteristic (ROC) A method for describing reliability via the 4 possible

outcomes

Measuring RTFI Process Reliability

Flaw/Defect size

Pro

bab

ilit

y o

f D

ete

cti

on

0

• The ROC can be related to the POD curve by plotting the true positives proportions as a function of flaw size

1

Methods for keeping RTFI on Target

Round Robin Testing (RRT) identifies keys: Training and certification Years of experience (ref. Fucsok et. al.)

Examined ROC vs. years of experience; 0-5, 6-22, 23-35 True Positive performance maximized in the 6 to 22 year

experience group tested False call performance maximized in the 23-35 year group More data needed to validate findings

Consensus, or peer review Strong & consistent influence on accuracy improvement in

multiple Probability of Detection Studies• Rummel• Suggs• Stanley

Effects of Peer Review on RTFI Reliability

Peer Review Illustration: Statistically significant series of Radiographs shown to

several certified inspectors Both flaws and unflawed samples used with at least 3:1

ratio of flawed to unflawed areas Objective: To determine correct interpretation &

consensus reliability

Which welds are acceptable? Which need to be repaired?

24A/6R 21A/9R 16A/14R

21A/9R

Effects of Peer Review on RTFI Reliability

Intersection of decisions produces most correct results via physical/metallurgical validations

24A/6R 16A/14R

Peer Review fever: Catch it!

Other Tools/Influences on RTFI Reliability

Digital/Quantitative Feedback Training Densitometer as density training aid

Usage results in increased accuracy of film density estimates

Film Digitization/Profiling

Improves quantitative skills

WeldPM

Example of Metrics to Observe RTFI Variation

Hawiyah RTFI Process Quality

0

10

20

30

40

50

60

70

80

90

100

110

6/16/2

000

6/30/2

000

7/14/2

000

7/28/2

000

8/11/2

000

8/25/2

000

9/8/20

00

9/22/2

000

10/6/

2000

10/20

/2000

11/3/

2000

Sam

ple

Siz

e &

Qu

alit

y P

erce

nta

ges

Total Welds Evaluated Submittal % Assessed Flaw Detection % Overcall % Film Quality %

Detection & Film Quality LCL

Overcall UCL

Other Tools/Influences on RTFI Reliability

Identification of Inspector Potential EPRI developed Dynamic Inspector Aptitude Testing, DIAT Testing motivated to quantify human factors which

influence inspector performance in POD studies Five (5) aptitudes measured:

General cognitive ability Abstract reasoning Spatial visualization Pattern recognition Stress tolerance

Saudi Aramco can participate in this testing via its association with EPRI through Henry Stephens

Conclusions & Benefits of on Target RTFI Reliability

Variation in the RTFI process should be managed Key variations & effects in the RTFI process:

Type II errors, “misses”, produce a right shifted POD curve Unique improvements, or creeping elegance, which can result

in a left shifted POD curve, but which are not uniform in effect• Produces an increase in Type I errors and repairs• Leads to an increase in the amount of inherent flaws

When the RTFI Process is On Target: Manufacturing has optimum knowledge of process health Accurate predictions can be made of the condition and life of

the structure Risk management is improved through accurate flaw

distribution inputs to the Risk Based Assessment modules

Reliability concepts should be extended to all NDE