1NSWC Corona-MS Interval DJ June 2002 Dr. Dennis Jackson 909-273-4492 DSN 933-4492...

1NSWC Corona-MS Interval DJ June 2002

Dr. Dennis Jackson909-273-4492DSN 933-4492

[email protected]


CALIBRATION INTERVAL ANALYSIS: CURRENT AND

FUTURE

CALIBRATION INTERVAL ANALYSIS: CURRENT AND

FUTURE

Dr. Dennis JacksonMS30A1

June 2002


Overview

Current Calibration Interval Methods

Interval Analysis Results

New Approaches to Calibration Interval Estimation


Current Methods:What Is a Calibration?

Compare the measurement values from a UUT with the measurement values from a calibrator.

– Deviation = UUT Measurement – Calibrator Measurement

A UUT is considered in tolerance if:

– Lower Tolerance < Deviation < Upper Tolerance

Measurement Reliability is the probability of being in tolerance.

A Calibration Interval is the amount of time between calibrations that will meet a measurement reliability target (keeps the UUT in tolerance).


Current Methods:Calibration Interval Determination

0 6 12 18 24 30 36 42 48

Test Equipment ReliabilityTest Equipment Reliabilityvs. Calibration Intervalvs. Calibration Interval

Calibration Interval (Months)

100

90

80

70

60

50

40

30

20

10

0

Mea

sure

men

t R

elia

bili

ty (

%)

72% EOP Reliability for GPTE

85% EOP Reliability for Safety-of-Flight and Mission Critical


Current Methods: Stages of the Calibration Interval

Process

EngineeringInterval Est.

No Further No Further ReviewReview

Gather Relevant

Data

StatisticalInterval Est.

IntegratedInterval

Est?

DivisionDivisionReviewReview

PolicyPolicyReviewReview

TR-6

QA

Yes

METRL

1 2 3 4 5

No


Interval Analysis Results:NAVSEA Interval Changes

INTERVAL ACTIONINTERVAL ACTION COUNTCOUNT

IN PROCESS 148

INITIAL INTERVALS 332

EXTENSIONS 113

DECREASES 24

NO CHANGE 361

TOTAL 978

(FY 2002 through April 2002)


Interval Analysis Results:Annual Calibration Cost Avoidance

NAVSEA NAVY

EXTENSIONS $153K 1918 (M/H)

$372K4644 (M/H)

DECREASES -$40K-495 (M/H)

-$60K-749 (M/H)

COST AVOIDANCE $113K 1423 (M/H)

$312K 3895 (M/H)

(Based on changes made in FY 2002 Through April 2002)


New Approaches to Calibration Interval Estimation

Near Term - Binomial Calibration Interval Estimation Methods– More accurate interval estimates

– Alternative reliability models

– Visual analysis methods

Long Term - Variables Data Calibration Interval Estimation Methods– Fixes data problems

– More information on measurement characteristics

– Less data required

– MEASURE 2 capability with automated data

10

NSWC Corona-MS Interval DJ June 2002

Traditional Reliability Methods

Assumptions: You know when the failure occurs.

R = 1.0 at time 0.Data: Failure Times.

0

0.2

0.4

0.6

0.8

1

Time Since Calibration

Me

as

ure

me

nt

Re

lia

bil

ity Exponential Model:

R = exp(-t)

11


Tolerance Testing Data

Characteristics: • The failure occurs during an interval.• R < 1.0 at time 0.

0

0.2

0.4

0.6

0.8

1


Me

as

ure

me

nt

Re

lia

bil

ity

Note: The points on this graph are observed in tolerance proportions.

12


Using Traditional Methods On Tolerance Testing Data

Problem: The estimates don’t match the data because the intercept must go through 1.0.

0

0.2

0.4

0.6

0.8

1


Me

as

ure

me

nt

Re

lia

bil

ity

13


0

0.2

0.4

0.6

0.8

1


Me

as

ure

me

nt

Re

lia

bil

ity

Reliability Methods For Tolerance Testing Data

Assumptions: The failure occurs during an interval.

R < 1.0 at time 0.Data: Success/Failure (Binomial)

Intercept Exponential ModelR = Ro exp(-t) = exp(0+ 1t)

14


Current Status of Near Term Efforts

2002 MSC Paper: “Calibration Intervals – New Models and Techniques”

– Binomial Analysis, New Models, Reliability Intercepts, Initial Variables Methods

Binomial Calibration Interval Analysis System

15


Benefits of Binomial Calibration Interval Estimation

Methods

The use of Binomial estimation methods provides more accurate calibration interval estimates based on current statistical estimation theory.

Binomial estimation methods allow for alternative measurement reliability models, including intercept and multivariable models.

Better graphical tools provide more understanding of test equipment behavior.

16


Long Term Approach: Variables Calibration Data

-15

-10

-5

0

5

10

15

20

0 3 6 9 12 15 18 21 24


Dev

iati

on

Fro

m S

tan

dar

d

Deviation

Nominal

U Limit

L Limit

17


Calibration Intervals Based on Variables Data

Compute a Drift Trend.

Compute a Variability Trend using residuals from the drift trend.

Obtain a Reliability Curve using the drift and variability trends.

Determine the Calibration Interval from the reliability curve.

Predict the Measurement Uncertainty using the drift and variability trends.

18


Drift Trend Analysis

E(d) = B0 + B1 t (Weighted Linear Regression on d)

-15

-10

-5

0

5

10

15

20

0 3 6 9 12 15 18 21 24


Dev

iati

on

Fro

m S

tan

dar

d

Deviation

Nominal

U Limit

L Limit

Pred Line

19


Variability Trend Analysis

E(res2) = C0 + C1 t (Linear Regression on res2)

-15

-10

-5

0

5

10

15

20

0 3 6 9 12 15 18 21 24


Dev

iati

on

Fro

m S

tan

dar

d Deviation

Nominal

U Limit

L Limit

Pred Line

U Pred

L Pred

20


A Basis for Increasing Variability

Generally, a single serial number does not show increasing variability

-6

-4

-2

0

2

4

6

8

0 10 20 30 40


Dev

iati

on

Fro

m S

tan

dar

d

21



However, several serial numbers could have slightly different slopes and intercepts:

-6

-4

-2

0

2

4

6

8

0 10 20 30 40


Dev

iati

on

Fro

m S

tan

dar

d

22



The overall effect is one of increasing variability for the population

-6

-4

-2

0

2

4

6

8

0 10 20 30 40


Dev

iati

on

Fro

m S

tan

dar

d

23


Reliability Curve Analysis

-20

-15

-10

-5

0

5

10

15

20

0 3 6 9 12 15 18 21 24


Dev

iati

on

Fro

m S

tan

dar

d

Deviation

Nominal

U Limit

L Limit

Pred Line

U Pred

L Pred

Norm 6

Norm 15

24


Determining Calibration Intervals From Variables Data

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 3 6 9 12 15 18 21 24


Mea

sure

men

t R

elia

bili

ty

Reliability Target

Calibration Interval

Reliability Curve

25


Current Statusof Long Term Efforts

2002 MSC Paper: “Calibration Intervals – New Models and Techniques”– Binomial Analysis, New Models, Reliability Intercepts,

Initial Variables Methods

2003 MSC Paper: “Calibration Intervals and Measurement Uncertainty Based on Variables Data”– NPSL, SCE

Variables Analysis Excel Tool– Estimates Trends, Calibration Intervals, Measurement

Uncertainty

MEASURE 2– Automated/Electronic data

26


Benefits of UsingVariables Data

MEASURE data is often suspect– In-Tolerance data is difficult to verify

(success/failure)– Engineering review required for nearly all

calibration interval determinations

Variables data is more trustworthy– This could significantly increase the number of

interval analyses

Variables data provides much more information– Requires fewer calibrations to accurately

determine a calibration interval than In-Tolerance data

Development of automated/electronic data recording could reduce calibration time.

27


Summary

Calibration intervals minimize the amount of calibration effort required to keep test equipment adequately in tolerance.

Recent adjustments to calibration intervals will result in significant cost avoidance.

Near-term improvements using Binomial methods will provide better visual analysis and more accurate estimation techniques.

Long-term improvements using variables data methods will:– Fix data problems– Provide faster analyses with less data– Possibly reduce administrative part of calibration

time

1NSWC Corona-MS Interval DJ June 2002 Dr. Dennis Jackson 909-273-4492 DSN 933-4492...

Documents

Transcript of 1NSWC Corona-MS Interval DJ June 2002 Dr. Dennis Jackson 909-273-4492 DSN 933-4492...