Calibration Fundamentals

Calibration

Fundamentals

An

Introduction to

Metrology

Presented by Jol Rameaux

Service & Support Unit

Agilent Technologies Europe

October 24, 2013

Calibration Fundamentals

1

October 24, 2013


2

Calibration Fundamentals Todays topics

Theres much more beneath the surface !

Contents

Metrology definition Measurements

confidence

What is calibration? Cal comparison Iceberg ahead ! How to calibrate Calibration of Agilent

equipment

Errors and uncertainty Summary

An

Introduction

to Metrology

The Importance of Measurement Metrology is the science of measurement

October 24, 2013


3

British scientist Lord Kelvin (William Thomson

1824-1907) is quoted from his lecture to the

Institution of Civil Engineers, 3 May 1883...

"I often say that when you can measure what you are

speaking about and express it in numbers you know

something about it;

but when you cannot express it in numbers your

knowledge is a meagre and unsatisfactory kind"

You cannot manage what you cannot measure. HP co-founder Bill Hewlett

Metrology : Wiki Definition

October 24, 2013


4

Metrology is the science of measurement.

Metrology includes all theoretical and practical aspects

of measurement. Applied metrology concerns the application of measurement

science to manufacturing and other processes and their use in

society, ensuring the suitability of measurement instruments,

their calibration and quality control of measurements. Although

the emphasis is on the measurements themselves, traceability

of the calibration of the measurement devices is necessary to

ensure confidence in the measurements.

Metrologist : Perform metrology

The Need for Good Measurement

October 24, 2013


5

We all make and use measurements

every day How much fuel will I get? What weight of fruit does the pie need? Will this nut fit the bolt? Does your mobile phone reliably connect to any network? Which drug is most effective? What is the time?

We have expectations

about the correctness of

these measurements

What are Specifications? Define a products expected performance

October 24, 2013


6

An example using a familiar product

Manufacturers view

Frequency: 900 MHz

Channel spacing: 200 kHz

Max power: 2 W

Rate/Modulation: 270 kbps/GMSK

Sensitivity: -97 dBm

etc.

Reason: Conforms with legal criteria (safety, interference); compatible with

operators networks.

Consumers view Reception quality

Reason: Can hear clearly when driving(*) a

car, while in an elevator

or walking through an

underpass.

(*) Via Bluetooth hands-free, of course!

Why Measure?

October 24, 2013


7

We take measurements to get information about the

manufactured or used product so that we can make

good decisions.

Measure the product

to see if it conforms

with specs or

requirements

We have a

good product Yes

But how do we know our

measurement tools are

giving good results?

No

Does it

comply?

Measurement decisions

are made during:

Research Product development Production Final testing Operation

Measurement Confidence

October 24, 2013


8

Measurements are made using a wide variety of

instruments, ranging from very simple to extremely

sophisticated.

We must have confidence that all those measuring

instruments are giving valid, useful data.

Because measured values are the basis of decisions.

That confidence is provided by an

effective program of calibration

Calibration Confidence

October 24, 2013


9

Reference Equipment used during calibration should be

calibrated and should match or exceed accuracy expectation.

Procedures used during calibration should be clear, validated

and repeatable.

Measurement results should be given at minimum as a

Pass/Fail result and preferably with measurement data and

uncertainties associated

Existing Traceability toward reference standards

People should know what they are doing

Preference should be given to automated calibration as it

increases confidence in overall calibration procedure

So, what is calibration ? (1)

October 24, 2013


10

A graphic illustration search Google Images for meal and see the variety

Are they all identical ?

So, what is calibration ? (2) Definition

October 24, 2013


11

Operation that, under specified conditions, in a first step establishes a relation between the quantity values with measurement

uncertainties provided by measurement standards and corresponding

indications with associated measurement uncertainties and, in a second

step, uses this information to establish a relation for obtaining a

measurement result from an indication.

From the ISO/IEC Guide 99 : 2007 -- International Vocabulary of Metrology

Or more simply...

The process of comparing a device with unknown performance to measurement standards traceable to the SI units and with determined

uncertainties to enable obtaining thereafter useful measurement results

from the device.

There is no detail on what should be done during calibration.

Calibration Provides Confidence

October 24, 2013


12

Confidence that a devices past performance was valid and that its future performance will continue to be good within

recommended calibration interval.

Unacceptable past performance may necessitate customer alert and product recall.

Due to drift, measuring equipment needs to be regularly calibrated to contain error to

acceptable limits.

Knowledge of the value of a calibrated parameter becomes less certain as time

elapses since calibration.

Some products require annual optimisation (adjustment) for high confidence of

compliance in future.

Time P

ara

mete

r valu

e

Performance

drift

Calibration &

adjustment

Spec limit

Calibration &

adjustment

Verification, Calibration, Adjustment, Repair

October 24, 2013


13

Specification Limit of normal equipment operation, defined by original manufacturer

Verification Test versus specifications. Result is Pass or Fail. Verification should include a data report with failed points, if any, and possibly measurement uncertainty.

Adjustment Mechanical or electrical tuning of equipment to bring it back within specifications.

Adjustment can be done in case of Failed verification or Passed verification (adjust always equipment). Adjustment should include a as-received data report and a post-

adjusted data report.

Calibration Verification + Adjustment combined (if needed)

Repair If equipment fail verification and cannot be adjusted, then unit is faulty and should be

repaired. Adjustment is not a repair !

Calibration Performance

Real data comparison MXG RF Digital Signal Generator

No

rma

lize

d M

eas

ure

d V

alu

e

Po

wer

An

alo

g

Po

we

r

Dig

ita

l

Ha

rmo

nic

Sp

uri

ou

s

Su

b

Ha

rmo

nic

No

n

Ha

rmo

nic

Dig

ita

l

Mo

du

lati

on

10 M

Hz

Fre

qu

en

cy

W-C

DM

A

Po

we

r

Ou

tpu

t

Po

we

r

Max

Min

Max

Min

Ag

ile

nt

Ca

l N

on

-Ag

ile

nt

Cal

N5182A Dig

ita

l M

od

Po

we

r

820 POINTS TESTED

257 POINTS TESTED

Calibration Performance

Real data comparison MXG RF Digital Signal Generator

No

rma

lize

d M

eas

ure

d V

alu

e

Po

wer

An

alo

g

Po

we

r

Dig

ita

l

Ha

rmo

nic

Sp

uri

ou

s

Su

b

Ha

rmo

nic

No

n

Ha

rmo

nic

Dig

ita

l

Mo

du

lati

on

10 M

Hz

Fre

qu

en

cy

W-C

DM

A

Po

we

r

Ou

tpu

t

Po

we

r

Max

Min

Max

Min

Ag

ile

nt

Ca

l N

on

-Ag

ile

nt

Cal

N5182A Dig

ita

l M

od

Po

we

r

Not measured (long or difficult tests)

Calibration -- The Iceberg Analogy

October 24, 2013


16

The calibration certificate and

measurement data report

What the customer sees

Calibration -- The Iceberg Analogy More to it than meets the eye

October 24, 2013


17

The calibration certificate and

measurement data report

What the customer sees

What the customer doesnt see Calibration procedure, software

Measurement adequacy

Traceability

Equipment recall system

Records

Labelling for identity & status

Interval management

Environment

Supplier evaluation

Technical competence

How to deliver Calibration? (1)

Calibration is verifying instrument specifications by measuring actual performance

using external lab standards [instruments] that in turn have better performance, and

are traceable to International Standard (NIST, NPL, PTB and NMIJ).

Agilent measures the actual

performance of every warranted

specification, for every installed

option, every time using same

procedure and including adjustments

You need :

- Systems/ETEs used as reference

- To follow OEM recommendations

- Appropriate Procedures

- Trained People

- Cal Laboratory

- To archive results

How to deliver Calibration ? (2)

October 24, 2013


19

Use OEM Calibration Labs

Built your own cal lab

Bring Cal Lab on your site(s)

Mix of all above

How to deliver Calibration ? (3)

October 24, 2013


20

TME Software Calibration Platform

Check ETEs (reference) needed

Check ETEs cal due date

Compute uncertainties as a function of

ETEs used

Define tests to be run as a function of configuration (option, frequency, etc), test operator setup

Give Pass/Fail summary and data reports with points out of specs

Print clean data report

Archive data and traceability

Validated by Agilent (operation, data points, uncertainties)

Calibration of Agilent Equipment?

October 24, 2013


21

Automatic Calibration with commercialy available software Reliable & Repeteable Automatic configuration check and uncertainty measurement Most new equipment

Automatic Calibration with proprietary software Reliable & Repeteable Automatic configuration check and uncertainty measurement Older equipment

Manual Calibration Rely on strickly following recommended test procedure Highly operator dependant Very old equipment, no computer interface

No-Calibration required Accessories, cables, adapters

30 %

40 %

20 %

10 %

What Needs Calibrating?

October 24, 2013


22

Any equipment used in incoming, final inspection, in-process or quality control (SPC), product calibration/testing.

Equipment used to perform lengthy or costly machine set-up.

Items whose indications impact upon controlled conditions (e.g. environment).

Any situation where the failure costs are too high (safety, one off testing, etc).

Equipment used in R&D where design decisions are made.

Used during failure or returned material analysis, where critical determinations could be affected (including repair troubleshooting).

Whenever someone acts upon an instrument's reading.

Equipment used for calibration (traceability) of any of the equipment used in the above situations.

In general, all equipment who are used as reference for other items

Error & Uncertainty

October 24, 2013


23

Error is the result of a measured

value minus the expected or

nominal value of the quantity. The

purpose of calibration is to

determine the magnitude of (and

possibly correct for) errors.

Error is by how much we miss the

target (i.e. the red bulls-eye).

The dispersion (spread of shots)

represents repeatability a component of uncertainty. Shots at a target

another illustration of error

Outlier

Average

error

Negative

error

Positive

error

Expected

value Measured

value

Measured

value

Error, Precision & Accuracy

October 24, 2013


24

Accurate and precise

Precise but inaccurate Inaccurate and imprecise

Quite accurate but imprecise

Accurate = True

Precise = Repeatable

Note: In reality we need lots of

data to determine the distribution

(5 shots is not enough).

Tru

e v

alu

e

Tru

e v

alu

e

Tru

e v

alu

e

Tru

e v

alu

e

Bad reference equipment

Bad methodoloy

Bad approach

State of the art

Concepts of Uncertainty Example : How long is a piece of string?

October 24, 2013


25

Contributors

Tape measure

Manufacturing tolerance

Bent or stretched

Temperature / humidity

Graduation resolution

String

Lying straight / flat

Under / Over-stretched

Temperature / humidity

Frayed ends

Method & Personnel

Lining-up string & tape measure

Horizontal or vertical

Repeatability

Eye report on graduation

Concepts of Uncertainty Sources of error & uncertainty

October 24, 2013


26

Measuring equipment

Specifications inc. imported uncertainty (traceability)

Readability

Properties of the thing being measured

Stability

Sensitivity to input change

Readability

Measurement method

Use of correctional data

Configuration

Data processing

Operator skill

Reaction time

Manual dexterity

Visual ability (meter parallax)

Environment

Temperature

Humidity

Altitude / Barometric pressure

EMI

Impedance effects

Mismatch

Loading (e.g. local / remote sensing)

Cables and connections

Torque

Materials

Cleanliness

Mechanical stress

Measurement non-repeatability

Due to properties of the test equipment and unit-under-test and also the variable influences

of test environment and operator.

Mistakes !

Cant be accounted for but can be statistically identified (outliers).

Uncertainty Example : Power sensor cal factor

October 24, 2013


27

Erse

te

Mi

Mt

Ms

MiKsKsKb

2

2

))(1(

))(1(

2

1)(

Measurement equation

Uncertainty calculation for a device

October 24, 2013


28

One spreadsheet/calculation to be created for each measurement setup and adjusted (uncertainty) for each measured point

-> Give uncertainty per measured point

-> Imply Pass/Fail criteria

Same analysis to be done for each measurement point

Same analysis to be done for each setup (alternate replacement)

Remember the MXG : 820 measurement points for 10 performance

setup ! Not including alternate ETEs analysis.

-> This analysis and validation is the pure role of the metrologist

Basic Model for Product Specification Where does uncertainty fit in?

October 24, 2013


29

Mean Production line

test limit

Production

process

distribution

(manufacturing

reproducibility)

Production

margin

Calibration

uncertainty

Delta environment

and drift

Specification

Guardband

Time

Para

mete

r valu

e

Performance

drift

Calibration &

adjustment

Spec limit

Calibration Intervals

October 24, 2013


30

May be expressed as a calendar period, usually months or years

between calibrations.

Or might be an interval of usage time such as 1000 hours of operation.

Or possibly the number of usage operations, especially mechanical

items subject to degradation through

wear.

Although a manufacturer may recommend an interval (the spec

implicitly has time dependency

owing to expected drift), the

equipment user is responsible for

assigning the appropriate

interval. Most of users follow

manufacturers recommended CI

Aim is to balance the cost of calibration (how often and the

extent of the testing) with the

risk of bad measurement (cost

of product recall, safety, etc.)

Better do a complete calibration every 3 years than a bad

calibration every year

Calibration Summary What you need to remember

October 24, 2013


31

Only manufacturer can define specs and what need to be tested.

All calibration are not equivalent. Compare what is really done to have a complete picture : function, number of points and accuracy.

Calibration certify past performance was correct and future performance is expected to be good

Some calibrations can be done by yourselves, some cannot.

Agilent is ready to help, in performing on-Site calibration for reduced down time and minimize risk in shipment

You can tune calibration interval. It is your decision, but you should be able to justify in front of auditors.

Metrology is really a specific and full time job reserved to specialists

Any question : [email protected] or contact Boris at IBIS

Calibration Summary

October 24, 2013


32

Questions ?

Calibration Summary

October 24, 2013


33

Thanks for your attention !

Metrological [measurement] Traceability Definition

October 24, 2013


34

Property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of

calibrations, each contributing to the measurement uncertainty.

From the ISO/IEC Guide 99 : 2007 -- International Vocabulary of Metrology

Or more simply as...

A series of calibrations of known accuracy that link measurements to recognised standards through an

unbroken chain of comparisons.

The need for records is to link a product or service event to a process, status or time.

If the process is measurement, detail about the equipment, procedure, test conditions and personnel are

all relevant to demonstrating traceability.

Traceability International System of Units (SI units)

October 24, 2013


35

http://physics.nist.gov/cuu/Units/index.html

Every measurement

is linked to one or

more of just 7 base

units !

The kilogramme is the

only unit defined by a

prototype (a specific physical item held at

BIPM in Paris, France).

Definitions (at NIST site)

Measurement Traceability A representation of the national measurement system

October 24, 2013


36

NATIONAL

METROLOGY INSTITUTE

STANDARDS

LABORATORY

CAL FACILITY

MANUFACTURING

PRODUCT SPEC

Uncertainty is the potential inaccuracy.

The levels do not represent consistent values of uncertainty.

Test uncertainty ratio (TUR) is the ratio of the unit-under-test's

specification to the measurement uncertainty.

Spec = 15%

TUR 5:1

Unc = 3%

TUR 10:1

Unc = 0.3%

TUR 3:1

Unc = 0.1%

TUR 2.5:1

Unc =

0.04%

NATIONAL

REALISATION OF

SI DEFINITION Unc =

0.008% International

comparisons

via BIPM

International

comparisons

via regional

accreditation

organisation

Another

stds lab

Another

cal lab And another

Another

production line

Another

factory And another

Traceability Why Bother? Summary

Common measurement references are critical to the world-wide

exchange and acceptance of

products, services and technology.

The aim is to transfer these references in a controlled manner

to the billions of individual

measurements made daily.

Traceable measurements ensure uniformity of manufactured goods

and industrial processes.

It supports equity in trade, compliance to regulatory laws and

standards.

If the calibration paper trail was followed from the unit under test up

through the working, reference and

primary standards, it would

eventually end at the record of an

experiment made to establish the

quantity in terms of one or more of

the SI base units.

It is essential to science and the development of technology.

October 24, 2013 37


Recommended Resources (Free) Find out more

October 24, 2013


38

NPL Good Practice On-line Modules on:

Measurement http://www.npl.co.uk/publications/good-practice-online-modules/measurement/

Uncertainty http://www.npl.co.uk/publications/good-practice-online-modules/measurement-uncertainty/

NPL -- A Beginner's Guide to Uncertainty of Measurement http://resource.npl.co.uk/cgi-bin/download.pl?area=npl_publications&path_name=/npl_web/pdf/mgpg11.pdf

UKAS M3003 -- The Expression of Uncertainty and Confidence in

Measurement http://www.ukas.com/library/Technical-Information/Pubs-Technical-Articles/Pubs-List/M3003.pdf

In or Out-of-Spec?

October 24, 2013


39

But it doesnt define how.

Most accreditation bodies (all of Europes) stipulate ILAC-G8 which requires conformity limits are set at spec minus uncertainty.

The US national implementation of 17025 is ANSI/NCSL-Z540.3 which

accepts a producer/consumer shared-risk approach:

The probability of false acceptance associated with any test point labelled Pass shall not exceed 2 %.

Learn more at: http://www.agilent.com/metrology/ncsli2009_dobbert.shtml

Measured values

with plus and minus

uncertainty bars

Ideal

value

Specification

limit A

B C

D E

ISO/IEC 17025 clause 5.10.4.2 :

When statements of compliance [with a specification] are made, the

uncertainty of measurement shall be

taken into account.

A= Pass

B,C,D = Indeterminate

(Agilent treats D as Fail)

E= Fail

International Laboratory Accreditation Cooperation (ILAC) G8 : Guidelines on the Reporting of Compliance with Specification

Calibration Fundamentals

Documents

Transcript of Calibration Fundamentals