Tolerance Intervals

Example 1: Asthma Inhaler

How often does an asthma inhaler deliver the right dose in a puff?

Is it enough that it delivers about the right dose on average?

Need to know about individual puffs

Example 2: Mental Health Testing

It is known that the top 10% of scorers on the OQ-45 test undergoing a particular therapy are likely to deviate from the recovery track.

A therapist gives his patient the test. How does he know if his patient’s score is in this 10% range?

Example 3: Releasing Land around A Uranium Enrichment Plant for Public Use

A state would like to make some of the land surrounding a uranium enrichment plant available for public use.

Is all of the land safe for public recreational use?

How Do We Determine The Appropriate Cut-off Example 1: FDA sets quality criteria for drug administration

85% coverage with high confidence

If the inhaler cannot meet criteria it cannot be sold

Are the points that mark the middle 85% of the data sufficient?

Example 2: Need to identify 10% that are at risk Is the 90th percentile for test scores sufficient?

Example 3: EPA insists that at least 95% of the land is within a safe range for uranium and uranium isotopes Soil samples collected from the area and analyzed for uranium

Can the 95th percentile be used as the cut-off?

Problem with Raw Percentiles

Would you make a decision about the mean of a drug based solely on the sample mean? Know that is not accurate

Accuracy is dependent on sample size

How do you know when the mean is too far away from a target to be due to random chance?

Use hypothesis tests to account for this uncertainty Can use confidence intervals to perform a hypothesis test

Allows a quick comparison between the critical value and sample mean

What is a Confidence Interval?

Interval that we are (1-α)% confident covers μ

Interval gets smaller as n increases

Provides a range for practical interpretation of the mean

Tells us nothing about individuals in the population

Will A Confidence Interval Work?

All three examples require knowledge about individuals, not averages Example 1: Each puff matters, not the average puff

Example 2: Need to know if my patient is at risk, not the average of everyone at risk

Example 3: How do I know if all, or at least 95%, of the land is safe?

Questions all deal with individuals, not averages

Confidence intervals are not appropriate

Tolerance Interval

Like a confidence interval for individuals

Can cover a certain proportion of the population with a certain degree of confidence

Example: a 99%/95% tolerance interval will include 99% of the population with 95% confidence.

Confidence means the same thing it does with confidence intervals

With a 90%/95% tolerance interval we want to capture 90% of the population with 95% confidence.

If this was the population only these two guys would be left out with 95% confidence.

In example 2 these are the two guys who are likely to deviate from the recovery track

Basic Calculation

/,

1 ,

p is the coverage

γ is the level of confidence

, is the χ2 with n‐1 degrees of freedom that is exceeded with probability γ

All other notation is standard

R Example: Uranium Enrichment Plant

Essential to examine distribution of the data

Central limit theorem cannot help because we are not dealing with averages or sums

This is close enough to work

Test Structure

EPA recreational user limit for uranium = 261 mg/kg

H0: More than 99% of the land in question is less than 261 mg/kg

Ha: Less than 99% of the land in question is less than 261 mg/kg

Compute upper 99%/95% tolerance limit

If the tolerance limit is greater than 261 mg/kg reject H0

R Example: Uranium Enrichment Plant99%/95% Tolerance Interval

Confidence level

Coverage

One-sided interval because we don’t care if there is too little uranium

R Example: Uranium Enrichment Plant99%/95% Tolerance Interval What does it mean?

Upper tolerance limit = 133 mg/kg

EPA recreational user limit for uranium = 261 mg/kg

The area is safe for the public with respect to uranium

SAS Example: Uranium Enrichment Plantdata uranium;

input Uranium @@ ; cards ;112 65.7 39.799.7 64.7 39.694.1 63.6 3982.4 55.8 38.977.7 53.9 37.775.1 51.7 30.573.6 48.3 27.271.8 46.7 21.367.3 ;

proc capability data=uranium noprint;intervals Uranium;

run;

PROC CAPABILITY computes many different coverage intervals including tolerance and confidence intervals

SAS Example: Uranium Enrichment Plant

• Two-sided intervals

• Want a one-sided interval

• Use 90% confidence to get one-side

• Many different computational methods so results differ slightly from R

What if the Data are Not Normally Distributed Non-parametric tests available

Use maximum observed value

Bootstrap

Tolerance package has intervals for other distributions

Notes on Tolerance Intervals

Research is not as developed as confidence intervals

Software typically uses other methods than the basic formula for computing tolerance intervals

There are many different computational methods just to get the basic intervals assuming normality so results may differ between software packages

Can be very useful, but it is wise to know the quirks and limitations of the intervals when using them

References

EPA (2006), Screening-Level Risk Assessment For Recreational Use Of Beaches Upper Columbia River Remedial Investigation and Feasibility Study. Retrieved from http://yosemite.epa.gov/r10/CLEANUP.NSF/82751e55bf4ef18488256ecb00835666/0d26a07060578af988256ecc0075d161/$FILE/Draft%20UCR%20Beach-Screen-RA.pdf

EPA, 2013, ProUCL Version 5.0.00 Technical Guide, EPA/600/R-07/041, U.S. Environmental Protection Agency, September 2013.

Gelo, O, et. al. (2014), Psychotherapy Research: Foundations, Process, and Outcome, New York, NY: Springer-Verlag.

Olsson, B. (2013) Parametric Tolerance Interval (PTI) Test for Improved Control of Delivered Dose Uniformity (DDU) in Orally Inhaled and Nasal Drug Products (OINDP), Retrieved from http://www.fda.gov/ohrms/dockets/ac/03/slides/3926S2_04_Olsson.ppt.