Tolerance  USU › jrstevens › biostat › Tolerance.pdf · Tolerance Interval Like a confidence...
Embed Size (px)
Transcript of Tolerance  USU › jrstevens › biostat › Tolerance.pdf · Tolerance Interval Like a confidence...

Tolerance Intervals

Example 1: Asthma Inhaler
How often does an asthma inhaler deliver the right dose in a puff?
Is it enough that it delivers about the right dose on average?
Need to know about individual puffs

Example 2: Mental Health Testing
It is known that the top 10% of scorers on the OQ45 test undergoing a particular therapy are likely to deviate from the recovery track.
A therapist gives his patient the test. How does he know if his patient’s score is in this 10% range?

Example 3: Releasing Land around A Uranium Enrichment Plant for Public Use
A state would like to make some of the land surrounding a uranium enrichment plant available for public use.
Is all of the land safe for public recreational use?

How Do We Determine The Appropriate Cutoff Example 1: FDA sets quality criteria for drug administration
85% coverage with high confidence
If the inhaler cannot meet criteria it cannot be sold
Are the points that mark the middle 85% of the data sufficient?
Example 2: Need to identify 10% that are at risk Is the 90th percentile for test scores sufficient?
Example 3: EPA insists that at least 95% of the land is within a safe range for uranium and uranium isotopes Soil samples collected from the area and analyzed for uranium
Can the 95th percentile be used as the cutoff?

Problem with Raw Percentiles
Would you make a decision about the mean of a drug based solely on the sample mean? Know that ̅ is not accurate
Accuracy is dependent on sample size
How do you know when the mean is too far away from a target to be due to random chance?
Use hypothesis tests to account for this uncertainty Can use confidence intervals to perform a hypothesis test
Allows a quick comparison between the critical value and sample mean

What is a Confidence Interval?
Interval that we are (1α)% confident covers μ
Interval gets smaller as n increases
Provides a range for practical interpretation of the mean
Tells us nothing about individuals in the population

Will A Confidence Interval Work?
All three examples require knowledge about individuals, not averages Example 1: Each puff matters, not the average puff
Example 2: Need to know if my patient is at risk, not the average of everyone at risk
Example 3: How do I know if all, or at least 95%, of the land is safe?
Questions all deal with individuals, not averages
Confidence intervals are not appropriate

Tolerance Interval
Like a confidence interval for individuals
Can cover a certain proportion of the population with a certain degree of confidence
Example: a 99%/95% tolerance interval will include 99% of the population with 95% confidence.
Confidence means the same thing it does with confidence intervals

With a 90%/95% tolerance interval we want to capture 90% of the population with 95% confidence.
If this was the population only these two guys would be left out with 95% confidence.
In example 2 these are the two guys who are likely to deviate from the recovery track

Basic Calculation
̅
/,
1 ,
p is the coverage
γ is the level of confidence
, is the χ2 with n‐1 degrees of freedom that is exceeded with probability γ
All other notation is standard

R Example: Uranium Enrichment Plant
Essential to examine distribution of the data
Central limit theorem cannot help because we are not dealing with averages or sums
This is close enough to work

Test Structure
EPA recreational user limit for uranium = 261 mg/kg
H0: More than 99% of the land in question is less than 261 mg/kg
Ha: Less than 99% of the land in question is less than 261 mg/kg
Compute upper 99%/95% tolerance limit
If the tolerance limit is greater than 261 mg/kg reject H0

R Example: Uranium Enrichment Plant99%/95% Tolerance Interval
Confidence level
Coverage
Onesided interval because we don’t care if there is too little uranium

R Example: Uranium Enrichment Plant99%/95% Tolerance Interval What does it mean?
Upper tolerance limit = 133 mg/kg
EPA recreational user limit for uranium = 261 mg/kg
The area is safe for the public with respect to uranium

SAS Example: Uranium Enrichment Plantdata uranium;
input Uranium @@ ; cards ;112 65.7 39.799.7 64.7 39.694.1 63.6 3982.4 55.8 38.977.7 53.9 37.775.1 51.7 30.573.6 48.3 27.271.8 46.7 21.367.3 ;
proc capability data=uranium noprint;intervals Uranium;
run;
PROC CAPABILITY computes many different coverage intervals including tolerance and confidence intervals

SAS Example: Uranium Enrichment Plant
• Twosided intervals
• Want a onesided interval
• Use 90% confidence to get oneside
• Many different computational methods so results differ slightly from R

What if the Data are Not Normally Distributed Nonparametric tests available
Use maximum observed value
Bootstrap
Tolerance package has intervals for other distributions

Notes on Tolerance Intervals
Research is not as developed as confidence intervals
Software typically uses other methods than the basic formula for computing tolerance intervals
There are many different computational methods just to get the basic intervals assuming normality so results may differ between software packages
Can be very useful, but it is wise to know the quirks and limitations of the intervals when using them

References
EPA (2006), ScreeningLevel Risk Assessment For Recreational Use Of Beaches Upper Columbia River Remedial Investigation and Feasibility Study. Retrieved from http://yosemite.epa.gov/r10/CLEANUP.NSF/82751e55bf4ef18488256ecb00835666/0d26a07060578af988256ecc0075d161/$FILE/Draft%20UCR%20BeachScreenRA.pdf
EPA, 2013, ProUCL Version 5.0.00 Technical Guide, EPA/600/R07/041, U.S. Environmental Protection Agency, September 2013.
Gelo, O, et. al. (2014), Psychotherapy Research: Foundations, Process, and Outcome, New York, NY: SpringerVerlag.
Olsson, B. (2013) Parametric Tolerance Interval (PTI) Test for Improved Control of Delivered Dose Uniformity (DDU) in Orally Inhaled and Nasal Drug Products (OINDP), Retrieved from http://www.fda.gov/ohrms/dockets/ac/03/slides/3926S2_04_Olsson.ppt.