Tolerance - USU › jrstevens › biostat › Tolerance.pdf · Tolerance Interval Like a confidence...

of 21 /21
Tolerance Intervals

Embed Size (px)

Transcript of Tolerance - USU › jrstevens › biostat › Tolerance.pdf · Tolerance Interval Like a confidence...

  • Tolerance Intervals

  • Example 1: Asthma Inhaler

    How often does an asthma inhaler deliver the right dose in a puff?

    Is it enough that it delivers about the right dose on average?

    Need to know about individual puffs

  • Example 2: Mental Health Testing

    It is known that the top 10% of scorers on the OQ-45 test undergoing a particular therapy are likely to deviate from the recovery track.

    A therapist gives his patient the test. How does he know if his patient’s score is in this 10% range?

  • Example 3: Releasing Land around A Uranium Enrichment Plant for Public Use

    A state would like to make some of the land surrounding a uranium enrichment plant available for public use.

    Is all of the land safe for public recreational use?

  • How Do We Determine The Appropriate Cut-off Example 1: FDA sets quality criteria for drug administration

    85% coverage with high confidence

    If the inhaler cannot meet criteria it cannot be sold

    Are the points that mark the middle 85% of the data sufficient?

    Example 2: Need to identify 10% that are at risk Is the 90th percentile for test scores sufficient?

    Example 3: EPA insists that at least 95% of the land is within a safe range for uranium and uranium isotopes Soil samples collected from the area and analyzed for uranium

    Can the 95th percentile be used as the cut-off?

  • Problem with Raw Percentiles

    Would you make a decision about the mean of a drug based solely on the sample mean? Know that ̅ is not accurate

    Accuracy is dependent on sample size

    How do you know when the mean is too far away from a target to be due to random chance?

    Use hypothesis tests to account for this uncertainty Can use confidence intervals to perform a hypothesis test

    Allows a quick comparison between the critical value and sample mean

  • What is a Confidence Interval?

    Interval that we are (1-α)% confident covers μ

    Interval gets smaller as n increases

    Provides a range for practical interpretation of the mean

    Tells us nothing about individuals in the population

  • Will A Confidence Interval Work?

    All three examples require knowledge about individuals, not averages Example 1: Each puff matters, not the average puff

    Example 2: Need to know if my patient is at risk, not the average of everyone at risk

    Example 3: How do I know if all, or at least 95%, of the land is safe?

    Questions all deal with individuals, not averages

    Confidence intervals are not appropriate

  • Tolerance Interval

    Like a confidence interval for individuals

    Can cover a certain proportion of the population with a certain degree of confidence

    Example: a 99%/95% tolerance interval will include 99% of the population with 95% confidence.

    Confidence means the same thing it does with confidence intervals

  • With a 90%/95% tolerance interval we want to capture 90% of the population with 95% confidence.

    If this was the population only these two guys would be left out with 95% confidence.

    In example 2 these are the two guys who are likely to deviate from the recovery track

  • Basic Calculation



    1 ,

    p is the coverage

    γ is the level of confidence

    , is the χ2 with n‐1 degrees of freedom that is exceeded with probability γ

    All other notation is standard

  • R Example: Uranium Enrichment Plant

    Essential to examine distribution of the data

    Central limit theorem cannot help because we are not dealing with averages or sums

    This is close enough to work

  • Test Structure

    EPA recreational user limit for uranium = 261 mg/kg

    H0: More than 99% of the land in question is less than 261 mg/kg

    Ha: Less than 99% of the land in question is less than 261 mg/kg

    Compute upper 99%/95% tolerance limit

    If the tolerance limit is greater than 261 mg/kg reject H0

  • R Example: Uranium Enrichment Plant99%/95% Tolerance Interval

    Confidence level


    One-sided interval because we don’t care if there is too little uranium

  • R Example: Uranium Enrichment Plant99%/95% Tolerance Interval What does it mean?

    Upper tolerance limit = 133 mg/kg

    EPA recreational user limit for uranium = 261 mg/kg

    The area is safe for the public with respect to uranium

  • SAS Example: Uranium Enrichment Plantdata uranium;

    input Uranium @@ ; cards ;112 65.7 39.799.7 64.7 39.694.1 63.6 3982.4 55.8 38.977.7 53.9 37.775.1 51.7 30.573.6 48.3 27.271.8 46.7 21.367.3 ;

    proc capability data=uranium noprint;intervals Uranium;


    PROC CAPABILITY computes many different coverage intervals including tolerance and confidence intervals

  • SAS Example: Uranium Enrichment Plant

    • Two-sided intervals

    • Want a one-sided interval

    • Use 90% confidence to get one-side

    • Many different computational methods so results differ slightly from R

  • What if the Data are Not Normally Distributed Non-parametric tests available

    Use maximum observed value


    Tolerance package has intervals for other distributions

  • Notes on Tolerance Intervals

    Research is not as developed as confidence intervals

    Software typically uses other methods than the basic formula for computing tolerance intervals

    There are many different computational methods just to get the basic intervals assuming normality so results may differ between software packages

    Can be very useful, but it is wise to know the quirks and limitations of the intervals when using them

  • References

    EPA (2006), Screening-Level Risk Assessment For Recreational Use Of Beaches Upper Columbia River Remedial Investigation and Feasibility Study. Retrieved from$FILE/Draft%20UCR%20Beach-Screen-RA.pdf

    EPA, 2013, ProUCL Version 5.0.00 Technical Guide, EPA/600/R-07/041, U.S. Environmental Protection Agency, September 2013.

    Gelo, O, et. al. (2014), Psychotherapy Research: Foundations, Process, and Outcome, New York, NY: Springer-Verlag.

    Olsson, B. (2013) Parametric Tolerance Interval (PTI) Test for Improved Control of Delivered Dose Uniformity (DDU) in Orally Inhaled and Nasal Drug Products (OINDP), Retrieved from