BASIC STATISTICS:AN OXYMORON?

(With a little EPI thrown in…)

URVASHI VAID MD, MSAUG 2012

OUTLINE

• Definition• Why do we need to know “statistics”?– Validity- external and internal– Reliability– Sensitivity– Specificity– Positive and negative predictive values

• ROC curves

STATISTICS

• “the science that deals with the collection, classification, analysis, and interpretation of numerical facts or data, and that, by use of mathematical theories of probability, imposes order and regularity on aggregates of more or less disparate elements”

BUT WHY?

BUT WHY?

• To effectively conduct research• To read and evaluate journals• To develop critical and analytical thinking• To be an informed consumer• To know your limits

Population

SampleTreatment group A

vs Treatment group B

InferenceSampling

POPULATION

Target PopulationEg smokers

Study Population (eligible)

Eg Ages 55-74 yrs,≥ 30 pack years

Study Sample (enrolled)Eg geographic area,

site, consent

Target Population

Study Population

Study Sample

Mean, Median and Mode

Measures of dispersion

• Range• Standard deviation- Bell’s curve• Others

NORMAL DISTRIBUTION

VALIDITY AND RELIABILITY-quality of measurements

VALIDITY AND RELIABILITY

GENERALIZABILITY

• Also “External Validity”• how representative of the target population is

the study population and study sample?• Based on inclusion/exclusion criteria and

standard of care• Where• When• Who (subjects)• What (treatments, outcomes)• How (assessments, data collection)

INTERNAL VALIDITY or no “bias”• Estimated effect in study sample is accurate reflection

of true effect in underlying study population

• Can be in jeopardy due “Systematic Error”– Selection or allocation bias– Information or recall bias or treatment related bias– Confounding– Cross over or lost to follow up

• Deal with this by creating a good study design and statistics

• Also- randomize, blind, quality control, adjust for co-variates

RANDOMIZATION• Balances groups on known and unknown risk

factors in the long run • Provides statistical basis for estimation and

testing • Effort, time, cost• Ethics

– patient wants “best” treatment– physician wants to tailor treatment to individual patient

• Clinical Equipoise?

HYPOTHESIS TESTING

HYPOTHESIS TESTING

HYPOTHESIS TESTING

• H0: Null Hypothesis--There is no association between an exposure and outcome

• H1: Alternative Hypothesis--There is an association between an exposure and outcome

• In general, epidemiologists attempt to reject the null

• One-sided (higher or lower) vs two-sided (different)

H0 True (No Association)

H1 True (There is an Association)

Do Not Reject H0

Correct Decision Type II Error (β)

Reject H0

Type I Error (α) Correct Decision

TRUTHDECISION

• Type I error (false positive, α)probability of rejecting null hypothesis, when it is true

• Type II error (false negative, β)probability of not rejecting null hypothesis, when it is false

• Power (1-β)probability of rejecting null hypothesis, when it is false

POWER AND SAMPLE SIZE

• Ensure study worth doing and feasible• Have to consider

– study design– outcome (distribution, variability, etc.)– hypothesis to be tested (superiority, equivalence, etc.)– statistical analysis to be performed– clinically relevant/meaningful effect size to be detected

• Larger the effect size, smaller the sample size needed

• More subjects more power!

Measures of diagnostic test performance

SENSITIVITY

• A test’s ability to identify diseased individuals• Probability of a positive test among those with

disease

SPECIFICITY

• A test’s ability to rule out disease• Probability of a negative test among those

without disease

SENSITIVITY AND SPECIFICITY ARE FIXED ASPECTS OF A TESTDO NOT CHANGE BASED ON DISEASE PREVALENCE

Disease + Disease - Total

Test + TP FP TP+FPTest - FN TN FN+TNTotal TP+FN FP+TN TP+FP+FN+TN

Sensitivity = True Positive /Those with disease = TP/(TP+FN)

Specificity = True Negative/Those without disease = TN/(TN+FP)

EXAMPLE

• 200 patients• D-dimer positive in 50• CT angio positive in 60• CT angio and d-dimer negative in 120• Construct a 2 X 2 table and calculate

sensitivity and specificity.

PE No PE

D-dimer positive 50

D-dimer negative 120

60 200

PE No PE

D-dimer positive 30 20 50

D-dimer negative 30 120 150

60 140 200

PE No PE

D-dimer positive 30 20 50

D-dimer negative 30 120 150

60 140 200

Sensitivity= TP/TP+FN= 30/30+30=50%

Specificity= TN/TN+FP= 120/120+20= 85%

POSITIVE PREDICTIVE VALUE

• The probability of having disease given a positive test: P(D+|T+)

• ‘How likely is it that this patient has the disease given that the test result is positive?’

• Calculated as: (TRUE POSITIVE)/(TRUE POSITIVE + FALSE POSITIVE)

• Sensitivity is the probability of test positive given disease: P(T+|D+)

Disease + Disease - Total

Test + 26 7 33Test - 97 1701 1798Total 123 1708 1831

What is the PPV?

NEGATIVE PREDICTIVE VALUE

• The probability of not having disease given a negative test: P(D-|T-)

• ‘How likely is it that this patient does not have the disease given that the test result is negative?’

• Calculated as: (TRUE NEGATIVE)/(TRUE NEGATIVE + FALSE NEGATIVE)

• Specificity is the probability of testing negative given no disease: P(T-|D-)

Disease + Disease - Total

Test + 26 7 33Test - 97 1701 1798Total 123 1708 1831

What is the NPV?

• Sensitivity and specificity are characteristics of tests themselves

• Predictive values (positive and negative) are functions of both test characteristics and prevalence

EFFECT OF PREVALENCE

• 4000 people, of which 2000 are ill and 2000 well• A screening test with 99% sensitivity and 99% specificity will

yield 1980 TP and 1980 TN and only 20 each for FP and FN• Hence PPV is 99%.• What if the number of ill people is only 200?• Then well people are 3800 and FP increase to 38 and PPV falls

to 84%

Disease + Disease -

Test + 1980 20 2000

Test - 20 1980 2000

2000 2000 4000

Disease + Disease -

Test + 198 38 236

Test - 2 3762 3764

200 3800 4000

ROC

• Receiver operator characteristic curves are a plot of false positives against true positives for all cut-off values

• The area under the curve of a perfect test is 1.0 and that of a useless test, no better than tossing a coin, is 0.5