Some help understanding and interpreting medical...
Transcript of Some help understanding and interpreting medical...
Some help understanding and interpreting
medical statistics …
John Norrie
Centre for Healthcare Randomised Trials (CHaRT)
University of Aberdeen, Scotland
Barcelona, June, 2013
www.eurordis.org
4
Acknowledgment
Thanks to Ferran Torres, Josep Torrent y Farnell and Julia Saperia for creating the previous versions of this workshop and providing the material that this version is based on.
Understanding and Interpreting medical statistics
5
Outline
• Why statistics
• Descriptive statistics
• Population and samples
• Hypothesis Tests and P-values
• Statistical “vs” clinical significance
• Statistical errors (a and b)
• Sample size
• Estimation of treatment effect: Confidence Intervals
• A worked example
Understanding and Interpreting medical statistics
6
What does statistics mean to you?
mean
information
percentage
variability data
median
average
graphs
measurement
probability
confidence
evidence
spread
Understanding and Interpreting medical statistics
7
Why Statistics?
• Statistics is the science of uncertainty – it is all about
investigating, understanding, and allowing for VARIABILITY
Understanding and Interpreting medical statistics
8
Variability
Understanding and Interpreting medical statistics
9
Why Statistics?
• Medicine is a quantitative science but not exact
– Not like physics or chemistry
• Variation characterises much of medicine
• Statistics is about handling and quantifying variation and
uncertainty
• Humans differ in response to exposure to adverse effects
Example: not every smoker dies of lung cancer
some non-smokers die of lung cancer
• Humans differ in response to treatment
Example: penicillin does not cure all infections
• Humans differ in disease symptoms Example: Sometimes cough and sometimes wheeze are presenting features for
asthma
Understanding and Interpreting medical statistics
10
Why Statistics Are Necessary
• Statistics can tell us whether events could have happened by
chance and to make decisions
• We need to use Statistics because of variability in our data
•Generalize: can what we know help to predict what will
happen in new and different situations?
Understanding and Interpreting medical statistics
11
Probability
• Probability is the methodological ‘glue’ that holds statistics together
• Given uncertainty is at the heart of statistical thinking, probability is our way of quantifying how likely various things that may happen will take place
• For example, a fair coin is tossed – in the long run, we would expect to get the same number of ‘heads’ as ‘tails’
If we threw the coin 10 times, intuitively we would think it much less likely to get 10 heads in a row (or for that matter ten tails) than say 5 heads and 5 tails (in any order) …
Why? Because there are lots of ways of getting 5 heads + 5 tails, only one way of getting 10 heads and no tails ….
• Probability is challenging – consider conditional probability …
Understanding and Interpreting medical statistics
12
draw a
sample
Population
sample
sample data
Inferential statistics
Draw conclusions for the
whole population based
on information gained
from a sample
Target of study design
representative sample
= ~ all "typical"
representatives are
included
Descriptive
statistics
(a population is a set of all
conceivable observations of a
certain phenomenon)
Understanding and Interpreting medical statistics
13
Population and Samples
Target Population
Population of the Study
Sample
Understanding and Interpreting medical statistics
14
Descriptive statistics (1)
• For continuous variables (e.g. height, blood pressure,
body mass index, blood glucose)
•Histograms
•Measures of location
–mean, median, mode …
•Measures of spread
–variance, standard deviation, range, interquartile
range, minimum, maximum
Understanding and Interpreting medical statistics
15
The histogram
•Taken from http://www.mathsisfun.com/data/histograms.html on 28 May 2012
Understanding and Interpreting medical statistics
16
Johann Carl Friedrich Gauss 1777-1855
68,3%
~95,5%
~99%,
mean mean + sd mean + 2sd mean - sd mean - 2sd
Picturing “normal” data
Understanding and Interpreting medical statistics
17
Descriptive statistics (2)
• For categorical variables (e.g. sex, dead/alive,
smoker/non-smoker, blood group)
•Bar charts, (pie charts)…
• Frequencies, percentages, proportions
–when using a percentage, always state the number it
is based on
–e.g. 68% (n=279)
–or perhaps 190/279 (68%)
Understanding and Interpreting medical statistics
18
Memorable quotes
• 50% of what you learn about therapy in the next 5 years is wrong.
(The trouble is we don’t know which 50%) (Anon)
• “…in this world there is nothing certain but death and taxes.” Benjamin Franklin (1706-1790). (also said by Woody Allen)
• 86% of all statistics are invented on the spot (Huff – How to Lie with Statistics)
• “There are lies, damn lies, and statistics” Benjamin Disraeli (1804-1881)
• And now we add ‘… lies, damned lies, statistics, and government
statistics’ !!
Understanding and Interpreting medical statistics
19
Error ? …
Unreliable
or Imprecise
Reliable (precise) but
not valid (accurate)
Reliable
& valid
Understanding and Interpreting medical statistics
20
130 150 170
01 02 03 04 05
True Value
Random vs Systematic error
Random Systematic (Bias)
130 150 170
01 05
02 03
04
True Value
Example: Systolic Blood Pressure (mm Hg)
Understanding and Interpreting medical statistics
21
Valid samples?
Population
Likely to occur
Unlikely to occur Invalid Sample and Conclusions
Understanding and Interpreting medical statistics
22
HYPOTHESIS TESTING
• Testing two hypotheses H0: A=B (Null hypothesis – no difference)
H1: A≠B (Alternative hypothesis)
•Calculate test statistic based on the assumption that H0 is true (i.e. there is no real difference)
• Test will give us a p-value: how likely are the collected data if H0 is true
• If this is unlikely (small p-value), we reject H0
Understanding and Interpreting medical statistics
23 1 23
P-value
• The p-value is a “tool” to answer the question:
Could the observed results have occurred by chance*?
Remember:
– Decision given the observed results in a SAMPLE
– Extrapolating results to POPULATION
*: accounts exclusively for the random error, not bias
p < .05
“statistically significant”
Understanding and Interpreting medical statistics
24
A intuitive definition
• The p-value is the probability of having observed our data when
the null hypothesis is true
• Steps:
1) Calculate the treatment differences in the sample (A-B)
2) Assume that both treatments are equal (A=B) and then…
3) …calculate the probability of obtaining a magnitude of at least the
observed differences, given the assumption 2
4) We conclude according the probability:
a. p<0.05: the differences are unlikely to be explained by random,
we assume that the treatment explains the differences
b. p>0.05: the differences could be explained by random,
we assume that random explains the differences
Understanding and Interpreting medical statistics
25
P-value
•A “very low” p-value do NOT imply:
Clinical relevance (NO!!!)
Magnitude of the treatment effect (NO!!)
With n or variability
p
•Please never compare p-values!! (NO!!!)
Understanding and Interpreting medical statistics
26
P-value
•A “statistically significant” result
(p<.05)
tells us NOTHING about clinical or scientific
importance. Only, that the results were
unlikely to be due to chance.
A p-value does NOT account for bias
only for random error
STAT REPORT
Understanding and Interpreting medical statistics
27
Significance level
• p-values are compared to a pre-specified significance level, alpha
• alpha is usually 5% (analogous to 95% confidence intervals)
if p ≤ 0.05 reject the null hypothesis (i.e., the result is statistically significant
at the 5% level) conclude that there is a difference in treatments
if p > 0.05 do not reject the null hypothesis conclude that there is not
sufficient information to reject the null hypothesis
(See Statistics notes: Absence of evidence is not evidence of absence:
BMJ 1995;311:485)
Understanding and Interpreting medical statistics
28
Type I & II Error & Power
Reality
(Population)
A=B A≠B
Conclusion
(sample)
“A=B” p>0.05 OK Type II error
(b)
A≠B p<0.05 Type I error
(a) OK
Understanding and Interpreting medical statistics
29
Type I & II Error & Power
• Type I Error (a)
False positive
Rejecting the null hypothesis when in fact it is true
Standard: a=0.05
In words, chance of finding statistical significance when in fact there truly
was no effect
• Type II Error (b)
False negative
Accepting the null hypothesis when in fact alternative is true
Standard: b=0.20 or 0.10
In words, chance of not finding statistical significance when in fact there
was an effect
Understanding and Interpreting medical statistics
30
Type I & II Error & Power
• Power
1-Type II Error (b)
Usually in percentage: 80% or 90% (for b =0.1 or 0.2, respectively)
In words, chance of finding statistical significance when in fact there is an
effect
Reality
(Population)
A=B A≠B
Conclusion
(sample)
“A=B” p>0.05 OK Type II error
(b)
A≠B p<0.05 Type I error
(a) POWER
Understanding and Interpreting medical statistics
31
Which sorts of error can occur when performing statistical tests?
H0 H1
H0 Correct
1-a
False negative
b
H1 False positive
a
Correct
1-b
Test
deci
sion
Hypoth
esi
s acc
epte
d
True state of nature
Significance level a is usually predetermined in the study protocol, e.g.,
a=0.05 (5%, 1 in 20), a= 0.01 (1%, 1 in 100), a= 0.001 (0.1%, 1 in 1000)
POWER 1- b : The power of a statistical test is the probability that the test will reject
a false null hypothesis.
Statistical mistakes/errors
Understanding and Interpreting medical statistics
32
Compare a statistical test to a court of justice
H0 Innocent
H1 Not Innocent
H0
Correct
1-a
False negative (i.e. guilty but not caught)
b
H1 False positive (i.e. innocent but convicted)
a
Correct
1-b
Test
deci
sion
Deci
sion o
f th
e c
ourt
True state of nature
False positive: A court finds a person guilty of a crime that they did not actually commit.
False negative: A court finds a person not guilty of a crime that they did commit.
Judged
"innocent"
Judged
“Not in
nocent"
Statistical tests vs court of law
Understanding and Interpreting medical statistics
33
Minimising statistical errors
Remember:
How do I gain adequate data?
Thorough planning of studies
• Defining acceptable levels of statistical error is key to the planning
of studies
• alpha (in clinical trials) is pre-defined by regulatory guidance
(usually)
• beta is not, but deciding on the power (1-beta) of the study is
crucial to enrolling sufficient patients
• the power of a study is usually chosen to be 80% or 90%
• conducting an “underpowered” study is not ethically acceptable
because you know in advance that your results will be inconclusive
Understanding and Interpreting medical statistics
34
Sample Size
• The planned number of participants is calculated on the basis
of:
Expected effect of treatment(s)
Variability of the chosen endpoint
Accepted risks in conclusion
↗ effect ↘ number
↗ variability ↗ number
↗ risk ↘ number
Understanding and Interpreting medical statistics
35
Sample Size
• The planned number of participants is calculated on the basis
of:
Expected effect of treatment(s)
Variability of the chosen endpoint
Accepted risks in conclusion
↗ effect ↘ number
↗ variability ↗ number
↗ risk ↘ number
ALTURA
202.5
197.5
192.5
187.5
182.5
177.5
172.5
167.5
162.5
157.5
152.5
147.5
142.5
137.5
132.5
127.5
122.5
ALTURA
Fre
cu
en
cia
300
200
100
0
Desv. típ. = 25.54
Media = 165.1
N = 2000.00
ALTURA
220.0
210.0
200.0
190.0
180.0
170.0
160.0
150.0
140.0
130.0
120.0
110.0
ALTURA
Fre
cu
en
cia
300
200
100
0
Desv. típ. = 26.94
Media = 165.0
N = 2000.00
ALTURA
250.0
240.0
230.0
220.0
210.0
200.0
190.0
180.0
170.0
160.0
150.0
140.0
130.0
120.0
110.0
100.090.0
80.0
ALTURA
Fre
cu
en
cia
120
100
80
60
40
20
0
Desv. típ. = 32.27
Media = 165.1
N = 2000.00
Understanding and Interpreting medical statistics
36
Sample Size
• The planned number of participants is calculated on the basis
of:
Expected effect of treatment(s)
Variability of the chosen endpoint
Accepted risks in conclusion
↗ effect ↘ number
↗ variability ↗ number
↗ risk ↘ number
Reality
(Population)
A=B A≠B
Conclusion
(sample)
“A=B” p>0.05 OK Type II error
(b)
A≠B p<0.05 Type I error
(a) POWER
Understanding and Interpreting medical statistics
37
Deciding how many patients to enrol
• the sample size calculation depends on:
– the clinically relevant effect that is expected (1)
– the amount of variability in the data that is expected (2)
– the significance level at which you plan to test (3)
– the power that you hope to achieve in your study (4)
• if you knew (1) and (2), you wouldn’t need to conduct a study
picking your sample size is a gamble
• the smaller the treatment effect, the more patients you need
• the more variable the treatment effect, the more patients you need
• the smaller the risks (or statistical errors) you’re prepared to take,
the more patients you need
Understanding and Interpreting medical statistics
38
95% (or 99%) Confidence Intervals
• Confidence Intervals are ‘better’ than p-values…
This is statistical estimation rather than hypothesis testing
…use the data collected in the trial to give an estimate of the treatment
effect size, together with a measure of how certain we are of our
estimate
• CI is a range of values within which the “true” treatment effect is believed to be found, with a given level of confidence.
95% CI is a range of values within which the ‘true’ treatment effect will lie 95% of the time
A 99% CI is wider than a 95% CI …
• Generally, 95% CI is calculated as
Sample Estimate ± 1.96 x Standard Error
Understanding and Interpreting medical statistics
39
Interval Estimation
Confidence
interval
Sample statistic
(point estimate)
Confidence
limit (lower)
Confidence
limit (upper)
A probability that the population parameter
falls somewhere within the interval.
Understanding and Interpreting medical statistics
40
Superiority study
d > 0 + effect
IC95%
d = 0 No differences
d < 0 - effect
Test better Control better
Understanding and Interpreting medical statistics
41
Superiority study
d > 0 + effect
IC95%
d = 0 No differences
d < 0 - effect
Test better Control better
Understanding and Interpreting medical statistics
42
Standard Deviation
• The Standard Deviation is a measure of how spread
out numbers are.
• Its symbol is σ (the greek letter sigma)
• The formula is easy: it is the square root of the
Variance
The average of the squared differences from the Mean.
Understanding and Interpreting medical statistics
43
Calculating a variance
To calculate the variance follow these steps:
• Work out the Mean (the simple average of the numbers)
• Then for each number: subtract the Mean
• and then square the result (the squared difference).
• Then work out the average of those squared differences.
Understanding and Interpreting medical statistics
44
Example
You and your friends have just measured the heights of your dogs (in
millimeters):
The heights (at the shoulders) are: 600mm, 470mm, 170mm, 430mm and 300mm.
Understanding and Interpreting medical statistics
45
First calculate the mean (or average) …
Mean = 600 + 470 + 170 + 430 + 300 =
1970 = 394
5 5
Understanding and Interpreting medical statistics
46
Next calculate the variance,
and then the standard deviation
To calculate the Variance, take each difference, square it, and then average the result:
So, the Variance is 21,704.
And the Standard Deviation is just the square root of Variance, so:
Standard Deviation: σ = √21,704 = 147.32... = 147 (to the nearest mm)
Understanding and Interpreting medical statistics
47
First calculate the mean (or average) …
And the good thing about the Standard Deviation is that it is useful.
Now we can show which heights are within one Standard Deviation (147mm) of the
Mean:
So, using the Standard Deviation we have a "standard" way of knowing what is
normal, and what is extra large or extra small.
Rottweilers are tall dogs. And Dachshunds are a bit short ...
Understanding and Interpreting medical statistics
48
Further reading
• www.consort-statement.org
standards for reporting clinical trials in the
literature • Statistical Principles for Clinical Trials ICH E9
useful glossary • http://openwetware.org/wiki/BMJ_Statistics_Notes_series
coverage of a number of topics related to
statistics in clinical research, mostly by Douglas
Altman and Martin Bland
Understanding and Interpreting medical statistics