Abc4
-
Upload
jonas-ranstam -
Category
Documents
-
view
84 -
download
0
Transcript of Abc4
![Page 1: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/1.jpg)
The ABC of statistics
Jonas Ranstam
![Page 2: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/2.jpg)
![Page 3: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/3.jpg)
![Page 4: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/4.jpg)
A scientific report
The idea is to try and give all the information to help others to judge the value of your contributions, not just the information that leads to judgment in one particular direction or another.
Richard P. Feynman
![Page 5: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/5.jpg)
![Page 6: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/6.jpg)
Uncertainty is ubiquitous
1. Random variation (precision)
a) measurement errors
b) sampling variation
2. Systematic deviation (validity)
a) selection bias
b) information bias
c) confounding bias
![Page 7: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/7.jpg)
![Page 8: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/8.jpg)
Random variation
Measurement errors
How uncertain is a body weight measurement?
![Page 9: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/9.jpg)
Error distributionVariation in observed body weight during 133 consecutive daily measurements (residuals, detrended using lowess)
Deviation from mean value (kg)
![Page 10: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/10.jpg)
How uncertain is an observed weight of 77kg?
Degree of uncertainty
68.0% ±1.5kg 95.0% ±3.0kg99.7% ±4.5kg
![Page 11: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/11.jpg)
How uncertain is a weight change, between two consecutive measurements?
Degree of uncertainty
68.0% ±2.1kg 95.0% ±4.2kg 99.7% ±6.4kg
![Page 12: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/12.jpg)
Random variation
Sampling variation
How uncertain is a mean value,
or, what does “statistically significant” mean?
![Page 13: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/13.jpg)
StatisticsNumerical descriptions
Observed sample
![Page 14: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/14.jpg)
StatisticsNumerical descriptions
Observed sample
Significance related statements
“There was no difference in...” “No difference in … could be observed” “There was a difference...” Etc.
![Page 15: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/15.jpg)
StatisticsIn singular: The scientific method of assessing
the uncertainty of generalizations
In plural: Numerical descriptions
Unobservedpopulation
Observed sample
![Page 16: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/16.jpg)
Problem
The sample is usually easy to identify, but what “population” are we talking about?
![Page 17: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/17.jpg)
To what population do experiment A belong?
Experiment A
![Page 18: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/18.jpg)
Experiment A Experiment A Experiment A Experiment A Experiment A
The mother of all possible realizations of
Experiment A
To what population do experiment A belong?
![Page 19: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/19.jpg)
The mother of all possible repetitions of
Experiment A
Experiment A Experiment A Experiment A Experiment A Experiment A
Sampling variability
μ
To what population do experiment A belong?
![Page 20: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/20.jpg)
The mother of all possible repetitions of
Experiment A
Experiment A Experiment A Experiment A Experiment A Experiment A
Sampling variability
μ
To what population do experiment A belong?
![Page 21: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/21.jpg)
The mother of all possible repetitions of
Experiment A
Experiment A Experiment A Experiment A Experiment A Experiment A
What is the sampling variability of these experiments?
Observed sampling variability after thousands of experiments
μ
![Page 22: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/22.jpg)
Experiment A
Sampling uncertainty?
μ
SDn
Can we say anything about sampling uncertainty if only one experiment is performed?
![Page 23: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/23.jpg)
Experiment A SDn
SEM = SD/√n
+1.96SEM-1.96SEM
Sampling uncertainty
Can we say anything about sampling uncertainty if only one experiment is performed?
![Page 24: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/24.jpg)
Unobservedpopulation
Observed sample
The properties of the population, like effect, risk, quality, etc., cannot be directly observed.
These can only be indirectly observed in the sample, contaminated by measurement error, sampling variation and bias (case-mix). This information needs to be statistically analyzed.
A number of the items in the sample may be interesting for the subjects in the sample, but cannot be interpret-ed in terms of independent effect, risk, quality, etc.
Is sampling uncertainty a problem only in experimental studies?
![Page 25: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/25.jpg)
Hospital A Hospital B Hospital C Hospital D Hospital E
Do different ranks in league tables represent differences in “hospital quality”?
Sampling variability?
![Page 26: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/26.jpg)
The mother of all possible repetitions of
Hospital A
Hospital A Hospital A Hospital A Hospital A Hospital A
Sampling variability
μ
Or do the differences just reflect sampling variation?
![Page 27: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/27.jpg)
It depends on the degree of uncertainty!
Sampling variability?
Hospital A Hospital B Hospital C Hospital D Hospital E
ICC ≈ 1.0
![Page 28: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/28.jpg)
Sampling variability?
Hospital A Hospital B Hospital C Hospital D Hospital E
ICC = 0
It depends on the degree of uncertainty!
![Page 29: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/29.jpg)
Evaluating uncertainty
Alt. 1. Hypothesis testing
H0: μ = 0
HA: μ ≠ 0
P( | H
0)
P < 0.05 → H0 is rejected
statistically significant
![Page 30: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/30.jpg)
Clinical vs. statistical significance
A clinically significant difference is important for all subjects, whether it is statistically significant or not.
Example
An increase in systolic blood pressure by 30mmHg.
![Page 31: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/31.jpg)
Clinical vs. statistical significance
A statistically significant finding is not necessarily clinically significant.
Example
A decrease (p = 0.023) in body weight of 0.1kg.
![Page 32: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/32.jpg)
ProbabilityYou know, the most amazing thing happened to me tonight. I was coming here, on the way to the lecture, and I came in through the parking lot. And you won't believe what happened. I saw a car with the registration plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see this particular one tonight? Amazing...
Richard P. Feynman
![Page 33: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/33.jpg)
Fundamentals of hypothesis testing
The p-value represents the probability of a false positive outcome of a pre-defined hypothesis test
Results from testing observed differences (in “fishing expeditions”) are unreliable.
![Page 34: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/34.jpg)
Fundamentals of hypothesis testing
The probability of a false positive outcome increases with thenumber of tests performed.
Results from performing multiple tests without recognizing the implicit multiplicity issues are unreliable.
![Page 35: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/35.jpg)
Fundamentals of hypothesis testing
Presenting only the statistical significance of findings is common but should be avoided.
The description “existing” or “not existing” (depending on their p-values)
- misleads the reader about what the investigator actually observed
- says nothing about the clinical relevance of the finding
- is misleading with respect to false positive and false negative findings.
![Page 36: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/36.jpg)
Fundamentals of hypothesis testing
H0: μ = 0
HA: μ ≠ 0
H0: μ > 0 or μ = 0
HA: μ < 0
Two-sided test
One-sided test
Example: The effect of an anti-hypertensive drug.
![Page 37: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/37.jpg)
Fundamentals of hypothesis testing
H0: μ = 0
HA: μ ≠ 0
H0: μ > 0 or μ = 0
HA: μ < 0
Two-sided test
One-sided test
Example: The effect of an anti-hypertensive drug.
Is the null hypothesis clinically meaningful?
![Page 38: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/38.jpg)
Fundamentals of hypothesis testing
The statistical power to detect a hypothetical difference is used when calculating the required sample size.
The post-hoc power of an observed difference is often calculated but is not meaningful. It does not reveal any other information than the p-value.
The statistical precision of an estimated parameter is best described by its confidence interval.
![Page 39: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/39.jpg)
Fundamentals of hypothesis testing
Statistical precision depends on the variability of subjects (independent observations) in the population and on thenumber of observations in the sample.
Testing body parts, e.g. hips, knee, feet, fingers, etc., (intraclass-correlated observations) usually leads to an under-estimation of between-subject variability and to an overestimation of the number of observations.
The consequence is usually too optimistic p-values.
![Page 40: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/40.jpg)
Fundamentals of hypothesis testing
Tests of imbalance are usually meaningless:
1. baseline imbalance in randomized trials
2. imbalance of matched sets in a matched case-control orcohort study
3. imbalance of exposure related risk factors for use inconfounding adjustments.
This imbalance is a property of the sample, and the tests are about properties of the population.
![Page 41: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/41.jpg)
Evaluating uncertainty
Alt. 2. Interval estimation
A 95% confidence interval
- 2SEM < µ < + 2SEM
Includes with 95% confidence the estimated parameter
![Page 42: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/42.jpg)
Note!
± 3SEM 99.7% confidence interval
± 2SEM 95% confidence interval
± 1SEM 68% confidence interval
![Page 43: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/43.jpg)
Note!
± 3SEM 99.7% confidence interval
± 2SEM 95% confidence interval
± 1SEM 68% confidence interval
± 1SD is a measure of observed dispersion
![Page 44: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/44.jpg)
Note!
± 3SEM 99.7% confidence interval
± 2SEM 95% confidence interval
± 1SEM 68% confidence interval
± 1SD is a measure of observed dispersion
± SD ≈ 95%Ci for the mean when n = 6
![Page 45: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/45.jpg)
0Effect
Statistically significant effect
Inconclusive
p < 0.05
n.s.
Information in p-values Information in confidence intervals[2 possibilities] [2 possibilities]
P-value and confidence interval
![Page 46: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/46.jpg)
0Effect
Clinically significant effects
Statistically and clinically significant effect
Statistically, but not necessarily clinically, significant effect
Inconclusive
Neither statistically nor clinically significant effect
Statistically significant reversed effect
p < 0.05
p < 0.05
n.s.
n.s.
p < 0.05
Information in p-values Information in confidence intervals[2 possibilities] [6 possibilities]
P-value and confidence interval
Statistically but not clinically significant effectp < 0.05
![Page 47: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/47.jpg)
0Control better
Margin of non-inferiorityor equivalence
Superiority shown
Superiority shown less strongly
Superiority not shownNon-inferiority not shown
Superiority not shown
Superiority vs. non-inferiority
New agent better
Non-inferiority shown Superiority not shown
Equivalence shown
![Page 48: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/48.jpg)
Experimental vs. observational studies
Experiments: Bias is eliminated by design:
“Block what you can, randomize what you cannot”.
Statistical analysis: Protect the type-1 error rate
Observation: Blocking and randomization is impossible.
The results must be adjusted in the statistical analysis.
Statistical analysis: Prioritize validity
![Page 49: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/49.jpg)
Observational studies
Validity
● Selection bias (systematic differences between comparison groups caused by
non-random allocation of subjects)
● Information bias (misclassification, measurement errors, etc.)
● Confounding bias (inadequate analysis, flawed interpretation of results)
![Page 50: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/50.jpg)
![Page 51: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/51.jpg)
![Page 52: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/52.jpg)
![Page 53: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/53.jpg)
Testing for confounding
Univariate screening for statistically significant effects, or stepwise regression, is often used to select covariates for inclusion in a regression model.
Confounding bias is a property of the sample, not of the population. What relevance have hypothesis tests?
![Page 54: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/54.jpg)
![Page 55: Abc4](https://reader034.fdocuments.in/reader034/viewer/2022050817/5567cb8bd8b42a2c098b49ed/html5/thumbnails/55.jpg)
Thank you for your attention!