Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo,...
Transcript of Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo,...
![Page 1: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/1.jpg)
Kaming Lo, M.P.H.Biostatistics Collaboration and Consulting Core
Division of BiostatisticsDepartment of Epidemiology and Public Health
![Page 2: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/2.jpg)
Introduction
High quality research results from a comprehensive plan which involve: Population selection Randomization Methodology Measurement Tools Power and Sample Size
![Page 3: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/3.jpg)
Why calculate sample size?
In statistics, the validity of the analysis depends upon: how much information can be used. how precise the information is.
Sample size calculations allow the investigator to: determine the minimum amount of information
needed for answering the research question.
![Page 4: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/4.jpg)
Outline
Power Concerns in Sample Size Calculations Common Formulae Collaboration with Statistician Other Useful Tools Take Away Messages
![Page 5: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/5.jpg)
Type I and Type II Errors
Type I Error, α When researcher rejects the Null Hypothesis
given the Null Hypothesis is true.
Type II Error, β When researcher does not reject/accepts
the Null Hypothesis given the Alternative Hypothesis is actually true.
![Page 6: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/6.jpg)
Power “Probability of rejecting the null hypothesis
given that the alternative hypothesis is true.”
Compliment of Type II Error.
Reject Null Do Not Reject Null
Null is TrueType I Error
(α)Confidence Level
(1- α)
Alternativeis True
Power(1- β)
Type II Error(β)
![Page 7: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/7.jpg)
Understanding Power (example) A population, based on previous studies, is known to have
a normal distribution with a mean of 20 and a standard deviation of 4.
An investigator was interested in another population with the same standard deviation.
He wanted to test if the mean is equal to 20.
He took a random sample of 44 from the studied population.
One sample t-test can be used to test the hypothesis:
H0: µ = 20 vs Ha: µ ≠ 20
![Page 8: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/8.jpg)
Understanding Power (example) cont’d Based on an α of 0.05, the investigator
calculated the critical values that would allow him reject the null hypothesis, which is 18.8 and 21.2. In another words, if the mean of his sample is below 18.8 or greater than 21.2, he would get a significant difference.
![Page 9: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/9.jpg)
Understanding Power (example) cont’d
Reject Null Do Not Reject Null
Null is TrueType I Error
(α)Confidence Level
(1- α)
Alternativeis True
Power(1- β)
Type II Error(β)
![Page 10: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/10.jpg)
Figure 1.
![Page 11: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/11.jpg)
What affects power? There are a few factors that would affect power:
Sample size (n)n increases Power increases
Type I error rates/significance level (α)α increases Power increases
Variability (σ)σ increases Power decreases
Effect size (Δ, it is the changes in magnitude of the outcome that is considered scientifically important.)Δ increases Power increases
Besides, power would be greater in a one-tailed test compared to a two-tailed test.
![Page 12: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/12.jpg)
How much power is needed
Achieving 80% power is generally acceptable.
Too much or too little power could be an issue.
![Page 13: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/13.jpg)
Underpower and Overpower
Consider a study to test for the difference between the effects of two drugs for diabetic patients. An investigator hypothesized that drug B would have a higher mean reduction on the Hemoglobin A1c than drug A by 1%.
![Page 14: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/14.jpg)
Underpower
Too little sample.
May results in no difference between drug A and B even there may actually be some significant differences.
Wasted funds in conducting the trial that returns no meaningful results.
![Page 15: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/15.jpg)
Overpower Too much sample.
May always find difference between drug A and B even the difference is not actually of scientific importance, e.g. the difference detect may actually be 0.3% instead of 1%.
Wasted funds in recruiting the extra subjects that are not really necessary.
![Page 16: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/16.jpg)
Outline
Power Concerns in Sample Size Calculations Common Formulae Useful Tools Collaboration with Statistician Take Away Messages
![Page 17: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/17.jpg)
Concerns in Sample Size Calculations
Hypotheses, both primary and secondary
Primary outcomes and variables of interest Continuous/categorical data
Effect size What is considered clinically importance?
Variability of the outcomes if continuous
![Page 18: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/18.jpg)
Study designs, for examples: Randomized controlled trial Non-randomized trial Observational study
Data structure, for examples: Parallel data Paired data Repeated measures
Concerns in Sample Size Calculations cont’d
![Page 19: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/19.jpg)
Outline
Power Concerns in Sample Size Calculations Common Formulas Useful Tools Collaboration with Statistician Take Away Messages
![Page 20: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/20.jpg)
Classical formula for testing difference of two means
H0: µ1 = µ2 vs. Ha: µ1 ≠ µ2
n =Sample size need in each group σ =Common standard deviation Z1-α/2 =Standardized value at desired α Z1-β =Standardized value at desired β Δ =Effect size
![Page 21: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/21.jpg)
Classical formula for testing difference of two proportions
H0: p1 = p2 vs. Ha: p1 ≠ p2
n =Sample size need in each group p1 =Proportion in group 1 p2 =Proportion in group 2 ṗ =(p1 + p2)/2 Z1-α/2 =Standardized value at desired α Z1-β =Standardized value at desired β Δ =Effect size
![Page 22: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/22.jpg)
Sample Size Calculation Classical formulas have many statistical
assumptions, such as normality, independent groups, equal variance, and more.
Often not the case in the reality.
If assumptions are violated or if studies involve complex study designs or statistical analyses. Simulation maybe needed. Simplify study design.
![Page 23: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/23.jpg)
Outline
Power Concerns in Sample Size Calculations Common Formulae Collaboration with Statistician Other Useful Tools Take Away Messages
![Page 24: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/24.jpg)
Why Statisticians?
A statistician understands: Which study design and statistical method
are more effective/powerful for answering the research question.
What statistical needs should be considered during the planning phase of the study.
How to help reducing the cost while maintaining the power of the study.
![Page 25: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/25.jpg)
What to prepare before meeting with statistician
An investigator should prepare as many of the following as possible: The study objectives The primary hypothesis Outcome variables Effect size
![Page 26: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/26.jpg)
What to prepare before meeting with statistician cont’d
Preliminary information, which can usually be obtained through previous literature: Means/proportions Standard deviation if continuous If not readily available, one might consider a
pilot study or consult with an expert in that research area to ask for an expectation on the values.
![Page 27: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/27.jpg)
Outline
Power Concerns in Sample Size Calculations Common Formulae Collaboration with Statistician Other Useful Tools Take Away Messages
![Page 28: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/28.jpg)
Other Useful Tools Java Applet developed by Lenth RV (2006)
http://www.cs.uiowa.edu/~rlenth/Power/ Epi Info by CDC
Commercial Software: SAS SPSS nQuery PASS
![Page 29: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/29.jpg)
Outline
Power Concerns in Sample Size Calculations Common Formulae Collaboration with Statistician Other Useful Tools Take Away Messages
![Page 30: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/30.jpg)
Take Away Messages Understanding what affects power is the key to
determine the best sample size. Different factors (α, σ, Δ, etc) Study designs and data structures. Statistical methods.
Formulas don’t always work! (Beware of the assumptions behind them).
![Page 31: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/31.jpg)
Take Away Messages cont’d Seek inputs from a statistician if in doubt
The earlier involvement of a statistician in a study the better.
Can assure a higher statistical power by choosing effective study design and analysis approach.
A researcher who comes to a statistician prepared will get the best results from the consultation Knowing the hypothesis, primary outcomes, effect size Research on any preliminary data (mean, sd, proportions,
etc)
![Page 32: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/32.jpg)
Biostatistics Collaboration and Consulting Core (BCCC)
Mission Statement:To assure that the appropriate statistical methodology is incorporated in research.
The BCCC operates as a cost center, offering support activities to faculty, staff, and students. All fees are based on UM policy B020 for Recharge or Cost Centers.
BCCC Activities:1. Study Design 7. Abstract/Manuscript Preparation2. Randomization Schemes 8. Grant Preparation3. Statistical Analysis Plan (SAP) 9. Survey/Questionnaire Design4. Sample Size Estimation or Power Analysis 10.Protocol Review5. Statistical Analysis 11.Safety Committee6. Consulting Statistician for Staff and Professional Meetings
BCCC Support: BCCC Free Support:Short Term Quick Consulting – 30 minutesGrantsOngoing Collaboration Plan Initial Meeting – 1 hours – Project Initiation and Agreement
![Page 33: Kaming Lo, M.P.H.biostat.med.miami.edu/documents/Power_and_Sample_Size_Consid… · Kaming Lo, M.P.H. Biostatistics Collaboration and Consulting Core. Division of Biostatistics. Department](https://reader033.fdocuments.in/reader033/viewer/2022052811/608af29cb1d4202dc5590d74/html5/thumbnails/33.jpg)
BCCC Contact Information:
Clinical Research Building, 10th Floor1120 N.W. 14th Street (R-669),
Miami, FL 33136
Contact person: Maria Jimenez-RodriguezTel: 305-243-4465
E-mail: [email protected]: www.biostat.med.miami.edu/core