Establishing Efficacy through Randomized Controlled Clinical Trials Ernst R. Berndt, Ph.D. MIT and...
-
Upload
aldous-washington -
Category
Documents
-
view
217 -
download
0
Transcript of Establishing Efficacy through Randomized Controlled Clinical Trials Ernst R. Berndt, Ph.D. MIT and...
Establishing Efficacy through Randomized Controlled
Clinical Trials
Ernst R. Berndt, Ph.D.
MIT and NBER
Goal of Today’s Presentation
• Help you to become critical, informed readers of reports/articles from randomized controlled trials (“RCTs”), and of associated research proposals
• Realize that “the devil is in the details”
BackgroundBaseline health status,demographics,
Change in medical intervention, lifestyle, education,Health Status = f comorbidities, other concomitant(symptom & medications, history of non-
responsivenessdisease metrics) to TX, host of other factors
or
HS = f (HS0, MI, everything else)
Issue: How to measure HS / MI ?
Possibilities & Problems
• Retrospective data and use of multivariate statistical analysis
• Prospective “naturalistic” data and use of multivariate statistical analysis
• Prospective randomized controlled (double blinded?) trial
• Other?
Why Randomize?
• Members of the treatment (“TX”) and control groups tend to be comparable on all variables, known and unknown
• Provides basis for statistical analysis
Pitfalls to Randomization
• Timing (before vs. after patient enters trial)
• Keeping track of drop-outs (non-random?)– “intent to treat” analysis– “last observation carried forward”
• Blinding
• Randomization need not generate “median person”
On External Validity
• What affects generalizability of study findings to population as a whole?– Exclusionary & inclusionary criteria
• How to assess generalizability?– Any rigorous test?
• Hawthorne effect present?
P-Values and Significance Levels
Definitions: Observe Hypothesize
MI Does Make a Difference 0+ H+
MI Does Not Make a Difference 0- H-
P-Value: P (O+/H-) -- false-positive rate, often .05 or .01
-- aka Type 1 or alpha error
Beta: P (0-/H+) -- false negative probability; for given n,
lower alpha error rate => higher beta
Sensitivity: P (0+/H+) -- true-positive rate, 1-beta or “power”
Specificity: P (O-/H-) -- true-negative rate
Implications
• P-value does not directly tell us the probability that H+ is true
• Classical statisticians “reject the H-” but cannot “accept the H+” (but Bayesians can go further than classical statisticians …)
• For given beta, why do “head to head” RCTs require larger sample sizes than “placebo-controlled” trials?
• Difference between “statistical significance” and “practical importance”
Related P-Value Issues
• Subgroups – must they be specified ahead of time?– Data mining and data dredging– Bonferroni adjustments to overall distinct sub-
group P-values
• Stopping rules– Sequential design vs. chance result– FDA “confirmatory trials”
Internal Validity
• Does credible physiological theory suggest mechanism of action?
• Persuasive studies in animals?• Replicated in other human studies?
– FDA requirement for two RCTs
• Study undertaken by team with good credentials?
• Was publication in peer-reviewed journal?
Single vs. Multi-Site Trials
• MS accelerates patient recruitment
• Less homogeneity of patients and treatments in MS trials, but greater external validity
• MS trials typically simpler
• MS trials give evidence of reproducibility
Quality Control of RCTs
• Very detailed protocol, including endpoint metrics, a hallmark of a good study – done ex ante, not ex post– Use of “surrogate markers” / intermediate outcomes
• Accuracy of case report forms• Trial design isolates effect of MI?• What if randomization “fails” in TX vs. control group
on some dimensions?– Additional statistical analysis– Site effects?
• Keeping track of drop-outs, and reasons for drop-outs
Related Class Topics
• Efficacy vs. effectiveness• Use of clinical research organizations (CROs) in
outsourcing clinical trials• Regulations on disseminating efficacy findings
from RCTs• Use of retrospective medical claims data bases• Other outcomes (quality of life, costs, economic
benefits)