Challenges and Opportunities for 21 st Century Pharmaceutical Statisticians
description
Transcript of Challenges and Opportunities for 21 st Century Pharmaceutical Statisticians
1
Challenges and Opportunitiesfor 21st Century
Pharmaceutical Statisticians
Christy Chuang-Stein, PhD
Statistical Research and Consulting Center
Pfizer Inc
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
2
Outline Evolving role of statisticians in pharmaceutical
industry
The external environment
Responses to the changes in the environment
Develop partnerships and collaborations
Examples of collaborations
A new metric for determining sample size for a confirmatory trial
Summary
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
3
Evolving Role of Stat in Pharma Industry
Little use of Statistics ==>
“Required” use of Clin Statistics ==>
Tactical use of Statistics ==>
Strategic use of Statistics & “Statistical
Thinking”
1955 2009 and beyond
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
4
Industry Perspective: “Then” Hired some statisticians to get things through the
regulatory agency (mostly in the US)
“Number crunchers” to get analyses done
“Blessed” clinical trial designs with minimal intellectual participation except sample size
Clinical and manufacturing focus
Very little input outside of “necessary”, low involvement in non-clinical areas
Statisticians played a secondary role
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
5
Statistician’s Role: Now and the Future
Full and equal partner with basic, clinical & regulatory scientists.
Focus on experimental design and development strategy.
Application of statistical thinking throughout the life cycle of a pharmaceutical product.
Parallel development in other disciplines such as epidemiology, genomics, biomarker development, and risk management expands statistician’s contributions.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
6
Statisticians in Pharma Industry
Being technically smart is not enough: Understand the broad clinical, regulatory and public-
health context Communicate statistical strengths and weaknesses
Proactive, not passive: Design: Options available, decision analysis Execution: Quality control and risk mitigation Analysis: Planned and unplanned, strengths/weaknesses Interpretation: Pre-planned or data-driven Presentations/Publications: Keeping audience in mind.
Statistics is a Collaborative Science!!
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
7
The External Environment Desire for more transparency in clinical research
and clinical reporting
A perceived need for more public “control” over the search for valuable medicines
Loss of public confidence in the clinical research process and the pharmaceutical industry
Loss of public confidence in the regulators
An increasing demand for product safety
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
8
Our External Stakeholders Health care providers
Patient care givers
Regulators
Journal editors
Shareholders
Health care payer
And most importantly - THE PATIENTS
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
9
Data Disclosure Desire for more “transparency” in the clinical
research process Protocols Results Logistics
Registries of trials (www.clinicaltrials.gov)
Results from trials involving marketed products are posted (www.clinicalstudyresults.org). Some companies have extended this to all trials and maintained their own trial results websites.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
10
How Can Statisticians Help?
Leverage the drive for transparency.
Develop partnerships with academia and government.
Within the limits allowed by internal rules and external regulations, share our processes and approaches to the outside world.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
11
Leverage the Drive for Transparency Improve study designs
All studies will be public and designs will be subject to public scrutiny.
Improve communication
Provide informative displays and presentations of results that are customized for the audience.
Endorse the use of common standards
Maximize the efficiency of collecting and sharing data via the use of common standards.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
12
Develop Partnerships Foster partnerships among academia, industry and
government.
Reasons: Solve complex problems by sharing resources and
knowledge Integrate ideas and increase credibility through peer
review Create solutions that take into consideration differing
perspectives
Belief: Working together is more effective than working in
isolation
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
13
Examples of Collaborations Adaptive Designs
Multiple Co-primary Endpoints
Active Control Study
QT Effect of a Pharmaceutical Product
Methods to Handle Missing Data
Dichotomizing Continuous Endpoint
Biomarker Quantification
Predictive Model (Pre-clinical to Clinical)
Safety Data Evaluation
Multi-regional Clinical Trials
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
14
Adaptive Designs
PhRMA Definition
uses accumulating data to decide on how to modify aspects of the study
without undermining the validity and integrity of the trial
ADWG: Adaptive Design Working Group
Validity means
providing correct statistical inference (such as adjusted p-values, estimates and confidence intervals)
minimizing operational bias
Validity means
providing correct statistical inference (such as adjusted p-values, estimates and confidence intervals)
minimizing operational bias
Integrity means
preplanning, as much as possible, based on intended adaptations
maintaining confidentiality of data
Integrity means
preplanning, as much as possible, based on intended adaptations
maintaining confidentiality of data
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
15
Drug Development ProcessL
evel
of
un
cert
ain
ty
Development process
PopulationEnd pointsDose rangeDosing frequencyExposureDrug formulation
Sample sizeAdd treatmentDrop treatmentRandomization fractionPopulation
Sample size Modify deltaRandomization fractionDrop treatmentPopulation
Exploratory phase Confirmatory phase
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
16
Adaptive Design Working Group The Group was formed in 2005 of industry and academic
members to investigate and facilitate opportunities for wider acceptance and usage of adaptive designs and related methodologies.
The Group met face-to-face about twice a year, has published numerous papers & given multiple presentations.
It has met with regulators in US, Canada, Europe and Japan to share experience and discuss issues of concern.
It continues to hold monthly telecons and sponsor a monthly Key Opinion Leader lecture series.
Is planning a workshop with the US Food and Drug Administration (FDA) to discuss a forthcoming FDA guidance on adaptive designs.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
17
10 Workstreams of ADWG Education Effort
ICH harmonization
Data monitoring issues and processes
Good adaptive practices
Case studies to share experience
Software user requirements
Material manufacturing and supply
Communications
Key Opinion Leader lecture series
Documentation
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
18
Multiple Co-Primary Endpoints
There are many disorders where regulators require a new treatment to demonstrate statistically significant benefit on multiple endpoints before accepting the efficacy of the new treatment.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
19
Definition of “Co-Primary” Endpoints These endpoints describe the disease in the sense
that an experimental drug that does not show superiority over placebo on all these endpoints is not a viable treatment for the disease.
This could arise because clinicians cannot agree on which endpoint (among several endpoints) is the most relevant one. So, they want evidence of effect on all of them.
In other words, they want statisticians solve the problem for them!
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
20
How Prevalent Is the Problem?Migraine (4) Alzheimer’s (2)
Acute Pain (3) Acne (3)
Fibromyalgia (3) Male Pattern Baldness (2)
Low Back Pain (3) Vaginal Atrophy (4)
Erectile Dysfunction (3) Organ Transplantation (2)
Osteoarthritis (2) BPH (2)
Menopausal Symptoms (4) Primary Biliary Cirrhosis (4)
Fracture Healing (2) Multiple Sclerosis (2)
Offen et al. Drug Information Journal (2007) 41(1):31-46.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
21
Two Examples Alzheimer’s Disease
Alzheimer’s Disease Assessment Scale - Cognitive (ADAS-Cog)
Clinician Interview-Based Impression of Change (CIBIC)
Vaginal Atrophy Self-reported frequency of the moderate or severe
symptoms that were most bothersome
Vaginal pH
% of vaginal parabasal cells from the maturation index
% of vaginal superficial cells from the maturation index
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
22
The Hypotheses Assume (…) represents the mean effect of a
new treatment over a placebo on K co-primary endpoints, positive i implying positive treatment effect.
The null (H0) and alternative (H1) hypotheses are
H0: 0 or 2 0 or … or 0
H1: > 0 and 2 > 0 … and K > 0
Usual requirement: false positive rate 2.5% (one-sided).
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
23
Current Regulatory Practice Test H0i vs H1i (the effect on the ith endpoint), each
at the one-sided 2.5% level, i = 1, …, K.
H0i: i 0
H1i: i > 0
Declare efficacy only if all H0i (i = 1, …, K) are rejected at the one-sided 2.5% level. This is the intersection-union test (IUT).
IUT is necessary to control the false positive rate at the one-sided 2.5% level over the entire null space.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
24
Overall Statistical Power
Number of co-primary endpoints
Correlation 2 3 4 9
0 64% 51% 41% 13%
0.2 66% 55% 47% 24%
0.5 69% 61% 56% 40%
0.8 73% 69% 66% 57%
*Power for each endpoint is 80%, one-sided =0.025
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
25
Correlation Coefficient (AD) Alzheimer’s Disease
Alzheimer’s Disease Assessment Scale - Cognitive (ADAS-Cog)
Clinician Interview-Based Impression of Change (CIBIC)
Correlation coefficient between ADAS-Cog and CIBIC is 0.22.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
26
Correlation Coefficient (VA)
Vaginal pH % Parabasal Cells
% Superficial Cells
Freq of Symptoms
0.20 0.11 -0.06
Vaginal pH __ 0.32 -0.13
% Parabasal Cells
0.32 __ -0.26
% Superficial Cells
-0.13 -0.26 __
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
27
% Increase on Sample Size
Number of co-primary endpoints
Correlation 2 3 4 9
0 31 49 62 96
0.2 29 46 58 91
0.5 25 39 49 74
0.8 17 27 32 48
Assume same effect size on each endpoint, tested at one-sided 2.5% level. The objective is to have an 80% overall power.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
28
Possible Solutions Optimal solution is a medical one
Reduce to a single dimension– Choose a single primary endpoint– Create a composite endpoint such as ACR20 for
rheumatoid arthritis
Prioritize endpoints, requiring only a positive “trend” in some endpoints. A positive trend could be, e.g. a two-sided P-value < 0.15.
Propose statistical approaches that could provide rationales for adopting a higher significance level for testing each endpoint.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
29
ACR20 (-50, -70) Definition At least 20% (50%, 70%) improvement in
# of swollen joint count, AND # of tender joint count, AND At least three of the following
– Patient’s global assessment of disease activity– Physician’s global assessment of disease activity– Patient’s assessment of pain (VAS, Likert, NRS)– Acute-phase reactants (ESR, CRP)– Patient’s assessment of disability (HAQ)
Often the primary endpoint to evaluate RA treatments.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
30
Adopting a Higher Sig Level (1-sided)Chuang-Stein et al (2007), Stat in Med, 26:1181-1192.
Correlation 2 3 4
0 0.036 0.055 0.082
0.2 0.036 0.052 0.075
0.4 0.035 0.048 0.066
0.6 0.032 0.043 0.055
0.8 0.030 0.037 0.044
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
31
Impact on Sample Size (K = 2) The entry in black (red) corresponds to % increase in
sample size under the new (current) approach.
Correlation 80% Power 90% Power
0 18% 31% 12% 23%
0.2 16% 29% 11% 22%
0.4 14% 26% 10% 20%
0.6 14% 22% 11% 18%
0.8 11% 16% 8% 13%
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
32
Impact on Sample Size (K = 3) The entry in black (red) corresponds to % increase in
sample size under the new (current) approach.
Correlation 80% Power 90% Power
0 19% 49% 11% 36%
0.2 18% 46% 11% 34%
0.4 17% 41% 11% 31%
0.6 15% 35% 11% 27%
0.8 12% 25% 9% 21%
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
33
A More Serious Issue A common question – Why can’t a sponsor just
increase the sample size to get an adequate overall power?
Statistical power gives the probability of concluding a treatment effect when the true treatment effect is the amount used to size the study.
For a trial to be successful, the new treatment needs to deliver the above assumed effect on all endpoints.
A greater demand of a new treatment!
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
34
A New Sample Size Paradigm Relevant to trials expected to provide confirmatory
evidence for product registration.
The primary objective of a confirmatory trial is to confirm that the effect of the new treatment is statistically better than a placebo. Recall that statistical power gives the probability of concluding a treatment effect when the true treatment effect is equal to the effect used to size the study.
But, we don’t know for sure if the treatment could deliver on the assumed effect.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
35
An Example A small PoC study, n = 25 per group, estimated
treatment effect over a placebo = 2.5 units, estimated standard deviation = 7.14 units. So, estimated effect size = 0.35 (2.5/7.14).
The temptation is to design the next study to detect an effect size of 0.35 based on a two-sided test at the 5% significance level.
Need 128 subjects per group for 80% power, or 172 per group for 90% power.
Source: Chuang-Stein (2006), Pharmaceutical Statistics, 5(4):305-309.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
36
Question
What is the probability that we will have a successful trial? Define a successful trial as obtaining a P-value less than 5%. Other criteria can be used.
Is it 80% if we enroll 128 subjects per group, or 90% if we enroll 172 subjects per group?
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
37
A Frequentist-Bayesian Approach
We know there is variability in our estimate for the treatment effect. Assume we can describe this variability by
| PoC data N(2.5 ; (2/25) (7.14)2)
We could obtain “average success prob” by
ddataPoCPvalueP ) |( )|05.0Pr( Success of Prob
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
38
A Frequentist-Bayesian Approach The success prob is 63.3% for 128 subjects per
group (80% power); and 67.7% for 172 subjects per group (90% power). These are much lower than the corresponding statistical power.
If the PoC study had 70 subjects per group (instead of 25) with the same estimated effect and standard deviation, then the prob of success is 69.2% for 128 subjects per group and 75.6% for 172 subjects per group.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
39
Observations If the required number is much higher than the number
obtained under the traditional approach, this could be a signal that our prior information on treatment effect is not sufficiently strong.
The above suggests that we need to select appropriate metrics to make decisions.
For proof-of-concept study, the metric might be the precision of the treatment effect estimate.
For a dose-response study, the metric might be the probability for selecting the right dose.
For a confirmatory trial, a better metric might be the probability of a successful trial.
delete these guides from slide master before printing or giving to the client
delete these guides from slide master before printing or giving to the client
40
Summary Pharmaceutical industry, as a whole, is facing
unprecedented challenges. These challenges also create a lot of opportunities.
A successful statistician needs to be responsive to these challenges. Statisticians need to be able to anticipate these challenges and offer solutions!
It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change. - Charles Darwin