Meandering journey towards routine trial adaptation: Survey …/file/... · 2015-06-11 ·...
Transcript of Meandering journey towards routine trial adaptation: Survey …/file/... · 2015-06-11 ·...
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
Meandering journey towards routine trial adaptation: Surveyresults on the use of adaptive designs in confirmatory trials
Munya Dimairo*, Susan Todd, Steven Julious and Jon Nicholl
Clinical Trials Research Unit, University of Sheffield
12 May 2015PSI Annual Conference (2015)
London
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
Outline
1 Some background information
2 Addressing the gap
3 The findings
4 Discussion and conclusions
5 Acknowledgements
6 References
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
Some background information
Introduction
The concept and methodological developments of ADs have come of age
... there can be no objection to the use of data, however meagre, as a guide to action required
before more can be collected ... Thompson,1933
1 However, practical implementation of AD has been too slow to gain traction
2 Understanding cross-disciplinary barriers and concerns is paramount
3 Some previous research (Quinlan et al,2010; Morgan et al,2012; Kairalla et al,2012;
Jaki,2013)
Lack of systematic approach to inform themes
Much focus on early phase trials
Setting
Little cross-sector and cross-disciplinary approach
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
Some background information
Aims
1 Explore barriers and concerns, and rank them with respect to perceived importance for
prioritisation
2 Explore ways to address some perceived barriers and propose recommendations
3 Comparison of private and public sector perspectives
4 Explore types of confirmatory ADs being implemented in practice
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
Addressing the gap
Survey approach
Uncovered some concerns which we could have overlooked
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
Addressing the gap
Survey instrument
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
Addressing the gap
Outline of statistical analysis
Descriptive
Rasch analysis using a Rating Scale Model (Andrich,1978)
ln(pnik
pni(k−1)
) = θn − (δi + τk ) (1)
k = 0, 1, ...,m are ordered responses and m is the number of choices
pnik - probability of respondent n with ability θn choose score k for item i
δi - perceived importance of item i
τk - threshold parameter corresponding to choice k
θn and τk are nuisance parameters
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
The findings
Snapshot of underlying distribution of the perception of barriers
Barrier item Not an Somewhat an Moderately Extremely
important important important important
barrier barrier barrier barrier
Lack of practical knowledge 1(8%) 3(23%) 5(38%) 4(31%)
Limited time to support planning 1(8%) 6(46%) 2(15%) 4(31%)
Practical complexities 1(8%) 3(23%) 6(46%) 3(23%)
Data management infrastructure 3(23%) 5(38%) 2(15%) 3(23%)
Lack of applied training 2(15%) 2(15%) 7(54%) 2(15%)
Lack of hands-on experience 1(8%) 4(31%) 6(46%) 2(15%)
Insufficient access to case studies 2(15%) 5(38%) 4(31%) 2(15%)
Preference for traditional designs 3(23%) 2(15%) 5(38%) 3(23%)
Lack of awareness of acceptable scope 1(8%) 4(31%) 6(46%) 2(15%)
Amount of time & effort to support planning 4(31%) 4(31%) 2(15%) 3(23%)
Regulatory fear 5(38%) 1(8%) 3(23%) 3(23%)
Marketing challenges to key stakeholders 2(15%) 6(46%) 4(31%) 1(8%)
. . . . .
. . . . .
. . . . .
Lack of motivational R & D support 6(46%) 5(38%) 2(15%) -
Negative regulatory experiences 5(38%) 3(23%) 2(15%) -
Lack of planning expertise 5(38%) 4(31%) 4(31%) -
Costing complexities 6(46%) 3(23%) 2(15%) 1(8%)
Negative implementation experiences 7(54%) 2(15%) 2(15%) -
Lack of R & D financial support 8(62%) 2(15%) 2(15%) 1(8%)
Lack of statistical expertise 8(62%) 2(15%) 3(23%) -
Worry about employment contracts 10(77%) 2(15%) - -
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
The findings
Private sector’s perception of barriers
Lack of practical knowledgeLimited time to support planning
Practical complexitiesData management infrastructure
Lack of applied trainingLack of practical hand-on experience
Insufficient access to case studies
Lack of awareness of acceptable scopeAmount of work and time required
Regulatory fearMarketing challenges to key stakeholders
Lack of awareness of when appropriateChallenges setting up decision-making criteria
Stats design complexitiesLack of bridge funding
Qutsourcing challengesTension in decision-making
Stats implementation complexitiesLack of awareness of benefits
Lack of awareness of implementation resourcesLack of knowledge of stats softwareLack of motivational R & D support
Negative regulatory experiencesLack of planning expertise
Costing complexitiesNegative implementation experiences
Lack of R & D financial supportLack of statistical expertise
Worry about employment contracts
Preference for traditional designs
-2 0 2 4 6Perceived relative importance
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
The findings
Lack of practical knowledge and experience
1 Learning from pacesetters - few organisations frequently implementing ADs
2 Encourage more accessible publication of case studies: rationale, design, regulatory hurdles,
statistical and practical aspects, implementation resources, facilitators, etc
3 Better indexing of ADs related publications
4 Transparency and optimal reporting of ADs in publications
ADs tailored CONSORT statement?
Practical complexities
1 Logistics of how the design will work in practice?
2 Degree varies considerably depending on the nature of the AD
Better communication of the complexity message
Practical oriented toolkit of design specific issues researchers need to consider when
planning confirmatory ADs
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
The findings
Fear of risking regulatory approval
1 Regulatory engagement: advice consultations
2 Adequate description of;
Rationale, type and scope of proposed AD
Procedures to minimise operational bias
Methodological issues (type I error rate control, inference after adaptation, potential
population drift, etc)
Totality of confirmatory evidence
Sponsor involvement (cross-sector of acceptable models needed)
Who is involved in interim decision making?
Consideration for important secondary objectives (e.g safety)
3 Adequate documentation of processes and procedures
4 Familiarisation with regulatory guidance and reflection papers
5 Improving regulatory awareness and experiences (Elsaßer et al., 2014)
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
The findings
Difficulties in selling ADs to key stakeholders
1 Better communication of the rationale of AD is imperative
2 How the design addresses the research question(s)?
3 Illustration of potential tangible benefits (patients, trial duration, resource utilisation, etc)
Lack of awareness acceptable scope and when appropriate
1 Cross-sector/disciplinary guidance document?
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
The findings
Contrast with public sector perspective
Lack of bridge fundingLack of practical knowledge
Lack of practical hand-on experience
Marketing challenges to key stakeholdersAmount of work and time required
Limited time to support planningLack of applied training
Insufficient access to case studiesPractical complexities
Stats design complexitiesChallenges setting up decision-making criteria
Capacity of proposal developersCosting complexities
Lack of awareness of acceptable scopeLack of awareness of when appropriate
Worry about employment contractsLack of awareness of implementation resources
Regulatory fearData management infrastructure
Stats implementation complexitiesLack of knowledge of stats software
Lack of statistical expertiseLack of awareness of benefits
Tension in decision-makingNegative implementation experiences
Negative reviewers experiences
Preference for traditional designs
-2 -1 0 1 2 3Perceived relative importance
Perception of barriers is consistent across sector
Few exceptions reflecting differences in organisational structures, experiences and nature of
study interventions
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
The findings
Private sector perceptions of concerns
Incr
easi
ng d
egre
e of
con
cern
Impact on secondary objectives
Potential of operational bias
Population drift
Robustness in decision-making
Acceptability to change practice
Non-inferiority early stopping
Efficacy early stopping
Futility early stopping
-1 -.5 0 .5 1 1.5Perceived relative importance
Small degree of separation in perception
More receptiveness towards futility early stopping
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
Discussion and conclusions
Implications of the findings
1 There are still multifaceted barriers requiring addressing
2 Cross-sector and cross-disciplinary engagement is key to alleviating some of the challenges
3 Most barriers emanate from the lack of practical knowledge and experience
4 Cross-sector/disciplinary receptiveness towards futility early stopping
Limitations
1 Average response rates
2 Sample representativeness
3 Findings may provide conservative perceptions on some of the barriers and concerns
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
Acknowledgements
Acknowledgements
1 Funding: NIHR Doctoral Research Fellowship (DRF-2012-05-182)
2 Fellowship Supervisors
Prof Steven Julious
Prof Susan Todd
Prof Jon Nicholl
3 Personal Tutor: Mike Bradburn
4 Advisory panel members
5 Participating organisations and individuals
Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials
References
References
Andrich, D. (1978) A rating formulation for ordered response categories. Psychometrika. 43
(4), 561–573
Elsaßer, A. et al. (2014) Adaptive clinical trial designs for European marketing authorization:
a survey of scientific advice letters from the European Medicines Agency. Trials. 15 (1), 383.
Jaki, T. (2013) Uptake of novel statistical methods for early-phase clinical studies in the UK
public sector. Clinical trials. 10 (2), 344–346
Kairalla, J. a et al. (2012) Adaptive trial designs: a review of barriers and opportunities.
Trials. 13145.
Morgan, C. C. et al. (2014) Adaptive Design: Results of 2012 Survey on Perception and Use
. Therapeutic Innovation & Regulatory Science . 48 (4 ), 473–481.
Quinlan, J. et al. (2010) Barriers and opportunities for implementation of adaptive designs
in pharmaceutical product development. Clinical trials. 7 (2), 167–173.
Thompson, W. R. (1933) On the Likelihood that One Unknown Probability Exceeds Another
in View of the Evidence of Two Samples. Biometrika. 25 (3/4), 285–294