Copyright 2011 Right Care

IDEALA rational approach to evaluation of complex treatments

Peter McCulloch

Chair, IDEAL Collaboration

March 2015

Online Learning Series

Right Care for Populations

Developing Evidence for TherapyPHARMA PARADIGM

• Theory• Lab demonstration • (animal studies)• First-in-man study• Toxicity study (Phase I)• Efficacy Study (Phase II)• RCT (Phase III)• Post-Marketing Surveillance (Phase IV)

SURGICAL PARADIGM

• Idea• (animal/bench trial)• First-in-man (CS)• Outcome experience (CS)• Confirmation/dispute from others (more CS)• RCT??

SPECIAL PROBLEMS OF SURGICAL TRIALS

1. Early “tinkering” 2. Defining the surgical intervention3. Accommodating surgical learning curves 4. Quality Control5. Developing standard outcome measures6. Strong preferences

IDEAL: What and Why?

AIM• Improve the methodological quality of research in

surgery and interventional therapiesSTEP 1: FRAMEWORK

• Describe the natural history of surgical innovationSTEP 2: RECOMMENDATION

• Define appropriate studies for each stage in the innovation pathway

Stage 1IDEA

Stage 2a DEVELOPMENT

Stage 2b EXPLORATION

Stage 3 ASSESSMENT

Stage 4 LONG TERM MONITORING

Initial report Innovation may be planned, accidental or forced Focus on explanation and description

“Tinkering”(rapid iterative modification of technique and indications) Small experience from one centre Focus on technical details and feasibility

Technique now more stable Replication by others Focus on adverse effects and potential benefits Learning curves important Definition and quality parameters developed

Gaining wide acceptance Considered as possible replacement for current treatment Comparison against current best practice (RCT if possible)

Monitoring late and rare problems, changes in use & quality of surgical performance

Identifying the IDEAL Stage

• Quick literature search• Review article TYPES, NUMBER, SOURCES and

PATIENT NUMBERS• Use the Framework to identify the Stage

Leads on to RECOMMENDATIONS for evaluation

Stage 1: IDEARECOMMENDATIONS

• Compulsory reporting of all new innovations on a Professional Innovation Database

• Confidential entry allowed to encourage reporting of failed innovations (CHRP system)

• “Light touch” local unified regulation

Stage 2a: Development RECOMMENDATIONS

• Prospective Development Studies:- Detailed description of selection criteria- Detailed technical description- Prospective account of ALL cases consecutively- Clear STANDARDISED definitions of outcomes reported- Description of ALL modifications and when made in

series

• Registration of PROTOCOL before study starts

PDS Example: Development of Robotic Oesophagectomy

Hospital Stay(days)

Stage 2b ExplorationRECOMMENDATIONS

Prospective Exploration Study

(collaborative uncontrolled

prospective study)

• To evaluate technique prospectively and co-operatively

• To develop a consensus over definition of the procedure, quality standards

• To accumulate data for power calculations

• To evaluate and monitor learning curves

• To evaluate preferences and values amongst patients and clinicians

• To achieve consensus on the trial question

• To develop a multi-centre randomised trial

PES Example:Feasibility Study for RCT of radical gastrectomy

• 8 North Italian surgical units

• Training scheme • Prospective Cohort • Review of short term

outcomes• 5 Eligible groups

enter RCT

Extended lymph node dissection for gastric cancer: results of a prospective, multi-centre analysis of morbidity and mortality in 118 consecutive cases.Degiuli M, Sasako M, Ponzetto A, Allone T, Soldati T, Calgaro M, Balcet F, Bussone R, Olivieri F, Scaglione D, Danese F, Morino M, Calderini P, Capussotti L, Fronda G, Garavoglia M, Locatelli L, Dellepiane M, Rossini FP, Calvo F.Eur J Surg Oncol. 1997 Aug;23(4):310-4.

Morbidity and mortality in the Italian Gastric Cancer Study Group randomized clinical trial of D1 versus D2 resection for gastric cancer.Degiuli M, Sasako M, Ponti A; Italian Gastric Cancer Study Group.Br J Surg. 2010 May;97(5):643-9. doi: 10.1002/bjs.6936.

Stage 3: AssessmentRECOMMENDATIONS

Surgical RCT Or Alternative…

• Consider blinding• Standardise Terms• Use Stage 2b data to decide;

• Power calculations• Definition of intervention• Quality measures• Learning curve eligibility

• Deal with preferences• Expertise-based trials• Qualitative research• Third party randomisation• Decision support aids• Cohort/RCT mixtures

• Feasibility trials(?)

• Stepped-wedge design

• Controlled interrupted time series

Stage 4 Long term studyRECOMMENDATIONS

• Prospective Registries• Best designed by clinical

community (with help)• Best curated by impartial

body (not manufacturer)• SPC used for quality

control (Shewart charts, CUSUM, VLAD)

IDEALAn integrated evaluation pathway

REGISTRATION OF 1st in MAN

PROSPECTIVE DEVELOPMENT STUDY

PROSPECTIVE EXPLORATION STUDY

RCT

REGISTRY

Uses and Implications• For designing and publishing studies of

invasive therapeutic techniques

• For defining the evidence required for licensing of therapeutic devices & implants

• For deciding what evidence is needed to justify commissioning of procedures and devices

Get involved

• Follow us on Twitter @IDEALCollab• Link to our website from your institution’s website• Send us text about your work or IDEAL related topics so

we can upload as news items on the websitewww.ideal-collaboration.net

• Become a member and subscribe to our newsletter• Collaborate on piloting & evaluating IDEAL study designs

Contact: allison.hirst@nds.ox.ac.ukIDEAL Project Manager

Peter.mcculloch@nds.ox.ac.ukIDEAL Chair

17

Online Learning Series

Right Care for Populations

Follow Right Care online• Subscribe to get a weekly digest

of our blog alerts in your inbox, • Receive occasional eBulletins • Follow us on Twitter

@qipprightcare

Find the full series at:

www.rightcare.nhs.uk/resourcecentre

Find more IDEAL resources at:

www.rightcare.nhs.uk/resourcecentre