Comparison of effect sizes associated with surrogate and final primary endpoints in

randomised clinical trials

1

Ciani O., Garside R., Pavey T., Stein K., Taylor R.S.

Background

Classic Definition for surrogatesDisease-centered characteristics Patient-centered characteristics

BiomarkersA characteristic that is objectively measured and evaluated as an indicator of normal, pathogenic or pharmacologic responses to a therapeutic intervention.

Final outcomeA characteristic

that reflects how patient feels, functions or

survives.

Surrogate outcomes

A biomarker that is intended to substitute and predict for a final

outcome.

e.g. LDL-cholesterolCardiovascular

Mortality

e.g. Intraocular pressure Loss of vision

Background

HTA-based Definition of surrogatesDisease-centered characteristics Patient-centered characteristics

BiomarkersA characteristic that is objectively measured and evaluated as an indicator of normal, pathogenic or pharmacologic responses to a therapeutic intervention.

Final outcomeA characteristic

that reflects how patient feels, functions or

survives.

Surrogate outcomes

A biomarker - or clinical or patient-relevant outcome - that is

intended to substitute and predict for a final

outcome, namely survival or HRQoL.

e.g. Rate of hip fracture Mortality/HRQoL

e.g. Event-free Survival Overall Survival

4

Objectives of the study

I. To study the association between primary endpoint (surrogate vs final) and treatment effect estimates in RCTs

II.To compare the risk of bias in trials reporting a surrogate endpoint vs trials reporting a final primary endpoint

5

Initial sample of abstracts (N = 639) Initial sample of abstracts (N = 639)

Excluded (N = 55)Not RCTs (N = 17)Economic evaluation studies (N = 11)Non interventional treatment (N = 25) Secondary analysis (N = 2)

Excluded (N = 55)Not RCTs (N = 17)Economic evaluation studies (N = 11)Non interventional treatment (N = 25) Secondary analysis (N = 2)

For outcomes classification (N = 584) For outcomes classification (N = 584)

Composite mixed outcomes (N = 73)

Composite mixed outcomes (N = 73)

Eligible for the study (N = 511) Eligible for the study (N = 511)

Surrogate outcomes based (N = 137)Surrogate outcomes based (N = 137) Final outcomes based (N = 137) Final outcomes based (N = 137)

Matching procedureMatching procedure

Final outcome trials (N = 101)Binary endpoint (N = 83)

Final outcome trials (N = 101)Binary endpoint (N = 83)

Excluded (N = 53)Equivalence/Non-inferiority study (N = 15) Unpooled Muliti-arm (N = 33)No analysable data (N = 5)

Excluded (N = 53)Equivalence/Non-inferiority study (N = 15) Unpooled Muliti-arm (N = 33)No analysable data (N = 5)

Surrogate outcome trials (N = 84)Binary endpoint (N = 51)

Surrogate outcome trials (N = 84)Binary endpoint (N = 51)

Excluded (N = 36)Composite mixed outcomes (N = 9)Early termination (N = 1)Equivalence/Non-inferiority study (N = 11)Unpooled Muliti-arm (N = 11)No analysable data (N = 4)

Excluded (N = 36)Composite mixed outcomes (N = 9)Early termination (N = 1)Equivalence/Non-inferiority study (N = 11)Unpooled Muliti-arm (N = 11)No analysable data (N = 4)

Methods

Study selection

6

Methods

Data extraction

Effect Size Binary endpoints: n/N data for each arm Continuous endpoints: SS, Mean, SD for each arm TEs(95%CI) as reported by authors

Study characteristics: sample size, follow-up, type of intervention, patient population, sponsor (i.e. FP, NFP and mixed), positive outcome in favour of the new treatment

Risk of bias: adoption of the intention to treat (ITT) principle, adequate randomized sequence generation and allocation concealment, double-blind/placebo-control

Surrogate outcomes: type of surrogate (i.e. imaging, histo/biochemical, instrumental, other), authors’ statement about validation and use of a substitute outcome

7

Methods

Data analyses

Primary Analysis Random-effects meta-analysis

Binary endpoints: TEs expressed as ORs Meta-regression models

Binary endpoints: Ratio of ORs (95%CI) ROR > 1 → greater TEs of the surrogate endpoints Adjustment for key trial characteristics

Sensitivity Analyses Pooled Relative Risk Ratios estimate (RRR) Combined continuous and binary endpoints ROR estimation Within-pair comparison of differences in ln(OR)

Secondary Analysis Logistic regression model

OR of reporting result in favour of the new treatment

Risk of bias assessment χ2 - test of methodological quality dimensions across groups

8

Results

Study characteristicsCharacteristics

Surrogate outcomes (N = 84)

Final outcomes (N = 101)

P-value

Intervention, N(%)     0.33

Pharmaceuticals 49 (58) 61 (60)

Medical Devices 7 (8) 7 (7)

Surgical procedures 4 (5) 8 (8)

Health promotion activities 7 (8) 2 (2)

Other therapeutic technologies 17 (20) 23 (23)

Sponsor, N(%)     0.86

Profit 24 (29) 28 (28)

Not-for-Profit 49 (58) 57 (56)

Mixed 11 (12) 16 (16)  

Sample size, Median (IQR)  371 (162-787)  741 (300-4731) <0.001

Follow up, [days] Median (IQR)  255 (137-540)  180 (40-730) 0.73*Chi-square test, Fisher exact test, Mann-Whitney U test

9

Results

Comparison of TEs – primary analysis

Method of Analysis

(Nr of Surrogate trials vs. Nr of

Final Outcome trials)

Surrogate

outcome Trials

OR (95% CI)

Final outcome

Trials

OR (95% CI)

ROR (95% CI)Adjusted^

ROR (95% CI)

Primary analysis

Binary outcomes

(51 vs. 83)

0.51

(0.42 to 0.60)

0.76

(0.70 to 0.82)

1.47

(1.07 to 2.01)

1.46

(1.05 to 2.04)

ORs = Odds ratios pooled using DerSimonian & Laird random effects meta-analyses. ROR: Relative Odds Ratio; ^Adjusted for trial-level characteristics of clinical area of intervention, patient population, type of intervention, sponsor, journal, mean sample size and mean follow up time

10

Results

Comparison of TEs – sensitivity analyses Method of Analysis

(Nr Surrogate trials vs.

Nr Final Outcome trials)

Surrogate Trials

RR (95% CI)

Final Trials

RR (95% CI)

ROR or RRR

(95% CI)

Adjusted^

ROR or RRR

(95% CI)

Inclusion of risk ratios as

reported by authors

(57 vs. 86)

0.56

(0.48 to 0.65)

0.80

(0.75 to 0.86)

1.38

(1.12 to 1.71)

1.36

(1.08 to 1.70)

Inclusion of continuous

outcomes

(84 vs. 101)

0.46

(0.39 to 0.54)

0.68

(0.62 to 0.74)

1.44

(0.83 to 2.49)

1.48

(0.83 to 2.62)

Binary outcomes

matched-pairs

(43 vs. 43)

0.48

(0.39 to 0.59)

0.68

(0.61 to 0.77)

1.38

(1.01 to 1.88)- 

RRR: Relative Risk Ratio; ^Adjusted for trial-level characteristics of clinical area of intervention, patient population, type of intervention, sponsor, journal, mean sample size and mean follow up time

11

Results

Risk of bias

 Risk of Bias Assessment, N(%)Surrogate

outcomes (N=84)Final outcomes

(N=101)P-value

ITT adoption 62 (74) 83 (82) 0.17

Adequate Randomization sequence generation 54 (64) 65 (64) 0.99

Adequate Randomization allocation concealment 61 (73) 74 (73) 0.92

Double Blinding/Placebo control 42 (50) 43 (43) 0.31

*Chi-square test

12

Between-trial comparison of treatment effects

Possible role of smaller trial sample size in surrogate outcome trials

~40% ‘overestimation’ of TEs in surrogate outcomes trials

Consistent result across sensitivity analyses, confirmed by secondary analyses

Findings not explained by methodological quality or other key trial characteristics

Discussion and limitations

13

Thanks for your attention

Q&A

oriana.ciani@pcmd.ac.uk

14

Main References1. Biomarkers Definitions Working Group. Biomarkers and surrogate endpoints:

preferred definitions and conceptual framework. Clinical Pharmacology and Therapeutics 2001; 69: 89–95.

2. Bucher, H. C. et al. Users' guides to the medical literature: XIX. Applying clinical trial results. A. How to use an article measuring the effect of an intervention on surrogate end points. Evidence-Based Medicine Working Group. JAMA, 1999: 282, 771-8.

3. Fleming TR, DeMets DL. Surrogate endpoints in clinical trials: Are we being misled? Annals of Internal Medicine 1996; 125: 605–13.

4. Lassere M. The Biomarker-Surrogacy Evaluation Schema: a review of the biomarker-surrogate literature and a proposal for a criterion-based, quantitative, multidimensional hierarchical levels of evidence schema for evaluating the status of biomarkers as surrogate endpoints .Statistical Methods in Medical Research 2007; 17: 303–340.

5. Taylor RS, Elston J. The use of surrogate outcomes in model-based cost-effectiveness analyses: a survey of UK Health Technology Assessment reports. Health Technol Assess 2009; 13(8).

6. Weir CJ, Walley RJ. Statistical evaluation of biomarkers as surrogate endpoints: a literature review. Stat Med 2006; 25: 183-203.