Endpoint Adjudication: effects of Adjudication in bias, variability, sample size and study power

© 2014 Syntax for Science SL

Juan V. Torres & Mimmo Garibbo

May 4-5, 2016 | Philadelphia

Effects of central adjudication in bias,

variability, sample size and study power

© 2016 Ethical2 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power

Introduction

Theoretical Background

– Bias

– Variability

– Sample size & power

Simulations

Conclusions & Further Work

AGENDA


INTRODUCTION


DEFINITION

INDEPENDENT REVIEW COMMITTEE

Independent Review Committees

(IRCs) review accumulating data in a

clinical trial and advise the sponsor

(directly or indirectly) on the future

management of the trial.

Endpoint adjudication is an important

task conducted by IRCs.

(EMA (2006) Guideline on data monitoring committees)


CURRENT USE

ENDPOINT ADJUDICATION

Endpoint adjudication is frequently used in clinical

development.

A survey conducted among 140 organizations showed that:

(Krumholz-Bahner et al. 2015; Ethical GmbH 2015)

69%

Adjudication

USA41%

Europe

41%


The main advantages of centralized endpoint adjudication are:

Reduce bias: markedly on unblinded studies (Walovitch et al. 2013) but also

present in blinded studies (Tang et al. 2008); also affected by the subjectivity

and complexity of the endpoint (Walovitch et al. 2013).

Reduce variability: central review by a small number of reviewers with

expertise in a specific area may lessen measurement variability. (Dodd et al.

2008).

Reduce sample size: measurement variability makes treatments appear more

similar than they really are, and therefore leads to reduced power to detect true

treatment effects. A reduction in the measurement variability involves a reduction

in sample size.

ADVANTAGES

CENTRALIZED ENDPOINT ADJUDICATION


Tang et al (2008) evaluated

differences between the

assessment of investigators and

IRCs at the time to measure

response rate (RR) and

progression free survival (PFS) in

phase III clinical trials.

Investigators generally over-

estimated RR compared to IRCs.

No significant differences on PFS.

EXAMPLES: REDUCE BIAS


Tang et al (2008)

18 studies


Krajewski et al (2014) found that interobserver variability

varied from 11% to 18% at the time to measure change from

baseline in lesion size (n=173 lesions).

Agreement between the IRC and INV assessments of PFS

status was 76.3% and 75.5% for paclitaxel alone and

paclitaxel + bevacizumab arms, respectively. (Genentech

2007, AVASTIN Briefing book)

Nagler et al (2013) found discrepancies varying up to 9.7%

among 360 measurements of fibrinogen measured by

different technicians.

EXAMPLES: REDUCE VARIABILITY



It is claimed that centralized endpoint adjudication:

– Increases study complexity

– Increases study cost

– It might be not necessary

DISADVANTAGES



Today’s presentation:

Present key concepts to understand the effect of bias and

variability in terms of sample size and study power.

In working progress:

Assess and provide estimates regarding the effect of bias

and variability in terms of study cost.

Is centralized endpoint adjudication really adding an extra

cost in the study?

OBJECTIVES


THEORETICAL

BACKGROUND


BIAS AND PRECISION


Design

Conduct

Analysis

Interpretation

…

Measurement error

Reader(s)

Subject

Site, time, treatment,…

SOURCES OF BIAS

SOURCES OF BIAS & VARIABILITY

SOURCES OF VARIABILITY


SAMPLE SIZE

Determine the optimum number of subjects required to be

able to arrive at ethically and scientifically valid results.

Given assumptions on:

– Expected treatment effect size

– Expected variability

we calculate how many subjects we need to guarantee a

certain study power preserving the type 1 error.


VARIABILITY, SAMPLE SIZE & POWER

50% Power

0

Expected

effect

Observed

effects

10Expected tx effect

Lack of efficacy Efficacy

90% Power

0 10Expected tx effect

n = 64, σ = 100 n = 172, σ = 100

n = 64, σ = 60


Two ways to reduce confidence intervals, i.e., increase power:


(n = 64, σ = 100)

Increase sample size Reduce variability

(n = 86, σ = 100)

(n = 116, σ = 100)

(n = 172, σ = 100)

(n = 64, σ = 100)

(n = 64, σ = 85)

(n = 64, σ = 73)

(n = 64, σ = 60)


Total sample size for a superiority study to compare two

independent groups can be obtained from the following

formula:


Variability

increment

Sample size

increment

1% 2%

5% 10%

10% 21%

20% 44%

It can be seen that an increment

of X% in variability involves an

increment of (X2+2X)% in sample

size.


SIMULATIONS


Iterations: 2000

Subjects: 130

Sites: 5

Treatment effect: 1

Total variability: 4.7

Variance components:

– Measurement error: 10%

– Site: 10%

– Subjects: 80%, 70%, 60%, 50%

– Readers: 0% , 10%, 20%, 30%

SIMULATION PARAMETERS

ASSUMPTIONS FOR THE SIMULATION ANALYSIS

Power ?


RESULTS

Variance components Scenario 1 Scenario 2 Scenario 3 Scenario 4

Sites 10% 10% 10% 10%

Measurement 10% 10% 10% 10%

Subjects 80% 70% 60% 50%

Readers 0% 10% 20% 30%

Power

Without Central reading 76% 77% 79% 83%

With Central reading 76% 79% 85% 90%


CONCLUSIONS

&

FURTHER WORK


The selection of the most efficient design should be mandatory from the

ethics point of view (avoid unnecessary patient involvement).

Use of centralized endpoint adjudication may reduce error in endpoint

assessment and therefore be used as a tool to reduce sample size or

increase power.

CONCLUSIONS


Assess a larger and more realistic variety of scenarios

– Different endpoints (continuous, ordinal, binary, time to event)

Incorporate costs in the simulations

– Total/partial endpoint adjudication

– Quality control (reassessment)

FURTHER WORK


Chen, E. X., G. R. Pond, and P. Tang. 2008. “Influence of Independent Review Committees (IRC) on Assessment

of Response Rate and Progression Free Survival in Phase III Clinical Trials.” ASCO Meeting Abstracts

26(15_suppl): 6567.

Dodd, Lori E. et al. 2008. “Blinded Independent Central Review of Progression-Free Survival in Phase III Clinical

Trials: Important Design Element or Unnecessary Expense?” Journal of Clinical Oncology 26(22): 3791–96.

EMA (2006), Committee for Medicinal Products For Human Use (CHMP). Guideline on data monitoring committees

Ethical GmbH. 2015. “Use of Adjudication Methods in Clinical Trials.”

https://www.ethicalclinical.com/eadjudication/adjudication-references (February 17, 2016).

Genentech 2007. AVASTIN Briefing book.

http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/OncologicDrugsAdvisoryC

ommittee/UCM219228.pdf

Krajewski, Katherine M. et al. 2014. “Intraobserver and Interobserver Variability in Computed Tomography Size and

Attenuation Measurements in Patients with Renal Cell Carcinoma Receiving Antiangiogenic Therapy: Implications

for Alternative Response Criteria.” Cancer 120(5): 711–21.

Krumholz-Bahner, S., M. Garibbo, K. A. Getz, and B. E. Widler. 2015. “An Overview and Analysis Regarding the

Use of Adjudication Methods in EU and US Drug Approvals.” Therapeutic Innovation & Regulatory Science 49(6):

831–39.

Nagler, Michael et al. 2013. “Variability between Laboratories Performing Coagulation Tests with Identical

Platforms: A Nationwide Evaluation Study.” Thrombosis journal 11(1): 6.

Tang P. A., Pond G.R., and Chen E. X. Influence of an independent review committee o assessment of response

rate and progression free survival in phase III clinical trials. Annals of Oncology 21: 19–26, 2010

Walovitch, Richard et al. 2013. “Subjective Endpoints in Clinical Trials: The Case for Blinded Independent Central

Review.” Open Access journal of clinical trials 5(september): 111–17.

REFERENCES


THANKS!

ANY QUESTIONS?

further Info at: www.ethicalclinical.com

http://www.ethicalclinical.com/

Endpoint Adjudication: effects of Adjudication in bias, variability, sample size and study power

Healthcare

Transcript of Endpoint Adjudication: effects of Adjudication in bias, variability, sample size and study power