Using Stata Graphs to Visually Monitor the

Progress of Multi-centre Randomized Clinical

Trials

Alexandra Whate & Glenn Jones

General Background• Clinical trials are increasingly multi-centre

and global• Routine meetings about trial status become

logistically difficult and expensive to hold• Investigators have different levels of training

and motivation in carrying out clinical trials• Monthly performance monitoring reports have

traditionally been text and table based -adaptations need to be made

Our Context• In 6 trials Investigators from over 30 countries

collect data and submit them to the DMC• Monthly Performance Monitoring Reports are

created by the DMC and submitted to Investigators and Technical Officers (TO)• Used to evaluate investigator performance • Used to monitor study progress

– Accrual, follow-up, survival, adverse events– Provide investigators with the “big picture” to

encourage continued accrual, adherence to protocol and patient follow-up

– Detection of emerging issues or bias

Principles to Optimize Monthly Reports:

• Visual – to overcome language barriers• Relatively Simple – to match the level of

training of Investigators• Discriminating – To easily identify those that

are adhering to protocol and those who are not

• Efficient – Quick to read and understand• Strategic – Motivating Investigators; minimize

bias; ensure patient safety

Applications of Stata

1. Monitoring Accrual2. Monitoring Survival 3. Monitoring Clinical Trajectories4. Monitoring Follow-up

1. AccrualAccrual should be relatively rapidAll centres should participate in

adding patients to the study on a regular basis

Using Stata To Monitor Accrual• Line plots show total accrual over the time of

the study (standard)• Bar graphs show monthly accrual• Bar graphs can also be made ‘by centre’ to

show accrual for each centre involved• These can demonstrate

– Trends in accrual – Interruptions due to issues with staffing and

equipment

0

20

40

60

80

100

120

140cu

mula

tive a

ccr

ua

l

13 3 3

56

7

11

56

2

54

56 6

3

9

6

2

12

6 68

02468

101214

nu

mbe

r of p

atien

ts a

ccru

ed

Oct

200

7

No

v 20

07

De

c 20

07

Jan 2

008

Feb

200

8

Marc

h 2

00

8

Ap

ril 20

08

May 2

00

8

June

200

8

July

20

08

Au

g 2

008

Se

pt 2

00

8

Oct

200

8

No

v 20

08

De

c 20

08

Jan 2

009

Feb

200

9

Marc

h 2

00

9

Ap

ril 20

09

May 2

00

9

June

200

9

July

20

09

Au

g 2

009

Se

pt 2

00

9

Total Accrual

Graph bar var, over(month var, label(angle(ver))) blabel(total) ytitle ylabel(0(2)14) ylabel(,angle(hor)) saving(graphstata) twoway line var var, clwidth(medthick) ylabel(,angle(hor)) xlabel(,nolabel) xtitle("") saving(graphlinestata)

graph combine graphstata.gph graphlinestata.gph, col(1) title(“Total Accrual”)

34

6

12

4

12

3

1

3

1

32

1111 12

12 2

1 11

3222

4

112

12

11

3

11

5

32

1

21

33 32

3

1

32

11

34

211 11

21

0

2

4

6

8

10

0

2

4

6

8

10

Oct

2007

Nov

2007

Dec

2007

Jan 2

00

8F

eb 2

00

8M

arc

h 2

008

April 2

008

Ma

y 2008

June 2

008

July

2008

Aug 2

008

Sept 2

008

Oct

2008

Nov

2008

Dec

2008

Jan 2

00

9F

eb 2

00

9M

arc

h 2

009

April 2

009

Ma

y 2009

June 2

009

July

2009

Aug 2

009

Sept 2

009

Oct

2007

Nov

2007

Dec

2007

Jan 2

00

8F

eb 2

00

8M

arc

h 2

008

April 2

008

Ma

y 2008

June 2

008

July

2008

Aug 2

008

Sept 2

008

Oct

2008

Nov

2008

Dec

2008

Jan 2

00

9F

eb 2

00

9M

arc

h 2

009

April 2

009

Ma

y 2009

June 2

009

July

2009

Aug 2

009

Sept 2

009

Oct

2007

Nov

2007

Dec

2007

Jan 2

00

8F

eb 2

00

8M

arc

h 2

008

April 2

008

Ma

y 2008

June 2

008

July

2008

Aug 2

008

Sept 2

008

Oct

2008

Nov

2008

Dec

2008

Jan 2

00

9F

eb 2

00

9M

arc

h 2

009

April 2

009

Ma

y 2009

June 2

009

July

2009

Aug 2

009

Sept 2

009

Oct

2007

Nov

2007

Dec

2007

Jan 2

00

8F

eb 2

00

8M

arc

h 2

008

April 2

008

Ma

y 2008

June 2

008

July

2008

Aug 2

008

Sept 2

008

Oct

2008

Nov

2008

Dec

2008

Jan 2

00

9F

eb 2

00

9M

arc

h 2

009

April 2

009

Ma

y 2009

June 2

009

July

2009

Aug 2

009

Sept 2

009

Oct

2007

Nov

2007

Dec

2007

Jan 2

00

8F

eb 2

00

8M

arc

h 2

008

April 2

008

Ma

y 2008

June 2

008

July

2008

Aug 2

008

Sept 2

008

Oct

2008

Nov

2008

Dec

2008

Jan 2

00

9F

eb 2

00

9M

arc

h 2

009

April 2

009

Ma

y 2009

June 2

009

July

2009

Aug 2

009

Sept 2

009

Oct

2007

Nov

2007

Dec

2007

Jan 2

00

8F

eb 2

00

8M

arc

h 2

008

April 2

008

Ma

y 2008

June 2

008

July

2008

Aug 2

008

Sept 2

008

Oct

2008

Nov

2008

Dec

2008

Jan 2

00

9F

eb 2

00

9M

arc

h 2

009

April 2

009

Ma

y 2009

June 2

009

July

2009

Aug 2

009

Sept 2

009

Country 1 Country 2 Country 3

Country 4 Country 5 Country 6

nu

mbe

r o

f pa

tien

ts a

ccru

ed

Graphs by centre

Accrual by Centre

graph bar var, by(centre) over(month var, label(labsize(vsmall) angle(ver))) ytitle("number of patients accrued") ylabel(0(2)10) ylabel(,angle(hor)) blabel(total, gap(.25)) title(“Accrual by Centre”)

Monitoring Randomization• We must ensure that randomization

processes are working– Require by the end of accrual that there are

equal number of patients on each treatment arm of the study

– Requires that patients are being placed at an equal rate on the study arms – real time randomization

Two-Arm Trial8

64

20

24

68

nu

mbe

r o

f pa

tien

ts r

an

do

miz

ed

01o

ct2

00

7

01n

ov2

00

7

01d

ec2

00

7

01ja

n2

00

8

01fe

b2

00

8

01m

ar2

00

8

01a

pr2

00

8

01m

ay2

00

8

01ju

n2

00

8

01ju

l20

08

01a

ug

20

08

01se

p20

08

01o

ct2

00

8

01n

ov2

00

8

01d

ec2

00

8

01ja

n2

00

9

01fe

b2

00

9

01m

ar2

00

9

01a

pr2

00

9

01m

ay2

00

9

01ju

n2

00

9

01ju

l20

09

01a

ug

20

09

01se

p20

09

01o

ct2

00

9

Date of Randomization

Arm 1 Arm 2

bihist var, by(var)

Multi-Arm Trial

0

.2

.4

.6

.8

1C

um

ulat

ive

Pro

babi

lity

Jul-

05S

ep-0

5N

ov-

05Ja

n-0

6M

ar-0

6M

ay-0

6Ju

l-06

Sep

-06

No

v-06

Jan

-07

Mar

-07

May

-07

Jul-

07S

ep-0

7N

ov-

07Ja

n-0

8M

ar-0

8M

ay-0

8Ju

l-08

Sep

-08

No

v-08

Jan

-09

Mar

-09

May

-09

Jul-

09S

ep-0

9N

ov-

09

DOrand

Arm 1 Arm 2 Arm 3 Arm 4

cdfplot var, by(txarm) legend(col(4)) xlabel(,angle(ver))

2. Monitoring Survival

Ethical requirement to monitor patterns of mortality over the span of the study

Monitoring Survival• Kaplan-Meier plots allow us to estimate event rates

over time– We create a “survival” variable with date of event or

last follow-up minus the date of randomization gen SURV=date of last follow-up – date of randomization stset SURV, failure(death==1)

• Stata allows for graphical demonstration of survival estimates for different baseline groups (different disease, stage of disease etc) especially important for sample size

0.000.100.200.300.400.500.600.700.800.901.00

41 36 34 28 20 16 13 9 8 7 7 7 5 4 3 3 3 2 2 1 1 1 1 1 1 1 1 1 1 0M1:69 66 62 57 52 46 41 34 31 26 25 23 21 20 18 15 14 12 11 10 8 8 7 5 3 2 0 0 0 0M0:

Risk

0 20

40

60

80

10

012

014

016

018

020

022

024

026

028

030

032

034

036

038

040

042

044

046

048

050

052

054

056

058

0

days from randomization

Disease 1 Disease 2

Kaplan-Meier survival estimates

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

0 50

10

0

15

0

20

0

25

0

30

0

35

0

40

0

45

0

50

0

55

0

60

0

days from randomization

110 88 62 41 32 25 18 13 9 6 3 1 0 Risk

95% CI Survivor function

Kaplan-Meier survival estimate sts graph, cim censored(single) risktable(,title(Risk)) ylabel(,angle(hor)) ylabel(#10) xlabel(#20) xtitle("days from randomization")xlabel(,angle(ver))

sts graph, cim censored(single) risktable(,title(Risk)) ylabel(,angle(hor)) ylabel(#10) xlabel(#20) xtitle("days from randomization")xlabel(,angle(ver))

1

3. Clinical Trajectory

Demonstrates the extent to which protocol-required activities are properly

ordered and on timeDemonstrates the homogeneity or patterns

in clinical trajectories and identifies clear outliers (protocol violation, patient choice)

Clinical Trajectory• In Stata, graphic representations of clinical

trajectories can be created by plotting dates of interest:

1. Date of diagnosis2. Dates of imaging/scans/biopsies3. Date of surgery4. Date of randomization5. Dates of treatment6. Dates of follow ups

• Plot deceased and living patients separately

Jul-07

Oct-07

Jan-08

Apr-08

Jul-08

Oct-08

Jan-09

Apr-09

Jul-09

Oct-09

Oct

-07

No

v-07

De

c-07

Jan

-08

Fe

b-0

8

Mar-

08

Apr-

08

May-

08

Jun

-08

Jul-

08

Aug

-08

Sep

-08

Oct

-08

No

v-08

De

c-08

Jan

-09

Fe

b-0

9

Mar-

09

Apr-

09

May-

09

Jun

-09

date of randomization

Follow-up CT scan Biopsy 1st Treatment2nd Treatment 3rd Treatment Randomization Death

Clinical Trajectories of Deceased Patients

twoway scatter var var var var var var var var if deathflag==1, xti("date of randomization")yti("") xlabel(,angle(ver)) ylabel(,angle(hor)) legend(col(4)) ti("Clinical Trajectories of Deceased Patients") || scatter var var if status==1,

msymbol(x) mcolor(gold) legend(lab(8 "Death")) xlabel(#20) ylabel(#10)

Jul-07Oct-07Jan-08Apr-08Jul-08Oct-08Jan-09Apr-09Jul-09Oct-09

Jul-07Oct-07Jan-08Apr-08Jul-08Oct-08Jan-09Apr-09Jul-09Oct-09

Oct

-07

No

v-0

7D

ec-

07

Jan-

08

Feb

-08

Mar-

08A

pr-0

8M

ay-

08

Jun-

08

Jul-0

8A

ug-0

8S

ep-0

8O

ct-0

8N

ov-

08

De

c-0

8Ja

n-0

9F

eb-0

9M

ar-

09A

pr-0

9M

ay-

09

Jun-

09

Oct

-07

No

v-0

7D

ec-

07

Jan-

08

Feb

-08

Mar-

08A

pr-0

8M

ay-

08

Jun-

08

Jul-0

8A

ug-0

8S

ep-0

8O

ct-0

8N

ov-

08

De

c-0

8Ja

n-0

9F

eb-0

9M

ar-

09A

pr-0

9M

ay-

09

Jun-

09

Oct

-07

No

v-0

7D

ec-

07

Jan-

08

Feb

-08

Mar-

08A

pr-0

8M

ay-

08

Jun-

08

Jul-0

8A

ug-0

8S

ep-0

8O

ct-0

8N

ov-

08

De

c-0

8Ja

n-0

9F

eb-0

9M

ar-

09A

pr-0

9M

ay-

09

Jun-

09

Country 1 Country 2 Country 3

Country 4 Country 5 Country 6

follow-up randomization CT scan biopsy

treatment 1 treatment 2 treatment 3 death

date of randomization

Graphs by centre

twoway scatter var var var var var var var if death==1 , xti("date of randomization")yti("") xlabel(,angle(ver)) ylabel(,angle(hor)) legend(col(4)) || scatter var var, msymbol(x) mcolor(gold) legend(lab(8 “death")) xlabel(#20) ylabel(#10) by(centre)

Scatter Plots for Living Patients

Dec-07Feb-08Apr-08Jun-08Aug-08Oct-08Dec-08Feb-09Apr-09Jun-09Aug-09Oct-09

Jan

-08

Fe

b-0

8

Mar-

08

Apr-

08

May-

08

Jun

-08

Jul-

08

Aug

-08

Sep

-08

Oct

-08

No

v-08

De

c-08

Jan

-09

Fe

b-0

9M

ar-

09

Apr-

09

May-

09

Jun

-09

Jul-

09

Aug

-09

Sep

-09

Oct

-09

No

v-09

Date of Randomization

Follow Up Treatment1 Treatment 2 Treatment 3CT scan Biopsy Randomization

Clinical Trajectories

twoway scatter var var var var var var var, msize(small) xti("Date of Randomization") xlabel(,angle(ver)) ylabel(,angle(hor)) legend(col(4) ti("Clinical Trajectories")yline(18048)xlabe(#25)ylabel(#10)

4. Tracking Follow-UpIdentify patients that are at risk of

being lost to follow-upFollow-up is required by protocol in

specific intervals

Ensuring Regular Follow-Up• Each month we are interested in monitoring the

date of last follow-up for each patient to determine whether follow-up submission is up to date – We can use a strip plot separated by country to show

the date of last follow-up for each patient– Dot plots can be used to isolate patients based on

follow-up date (mlabel)– Goal is to inform Investigators about the specific cases

that are missing follow-up

• Allows synchronization of records

Strip Plot: Follow-Up

stripplot var, over(var) xlabel(,angle(ver)) xline(18110) separate(centre) legend(col(3))

Oct

-07

No

v-0

7

De

c-0

7

Jan

-08

Fe

b-0

8

Ma

r-0

8

Ap

r-0

8

Ma

y-0

8

Jun

-08

Jul-

08

Au

g-0

8

Se

p-0

8

Oct

-08

No

v-0

8

De

c-0

8

Jan

-09

Fe

b-0

9

Ma

r-0

9

Ap

r-0

9

Ma

y-0

9

Jun

-09

Jul-

09

Au

g-0

9

Se

p-0

9

Oct

-09

DOlfu

Country 1 Country 3 Country 5 Country 7 Country 9

Country 2 Country 4 Country 6 Country 8

Dot Plots

IN025

CR009

CR012

IN024 CR010

IN015

IN012

IN018

IN029

INO16

IN027

CH021

IN030 CH001 IC002

IN031 IN032 IN033

IN034

IN035

IN039

IN036

SA015 IN037

IC003

IN038

TL006 CH029

CH026

TL005 CH027 CH025

CH008

CH031 CR013 CR003 CR007

CH019 CH017 CH020 SA025 CH028

CH010 CH012 CH030

SA027 CH006

IN040 CR015 SA023 CH032 CR011 CH033 SA031 SA026 SA033 SA028 SA032 SA034 SA029

CH034

SA030

May-09

Jun-09

Jul-09

Aug-09

Sep-09

date

of l

ast

follo

w-u

p s

ubm

issi

on

dotplot var, ylabel(,angle(hor)) ylabel(#5) mlabel(patid) mlabsize(tiny) xscale(off) yline(18110)

Strategic Monitoring of Follow-up

• Scatter plots and strip plots give a visual clue to the proportion of patients that are out of date for follow-up

• A more sophisticated look at combined follow-up for the whole trial allows us to determine the proportion exactly– Plotting cumulative date of last follow-up with

CDF plots estimates permanent loss to follow-up

Ideal Cumulative Follow-up

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

%

Apr

-09

May

-09

Jun

-09

Jul-

09

Aug

-09

Sep

-09

date of last follow-up

Cumulative Follow-up

CDF Plot - Cumulative Follow-up

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

%

No

v-07

De

c-07

Jan

-08

Fe

b-0

8M

ar-

08

Apr-

08

May-

08

Jun

-08

Jul-

08

Aug

-08

Sep

-08

Oct

-08

No

v-08

De

c-08

Jan

-09

Fe

b-0

9M

ar-

09

Apr-

09

May-

09

Jun

-09

Jul-

09

Aug

-09

Sep

-09

Oct

-09

No

v-09

date of last follow-up

Cumulative Follow-up

Line to denote the time since

which follow-up forms should

have been received

cdfplot var, ylabel(#10) xlabe(#5) xlabel(,angle(ver)) xline(18130) xtitle(date of last follow-up) ytitle (%) legend(col(3))

Determining Contribution• CDF plots can be made by centre to show how each

centre is contributing to the global CDF• Then scatter plots can also be modified to list

specific patients that are behind on follow-up– We generate cumulative date variables cumul var, gen(newvar) and scatter these dates to create a “Scatter CDF plot” – By plotting only those patients that are overdue for

follow-up we can “zoom in” on the CDF plot to identify the specific patients that are contributing to sections 1 or 2 of the overall CDF plot.

Cumulative Follow-Up by Centre

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

%

No

v-07

De

c-07

Jan

-08

Fe

b-0

8M

ar-0

8

Apr

-08

May

-08

Jun

-08

Jul-

08

Aug

-08

Sep

-08

Oct

-08

No

v-08

De

c-08

Jan

-09

Fe

b-0

9M

ar-0

9

Apr

-09

May

-09

Jun

-09

Jul-

09

Aug

-09

Sep

-09

Oct

-09

No

v-09

date of last follow-up

Country 1 Country 3 Country 5 Country 7 Country 9

Country 2 Country 4 Countrry 6 Country 8

Cumulative Follow-up

cdfplot var, by(centre) ylabel(#10) xlabe(#5) xlabel(,angle(ver)) xline(18110) xtitle(date of last follow-up) ytitle (%) legend(col(3))

Scatter CDF

CH021

CH001

CR009CR012

CR010

IN025

IN024

IN015

IN012

IN018IN029

INO16

IN027

IN030

IN031IN032IN033

IN034

IN035IN039

IN036

IN037

IN038

SA015

TL006

.4

.45

.5

.55

.6

.65

.7

cdf d

ate

of la

st fo

llow

-up

Apr

-09

May

-09

Jun

-09

Jul-

09

Aug

-09

date of last follow-up

Country 1 Country 2 Country 3 Country 4 Country 5 Country 6

twoway (scatter cdf var if centre==1, mlabel(patid)) (scatter cdf var if centre==2, mlabel(patid)) (scatter cdf var if centre==3, mlabel(patid)) (scatter cdf var if centre==4, mlabel(patid)) (scatter cdf var if centre==5, mlabel(patid)) (scatter cdf var if centre==6, xlabel(,angle(ver)) legend(col(6)) xlabel(#5) ylabel(,angle(hor)) mlabel(patid))

Follow-Up Over Time

• We can merge serial .dta files (just key variables) to enable simultaneous plotting of each months CDF for dates of last follow-up in patients that are/were alive

• This shows the structure of follow-up across the months of the study

CDF Plot-Merged Over Time

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

Cu

mula

tive

Pro

ba

bili

ty

May-

08

Jun

-08

Jul-

08

Aug

-08

Sep

-08

Oct

-08

No

v-08

De

c-08

Jan

-09

Fe

b-0

9

Mar-

09

Apr-

09

May-

09

Jun

-09

Jul-

09

Aug

-09

Sep

-09

Oct

-09

DOlfu

Jan 2009 Feb 2009 March 2009 April 2009

May 2009 June 2009 July 2009 Aug 2009

Sept 2009

Summary• Stata’s graphing capabilities can be used to monitor

accrual, survival and follow-up status and other aspects of protocol adherence

• The time interval between reports can be short– reports can be produced efficiently using do files

• Minimum Investigator training is required to interpret reports about the whole study and their own performance

• The trial can be visually monitored by TO’s and the DMC

Summary

• Rapid identification of problems by centre and patient can lead to more timely responses

• Different patterns of problems can be detected for which different strategies for resolution can be pursued – this can give direction to Investigators as to how to solve problems

• Using Stata to visually monitor trails helps to improve patient safety and trial quality

Questions?