Don’t be the fool on the hill! The role of a ...Don’t be the fool on the hill! The role of a...

Don’t be the fool on the hill!The role of a biostatistician in the medical research processGeorg Heinze

Medical University of Vienna

Center for Medical Statistics, Informatics, and Intelligent Systems

CeMSIIS - Section for Clinical Biometrics

Georg Heinze

1

The fool on the hill (P. McCartney 1968)


Georg Heinze

2

Lyrics Analogy

Day after dayAlone on a hillThe man with the foolish grinIs keeping perfectly stillBut nobody wants to know himThey can see that he's just a foolAnd he never gives an answer

The statistician working in the academic ivory tower

…doing strange things…

Researchers do not want to work with the statistican

He does not provide useful answers

But the fool on the hillSees the sun going downAnd the eyes in his headSee the world spinning round

Statistician knows what would be right

The fool on the hill (P. McCartney 1968)


Georg Heinze

3

Lyrics Analogy

Well on the wayHead in a cloudThe man of a thousand voicesTalking perfectly loud

But nobody ever hears himOr the sound he appears to makeAnd he never seems to notice

It needs ‚speaking in the language of the medicalresearchers‘

And nobody seems to like himThey can tell what he wants to doAnd he never shows his feelings

Empathy for the medical research project!

He never listens to themHe knows that they're the foolsThey don't like himBut the fool on the hill…

What is the medical research question?

Don‘t be the fool on the hill!


Georg Heinze

4

• The fool on the hill does not provide consulting services

• … loses connection to medical science

• … has no empathy for the medical research project

• Though he has a good knowledge on what would be right

• He does not take the effort to translate his knowledge to the medical researcher

• He does some research, but this is mostly ignored by the medical community

Personal statement


Georg Heinze

5

• Research:

• Clinical collaborative papers

• Methodological papers

• Interdisciplinary research projects

• Methodological research projects

• Teaching activities

• Advice to undergraduate and graduate students

• Practical ‚Statistical Consulting‘ for students of statistics

• Regular advising activities


Georg Heinze

6

A roadmap: the role of a biostatistician


Georg Heinze

7

• 1. Creative and technical aspects

• 2. Personal aspects – about our and their expectations

• 3. Infrastructure: possible models

• 4. Experience reports

• 5. Final remarks

On the role of the biostatistician


Georg Heinze

8

• The biostatistician‘s input to medical research needs some skills:

• A creative aspect:

• A technical aspect:

Experience with data

Common sense

Mathe-matics

Communication FormalizationTranslation

Examples of a biostatistician‘s input: design phase


Georg Heinze

9

• Creative aspect:

• The ideal clinical endpoint is often binary, but resources do not allow for a large study

• Finding a suitable continuous endpoint

• Is a control group always needed?

• Technical aspect:

• Translates the meaning of the expected effect size as ‚clinically relevant difference‘

• Estimates SD from literature data or from logical reasoning

• Performs sample size calculation

• Describes sample size calculation concisely for study protocol

Examples of a biostatistician‘s input: data analysis


Georg Heinze

10


• Knows about properties of methods (repeated-sampling properties)

• Selects a method with good properties

• Tries to avoid common mistakes(dichotomania, no proper validation, no adjustments for multiple testing, …)


• Is able to use computer code

• Can perform ‚looped‘ analyses (e.g. bootstrap, CV)

• Can implement complex analyses

• Sets up R project with markdown for reproducibility

Examples of a biostatistician‘s input: inference


Georg Heinze

11


• Data-driven decisions can severely impact validity

• E.g. reporting only ‚most significant‘ outcome, variable selection, missing data

handling, …

• Has a feeling of which approach is optimal (without checking with data)

(uses background knowledge efficiently)


• Accurate description of assumption, limitations, data-driven decisions etc.

• Methods, results, tables, figures are prepared such that misinterpretation is avoided

Examples of a biostatistician‘s input: discussion


Georg Heinze

12


• Recognizes limitations, dependency on assumptions etc.

• Can make conclusions on generalizability of the results


• Can perform sensitivity analysis to check robustness

• Can perform simulation studies to evaluate the dependency of main results on

assumptions

• ‚Translates‘ limitations etc. to medical researcher

I am a medical researcher. I need …


Georg Heinze

13

• Support in study design, sample size justification

• Support in writing statistical methods parts for my study protocol

• Support in revising my study protocol after IRB evaluation

• Support in setting up randomization

• Support in writing statistical analysis plans

• Support in data analysis

• Support in writing my manuscript (methods&results)

I am a biostatistician. I want …


Georg Heinze

14

• to do research on study design and data analysis

• to derive new mathematical procedures

• understand the asymptotic and small-sample properties of my method

• some data as example to apply my theory

• a tenure track position in academia

• some funding for PhDs

• to learn about the relevance of methodology from real studies

A conflict of interest


Georg Heinze

15

Medical research

Service

♥ Methodologicalresearch! ♥

Service Methods research

• ‚Fame‘ among statisticians

• My model is used by many others

• Mathematical research, a pure pleasure

• Defining my own real world

• Produces credits for tenure-tracks

• Tedious, often with no success

• Difficult to raise funding

• ‚Fame‘ among medical researchers

• Often with quick success

• Hop on projects

• Coping with the real world

• Reproducing others‘ models

• Medical research, I don‘t understand it anyway

• Does not produce credits for tenure-tracks


Georg Heinze

16

Possible models I


Georg Heinze

17

• Biostatistician is service provider

• No academic career possible, only ‚staff scientist‘

• Still, this can be a safe and attractive position, if some conditions are given:

Medical research

Service

Expectations: the ideal case


Georg Heinze

18

• What a consulting statistician expects

• A relevant research question

• Finding the ‚best‘ methodology: a plausible model, meaningful assumptions,

estimation with high accuracy

• Focusing on a few, not many projects

• A clear research plan and interpretation of results according to it

• A clean data base to work on

• A junior statistical programmer doing the hard work

Expectations: the no-gos


Georg Heinze

19

• What a consulting statistician doesn‘t expect

• Searching a data base for a publishable research question

• Having to find methodology

that gives the ‚most significant‘ result no matter which assumptions

• Having to use outdated methodology

because ‚the reviewers won‘t understand new stuff‘

• Stupid cross-tabulating and 100‘s of t-tests

• Fine-formatting graphics in 1200 dpi

• Solving all kinds of data base and computer problems

Possible models II


Georg Heinze

20

• Biostatistician only performing methods research

• If successful, biostatistician will pursue academic career

• Difficult to realize in a medical school

Medical research

Methodologicalresearch


Georg Heinze

21

• Relevance of position will be questioned

• Risk of becoming a ‚fool on the hill‘

• Or better to contribute to a properly conducted medical study?

Medical research

What‘s he doing?

Hooray! I finallypublished the

10th iteration ofthe extendedversion of …

After some years …

…or should I better collaborate

with him?

Possible models III


Georg Heinze

22

• Joining forces for both medical and methodological research projects

Medical research


Possible models III


Georg Heinze

23

• Joining forces for both medical and methodological research projects

Medical research


stimulate

enhance

Experience report 1: partnership in an interdisciplinary research project


Georg Heinze

24

• ‚Syskid‘ was a EU-funded project involving 25 partners receiving 16M€ funding

• I was involved in setting up the consortium and the steering group

• I led one of two statistical partner groups (~400k€)

• It was a huge effort

• Although the goal of the project was biomarker discovery for kidney disease,

we could publish 17 papers with a statistician as 1st author

• A temporary position was turned into a tenure track, finally permanent position

• Several follow-up projects: e.g. PROVALID, BeatDKD, CaReSyAn

Rainer Oberbauer, MD

Experience report 2:Prediction models

CeMSIIS - Section for Clinical Biometrics cemsiis.meduniwien.ac.at/kb

Georg Heinze

25

• What I have predicted in my career:

• Probability of preterm birth

• Survival after:

• Cervical cancer diagnosis

• Kidney donation

• Kidney transplantation

• Ovarian cancer diagnosis

• Cardiovascular events

• Progression of kidney disaeas

• Monitoring of kidney function in diabetics

• Adenomae in coloscopies

• …

• Recurrence of deep vein thrombosis

Goscinny & Uderzo (1975): Asterix and the soothsayer, Orion paperback

Experience report 2:Prediction of recurrence of venous thromboembolism


Georg Heinze

26

• Goal: identify patients at low risk

for recurrence, as they might

unneccessarily suffer from

the side effects of

anticoagulation

• A large prospective cohort was

available

Dr. Sabine Eichinger

The Viennaprediction model


Georg Heinze

27

Points0 10 20 30 40 50 60 70 80 90 100

Sexfemale

male

Localizationdistal pulmonary embolism

proximal

DDimer100 150 200 250 400 500 750 1000 1500 2000

Total Points0 50 100 150 200 250 300 350

12 months cumulative recurrence rate0.02 0.04 0.06 0.08 0.1 0.12 0.15

60 months cumulative recurrence rate0.1 0.2 0.3 0.4 0.5

Three variablesselected from ~8

Vienna Prediction Model


Georg Heinze

28

2008 2010 2012 2014 2016 2017

First trials with regression trees and simple Cox models.Model development: only three variables selected.

Publication in Circulation. Editor‘s Pick.



Georg Heinze

29


Georg Heinze

30



Georg Heinze

31

2008 2010 2012 2014 2016 2017

First trials with regression trees and simple Cox models.Model development: only three variables selected.

Publication in Circulation. Editor‘s Pick.Web implementation.Press echo.

Production of an App.First external validation studies.

Alternative models published.Started VALID validation study.Update: Dynamic VPM (JAHA).

Systematic Review.NL-Validation study.


Georg Heinze

32

Validating the clinical benefit


Georg Heinze

33

First unprovoked venous thromboembolism

OAC for >3 and <6months

Stop of OAC Patient information/informed consent

3 weeks after stop of OAC D-Dimer testing and calculation of recurrence risk according to VPM

VPM > 180 points VPM < 180 points

Treatment accordingto current guidelinesleft at the discretion

of the treatingphysician

No furtheranticoagulant

treatment

Prospective follow-upClinic visits after 3, 12, and 24 months

Figure 3: Flowchart of the study plan

Recurrence rate at 12 months <5% (??)

Lessons from Vienna Prediction Model project


Georg Heinze

34

• ‚Stimulating Environment‘: partnership with mutual respect

• Publications: Quality > Quantity

• „Simple“ model better than „complex“ model:

• Clinical application

• Stability vs. flexibility

• Further collaboration, funding a statistical PhD

• Model was discussed among the experts (Moons, Harrell)

• DVPM stimulated further research on shrinkage factors (Dunkler JStatSoft 2016)

• International cooperation, R-Software, …

Uexküll‘s notion of ‚Umwelt‘


Georg Heinze

35

• How do you train a guide dog (for the blind)?

• Umwelt = ‚Self-centered world‘

Jakob von Uexküll(1864-1944)

Recipe for successful partnership


Georg Heinze

36

• Statisticians recognize the relevance of the research project

• Medical researchers recognize the added value that statisticians may give to their

project, but also their expectations

• Both extend their ‚Umwelt‘ to include also their partner

Don’t be the fool on the hill! The role of a ...Don’t be the fool on the hill! The role of a...

Documents

Transcript of Don’t be the fool on the hill! The role of a ...Don’t be the fool on the hill! The role of a...