STUDY DESIGNS – WHICH ONE IS BEST?

David Nunan, PhDResearch Fellow and Tutor

Nuffield Department of Primary Care Health Sciences and

Oxford Centre for Evidence Based Medicine

University of Oxford

Exercise

• What is the best study design?

• Why?

• Compare with your neighbour

Does thalidomide cause phocomelia?

You should be happy if when you leave today, you understand…

• Observational studies• Descriptive studies

• Case studies, case series

• Analytic studies

• Cohort, case-control, cross-sectional

• Experimental studies (randomised trials)

• Qualitative studies

Basic

principles of

study design

THE LANCET 2002;359:57- 61

Bias in study design

Randomised controlled trials

Cohort studies

Case-control studies

Cross-sectional studies

Clinical observation (case

reports, case-series)

Least biased

Most biased

Experimental

Observational

Practicing EBM – the 4 A’s

Ask a

clinical

question

Acquire

the best

evidence

Appraise

the

evidence

Apply

the

evidence

Step 1

Step 2

Step 4

Step 3

Clinical questions:

Interventions

Aetiology and risk factors

Diagnosis

Prognosis

Frequency and rate

What study designs should you be looking for?

Question Question type Best study design

What should I do about

this condition or

problem?

INTERVENTION RANDOMISED

CONTROLLED TRIAL

(RCT) > cohort

What causes the

problem?

AETIOLOGY AND RISK

FACTORS

RCT (?) > cohort > case

control

Does this person have

the condition or problem?

DIAGNOSIS CROSS-SECTIONAL

(random or consecutive

sample)

What will happen with

this problem?

PROGNOSIS AND

PREDICTION

COHORT (survival) >

case control > case

series

How common is the

problem?

FREQUENCY AND

RATE

CROSS SECTIONAL

(prevalence)

COHORT (incidence)

CASE REPORT/SERIES

Case reports

Conclusions – case reports/seriesAdvantages• Can help identify new trends or diseases• Can have significant influence on subsequent literature and

possibly on clinical practice• Often, report rare conditions for which trials may not be feasible.

Disadvantages• Generally short term• Investigator selection bias• Generally no controls

Design pitfalls to look out for• Patient(s) should be described in detail• Carefully reported, unbiased observations• Explore and infer, not confirm, deduce or prove

Overview

• Observational studies

• Descriptive studies• Case studies, case series

• Analytic studies• Cohort, case-control, cross-sectional

• Experimental studies (randomized trials)

• Qualitative studies

Analytical Studies

What is the study design?

Analytical Studies

Lowest = furthest from base station

Highest = nearest to base station

Is there a better study design to answer the question –

Mobile phone masts and childhood cancer?

Advantages• Good for studying rare conditions or diseases• Useful as initial studies to establish association• Can answer questions that can’t be answered by other designs

Disadvantages• Retrospective studies have more problems with data quality

(recall bias)• Limited to examining one outcome• Not good for study of rare exposures• It can be difficult to find a suitable control group

Design pitfalls to look out for• Confounding – exposure and outcome related to third variable

Conclusions – case control studies

What is the study design?

Analytical Studies

Bias in cross-sectional studies

Risk of ASthma in Poultry breeders

The RASP study – a cross sectional survey

How might risk be underestimated?

Bias in cross-sectional studies

Peptic Ulcers and milK ingEstion

The PUKE study – a cross sectional

study

R² = 0,9713

0

1

2

3

4

5

6

7

8

9

0 2 4 6 8

# o

f p

ep

tic

ulc

er

ca

se

s (

we

ek

ly)

Milk consumption (litres/day)Does milk cause peptic

ulcers?

Advantages• Good for assessing prevalence/burden of disease• Quick and easy to conduct (no follow-up)• Multiple outcomes and exposures can be studied• Useful to identify hypothesis/relationships that can be followed

up in experimental studies

Disadvantages• Cannot measure change in variables over time• Inappropriate for demonstrating causal relationships• Not suitable for rare diseases or disease of short duration• Non-responder bias (surveys)

Conclusions – cross sectional studies

Analytical Studies

• Prospective or retrospective

• Controlled

• Can determine causes and incidence of diseases as well as identify risk factors

• Generally expensive and difficult to carry out (prospective)

• Cheaper and quicker = retrospective

Cohort studies

Cohort Design - Prospective

Time

Study begins here

Study

population

free of

disease

Exposure

present

Exposure

absent

Disease/outcome

No disease/outcome

Disease/outcome

No disease/outcome

present

future

Cohort Design - Retrospective

Time

Study begins here

Study

population

free of

disease

Exposure

present

Exposure

absent

Disease/outcome

No disease/outcome

Disease/outcome

No disease/outcome

present

future

• 2748 men and women, 50 – 79 years• HDL-C measured at baseline• Followed for 12 years• Death from all causes, CHD, and CVD• No loss to follow-up

• Low levels of HDL-C associated with increase mortality

• Risk of death from all-causes 2x higher in low HDL-C versus high HDL-C

• Risk of death from CVD and CHD 4x higher in low versus high HDL-C

Advantages• Subjects in cohorts can be matched, which limits the influence of

confounding• Standardisation of criteria/outcome is possible• Easier and cheaper than a RCT

Disadvantages• Cohorts can be difficult to identify due to confounding variables• No randomisation, which means that imbalances in patient

characteristics could exist• Blinding/masking difficult• Can take long time for outcome of interest to occur

Conclusions – cohort studies

Confounding

• Other patient features/causal factors, apart from the one being measured, that can affect the outcome of the study e.g..

Why random allocation?

Odds ratios were exaggerated by 41% for inadequately concealed

trials and by 30% for unclearly concealed trials (adjusted for other

aspects of quality).

Non-randomised controlled trials: • Can detect associations between an intervention and an

outcome

• But, they cannot rule out the possibility that the association was caused by a third factor linked to both intervention and outcome.

• Failure to conceal random allocation exaggerates estimates of treatment effects

Time

Study begins here

Study

population

free of

disease

Exposure

present

Exposure

absent

Disease/outcome

No disease/outcome

Disease/outcome

No disease/outcome

present

future

RANDOMISATION

Difference between randomized trials and non-randomized studies

Randomized trials

• Random (and hopefully concealed) procedure decides which participants go in which arm. (Rules out allocation bias)

• Placebo controls and double blinding possible.

Non-randomized/Cohort studies

• Participants ‘self-select’ or are selected to receive an intervention/exposure

• Placebo controls and double blinding not possible.

What study design(s) would you choose?

1. What is the importance of patient preferences in the choice of treatment for benign prostatic hyperplasia? (1 min)

2. How will you identify and describe misunderstandings between patients and doctors associated with prescribing decisions in general practice? (1 min)

3. How can you explore how men and women with cancer talk about using the internet? (1 min)

Overview

• Observational studies• Descriptive studies

• Case studies, case series

• Analytic studies

• Cohort, case-control, cross-sectional

• Experimental studies (randomized trials)

• Qualitative studies

What is qualitative research?

• Study of “things” (phenomena) in their natural settings

• Make sense of, or interpret, phenomena in terms of meanings people bring to them

• Aim to get a “deeper significance” of what study participants ascribe to the research topic

• Interpretive, naturalistic approach

• Priority is how participant(s) “experiences” contribute to the research question

What are some qualitative research methods?

• Participant observation – data on naturally occurring behaviours in their usual contexts

• Focus groups – effective in eliciting data on cultural norms of a group and generating broad overviews of issues in groups

• In-depth interviews – optimal for collecting data on individuals’ personal histories, perspectives, and experiences

You should be happy if when you leave today, you understand…

• Observational studies• Descriptive studies

• Case studies, case series

• Analytic studies

• Cohort, case-control, cross-sectional

• Experimental studies (randomised trials)

• Qualitative studies

THANK YOU FOR LISTENING!

david.nunan@phc.ox.ac.uk

THANK YOU FOR LISTENING!