MRCPsych10 - How to analyse diagnostic and prognostic studies

MRCPsych 2010

A. Critical Appraisal of Diagnostic Tests

Alex J MitchellConsultant in Liaison PsychiatryUniversity of Leicester

MRCPsych Teaching 2010 www.slideshare.net/ajmitchell

Studies of Accuracy, Validity, Screening & Case finding

1. Importance of understanding diagnostic tests

1. Importance of diagnostic tests

2. Concept of diagnostic tests: traits to diseases

3. Statistics of diagnostic tests

4. Clinical Value of diagnostic tests

5. Worked examples

6. Advances techniques

MRCPsych 2010

What Is a Diagnostic Test in Psychiatry?

• CT/MRI• CSF• Blood tests eg TFTs• SCAN/SCID/PSE/MINI• Neuropsychological Testing• MMSE• HADS/BDI/CESD?• Clinical Judgement• Self-report

MRCPsych 2010

Why Is a HADS score not a diagnosis?

1. No core features2. No symptom ranking3. No functional assessment4. Duration unclear5. What if Missing items?6. Imprecise

MRCPsych 2010

Defining Diagnostic Testing

INTENTION• Screening

– The systematic application of a test or inquiry, to identify individuals at sufficient risk of a specific disorder to warrantfurther actions among those who have not sought medical help for that disorder

• Case-Finding– The selected application of a test or inquiry, to identify

individuals with a suspected disorder and exclude those without a disorder, usually in those who have sought medical help for that disorder

Adapted from Department of Health. Annual report of the national screening committee. London: DoH, 1997.

MRCPsych 2010


PRACTICAL• Screening

– Rule out those without the disorder with high accuracy (high NPV)

• Case-Finding– Rule in those with the disorder with high accuracy

(high PPV)

MRCPsych 2010


APPLICATION• Routine Screening

– The systematic application of a test or inquiry, to all individuals who may have (or who have not sought medical help for that disorder)

• Targeted (High Risk)– The highly selected application of a test or inquiry, to identify

individuals at high risk of a specific disorder by virtue of known risk factors

Adapted from Department of Health. Annual report of the national screening committee. London: DoH, 1997.

MRCPsych 2010


COMPARATOR

• Accuracy (aka convergent validity)– The degree of approximation (veracity) to a robust comparator

• Validity (aka criterion validity)– The degree of approximation (veracity) to a criterion reference

• Precision– The degree of predictability (low SD) in the measure

MRCPsych 2010

Aims of Detection

• Screening:– Short; Easy; some false +ve (low SpS PPV), few false

–ve (High Sens, NPV)

• Diagnosis (case-finding)– Accurate, Few false +ve or –ve

• Rating– Simple, patient rated, correl. With QoL and other

outcomes

MRCPsych 2010

UK National Screening Committee Guidelines

• The condition should:• • Be an important health issue• • Have a well-understood history, with a detectable

risk factor or disease marker• • Have cost-effective primary preventions

implemented.

• The screening tool should:• • Be a valid tool with known cut-off• • Be acceptable to the public• • Have agreed diagnostic procedures.

• The treatment should:• • Be effective, with evidence of benefits of early

intervention• • Have adequate resources• • Have appropriate policies as to who should be

treated.

• The screening program should:• • Show evidence that benefits of screening

outweighing risks• • Be acceptable to public and professionals• • Be cost effective (and have ongoing evaluation)• • Have quality-assurance strategies in place.• Adapted from: UK National Screening Committee

Criteria for appraising the viability, effectiveness and appropriateness of a screening programme

• http://www.nsc.nhs.uk/pdfs/criteria.pdf

MRCPsych 2010

In this last step the screening tool /method is introduced clinically but monitored to discover the effect on important patient outcomes such as new identifications, new cases treated and new cases entering remission.

Screening implementation studies using real-world outcomes

ImplementationPhase IV_screen

This is an important step in which the tool is evaluated clinically in one group with access to the new method compared to a second group (ideally selected in a randomized fashion) who make assessments without the tool.

Screening RCT; clinicians using vs not using a screening tool

ImplementationPhase III_screen

The aim is to assess the refined tool against a criterion (gold standard) in a real world sample where the comparator subjects may comprise several competing condition which may otherwise cause difficulty regarding differential diagnosis.

Diagnostic validity in a representative sample

Diagnostic validityPhase II_screen

The aim is to evaluate the early design of the screening method against a known (ideally accurate) standard known as the criterion reference. In early testing the tool may be refined, selecting most useful aspects and deleting redundant aspects in order to make the tool as efficient (brief) as possible whilst retaining its value.

Early diagnostic validity testing in a selected sample and refinement of tool

Diagnostic validityPhase I_screen

Here the aim is to develop a screening method that is likely to help in the detection of the underlying disorder, either in a specific setting or in all setting. Issues of acceptability of the tool to both patients and staff must be considered in order for implementation to be successful.

Development of the proposed tool or test

DevelopmentPre-clinical

DescriptionPurposeTypeStage

Development of Diagnostic Tests

Citation: Mitchell AJ. Screening for depression in clinical practice: evidence based approach

2. Concepts of Diagnostic Tests: Trait / Syndrome / Disease

MRCPsych 2010

Graphical – Screening principles

Non-Depressed

Depressed

# ofIndividuals

# ofIndividuals

Severity of Depression

MRCPsych 2010


Non-Depressed

Depressed

# ofIndividuals

Cut-Off

# ofIndividuals


HighLow

High Sensitivity >>>>

<<<< high Specificity

MRCPsych 2010


Non-Depressed

Depressed

# ofIndividuals

Cut-Off

# ofIndividuals


HighLow

High Sensitivity >>>>

<<<< low Specificity

MRCPsych 2010

Graphical – Definition of NPV

Non-Depressed

Depressed

Cut-OffHighLow

True +ve

True -ve

True +ve / ALL +ve = PPV

False alarms

MRCPsych 2010

Graphical – Definition of PPV

Non-Depressed

Depressed

Cut-OffHighLow

True +ve

True -ve

True –VE / ALL -ve = NPV

Missed cases

MRCPsych 2010

Theory of Diagnostic Tests

Non-Depressed

Depressed# ofIndividuals

TestResult

Cut-off value

False +veFalse -ve

True -ve

True +ve

MRCPsych 2010

Low Prevalence (Se Sp = same)

Non-Depressed

Mj Depression# ofIndividuals

TestResult

Cut-off value

False +veLARGE

False –veSMALL

MRCPsych 2010

High Prevalence (Se Sp = same)

Non-Depressed Mj+Mn Depression

# ofIndividuals

TestResult

Cut-off value

False +veSMALL

False –veLARGE

Can This Help establish a syndrome?

Example: A Clear Disease [#1]

Disorder

Number ofIndividuals

False +veFalse +ve

True ‐veTrue ‐ve

Point of Partial Rarity

Test Result

No Disorder

False ‐veFalse ‐ve

True +veTrue +ve

Example: A Probable Syndrome [#2]

Disorder


False +veFalse +ve False ‐veFalse ‐ve


True +veTrue +ve

MMSE Cognitive Score

No Disorder

Example: A Normally Distributed Trait [#3]

Disorder


False +veFalse +ve False ‐veFalse ‐ve


True +veTrue +ve

MMSE Cognitive Score

No Disorder

MRCPsych 2010

Example: Dementia

Disease?Syndrome?Trait?

MRCPsych 2010

Hubbert et al (2005) BMC Geriatrics

MMSE scores for dementia (n=72)and non-dementia (n=2735)

Huppert et al BMC Geriatrc 2005

MRCPsych 2010

Example: Depression

DiseaseSyndromeTrait

MRCPsych 2010

Thompson et al (2001) n=18,414

0

500

1000

1500

2000

2500

3000

Zero One

TwoThree Four

Five SixSev

en

eight

Nine

TenEleve

nTwelv

eThirt

een

Fourtee

nFifte

enSixtee

nSev

entee

nEightee

n

MRCPsych 2010

Mitchell, Coyne et al (2008)

0

10

20

30

40

50

60

70

80

90

100

110

Early Pregnancy3months Post-Partum12months Post-Partum

Scores on the CES-D during Pregnancy, 3 and 12 months Post-partum in 947 Women

Depressive Symptoms Moderate to Severe DepressionHealthy Mild Depression

MRCPsych 2010

PHQ9 Linear distribution

0

5

10

15

20

25

30

35

Zero One Two

Three

Four

Five Six

Seven

Eight

Nine

TenElev

enTwelveThir

teen

Fourte

enFifte

enSixt

een

Sevente

enEigh

teen

PHQ9 (Major Depression)PHQ9 (Minor Depression)PHQ9 (Non-Depressed)

Baker-Glen, Mitchell et al (2008)

3. Statistics of Diagnostic Tests: 2x2s

MRCPsych 2010

Accuracy 2x2 Table

PrevalenceSpecificitySensitivity

NPVTrue -VeFalse -VeTest -ve

PPVFalse +veTrue +veTest +ve

DepressionABSENT

DepressionPRESENT D / B + D

SpA / A + C

SnTotal

D/C + DNPV DC

Test-ve

A/A + BPPV BA

Test+ve

Reference StandardNo Disorder

Reference StandardDisorder Present

MRCPsych 2010

Accuracy 2x2 Table


NPVTNFNTest -ve

PPVFPTPTest +ve

DepressionABSENT

DepressionPRESENT

MRCPsych 2010

Basic Measures of Accuracy

• Sensitivity (Se) a/(a + c) TP / (TP + FN)

• A measure of accuracy defined the proportion of patients with disease in whom the test result is positive: a/(a + c)

• Specificity (Sp) d/(b + d) TN / (TN + FP)• A measure of accuracy defined as the proportion of patients without disease in

whom the test result is negative

• Positive Predictive Value a/(a+b) TP / (TP + FP)• A measure of rule-in accuracy defined as the proportion of true positives in

those that screen positive screening result, as follows

• Negative Predictive Value c/(c+d) TN / (TN + FN)• A measure of rule-out accuracy defined as the proportion of true negatives in

those that screen negative screening result, as follows

MRCPsych 2010

Accuracy in words• Sensitivity

– The chance of testing positive among those with the condition – The chance of rejecting the null hypothesis among those that do not satisfy the null hypothesis

• Specificity– The chance of testing negative among those without the condition– The chance of accepting the null hypothesis among those that satisfy the null hypothesis

• Positive Predictive Value – The chance of having the condition among those that test positive – The chance of not satisfying the null hypothesis among those that reject the null hypothesis

• Negative Predictive Value – The chance of not having the condition among those that test negative – The chance of satisfying the null hypothesis among those that accept the null hypothesis

• Type I Error or α (alpha) or p-Value or false positive rate – The chance of testing positive among those without the condition– The chance of rejecting the null hypothesis among those that satisfy the null hypothesis

• Type II Error or β (beta) or false negative rate – The chance of testing negative among those with the condition – The chance of accepting the null hypothesis among those that do not satisfy the null hypothesis

• False Discovery Rate or q-Value – The chance of not having the condition among those that test positive – The chance of satisfying the null hypothesis among those that reject the null hypothesis

• False Omission Rate – The chance of having the condition among those that test negative – The chance of not satisfying the null hypothesis among those that accept the null hypothesis

MRCPsych 2010

Rule-in Accuracy

PrevalenceSpecificitySensitivity(occurrence)

NPVTrue -VeFalse –Ve

(type II error)

Test -ve

PPV(discrimination)

False +ve(type I error)

True +veTest +ve

DepressionABSENT

DepressionPRESENT

MRCPsych 2010

Rule-Out Accuracy

PrevalenceSpecificity(occurrence)

Sensitivity

NPV(discrimination)

True -VeFalse –Ve(type II error)

Test -ve


DepressionABSENT

DepressionPRESENT

MRCPsych 2010

Likelihood RatiosLikelihood Ratio for Positive TestsThe chance of testing positive among those with

the condition; divided by the chance of testing positive among those without the condition

Sensitivity / (1 - Specificity) [ TP / (TP + FN) ] / [ FP / (FP + TN) ]

= PPV / Prevalence

Likelihood Ratio for Negative TestsThe chance of testing negative among those with

the condition; divided by the chance of testing negative among those without the condition

Specificity / (1 – Sensitivity)[ FN / (FN + TP) ] / [ TN / (TN + FP) ]

= NPV / Prevalence

MRCPsych 2010

Summary Measures

• Youden's J– Sensitivity + Specificity – 1

• Predictive Summary Index– PPV + NPV – 1

• Overall accuracy (fraction correct)– TP+TN / TP+FP+TN+FN

MRCPsych 2010

Reciprocal Measures

• Number Needed to Diagnose (NND)– 1 / (Youden's J)

• Number Needed to Predict (NNP)– 1 / (PSI)

• Number Needed to Screen (NNS)– 1/(FC-FiC)

Murphy JM, Berwick DM, Weinstein MC, Borus JF, Budman SH, Klerman GL 1987 : Performance of screening and diagnostic tests: Application of Receiver Operating Characteristic ROC analysis. Arch Gen Psychiatry 44:550-555

Receiver Operating Characteristic

MRCPsych 2010

Accuracy 2x2 Table


NPVTrue -VeFalse -VeTest -ve


DepressionABSENT

DepressionPRESENT

MRCPsych 2010

Test vs Major Depression

700060001000

50004500500Test -ve

20001500500Test +ve

DepressionABSENT

DepressionPRESENT

Sensitivity50%

PPV 25%

Specificity75%

NPV 90%

Prevalence 14%

MRCPsych 2010

Test vs Major + Min Depression

300020001000

1000500500Test -ve

20001500500Test +ve

DepressionABSENT

DepressionPRESENT

Sensitivity50%

PPV 25%

Specificity33%

NPV 50%

Prevalence 33%

4. Clinical Value of Diagnostic Tests

MRCPsych 2010

Added Value

• Definition 1:– The additional ability of a test to rule-in or rule-out

compared with the baseline rate– PPV minus Prevalence– NPV minus prevalence

• Definition 2:– The additional of a test to rule-in or rule-out compared

with the unassisted rate– PPV test minus PPV no test (assuming equal prevalence)

– LR+ test minus LR+ no test

– AUC test minus AUC no test

MRCPsych 2010

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Loss

of

ener

gy

Dim

inis

hed

driv

e

Slee

p di

stur

banc

e

Con

cent

rati

on/i

ndec

isio

n

Dep

ress

ed m

ood

Anx

iety

Dim

inis

hed

conc

entr

atio

n

Inso

mni

a

Dim

inis

hed

inte

rest

/ple

asur

e

Psyc

hic

anxi

ety

Hel

ples

snes

s

Wor

thle

ssne

ss

Hop

eles

snes

s

Som

atic

anx

iety

Tho

ught

s of

dea

th

Ang

er

Exce

ssiv

e gu

ilt

Psyc

hom

otor

cha

nge

Inde

cisi

vene

ss

Dec

reas

ed a

ppet

ite

Psyc

hom

otor

agi

tati

on

Psyc

hom

otor

ret

arda

tion

Dec

reas

ed w

eigh

t

Lack

of

reac

tive

moo

d

Incr

ease

d ap

peti

te

Hyp

erso

mni

a

Incr

ease

d w

eigh

t

All Case ProportionDepressed ProportionNon-Depressed Proportion

Mitchell, Zimmerman et al MIDAS Database. Psychol Med 2007 Submitted

MRCPsych 2010

-0.10

0.00

0.10

0.20

0.30

0.40

0.50A

nger

Anx

iety

Dec

reas

ed a

ppet

ite

Dec

reas

ed w

eigh

t

Dep

ress

ed m

ood

Dim

inis

hed

conc

entr

atio

n

Dim

inis

hed

driv

eD

imin

ishe

d in

tere

st/p

leas

ure

Exce

ssiv

e gu

ilt

Hel

ple

ssne

ss

Hop

eles

snes

s

Hyp

erso

mni

a

Incr

ease

d ap

peti

te

Incr

ease

d w

eigh

t

Inde

cisi

vene

ss

Inso

mni

aLa

ck o

f re

acti

ve m

ood

Loss

of

ener

gy

Psyc

hic

anxi

ety

Psyc

hom

otor

agi

tati

on

Psyc

hom

otor

cha

nge

Psyc

hom

otor

ret

arda

tion

Slee

p di

stur

banc

e

Som

atic

anx

iety

Thou

ghts

of

deat

h

Wor

thle

ssne

ss

Rule-In Added Value (PPV-Prev)Rule-Out Added Value (NPV-Prev)

MRCPsych 2010

Accuracy of Tests: Visual Post-test Probabilities

0% 100%25% 75%

Very unlikely Very likelylikelyunlikely

2 Questions

Overall

PHQ-2

WHO5 (1+3)

1 Question3% - (37) - 63% = 60%

3% - (16) - 32% = 29%

3% - (16) - 32% = 29%

10% - (22) -50% = 54%

32% - (37) - 96% = 64%

Henckel et al (2004) Eur Arch Psychiatry Clin Neurosci

CIDI (computer) Any Depression

Henckel et al (2004) Eur Arch Psychiatry Clin Neurosci

CIDI (computer) Any Depression

Arroll B et al (2003) BMJ

CIDI (computer) Mj Depression

CIDI (computer) Mj Depression

MRCPsych 2010

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Pre-test Probability

Pos

t-tes

t Pro

babi

lity

Clinician Positive (Fallowfield et al, 2001)

Clinician Negative (Fallowfield et al, 2001)

Baseline Probability

HADS-D Positive (Mata-analysis)

HADS-D Negative (Meta-analysis)

MRCPsych 2010

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Post

-test

Pro

babi

lity

Depression Present (Routine)

Depression Absent (Routine)

Depression Scales +ve (Median)

Depression Scales -ve (Median)

Prior Probability

PPV=0.41

NPV=0. 97

Prevalence of 0.15

5. Worked Examples of diagnostic tests

MRCPsych 2010

PostStroke Mj Depression vs NonMj

• Clinicians diagnosis using DSMIV vs SCAN/PSE

– 50 people with major depression

– 150 healthy people

– 50 with subsyndromal depression

MRCPsych 2010

Clinicians using DSMIV

• IF: Clinicians diagnosed 50 cases with Mj depression• IF: Their specificity was 95%

• Q. What was the sensitivity?• Q. What was the prevalence?• Q. What was the PPV?• Q. What was the % correctly identified per every 100

screened?

MRCPsych 2010


20050

??Test -ve

50??Test +ve(Clinician)

DepressionABSENT

DepressionOn SCAN

Sensitivity50%

PPV ??%

Specificity95%

NPV ??%

Prevalence ??%

MRCPsych 2010


20050

20019010Test -ve

501040Test +ve(Clinician)

DepressionABSENT

DepressionOn SCAN

Sensitivity80%

PPV 80%

Specificity95%

NPV 95%

Prevalence 20%

6. Advanced Techniques

sROCReal World NumbersNND; NNSBivariate meta-analysisEconomics

MRCPsych 2010 ROC Plot

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00

1 - Specifity

Sens

itivi

ty Low Mood

DSMIV

Low mood & loss interest

MRCPsych 2010

MRCPsych 2010

Bivariate Diagnostic meta-analysis

MRCPsych 2010

Further Reading

• David A Grimes, Kenneth F Schulz Uses and abuses of screening tests Lancet 2002; 359: 881–84

• Jonathan J Deeks, Douglas G Altman Diagnostic tests 4: likelihood ratios BMJ VOLUME 329 17 JULY 2004

• Patrick M Bossuyt, Les Irwig, Jonathan Craig and Paul Glasziou Comparative accuracy: assessing new tests against existing diagnostic pathways. BMJ

• 2006;332;1089-1092

• Reitsma JB et al Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. Journal of Clinical Epidemiology 58 (2005) 982–990

MRCPsych 2010

B. Critical Appraisal of Prognostic Tests

Alex J MitchellConsultant in Liaison PsychiatryUniversity of Leicester

MRCPsych Teaching 2010

Risk, predictors, measuring outcomes

MRCPsych 2010

Measuring Risk

Risk– the probability of some untoward event

•e.g., disease, death

Risk Factor– characteristics or behaviours associated with an

increased risk of becoming diseased

Dementia

Healthy

HealthyWith SMC

MCIWith SMC

VaD

ADMixed

LBD

FTD

Healthy

MRCPsych 2010

Modelling Progression on MCI-DementiaD

isea

se S

ever

ity

Time in Years

T4T0 T+8

30

Severe Dementia

Moderate Dementia

Mild Dementia

MCI

Healthy

23v24

20v21

11v12

MM

SE

0

T+12

MRCPsych 2010

Modelling Progression on MCI-DementiaD

isea

se S

ever

ity

Time in Years

T4T0 T+8

30

MCI-ProgressiveModerate Risk

Severe Dementia

Moderate Dementia

Mild Dementia

MCI

Healthy

MCI-ProgressiveHigh Risk

23v24

20v21

11v12

MM

SE

0

T+12

MRCPsych 2010

Accuracy 2x2 Table

PrevalenceSpecificity of predictor

SensitivityOf predictor

NPV of predictor

True -VeFalse -VeRISK -ve

PPV of predictor

False +veTrue +veRISK +ve

OUTCOMEABSENT / GOOD

OUTCOMEPRESENT/ POOR

MRCPsych 2010

Test for AD vs HC…fill in missing cells

1800 (44%)

1000800

Test -ve

700100600Test +ve

MCIAD

Sensitivity75%

PPV 85.0%

Specificity90%

NPV 81%

Mitchell (2005)Meta-analysis

N=14x

MRCPsych 2010

Test for AD vs HC…..good test?

1800 (44%)

1000800

1100900200Test -ve

700100600Test +ve

MCIAD

Sensitivity75%

PPV 85.0%

Specificity90%

NPV 81%


N=14x

MRCPsych 2010

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Post

-test

Pro

babi

lity

Test+

Test-


MRCPsych 2010

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Post

-test

Pro

babi

lity

Test+

Test-


Clinician+

Clinician-

MRCPsych 2010

AD vs HC – P-Tau181

1541689852

833576257Test -ve

708113595Test +ve

MCIAD

Sensitivity69.8%

PPV 84.0%

Specificity83.6%

NPV 69.1%


N=14x

MRCPsych 2010

Classifying Predictors

Demographic

• Age• Gender• Education

Service Related

• Recruitment Setting• Education• Length of follow-up• Delay in diagnosis• Treatment• Size of study

Disease Related

• MCI Type• MCI Subtype

• Structural Imaging• Functional Imaging• CSF Studies• Genetic testing (ApoE4)• Cognitive Testing• Non-memory impairment• Depression/anxiety• Subjective Performance• Functional status• Vascular status

MRCPsych 2010

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

CP1183493-11

Years after enrollmentYears after enrollment

Alive (%)

Alive (%)

NormalsNormals

A-MCI single domainA-MCI single domainAll amnestic MCIAll amnestic MCI

A-MCI multidomainA-MCI multidomain

P<0.023P<0.023

Mayo Data Survival (Kaplan-Mayer)

MRCPsych 2010

Measuring Risk

• Absolute Risk Reduction (ARR)– is the absolute difference in event rates between

the experimental and control patients.

ARR = CER - EER

• Number Needed to Treat (NNT)– is the number of patients a clinician needs to

treat in order to prevent one additional adverse outcome

MRCPsych 2010

Measuring Risk - Examples

C E R E E R R R R A R R N N T

0 .6 0 .4 3 3 % 2 0 % 5

0 .0 6 0 .0 4 3 3 % 2 % 5 0

0 .0 0 6 0 .0 0 4 3 3 % .2 % 5 0 0

RRR remains the same despite differences in absolute rate of events.

ARRs reflect underlying susceptibility of patients

NNTs provide a measure of the clinical effort that must be expended

MRCPsych 2010

Checklist criteria

• Are the study population similar to our patients?

• Is the study design appropriate for the research question?

• Were the study subjects representative of patients with the disease in question?

• Were the patients at a similar point in the course of their disease?

• Were the outcomes objectively-defined, and were the people recording the outcomes blinded to the prognostic factors?

• Were patients followed long enough for outcomes to occur?

• Was the dropout rate excessive?

• Did the authors adjust for differences between groups?

MRCPsych10 - How to analyse diagnostic and prognostic studies

Health & Medicine

Transcript of MRCPsych10 - How to analyse diagnostic and prognostic studies