[Workshop] Implementation of screening (Oct10)

Alex Mitchell www.psycho-oncology.info/workshop

Department of Cancer & Molecular Medicine, Leicester Royal Infirmary

Department of Liaison Psychiatry, Leicester General Hospital

IPOS 2010IPOS 2010

WORKSHOP Day 2

Implementation of Screening:Screening studies, Short methods, HADS and longer methods, implementation, future of screening

WORKSHOP Day 2

Implementation of Screening:Screening studies, Short methods, HADS and longer methods, implementation, future of screening

Schedule Day 2Schedule Day 2

930-10.00 – Introduction to research task 1. design 2. evaluation

10.00-11.00 – T3 Screening in Cancer: Instruments & Validity

Break

11.30 – 12.30 – Group work #2

Lunch

1.30-2.30 – T4 Screening in Cancer: Implementation and future

Break

3.00 – 4.00 – Presentation of Research task

Group Work #2Group Work #2

930-10.00 – Introduction, groups and issues

10.00-11.00 – T1 Basic science of screening

Break

11.30 – 12.30 – Group task #1

Lunch

1.30-2.30 – T2 Symptoms, Burden, Help, Needs in Cancer

Break

3.00 – 4.00 – Evaluation of a screening paper

Group Work #2Group Work #2

Read paper in your group……..

1.What is being tested?

2.What is the comparison?

3.Is the tool effective?

4.Is the tool acceptable?

5.Did the tool make a difference?

T1. Are We Looking for Distress?T1. Are We Looking for Distress?

How Often

What method?

n=226Comment: Frequency of cancer specialists enquiry about depression/distress from Mitchell et al (2008)

1,2 or 3 Simple QQ15%

Clinical Skills Alone73%

ICD10/DSMIV0%

Short QQ3%

Other/Uncertain9% Other/Uncertain

2%

Use a QQ15%

ICD10/DSMIV13%



Cancer StaffCurrent Method (n=226)

Psychiatrists

Comment: Current preferred method of eliciting symptoms of distress/depression



ICD10/DSMIV24%

Short QQ24%

Long QQ8%

Algorithm26%

Short QQ23%

ICD10/DSMIV0%



Cancer StaffIdeal Method (n=226)

Psychiatrists

Effective?

Comment: “Ideal” method of eliciting symptoms of distress/depression according to clinician

T2. Are We finding it?T2. Are We finding it?

How successful are we (routinely)?

Comment: Slide illustrates diagnostic accuracy according to score on DT

11.815.4

30.4 28.9

41.9 42.9 40.7

57.1

82.4

66.771.4

15.8

25.0

26.124.4

19.4 19.0

33.3

21.4

11.8

22.2 14.3

72.4

59.6

43.546.7

38.7 38.1

25.921.4

5.911.1

14.3

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

Zero One Two Three Four Five Six Seven Eight Nine Ten

Judgement = Non-distressedJudgement = UnclearJudgement = Distressed

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Pre-test Probability

Pos

t-tes

t Pro

babi

lity

CHEMO+

CHEMO-

Baseline Probability

COMMU+

COMMU-

Detection sensitivity = 50.6%Detection specificity = 79.4%Overall accuracy = 65.4%.

Comment: Slide illustrates performance of chemotherapy vs community nurses in oncology T125 – Sat am

0

10

20

30

40

50

60

70

80

90

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

GP Accuracy – Detection of Distress by GHQ ScoreGP Accuracy – Detection of Distress by GHQ ScoreMcCall et al (2007) Primary Care Psychiatry - Recognition by Severity

Comment: Slide illustrates raw number of people identified by severity on the GHQ. Although the % detection increases with severity, the absolute number decreased due to falling prevalence

0

0.05

0.1

0.15

0.2

0.25

0.3

Eight

Nine Ten

Eleven

Twelv

eTh

irtee

nFo

urtee

n

Fiftee

nSixt

een

Seven

teen

Eighteen

Ninetee

n

Twen

tyTw

enty-

one

Proportion MissedProportion Recognized

HADS-D

Testing Clinicians: A Meta-AnalysisTesting Clinicians: A Meta-Analysis

Methods (currently unpublished)

12 studies reported in 7 publications. 2 studies examined detection of anxiety, 8 broadly defined depression (includes HADS-T)3 strictly defined depression and 7 broadly defined distress.

9 studies involved medical staff and 2 studies nursing staff.

Gold standard tools including GHQ60, GHQ12 HADS-T, HADS-D, Zung and SCID.

The total sample size was 4786 (median 171).

Testing Clinicians: A Meta-AnalysisTesting Clinicians: A Meta-Analysis

All cancer professionalsSE =39.5% and SP =77.3%.

OncologistsSE =38.1% and SP = 78.6%; a fraction correct of 65.4%.

By comparison nursesSE = 73% and SP = 55.4%; FC = of 60.0%.

When attempting to detect anxiety oncologists managedSE = 35.7%, SP = 89.0%, FC 81.3%.

Presented at IPOS2009

GPs vs Oncologists vs NursesGPs vs Oncologists vs Nurses

Who is better?

Bayesian analysis

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Post

-test

Pro

babi

lity

GP+GP-Baseline ProbabilityNurse+Nurse-Oncologist+Oncologists-

Comment: Doctors appear to be more successful at ruling-in or giving a diagnosis, nurses more successful at ruling out

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Post

-test

Pro

babi

lity

Ave Confidence+

Ave Confidence-


Above Ave Confidence+

Above Ave Confidence-

High Confidence+

High Confidence-

Low confidence = more cautious, fewer false positives, more false negatives

High confidence = less cautious, more false positives, low false negatives

p180

T3. Screening Tools in CancerT3. Screening Tools in Cancer

Clinician Opinion

Patient Opinion

Observation

Interview

Visual

Self-Report

DepressionScreening

DISCS

VA-SES

ET/DT

HAMD-D17

PhysicalGeneral

Signs ofDS

6

CDSS#10

MADRAS10

Trained

ConfidentSkilledClinician

Alone

YALE

SMILEY

Clinicians Methods to Evaluate Depression

Unassisted Clinician Conventional Scales

Ultra-Short (<5) Short (5-10) Long (10+)Untrained Trained

Routine Implementation

Acceptability ?

Accuracy? Accuracy?

vsComment: schematic overview of methods to evaluate depression

example

Clinicians Methods to Evaluate Depression

Conventional Scales

Short (5-10) Long (10+)

HADS-D BDIexample example

Comment: This is a reminder of the structure of the HADS scale, this version adapter for cancer.

HADS – Pros vs ConsHADS – Pros vs ConsADVANTAGES DISADVANTAGES

HADS – Pros vs ConsHADS – Pros vs ConsADVANTAGES

Well knownShort (7 items)Well testedDepression & anxiety coveredSelf-report

DISADVANTAGES

Can be too longValidation stats not goodWhich version?Distress, anger, needs not coveredScoring complexHADS-t not recommendedRoyalty fee

Inadequate Data(n=11)

No data (n= 250)

No reference standard(n= 293)

Accuracy or Validity Analyses(n= 210)

HADS Validity Analyses(n=50)

HADS in CancerInitial Search (n= 768)

ScaleTypes

Sample Size (cases)

HADS-T(n=26)

HADS-D(n=14)

HADS-A(n=10)

Less than 30(n=22)

More than 100(n=8)

30 to 100(n=20)

Review articles (n= 16)

Depression(n=22)

Any Mental Ill Health(n=24)

Anxiety(n=4)

OutcomeMeasure

No interview standard(n=149)

Validity of HADS vs depression (DSMIV)Validity of HADS vs depression (DSMIV)

SE 71.6% (68.3)

SP 82.6% (85.7)

Prev 13%

PPV 38%

NPV 95%

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Post

-test

Pro

babi

lity

HADS+

HADS-


HADS7v8+

HADS7v8-

Depression_HADS-d (7v8)

British Journal of Cancer (2007) 96, 868 – 874

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

DistressThermometer

AnxietyThermometer

DepressionThermometer

AngerThermometer

TenNineEightSevenSixFiveFourThreeTwoOneZero

Comment: Slide illustrates scores on ET tool

ET - Table of Cut-PointsET - Table of Cut-Points

Distress Thermometer

Anxiety thermometer

Depression Thermometer

Anger Thermometer

Help Thermometer Cut-point

Insignificant 39.0 25.6 50.1 55.7 54.3 0,1

Minimal 20.1 22.5 18.3 13.6 15.4 2,3

Mild 16.9 16.5 12.2 10.5 12.2 4,5

Moderate 12.0 14.5 9.8 6.6 6.6 6,7

Severe 11.9 20.8 9.5 13.6 11.2 8,9,10

p130

8%

DT37%

DepT23%

AngT18%

AnxT47%

4%

7%

1%

1%

9%

3%

0%

2%

4%

15%

3%

2%

Nil41%

Non-Nil59%

DT

AnxT AngT

DepT

T4. How Valid Are the ToolsT4. How Valid Are the Tools

Validity of Methods to Evaluate Depression

Unassisted Clinician Conventional Scales

Ultra-Short (<5) Short (5-10) Long (10+)Untrained Trained

DT vs HADS-T Validity (n=660)DT vs HADS-T Validity (n=660)

SE SP AUC CUT

DT – 71.9% 78.4% 0.814 cut point >=4

AnxT – 75.7% 73.4% 0.821 cut point >=5

DepT – 77.6% 82.2% 0.855 cut point >=3

AngT – 77.5% 77.6% 0.823 cut point >=2

HelpT - 69.1% 80.8% 0.809 cut point >=3

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Pos

t-tes

t Pro

babi

lity


HADSd+

HADSd-

HADS-T+

HADS-T-

HADS-A+

HASD-A-

Depression_HADS

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Post

-test

Pro

babi

lity

1Q+1Q-Baseline ProbabilityDT+DT-2Q+2Q-HADSd+HADSd-HADS-T+HADS-T-BDI+BDI-EPDS+EPDS-HADS-A+HASD-A-

Depression_all

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Post

-test

Pro

babi

lity

DT+ [N=4]DT+ [N=4]Baseline Probability1Q+ [N=4]1Q- [N=4]2Q+2Q-DT/IT+DT/IT-HADST+ [N=13]HADST+ [N=13]PDI+PDI-

Mitchell AJ. Short Screening Tools for Cancer Related Distress A Review and Diagnostic Validity Meta-analysis JNCI (2010) in press

Distress

Validity of DT vs depression (DSMIV)Validity of DT vs depression (DSMIV)

SE 80%

SP 60%

PPV 32%

NPV 93%

DT vs DSMIV DepressionDT vs DSMIV Depression

SE SP PPV NPV

DTma 80.9% 60.2% 32.8% 92.9%

DTLeicesterBW 82.4% 68.6% 28.0% 98.3%

DTLeicesterBSA 100% 59.6% 26.8% 100%

BSA = British South Asian BW= British White

T5. How to Choose A Cut-OffT5. How to Choose A Cut-Off

Distress ThermometerDistress Thermometer

Distress Thermometer – Pooled TableDistress Thermometer – Pooled Table

ScoreRansom 2006

Tuinman2008

Mitchell 2009

Lord 2010

Hoffman 2004

Gessler2009

Clover 2009

Jacobsen 2005 Sum

Proportion

Zero 68 38 61 123 14 27 65 71 467 18.4%

One 72 31 42 68 5 26 39 46 329 12.9%

Two 77 22 35 44 5 18 30 54 285 11.2%

Three 65 37 42 46 8 23 45 46 312 12.3%

Four 51 29 29 30 8 7 21 31 206 8.1%

Five 41 46 62 40 11 13 41 48 302 11.9%

Six 38 32 23 28 2 16 26 31 196 7.7%

Seven 36 21 23 38 2 15 32 16 183 7.2%

Eight 18 12 18 29 6 9 19 15 126 5.0%

Nine 16 5 8 14 3 3 13 9 71 2.8%

Ten 9 4 7 20 4 0 9 13 66 2.6%

Sum 491 277 350 480 68 157 340 380 2543

Proportion 19.3% 10.9% 13.8% 18.9% 2.7% 6.2% 13.4% 14.9%

Distress Thermometer – Pooled

Proportion

18 .4 %

12 .9 %

11.2 %12 .3 %

8 .1%

11.9 %

5.0 %

2 .8 % 2 .6 %

7.7% 7.2 %

0.0%

2.0%

4.0%

6.0%

8.0%

10.0%

12.0%

14.0%

16.0%

18.0%

20.0%

Zero One Two Three Four Five Six Seven Eight Nine Ten

Insignificant SevereModerateMildMinimal

p124

50%

British Journal of Cancer (2007) 96, 868 – 874

SampleSample

We analysed data collected from Leicester Cancer Centre from 2008-2010 involving 531 people approached by a research nurse and two therapeutic radiographers.

We examined distress using the DT and daily function using the question:

“How difficult have these problems made it for you to do your work, take care of things at home, or get along with other people?”

“Not difficult at all =0; Somewhat Difficult =1; Very Difficult =2; and Extremely Difficult =3”

Dysfunction in 531 cancer patientsDysfunction in 531 cancer patients

55.7%

34.3%

7.3%

2.6%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Unimpaired Mild Moderate Severe

Unimpaired by DT ScoreUnimpaired by DT Score

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

1 2 3 4 5 6 7 8 9 10 11

18%

DepT23%

Distress69%

Dysfunction76%

0.3%

3% 2%

26%28% 22%

Of the 293 Non-Nil

DysfunctionDistress

DepT

Mean DT Scores?Mean DT Scores?

Unimpaired Mild Moderate Severe

Mean DT Score 2.1 4.1 5.9 6.5

Std Deviation 2.54 3.0 2.56 3.59

Sample Size 296 182 39 14

Simplified DT Range* 0-3 4-5 6-7 8-10

DT distribution by ImpairmentDT distribution by Impairment

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0 1 2 3 4 5 6 7 8 9 10

Typically severely impared

Typically mod impared

Typically mildly impared

Typically unimpared

None at all

Extreme and incapacitating

Very Severe and very disabling

Moderately Severe and disabling

Moderate and quite disabling

Moderate and somewhat disabling

Mild-Moderate and slight disabling

Mild but not particularly disabling

Very mild and not disabling

Minimal but bearable

Minimal and not problematic

None at all

Dt vs DysfunctionDt vs Dysfunction

ROC plot from Book 1

0.00 0.25 0.50 0.75 1.000.00

0.25

0.50

0.75

1.00Sensitivity

1-Specificity

Distress Thermometer(+ve), M(-ve)

Optimal Cut to Define Distress on DTOptimal Cut to Define Distress on DT

At a cut-off of 2v3 (>=3)Sensitivity =67.8%; PPV =60.3%; UI+ = 0.409Specificity = 68.9%; NPV = 70.3%; UI- = 0.484

At a cut-off of 3v4 (>=4)Sensitivity =58.9%; PPV =65.6%; UI+ = 0.386Specificity = 75.9%; NPV = 70.3%; UI- = 0.534

At a cut-off of 4v5 (>=5)Sensitivity =50.9%; PPV =67.85; UI+ = 0.345Specificity = 81.1%; NPV = 67.9%; UI- = 0.55

T6. Screening in Cancer: ImplementationT6. Screening in Cancer: Implementation

Clinician Opinion

Patient Opinion

Comment: Slide illustrates actual gain in meta-analysis of screening implementation in primary care

ScreenRoutine vs At-Risk vs Identified

Low High

Follow-up Care

?? Desire for Help

Meetable Unmet Needs

800 Patients Approached

100 Not Willing (13%) 700 Patients Willing (87%)

500 Staff Willing (71%)TAU

402 Data Collected (80%)Screen Data

Leicester: DT/ET ImplementationLeicester: DT/ET Implementation T177 t680

Pre-Post Screen - DistressPre-Post Screen - Distress

Before After

Sensitivity of 49.7% 55.8% =>+5%

Specificity of 79.3% 79.8% =>+1%

PPV was 67.3% 70.9% =>+4%

NPV was 64.1% 67.2% =>+3%

There was a non-significant trend for improve detection sensitivity (Chi² = 1.12 P = 0.29).

Qualitative Aspects: CommunicationQualitative Aspects: Communication

DISTRESS

43% of CNS reported the tool helped them talk with the patient about psychosocial issues esp in those with distress

28% said it helped inform their clinical judgement

DEPRESSION

38% of occasions reported useful in improving communication.

28.6% useful for informing clinical judgement

2x2 Clinician Help Table : ACTUAL HELP2x2 Clinician Help Table : ACTUAL HELPClinician thinks:Unmet Needs

Clinician thinks no Unmet Needs

Patient Says:Help Wanted

=> Intervention => Low grade

Patient Distressed => Intervention =>??

Patient Not distressed orHelp Not Wanted

=> Monitor? => discharge?

2x2 Clinician Help Table : ACTUAL HELP2x2 Clinician Help Table : ACTUAL HELP

Clinician thinks:Unmet Needs

Clinician thinks no Unmet Needs

Patient Says:Help Wanted (60)

Helped 21/35 (60%)

Helped 11/23(48%)

Patient Distressed

Helped 65/102(63%)

Helped 31/62(50%)

Patient Not distressed orHelp Not Wanted

Helped 8/35(23%)

Helped 20/117(17%)

b. Intervention and helpb. Intervention and helpPREDICTORS

1. patient desire for help

2. number of unmet needs

3. clinicians confidence

4. patient reported anger

p179

RCT using DT Carlson et al 2010RCT using DT Carlson et al 2010

Screening for Distress in lung and breast cancer outpatients: A randomized controlled trial Linda Carlson Tom Baker Cancer Centre, University of Calgary

1) Minimal Screening: the Distress Thermometer (DT) [n=365]

2) Full Screening: DT, Problem Checklist, Psychological Screen for Cancer (PSSCAN) [n=391] a personalized report

3) Triage: Full screening plus optional personalized phone triage [378]

Advanced AspectsAdvanced Aspects

Algorithms

Structured interviews

Computerized testing

Item-banking

Screening in subgroups

p643

p454

T7. ExtrasT7. Extras

Unfiled

Cancer Population

CNS Assessment

Possible case

Depression

Screen #1+ve

n = 200 No Depression

Sp 55%

Se 70%

n = 800

N = 1000

TP = 140

FP = 360Probable Non-Case TN =440

FN = 60

PPV 28% NPV 88%

Screen #1-ve

YieldTP = 140

TN = 440

FN = 60

FP = 360

NPV 88%

PPV 28%

Sp 55%

Se 70%

Cancer Population

CNS Assessment

Possible case

Depression

Screen #1+ve

n = 200 No Depression

Sp 55%

Se 70%

n = 800

N = 1000

TP = 140


FN = 60

PPV 28%

Oncologist Assessment Sp 80%

Sp 40%

NPV 88%

Probable Depression TP = 56


FN = 84

PPV 44% NPV 77%

Screen #1-ve

Screen #2+ve

Screen #2+ve

Cumulative YieldTP = 56

TN = 728

FN = 144

FP = 72

NPV 83%

PPV 44%

Sp 91%

Se 28%

Credits & Acknowledgments

Elena Baker-Glenn University of NottinghamPaul Symonds Leicester Royal InfirmaryChris Coggan Leicester General HospitalBurt Park University of NottinghamLorraine Granger Leicester Royal InfirmaryMark Zimmerman Brown University, Rhode IslandBrett Thombs McGill University CanadaJames Coyne University of PennsylvaniaNadia Husain University of Leicester

For more information www.psycho-oncology.info

FURTHER READING:

Screening for Depression in Clinical Practice An Evidence-Based guide

ISBN 0195380193 Paperback, 416 pagesNov 2009Price: £39.99

[Workshop] Implementation of screening (Oct10)

Health & Medicine

Transcript of [Workshop] Implementation of screening (Oct10)