Session 4 - ISCTM · 2017-10-26 · Session 4 Suicidal Ideation and Behavior: Methodological and...

Session 4 Suicidal Ideation and Behavior:

Methodological and Technical Challenges

2013 Annual ISCTM Meeting Washington, DC

February 20, 2013

ISST-Plus Overview

• A modified version of the ISST (1997), was developed by Alphs and Lindenmayer in 2009

• Components – Part I: Suicidal Ideation—13 items (Time: Last 7 Days) – Part II: Suicidal Behavior (Time: Since last visit) – Part III: Global Suicidal ideation and behavior (Time: Now)

• Severity • Status if subject misses assessment

– Suicide Narrative (optional) (Time: Since last visit) • If the subject reports suicidal behavior (e.g., preparation,

injuries, accidents, passive or active attempt) in Part II, or if the subject completes suicide (documented in Part III), all available information will be reported in the Suicide Attempt Narrative section

• The ISST-Plus takes 15-20 minutes to complete confidential

Results of a Prospective Validation

for Mapping 3 SIB Assessment Instruments to the C-CASA

Larry Alphs, MD, PhD Janssen Scientific Affairs

February 21, 2013

• Employed by J&J • Stock in J&J • Developer of ISST-Plus

Disclaimer

4

• Valuable to have alternative scales to assess suicidal ideation and behavior (SIB)

• These scales must meet FDA requirements and map to the Columbia Classification Algorithm of Suicide Assessment (C-CASA) similar to the Columbia Suicide Severity Rating Scale (C-SSRS)

• Alternatives to the C-SSRS – InterSePT Scale for Suicidal Thinking–Plus (ISST-Plus) – Sheehan Suicidality Tracking Scale (S-STS)

Introduction

5

• To examine the concurrent validity for mapping symptoms of suicidal ideation, self-harm, and suicidal behavior to C-CASA (2010 and 2012) for – ISST-Plus – S-STS (clinician-rated) – S-STS (patient-rated) – C-SSRS

Objectives

6

• Subjects with full range of suicidality from “not present” to “extremely severe”

• Identified from inpatient and outpatient settings • Signed informed consent • Interviewed on the same day on the ISST-Plus, the S-

STS, and the C-SSRS • Independent raters administered the 3 scales in random

sequence for each subject (Figure below) • Time frame: Past 7 days • Separate consent for video recordings

Study Design

1st Interview Rater N

(Scale X)

2nd Interview Rater M (Scale Y)

3rd Interview Rater Q

(Scale Z)

7

• Mental health professionals from one academic site • Experienced in working with suicidal patients • Training provided by 3 scale authors

– semi-structured interviews – training slides and materials – standardized and uniform definitions for all critical SIB

terms • Trained to administer all 3 scales • Selected for participation on an “as available” basis

Raters

8

• Ratings from subjects who completed responses on all 3 instruments were mapped to the C-CASA (2010 and 2012)

– C-SSRS:

• Completed on paper by a trained individual

• Used predefined C-CASA mapping algorithms from the author

– ISST-Plus and S-STS:

• Direct data capture and mapping with computers

• Eliminates the possibility of missing values, double entries, and legibility problems

Instrument Mapping

9

Mapping of Assessment Instrument to C-CASA (2010)

CSSRS, ISST-Plus, S-STS

Completed suicide Suicide attempt Preparatory suicidal behavior Suicidal ideation Other, no deliberate self-harm Fatal, not enough information to classify Self-injurious act, no suicidal intent (NSSI) Self-injurious behavior, intention unknown Nonfatal, not enough information to classify

10

Mapping Algorithms C-CASA 2010 C-CASA

Code Number

C-CASA (Version 2010) Classification

ISST-Plus Map to

C-CASA* (Version 2012)

S-STS Map to

C-CASA (Version 2012)

C-SSRS Map to C-CASA

(Version 2012)

1 Completed Suicide Yes if: Part III coded as 6 A Yes response to 13 A Yes response to Completed

Suicide

2 Suicide attempt Yes if: Yes on Part II, Item 1.2 OR Yes on Part II, Item 1.3

A positive response to 10 or 16 OR A Yes response to 1b

A Yes response to Actual Attempt

3 Preparatory suicidal behavior Yes if: Part II, Item 1.4 = Yes

A positive response to 8 with at least one Level 1 on 12

A Yes response to Preparatory Acts or Behavior

4 Suicidal Ideation

Yes if: (Part I, Item 4 ≥1 OR Part I, Item 5 ≥1) AND Items 10 and 12 ≥1; AND Item 11 = 0

A positive response to 3 and 4 (and 6 or 7) AND A negative response to 5

A Yes response to Item 4 (Active Suicidal Ideation With Some Intent to Act, Without Specific Plan)

5 Other, no deliberate self-harm

Yes if: Part II, Item 1.1 = Yes; AND Part II, Item 1.2 = No

A positive response to 9 OR A positive response to 1a and a negative response to 1b

A Yes response to Has subject engaged in Nonsuicidal Self-injurious Behavior?

6 Fatal, not enough information to classify Yes if: Part III scored 8 A Yes response to 14 —

7 Self-injurious act, no suicidal intent (NSSI)



A Yes response to “Has subject engaged in Nonsuicidal Self-injurious Behavior?”

8 Self-injurious behavior, intention unknown

Yes if: Part II, Item 1.1 = Yes; AND Part II, Item 1.2 = Unknown/NA

A positive response to 1a with 1b and 6 and 7 and 10 unanswered —

9 Nonfatal, not enough information to classify

Yes if: Part II, Item 1.0 = Unknown/NA OR Part II, Item 1.1 = Unknown/NA

Yes if question 17 is yes and C-CASA1-C-CASA13 are ALL No —

11

Mapping Algorithms C-CASA 2010 C-CASA

Code Number


ISST-Plus Map to


S-STS Map to



(Version 2012)


Suicide

2 Suicide attempt Yes if: Yes on Part II, Item 1.2 OR Yes on Part II, Item 1.3



3 Preparatory suicidal behavior Yes if: Part II, Item 1.4 = Yes



4 Suicidal Ideation




5 Other, no deliberate self-harm




6 Fatal, not enough information to classify Yes if: Part III scored 8 A Yes response to 14 —

7 Self-injurious act, no suicidal intent (NSSI)



A Yes response to “Has subject engaged in Nonsuicidal Self-injurious Behavior?”

8 Self-injurious behavior, intention unknown

Yes if: Part II, Item 1.1 = Yes; AND Part II, Item 1.2 = Unknown/NA

A positive response to 1a with 1b and 6 and 7 and 10 unanswered —

9 Nonfatal, not enough information to classify


Yes if question 17 is yes and C-CASA1-C-CASA13 are ALL No —

12

• Issue identified: – 5 subjects who had been injured in accidents, but

were not suicidal. • C-SSRS was not designed to categorize such subjects

and thus could not be used as a standard for them.

• Categorization of these 5 subjects by ISST-Plus and S-STS is compared separately

Instrument Mapping

13

• Baseline demographics and clinical characteristics described descriptively

• Levels of agreement between mappings to the C-CASA of the same subject from ISST-Plus, S-STS, and C-SSRS were summarized using diagnostic characteristics measures – sensitivity – specificity – positive predictive value [PPV] – negative predictive value [NPV] – area under the receiver operating characteristic [ROC] curve)

• Tested using McNemar’s test • Kappa coefficients were computed for each mapping variable

Statistical Analysis

14

• Sensitivity: When present according to the standard reference scale, how often does the diagnostic detect the condition? A/(A + C)

• Specificity: When absent according to the standard reference scale, how often does the diagnostic say the condition is absent? D/(D + B)

• Positive Predictive Value: When present according to the diagnostic, how often does the standard reference scale agree? A/(A + B)

Definitions

Standard Reference (C-SSRS)

Present Absent Diagnostic

Scale Present A B

Absent C D

15

• Negative Predictive Value: When absent according to the diagnostic, how often does the standard reference scale agree? D/(C + D)

• Area Under the ROC Curve: What is the probability that a randomly selected subject with the trait is more likely to be categorized as having the trait relative to a randomly selected subject without the trait? When both the predictor and the criterion variables are binary, this is calculated by (Sensitivity + Specificity)/2

Definitions

Standard Reference (C-SSRS)

Present Absent Diagnostic

Scale Present A B

Absent C D

16

Subject Characteristics

Characteristic Description

Sample size 45

Raters 5

Videotapes 40

Age 39.9 (15.0) years (range, 19-73)

Sex 65.6% female

Race 73.3% white, 24.4% black, and 2.2% mixed race

Age 1st psychiatric symptoms 16.6 (10.4) years (range, 5-42)

Age 1st psychiatric treatment 24.5 (13.3) years (range, 5-58)

Age 1st psychiatric hospitalization 30.8 (14.7) years (range, 10-68)

Number past suicide attempts Mean (SD) 2.1 (2.8) (range, 0-13)

17

Sensitivity of Each Scale by C-CASA (2010) on Classification Relative to the C-SSRS

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

2 3 4 7 8

Excellent

Good

Acceptable

ISST+ S-STSp S-STSc

18

Suicide Attempt Preparatory Acts Suicidal Ideation Self Injury, No Suicide Intent

Self Injury, Intent Unknown

Specificity of Each Scale by C-CASA (2010) on Classification Relative to the C-SSRS

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

2 3 4 7 8

Excellent

Good

Acceptable

ISST+ S-STSp S-STSc

19



NPV of Each Scale by C-CASA (2010) on Classification Relative to the C-SSRS

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

2 3 4 7 8

Excellent

Good

Acceptable

ISST+ S-STSp S-STSc

20



PPV of Each Scale by C-CASA (2010) on Classification Relative to the C-SSRS

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

2 3 4 7 8

Excellent

Good

Acceptable

ISST+ S-STSp S-STSc

21



Mapping to C-CASA 2010 % Disagreement with C-SSRS

22

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

CCA

SA 1

CCA

SA 2

CCA

SA 3

CCA

SA 4

CCA

SA 5

CCA

SA 6

CCA

SA 7

CCA

SA 8

CCA

SA 9

ISST

SSTS

• The ISST-Plus, the S-STS, and the C-SSRS are all potentially valuable instruments for identifying and categorizing SIB

• None of the 3 scales is clearly and consistently superior to the other 2

Discussion: C-CASA 2010 Mapping

23

• Differences in categorization observed for the 3 scales may be related to – Variability in patient reports – Differences in rater reliability – Differences in the instruments’ approaches to collecting

SIB data – The relatively low base rates of most categorical hits

• NPVs are relatively better than the PPVs. • Inherently tied to base rates • Would change (potentially by a substantial amount ) if the

frequency of the constructs of interest changed


24


25


26

Additional highly desirable categories from C-CASA 2010

Mapping Algorithms

C-CASA Code Number


ISST-Plus Map to


S-STS Map to



(Version 2012)

1 Passive Suicidal Ideation

Yes if: Part I, Item 1 ≥1 A positive response to 2 A Yes response to Item 1 (Wish

to be Dead)

2 Active Suicidal Ideation: Nonspecific (no method, intent, or plan)

Yes if: (Part I, Item 4 ≥1 OR Part I, Item 5 ≥1) AND Part I Items 10, 11, 12 = 0

A positive response to 3 AND A negative response to 4 and 5 and 6 and 7

A Yes response to Item 2 (Nonspecific Active Suicidal Thoughts)

3 Active Suicidal Ideation: Method, but no intent or plan

Yes if: (Part I, Item 4 ≥1 OR Part I, Item 5 ≥1) AND Part I, Item 10 ≥1; AND Part I, Items 11, 12 = 0

A positive response to 3 and 4 AND A negative response to 5 and 6 and 7

A Yes response to Item 3 (Active Suicidal Ideation With Any Methods (no plan) Without Intent to Act)

4 Active Suicidal Ideation: Method and intent, but no plan




5 Active Suicidal Ideation: Method, intent, and plan

Yes if: (Part I, Item 4 ≥1 OR Part I, Item 5 ≥1) AND Part I, Items 10, 11, and 12 ≥1

A positive response to 3 and 4 and 5 (and 6 or 7)

A Yes response to Item 5 (Active Suicidal Ideation With Specific Plan and Intent)


Suicide

7 Suicide Attempt

Yes if: Yes on Part II, Item 1.2 OR Yes on Part II, Item 1.3



8 Interrupted Suicide Attempt

Yes if: Part II, Item 2.3 ≥1


A Yes response to Interrupted Attempt

27

Mapping Algorithms

C-CASA Code Number


ISST-Plus Map to


S-STS Map to



(Version 2012)

9 Aborted Suicide Attempt

Yes if: Part II, Item 2.2 ≥1


A Yes response to Aborted Attempt

10

Preparatory acts toward imminent suicidal behavior—not counting Aborted or Interrupted Attempts

Yes if: Part II, Item 1.4 = Yes



11 Self-injurious Behavior Without Suicidal Intent




12 Self-injurious Behavior, Intent Unknown Yes if: Part II, Item 1.1 = Yes; AND Part II, Item 1.2 = Unknown/NA

A positive response to 1a with 1b and 6 and 7 and 10 unanswered

Cannot Evaluate —

13 Not enough information (fatal) Yes if: Part III scored 8 A Yes response to 14 Cannot Evaluate —

14 Not enough information (nonfatal)


Yes if question 17 is yes and C-CASA1-C-CASA13 are ALL No

Cannot Evaluate —

15 Other (accidental, psychiatric medical), no deliberate self-harm


(A positive or blank response to 1 and a negative response to questions 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, and 14) OR A positive response to 15

Cannot Evaluate —

28

Mapping Comparison • Not all categories depicted

– Completed suicide; fatal, not enough information; and nonfatal, not enough information, respectively) are not depicted here; these items did not occur in the subjects surveyed

– Therefore, 100% agreement

• The 5 subjects identified as “recent accident without suicide intent”

– On ISST-Plus and S-STS Clinician Form, all these subjects mapped with 100% agreement to Category 15 (other injury or overdose, no suicide intent)

Results: C-CASA 2012

29

Area Under ROC Curve for Each Scale by C-CASA (2012) Classification Relative to the C-SSRS

Excellent

Good

Acceptable

ISST+ S-STSp S-STSc

30

Ideation Passive

Ideation Nonspecific

Active Method

Method, Intent

Method, Intent, plan

Suicide Attempt

Interrupted Aborted

Self Injury Not suicide

Preparatory Acts

Area Under ROC Curve for Each Scale by C-CASA (2012) Classification Relative to the C-SSRS

Excellent

Good

Acceptable

ISST+ S-STSp S-STSc

31

Ideation Passive


Active Method

Method, Intent


Suicide Attempt

Interrupted Aborted


Preparatory Acts

Sensitivity of Each Scale by C-CASA (2012) Classification Relative to the C-SSRS

Excellent

Good

Acceptable

ISST+ S-STSp S-STSc

32

Ideation Passive


Active Method

Method, Intent


Suicide Attempt

Interrupted Aborted


Preparatory Acts

Specificity of Each Scale by C-CASA (2012) Classification Relative to the C-SSRS

Excellent

Good

Acceptable

ISST+ S-STSp S-STSc

33

Ideation Passive


Active Method

Method, Intent


Suicide Attempt

Interrupted Aborted


Preparatory Acts

NPV for Each Scale by C-CASA (2012) Classification Relative to the C-SSRS

Excellent

Good

Acceptable

ISST+ S-STSp S-STSc

34

Ideation Passive


Active Method

Method, Intent


Suicide Attempt

Interrupted Aborted


Preparatory Acts

PPV for Each Scale by C-CASA (2012) Classification Relative to the C-SSRS

Excellent

Good

Acceptable

ISST+ S-STSp S-STSc

35

Ideation Passive


Active Method

Method, Intent


Suicide Attempt

Interrupted Aborted


Preparatory Acts

Mapping to C-CASA 2012 % Disagreement with C-SSRS

36

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

CCA

SA2

-1

CCA

SA2

-2

CCA

SA2

-3

CCA

SA2

-4

CCA

SA2

-5

CCA

SA2

-6

CCA

SA2

-7

CCA

SA2

-8

CCA

SA2

-9

CCA

SA2

-10

CCA

SA2

-11

CCA

SA2

-12

CCA

SA2

-13

CCA

SA2

-14

CCA

SA2

-15

ISST

SSTS

• The ISST-Plus, the S-STS, and the C-SSRS remain potentially valuable instruments for identifying and categorizing SIB

• Reliability of mapping the C-SSRS, the ISST-Plus, and the S-STS scales to the C-CASA (2012) was considerably worse than reliability of mapping to the C-CASA (2010)

• The small sample size highlights the problems of ratios as psychometric descriptors

• The domains of the C-CASA (2012) are not empirically derived and may inadequately represent the reality of SIB


37

• Greatest disagreement between the C-SSRS and the ISST-Plus and the S-STS

– At “intermediate levels” of active suicidal ideation (2, 3, and 4) (i.e., short of including method, with intent and plan)

– Agreement between the scales was high when the C-SSRS fails to endorse these 3 categories.

– Conversely, agreement between the 3 scales was low when the C-SSRS endorsed any these 3 categories

• ISST-Plus and the S-STS endorsed categories 2, 3 and 4, 0% to 15% of the time

• C-SSRS endorsed 55% to 70% of the time


38

• Differences in categorization may be related to – Variability in patient reports – Differences in rater reliability – Differences in the instruments’ approaches to

collecting SIB data and navigating through possible alternatives

– The relatively low base rates of most categorical hits

• NPVs are relatively better than the PPVs. • Inherently tied to base rates • Would change (potentially by a substantial amount ) if

the frequency of the constructs of interest changed


39

• The sample size of the study was modest

• Kappa scores affected by low base rates for endorsement

• Lack of variation in C-CASA Categories 6, 12, 13, and 14 precludes analysis of these categories.

• Emphasis of the C-SSRS on suicidal patients (without addressing of non-suicidal patients) precluded its use as a standard reference scale for C-CASA Category 15

• Assumes a unidimensional construct

Limitations

40

• Definitions of terms like “intent,” “plan,” “aborted attempt,” “interrupted attempt,” are inadequate: – Unclear, idiosyncratic, overlapping, not consistent with field or general

population usage and, so, lead to confusion • Unclear what information should be counted in each category • Domains like “intent,” “plan,” “method” are not mutually exclusive and are

not empirically derived. Important domains may be missing • Important domains of classification for clinical trials are missing • Even with the identified groupings 11 of 16 possible classifications are

missing • Assumes a simple uni-dimensional construct that does not reflect the

clinical situation • Problems in defining the baseline

Problems Identified with C-CASA 2012

41

Limitations: Combinations of Active Ideation, Method, Intent and Plan

42

16 combinations possible. Only 5 of these are covered by C-CASA and by C-SSRS.

Combin-ation

Passive Ideation

Active Ideation Intent Method Plan C-CASA 1

C-CASA 2 C-CASA 3 C-CASA 4 C-CASA 5

? covered by C-CASA

Can this occur?

1 0 0 0 0 0

Covered by a No to

1,2,3,4,5

2 0 1 0 0 0 X Covered in

Q 2 3 0 0 1 0 0 Not covered Yes 4 0 1 1 0 0 X x 5 0 0 0 1 0 Not covered Yes 6 0 1 0 1 0 X x 7 0 0 1 1 0 Not covered Yes 8 0 1 1 1 0 X x 9 0 0 0 0 1 Not covered Yes

10 0 1 0 0 1 X Not covered Yes 11 0 0 1 0 1 Not covered Yes 12 0 1 1 0 1 X x 13 0 0 0 1 1 Not covered Yes 14 0 1 0 1 1 X Not covered Yes 15 0 0 1 1 1 Not covered Yes 16 0 1 1 1 1 X x

Based on C-CASA 2012 Definitions

• The ISST-Plus, the S-STS, and the C-SSRS remain potentially valuable instruments for identifying and categorizing SIB

• Additional psychometric validation work is necessary – Assessment of inter-rater and intra-rater reliability – Sensitivity to change

• The C-CASA 2012 should be clarified to meet needs of clinical trials

Conclusions: C-CASA Mapping

43

• How do you distinguish one thought from another? • When does one thought end and another begin? • Can we distinguish between conscious and unconscious

thought or is that on a continuum? • Do unconscious thoughts count? Dreams? • Can you have more than one thought at a time? • Can you hold opposing thoughts at the same time? • Is the vocabulary being used by the rater understood in

the same way by the subject? • Is the vocabulary being used understood in the same

way by all subjects?

Methodological Questions

44

• David V. Sheehan, USF

• David Williamson, Janssen

• Lian Mao, Janssen

• Christopher Gray, Consultant

• Qin Li, Janssen

• Richard Shelton, UAB

• Roberta May, UAB

• Xiaohua Li, UAB

• Cheryl McCullumsmith, UAB

• Emily Hammond, UAB

Thank you

45

Session 4 - ISCTM · 2017-10-26 · Session 4 Suicidal Ideation and Behavior: Methodological and...

Documents

Transcript of Session 4 - ISCTM · 2017-10-26 · Session 4 Suicidal Ideation and Behavior: Methodological and...