Validation of self-administered Validation of self-administered single item screening questions single item screening questions

(SISQs) for unhealthy alcohol and (SISQs) for unhealthy alcohol and other drug use in primary care other drug use in primary care

patients at two sitespatients at two sites

Jennifer McNeely, Charles M. Cleland, Shiela M. Strauss, Joseph J. Palamar, John Rotrosen, Marc N. Gourevitch, Richard Saitz

No relevant financial relationships to disclose

ObjectivesObjectives

1. Describe the need for a self-administered approach to substance use screening

2. Single Item Screening Questions (SISQs) for alcohol and drug use

3. Present results of a validation study in primary care

4. Discuss feasibility and application

Screening for Screening for substance use in substance use in

primary careprimary care• Medical providers fail to identify

clinically relevant substance use• Barriers to screening:

o Timeo Workflowo Knowledge/Trainingo Discomforto Attitudes

D’Amico, Medical Care 2005

Sterling, Addict Med Clin Pract 2012Friedmann, J Gen Int Med 2000Friedmann, Arch Int Med 2001Anderson, Alcohol Alcoholism 2004McCormick, J Gen Int Med 2006

Self-administered screening Self-administered screening is a more feasible approach is a more feasible approach

Education Monitoring

Office-based counseling

Treatment

Screening

Assessment

Low Risk Moderate RiskHigh Risk or Dependence

+

Screening: SISQ-alcohol and SISQ-drug

Single Item Screening Single Item Screening

QuestionsQuestions

• SISQ-alcoholHow many times in the past year have you had X or

more drinks in a day?(X=5 for men, and X=4 for women)

• SISQ-drugHow many times in the past year have you used an

illegal drug or used a prescription medication for non-medical reasons (for example, because of the experience or feeling it caused?

Prior validation of SISQsPrior validation of SISQs

• Adult primary care patients (N=286)• Single site, urban safety net medical

center• Good sensitivity and specificity for

detection of unhealthy use • SISQ-alcohol: Sensitivity 82%, Specificity

79% • SISQ-drug: Sensitivity 85%, Specificity

96%Smith et al., JGIM 2009Smith et al., Arch Int Med 2010

Current Study Current Study

Screening (computer) Validation Measures (interviewer)

ReferralsIncentive

• Timeline Follow Back

• SIP-A and SIP-D• MINI-Plus

• REALM• Demographics

Saliva drug screen

Second Consent

• SUBS• SISQ-alcohol, SISQ-

drug

Reference standard Reference standard measuresmeasures

 

Timeline follow-back (30d)

SIP-A SIP-DMINI-Plus screening

MINI-Plus abuse or dep

Intercept oral fluid test*

Alcohol Unhealthy use

+ + +

Disorder +

DrugsUnhealthy use

+ + + +

Disorder +

* Collected at Site A only

Statistical AnalysisStatistical Analysis

1.Comparison of SISQs to composite reference standards

2.Examined site differences3.Calculate sensitivity, specificity, AUC:oUnhealthy useoSubstance use disorder

4. Subgroup analyses

Study Sites and Study Sites and RecruitmentRecruitment

• Adult primary care clinics• 2 urban safety net hospitals • Patients presenting for medical visits• Consecutive recruitment

Eligibility Criteria: • Age 21-65• Current clinic patient• Fluent in English• No disability preventing computer use

Participant Recruitment

Screened: N = 2131

Eligible: N = 915

1216 were excludedLanguage: 679Age: 306Not a patient: 168Other: 115

Completed interview: N = 459Site A: 265*Site B: 194

453 declinedNo time: 363Other: 90

1 lost data

*230 (87%) Site A participants agreed to saliva test

Characteristics of the 459 Characteristics of the 459

participantsparticipants

Age (years) Mean = 46, SD = 12Range = 21-65

Sex (%) MaleFemaleTransgender

48.451.20.4

Race/Ethnicity (%) Black/African American HispanicWhite/Caucasian Other

51.820.219.18.6

Country of Birth (%)

United StatesOutside of United States

64.635.3

Education and Health LiteracyEducation and Health Literacy

Highest Level of Education Health Literacy Level

Prevalence of substance usePrevalence of substance use

Substance Past year use (MINI)

 N (%)

Past month use(TLFB)

 N (%)

Alcohol 103 (22.3)a 89 (19.3)b

Drugs 114 (24.7)c 73 (15.8)c

Specific drug categoriesIllicit drugs 108 (23.4)  

Marijuana -- 58 (12.6)Cocaine -- 12 (2.6)Heroin -- 10 (2.2)Hallucinogens -- 1

Prescription drugs (non-medical use)

21(4.6)  

Opioids -- 5 Benzodiazepines -- 3 Stimulants -- 2

Unhealthy useUnhealthy use

Substance + on SISQ

N (%)

+ on Reference

N (%)

Sensitivity

%(95% CI)

Specificity

%(95% CI)

AUC

(95% CI)

Alcohol 155 (34) 146 (32) 73.3 (65.3, 80.3)

84.7 (80.2, 88.5)

0.79 (0.75, 0.83)

Drugs 107 (23) 122 (27) 71.3 (62.4, 79.1)

94.3 (91.3, 96.6)

0.83 (0.79, 0.87)

Oral fluid test results: 8 tested positive, all reported use on SISQ

No change to results

Substance use disorderSubstance use disorder

Substance + on SISQ

N (%)

+ on Reference

N (%)

Sensitivity

%(95% CI)

Specificity

%(95% CI)

AUC

(95% CI)

Alcohol 155 (34) 60 (13) 86.7 (75.4, 94.1)

74.2 (69.6, 78.4)

0.80 (0.76, 0.85)

Drugs 107 (23) 74 (16) 85.1 (75.0, 92.3)

88.6 (85.0, 91.6)

0.87 (0.83, 0.91)

Subgroup AnalysisSubgroup AnalysisSubgroups anticipated to have greater difficulty with self-administered screening:•Male•Age greater than 50•Hispanic/Latino•Primary language other than English•Born outside US•Education or health literacy lower than high school level

Subgroup AnalysisSubgroup Analysis• No differences for SISQ-alcohol• Lower sensitivity of SISQ-drug among:Primary language other than English (p<0.01)

Less than high school education (p<0.01)

Sensitivity Specificity

English 74.3 (65.1, 82.2) 94.4 (90.7, 96.9)

Non-English 46.2 (19.2, 74.9) 94.3 (87.1, 98.1)

Sensitivity Specificity

High school 79.0 (66.8, 88.3) 95.2 (91.0, 97.8)

< High school 63.3 (49.9, 75.4) 93.3 (88.0, 96.7)

LimitationsLimitations

• Safety net primary care populations• English speaking only• Tested in research context, with

assurance of confidentiality

ConclusionsConclusions• SISQs accurately identified unhealthy

substance us in primary care patients

• Lower sensitivity and specificity than interviewer-administered versions

• Efficiency, fidelity, and patient comfort may be advantages to self-administered approach

AcknowledgementsAcknowledgementsFunding:• K23 Career Development Award

NIDA K23 DA030395• NYU-HHC CTSI Pilot Grant

NIH/NCATS UL1 TR000038 • The MITRE Corporation (contract

from ONC and SAMHSA)

Staff and others:•Seville Meli•Jacqueline German•Ritika Batajoo•Catherine Federowicz•Marshall Gillette•Charlie Jose•Emily Maple•Keshia Toussaint•Julianne Cameron•Arianne Ramautar•Derek Nelsen•Linnea Russell•Study participants