Medical Interpreting: Outcomes, Errors, and Understanding

The Center for Immigrant HealthNew York University School of Medicine

Thanks!

UHFAltmanNew York Community TrustCommonwealth FundCalifornia Endowment

The Center for Immigrant Health NYU School of Medicine

Founded in 1989Network of community members/CBOs/FBOs, providers, researchers, facilities and administrators, program and policymakersMission: To facilitate the delivery of linguistically, culturally, and epidemiologically sensitive healthcare services to newcomer populations to reduce health disparitiesResearch, Education, Program/Policy Dvpt

PARTNERS

Linguistic Diversity: United States

speaks alanguage other

than English

consideredlimited English

proficient

1990 20

00

17.9%

8.1%13.8%

6.0%

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%1990 Census

• 31 million spoke a language other than English • 14 million considered limited English proficient

2000 Census• 47 million speak a language other than English • 21 million considered limited English proficient

% LEP Change by State

Source: Access Project & National Health Law Program

Filipino 21%

Indian 27%

Italian 36.1%

Pakistani 48%

Spanish 50%

Russian 64.5%

Chinese 70%

Proportion of NYC Immigrant Population that is LEP

Source: New York City Department of Planning

Study Series: A Series of Firsts

Randomized study: impact of MI modes upon medical outcomes and costs

Comparative study of accuracy

Determination of efficiencies across modalities

Intervention:Remote Simultaneous Medical Interpreting System(RSMI)

Trained Simultaneous Medical Interpreters

Remotely Located, Pooled Resource

Spanish, Mandarin, Cantonese, Bengla

Research Questions

Does RSMI Improve Timely Diagnosis of Depression?

Does RSMI Facilitate Appropriate Follow-up Care?

Research Questions

Does RSMI Improve Adherence to Screening Guidelines?

Does RSMI Improve Outcomes for Chronic Diseases?

Diabetes, Hypertension, and Hypercholesterolemia

Research Questions

Does RSMI lead to fewer interpreting errors?

Is RSMI a more efficient form of interpreting?

Does RSMI lead to improved understanding of exit instructions

Cost Analysis Research Questions: Pending

Are visit lengths different across different modalities?Are there fewer repeat visits to achieve the same outcomes?Are there differences in test ordering behaviors, hence, costs?What are the opportunity costs vis-a-vis staff time?Hospitalizations/ER visits prevented

Error Analysis and Efficiency

Scripted Encounters:Spanish and Chinese: TB, Menopause, Diabetes, DepressionBengali: Breast Cancer

RSMI, Proximate Consecutive, Over-the- telephone Consecutive, Ad Hoc

Patient/Doctor ActorsEncounters Audiotaped and Transcribed

Error Analysis ToolWord-by-word, and by concept

Linguistic errors: meaningful and non-meaningful

Medical errors: no, mild, moderate, high, and life-threatening significance

HPI, meds/allergies, family history, diagnosis, plan, psychosocial, F/U, patient education

Error Analysis Panel

Linguist and 3 physicians, at least 2 bilingual

Scored separately, then discussed differences until consensus

Error Analysis

Error rate per utterance

Medically significant/Category

Time

Control for training

Spanish Error Analysis

RSMI versus non-RSMI

RSMI 30% as likely to result in potential medical error

**p<0.05

Spanish Error Analysis: Odds Ratio of a moderately significant to life-threatening error

Trained Proximate ConsecutiveTrained Remote ConsecutiveAd Hoc(18 yrs experience)Trained Remote Simultaneous

6.3***

7.54***

1.71

1.00

***p<0.001

Error Analysis Efficiency Results

Mean N Method

1420.75 4 Distal Consecutive

1174.75 4 Proximal Consecutive

1095.00 4 Proximal Ad-Hoc

762.00 4 RSMI

Mean time (in seconds) for each group

Spanish Efficiency

RSMI is 30% faster than the next fastest mode(ad hoc)

RSMI two times faster than over-the-phone consecutive

Spanish encounters more accurate and efficient with RSMI

Different Languages May be Different

Mandarin Analyses in Progress

Bengali Error Analysis

Standardized Training

Standardized Practice

One Script Across All Modes

Results/Bengali:TrainingMultivariate Regression Chi-Square Analyses

Variable Unadjusted OR Adjusted OR(Confidence

Intervals)

Trained Untrained Total

Any concept error Milderrors

Moderateor greater

errors

TotalConcept

Utterances

Milderrors

Moderateor greater

errors

TotalConcept

Utterances

Milderrors

Moderateor greater

errors

TotalConcept

Utterances

Interpreters3

Trained 0.13*** 0.11***(0.06, 0.21)

95(8%)

94(8%)

1215 95(8%)

94(8%)

1215

Ad hoc 1.00 1.00 159(26%)

166(27%)

608 159***(26%)

166***(27%)

608

Vocabulary Precision Rate3

<40.0%*** 13.10*** 4.84*** (3.12,7.50)

47(25%)

36(19%)

191(18%)

98(29%)

121(36%)

338(64%)

145***(27%)

157***(30%)

529

41 - 79%** 3.98*** 2.65*** (1.75,4.00)

31(8%)

33(9%)

366(35%)

49(32%)

34(22%)

152(29%)

80***(15%)

67**(13%)

518

>79% 1.00 1.00 15(3%)

23(5%)

500(47%)

4(10%)

6(15%)

39(7%)

19(4%)

29(5%)

539

1 Training p<0.05, 2 Training p<0.01, 3 Training p<0.001

* Variable p<0.05, ** Variable p<0.01, *** Variable p<0.001

Note: Multivariate logistic regression model also adjusted for mode of interpreting and time per utterance.

Training Matters

27% of errors made by untrained interpreters were of moderate or greater clinical significance vs. 8.5% of errors made by trained interpreters

Vocabulary precision rate .69 for trained vs. 0.34 for the untrained

Training: Error ExamplesDr: The results were positive which means that you carry the gene that puts you at risk for developing breast cancerInt: The results were correct

Dr: One important thing that you have going for you is the fact that the cancer has probably been caught earlyInt: One important thing is the fact that the cancer is working quickly in your body

Dr: The doxy could hurt your heartInt: The doxy can give you pain

Study Design: Outcomes

Randomized Control for Discordant(Spanish-English, Mandarin-English, Cantonese-English)

RSMI

Usual and Customary

Language Concordant Encounters: English-English, Spanish-Spanish, Chinese-Chinese

Data Collection:Depression, Medical Outcomes,Knowledge

Clinic Intake Questionnaire,

including Beck Depression Index

Chart Reviews and Computerized Tracking for 1 Year after Enrollment

Exit Interviews Several Hundred

Enrolled

ER Intake Questionnaire Audiotaped Visits Exit Interviews Patient

Understanding Scale

Clinic Population

782 patients enrolledRSMI and U&C comparable in demographics including

age, gender, education, years in U.S., primary language, English proficiency, acculturation, and self-reported health status

Randomization Assignment and Exposure Group

Exposure Group Randomized Group

RSMI

(n=256) U&C

(n=271) LC

(n=255) RSMI 144 (56) 9 (3) 0 U&C 23 (9) 153 (56) 1 (0) LC-All 85 (33) 106 (39) 252 (99) Missing 4 (2) 3 (1) 2 (1)

Spanish-speaking Clinic Study Patients, n=465

Patient Gender

60%40%

Male Female

n=282 n=

0

20

40

60

80

100

Per

cen

tag

e

Cancer

Blood Sugar

Cholesterol

Depression

Blood Pressure

Reported Medical History

Patient Ages

14%

1%6%

21%37%

21%

18-25 26-3536-45 46-5556-65 66 and over

2%

12%

24%

7%4% 3% 2% 4% 6%

36%

U.S.P.R.MexicoD.R.EcuadorColombiaPeruHondurasGuatemalaOther

Country of Origin

12%

28%

47% 13%

Less than 1Year

1-2 Years

3-5 Years

Over 5 Years

Years in the U.S

Gender

Age

Chinese-speaking Study Patients, n=208

Patient Gender

40%60%

Male Female

0

20

40

60

80

100

Per

cen

tag

e

Cancer

Blood Sugar

Cholesterol

Depression

Blood Pressure

Reported Medical History

Patient Ages

11% 2%

26%

14%

20%24%

18-25 26-3536-45 46-5556-65 66 and over

10%

28%

48%14%

Less than 1Year

1-2 Years

3-5 Years

Over 5 Years

.

Years in the U.S

Gender

Age

Spanish-speaking ER Patients, n=225

0

20

40

60

80

100

Pe

rce

nta

ge

Ear Pain

Lower Back Pain

Musculoskeletal

Sore Throat

UTI

Presenting Condition

Patient Gender

38%62%

Male Female

5%13%

75%

7%

Less than 1Year

1-2 Years

3-5 Years

Over 5 Years

Length of Stay in U.S.

26%

11%

2%

4% 2% 3%8%

22%

22%

D.R.EcuadorMexicoP.R.U.S.ColombiaEl SalvadorHondurasOther

Country of Origin

Patient Ages

20%

16%

29%

27%

5%3%

18-25 26-3536-45 46-5556-65 66 and over

Age

Gender

Immigrants at Risk: Language and Influenza Vaccination

462 patients were enrolled in the study between November 2003 and July 2004102 were at the highest risk of complications from influenza (chronic medical condition, age, or pregnancy) Only 10 patients in this group were referred for vaccination

9 received vaccination54 patients aged 50 to 64 years without underlying medical conditions composed a second group who were eligible

4 in this group were referred for and received flu vaccination

None of the Cantonese or Mandarin-speaking patients in either group received vaccination.

CLEAN:

RSMI associated with a higher referral rate for screening colonoscopy (OR of 1.7) compared with U&CPhysicians in “language concordant” encounters had lower rates of referrals for screening colonoscopy than language discordant

Instructions Given Audiotape analyses of 214 ER Spanish language encounters Spanish language concordant, RSMI, U&CTrained RSMI

mean # instructions per encounter: 14.29,std dv 6.9 equal to Spanish language concordant

(14.33, std dev 6.33) Usual and customary interpreting

significantly fewer instructions: mean # 11.9, std dev 6.17

Diabetes Management

Research question: Does RSMI lead to improved management of diabetes mellitus?74 patients with DM

Young patient population at clinic

Guidelines for DM management as per the American Diabetes Association (ADA)

Methodology

Score computed based on ADA guidelines considering following:A1C 1st visit Podiatry referralA1C 3 months later Eye doc referralBP recorded Nutrition referralLDL ordered Flu shot referralWeight recorded Pneumovax referralUrine spot or 24h ordered

Smoking cessation referral (if eligible)Prescribed aspirin (if eligible)

Results

RSMI U&C

LC P value

Randomized Score

Mean 0.31 0.28

0.30

3 gps 0.80

SD 0.18 0.14

0.17

2 gps 0.50

Exposure Score

Mean 0.38 0.22

0.30

3 gps 0.02

SD 0.19 0.12

0.15

2 gps 0.01

Diagnosis of Depression

Research question: Does RSMI Improve Timely Diagnosis of Depression?

Why depression?Common disorder in the primary care settingAssociated with significant morbidityEffective communication is key in diagnosis

Methodology

Beck’s Depression InventoryValidated in Spanish and ChineseScreening tool

Administered to patients at intake by research assistantBDI score of 4 or more considered as positiveRSMI, U&C, LC compared in terms of rate of diagnosis by physicians and matched against the expected rate from the BDI score

ResultsBDI +ve rates

BDI +ve 153BDI -ve 309BDI not done 320

• intake process, BDI at end of interview

Demographics comparable in BDI +ve, -ve, and not doneSignificant trends observed:

Diagnosis rate best in LC, followed by RSMI and then U&CTime to medication in new diagnosis of depression better in RSMI compared with U&C

Patients with a positive BDI with new diagnoses of depression RSMI U&C LC Randomization 27% 20% 26% RSMI U&C LC Exposure 29% 23% 24%

  RandomizationRandomization

 * Comparing all 3 groups:  Chi-square P=0.66, Fisher P=0.65   * Comparing all 3 groups:  Chi-square P=0.66, Fisher P=0.65   ** Comparing RSMI vs U&C:  Chi-square P=0.41, Fisher P=0.47     ** Comparing RSMI vs U&C:  Chi-square P=0.41, Fisher P=0.47    

ExposureExposure

           * Comparing all 3 groups:  Chi-square P=0.83, Fisher P=0.85      * Comparing all 3 groups:  Chi-square P=0.83, Fisher P=0.85      ** Comparing RSMI vs U&C:  Chi-square P=0.58, Fisher P=0.76  ** Comparing RSMI vs U&C:  Chi-square P=0.58, Fisher P=0.76

Patient Satisfaction/Understanding Perception

ER and Clinic Combined

First Visits

Language Concordant(E-E,Sp-Sp, M-M, Ca-Ca), RSMI, Usual and Customary

Understanding MD Understands Pt Underst Explan Pt Under Instructions

LC RSMI

UC LC RSMI

UC LC RSMI

UC

VW 69% 49% 35% 59% 39% 34% 63% 38% 32%

W 33% 49% 55% 33% 52% 50% 33% 54% 59%

NW

2% 2% 10% 4% 9% 15% 3% 8% 9%

Satisfaction

How would you rate the MD overall?

LC RSMI UCEx 63% 56% 49%Gd 32% 40%

44%Fa 4% 4% 4%

How satisfied with care overall?

LC RSMI UCVery 57% 57% 47%Swht 38% 40% 48%SwtDis 4% 3% 5%

How well did the method protect your privacy?

RSMI U&C

Very Well 49% 40%Well 44% 49%Not Well 7% 10%Poor 0% 1%

No Difference

How well did the interpreter understand you?Did the interpreter listen carefully(yes/no)?Did the interpreter treat you with respect?(trend)How well did the interpreter interpret?(trend)

Implications for PolicyTraining matters….training programs should be systematized.

Investment in dissemination of RSMI to users of telephone interpreting services will provide cost savings even without consideration of seemingly improved outcomes(?).

For Spanish-language encounters, RSMI will likely provide the most accurate results, and better patient outcomes.

First ever randomized trial of impact of varying modes of interpreting. Results can provide basis for institutional and federal/state/local policy evidence-based decision-making.