Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of...
18
Sunghee Lee Julia Lee University of Michigan Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012
Transcript of Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of...
Sunghee Lee
Julia Lee
University of Michigan
Quantitative Evaluation
of Questionnaire
Translation with
Bilingual Speakers
AAPOR 2012
Literal Equivalency
Functional Equivalency
Measuring the same intended construct
Cultural relevance
Question wording + Response scale
QUESTIONNAIRE TRANSLATION
2
Yes/No
Vague quantifiers
RESPONSE SCALE
3
Mostly qualitative works
To quantitatively evaluate the
translation, need randomized
experiments
- Perfect bilingual speakers
- Randomizing interview language
- Any observed differences attributable
imperfect translation
DIFFICULTY IN EVALUATION
4
National Latino and Asian American
Study (NLAAS), 2003
Target populations: Hispanics and Asians
Our Focus: English-Spanish bilingual
Hispanics (Self-rated equal proficiency
on both languages)
5
DATA
Total %
Spanish Only 72 4.5%
Mostly Spanish 521 32.7%
20.8%
Mostly English 647 40.6%
English Only 21 1.3%
Total 1,593
DATA
6
Interview
language
randomization
182 English
150 Spanish
NLAAS Hispanic Sample
Spanish/English 332
One-to-One Translation
Poor, fair, good, and excellent
Spanish/English Proficiency
One-to-Two Translation
Frequency scale
Often, Sometimes, Rarely, Never
Response Style
7
ANALYSIS
SELF-RATED SPANISH PROFICIENCY
8
SELF-RATED ENGLISH PROFICIENCY
9
Translation incurred measurement error
RESPONSE SCALE DIFFERENCES
10
English
Spanish
Poor Fair Good Excellent
Pobre Regular Bien Excelente
Incomparable measurement
Measurement error
FREQUENCY SCALE
11
English
Spanish
Translation
A
Spanish
Translation
B
Often Muchas veces Muchas veces
Sometimes Alguna veces Alguna veces
Rarely Pocas veces Casi nunca
Never Nunca Nunca
How often does your (spouse/partner) make too many demands on you, often, sometimes, rarely, or never ?
muchas veces, algunas veces, pocas veces o nunca?
How often do your relatives or children make too many demands on you, often, sometimes, rarely, or never?
muchas veces, algunas veces, casi nunca o nunca?
FREQUENCY SCALE
12
FREQUENCY SCALE: MARRIAGE Qs
13
FREQUENCY SCALE: OTHER Qs
14
Literature suggests Hispanics’ higher tendency on
Extreme response style
In NLAAS, 10 questions with “Strongly Agree,” to “Strongly Disagree” scale
Bilingual Spanish respondents consistently have higher frequency on “Strongly Agree”
RESPONSE STYLE
15
Why did we find these differences?
Functionally equivalent?
Cultural norm invoked by language?
Translation differences?
DISCUSSION
16
Analyze variables with different
response scales.
Implement various measurement
models to adjust for incomparable
measurement error.
17
WHAT’S NEXT?
