mmpi teona 2

ARTICLE IN PRESS

International Journal of Intercultural Relations

30 (2006) 531–543

0147-1767/$ -

doi:10.1016/j

�CorrespoE-mail ad

www.elsevier.com/locate/ijintrel

Evaluating cross-cultural equivalence of the KoreanMMPI-2 via bilingual test–retest

Junmo J. Chunga, Nathan C. Weedb, Kyunghee Hanb,�

aDepartment of Psychology, University of Mississippi, USAbDepartment of Psychology, Central Michigan University, 103 Sloan Hall, Mt. Pleasant MI 48859, USA

Received 13 May 2005; received in revised form 16 August 2005; accepted 23 August 2005

Abstract

Previous efforts to establish the cross-cultural equivalence of the Korean MMPI-2 have focused on

analysis of internal structure and peer behavioral correlates using only a Korean college sample. The

intent of this study was to provide further evidence of the cross-cultural equivalence of the Korean

MMPI-2 using a bilingual test–retest method. Fifty-three self-reported bilinguals living in the US

completed both the MMPI-2 in Korean and English within a 1-week interval. Their profiles were

compared to previously established test–retest reliabilities. Cross-language correlations were sizable,

but substantially lower than the published within-language test–retest correlations. However, when a

proficient bilingual subsample was selected from the original sample post hoc, cross-language

correlations were more similar to test–retest reliabilities. Cross-language item analysis revealed that

items with simple and direct grammar had the highest item agreement. Complicated sentences, such

as those containing double negatives or unique American idioms, had the least concordance.

r 2005 Elsevier Ltd. All rights reserved.

1. Introduction

Translated into numerous languages and adapted for a wide variety of cultures(Butcher, 1996; Butcher & Pancheri, 1976), the MMPI (Minnesota Multiphasic PersonalityInventory) is the most widely used personality instrument in the world. The MMPI hasbeen used in Korean populations as a clinical assessment and research instrument since its

see front matter r 2005 Elsevier Ltd. All rights reserved.

.ijintrel.2005.08.009

nding author. Tel.: +1989 774 6496; fax: +1 989 774 2553.

dress: [email protected] (K. Han).

www.elsevier.com/locate/ijintrel

ARTICLE IN PRESSJ.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531–543532

translation in 1963 (Cheung, Lee, & Jin, 1963). Despite wide use and acceptance by theKorean psychological community the validity and reliability of this original versionremained largely untested, and serious translation problems (including miskeying) wereidentified (Han, 1993). In 1989, the Korean MMPI was retranslated and standardized witha large normative sample (Kim et al., 1989). Although all items were properly keyed in thesame direction as the original English version, the new translation appeared not to correctmany distortions present in the original translation, and added others. Further, thesimultaneous release of the English MMPI-2 (Minnesota multiphasic personalityinventory-revised) made this version obsolete as it was released.Shortly after the MMPI-2 was published, the MMPI-2 was translated into Korean by

Han (1990). Han and one bilingual student independently translated the MMPI-2 fromEnglish into Korean and then compared the two versions, solving discrepancies betweenthem by mutual consensus. The Korean items were then submitted to a different bilingualstudent for back translation into English. The two English versions, the original Englishitems and the back translated English items, were then examined for discrepancies by anAmerican psychologist who was an expert in MMPI cross-cultural work. Discrepanciesbetween the two versions guided a revision of the affected Korean items. Out of the totalMMPI-2 item pool, 20 items were judged as non-equivalent in the two English versions.The 20 inadequate items were reviewed and re-translated. The corrected version of theseitems was presented to the same back translator, who then produced a back-translationjudged to be equivalent to the original English version (Han, 1993).To measure its psychometric properties, the Korean MMPI-2 was administered to 726

Korean college students from eight universities across Korea, and validation effortsconsisted of equivalence data in the form of factor structure, 1-week test–retest, and peerbehavior correlates (Han, 1993). Although these efforts produced the best data to date insupport of the cross-cultural equivalence of the Korean MMPI-2, additional data areneeded to support these early efforts, particularly involving: (a) samples other than Koreancollege students, and (b) additional analytic strategies. The present study attempts toprovide additional validation support for the Korean MMPI-2 by examining responsesfrom Korean–English bilinguals on both translations of the MMPI-2.The bilingual retest technique is considered an important method for evaluating the

accuracy and adequacy of a translation of a test (Butcher, 1996). In this method, both theoriginal language form and the translated version are administered to a selected group ofbilinguals who are familiar with both cultures. Then, scale mean differences or itemendorsement frequencies across the two versions are compared, or cross-language correlationsare computed. Recently more rigorous statistical techniques, such as those based on IRT (itemresponse theory) have been developed to evaluate the equivalence of two versions usingsamples of bilingual respondents (e.g., Sireci & Berberoglu, 2000), but they have not beenwidely used due to strict assumptions and requirement of a large sample size.A number of studies have used the bilingual technique for checking translation adequacy of

the MMPI and MMPI-2 items (e.g., Butcher & Gur, 1974; Tran, 1996; Velasquez et al., 2000).Butcher and Gur (1974) administered the English-language and Hebrew translation of

the MMPI to 28 bilinguals. The cross-language correlations for the 13 basic scales rangedfrom .51 to .91, with a mean of .74. In a sample of 32 college students or recent universitygraduates, Tran (1996) found that the mean correlation coefficient between scores on theEnglish MMPI-2 and those on the Vietnamese translation of the MMPI-2 was .72, withcoefficients ranging from a low of .35 for the Hy scale and a high of .88 for the Mf scale. In

ARTICLE IN PRESSJ.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531–543 533

a Hmong adaptation study involving 30 bilinguals, Deinard, Butcher, Thao, Vang, andHang (1996) reported that the mean cross-language correlation was .59, with scales Maand D showing the lowest (.38) and highest (.80) correlations, respectively. The latter wassomewhat of an unexpected finding because the authors used very rigorous criteria toevaluate fluency in both languages. In contrast, exceptionally high cross-languagecorrelations (M ¼ :79) were found in a sample of 148 bilingual Icelandic adults (Konraos,1996), a finding that may owe to the linguistic commonalities between English andIcelandic languages. In Velasquez et al. (2000), 27 bilingual participants were administeredboth the English version and the Spanish translation of the MMPI-2. No significant meandifferences on the basic scales were found, and the mean cross-language correlation was.71, with coefficients ranging from .60 (Hy) to .77 (D, Pa, Pt, Sc, and Ma).

The present study attempts to provide additional validation support for the KoreanMMPI-2 by examining the responses from Korean–English bilinguals on both translationsof the MMPI-2. If the Korean translation of the MMPI-2 is accurate, bilinguals shouldrespond similarly to the two versions. Consequently, we expect to find that a sample ofKorean–English bilinguals will produce a similar pattern of mean scores on MMPI-2scales regardless of translation and demonstrate high correlations between their individualscores on the Korean and English versions of MMPI-2 scales.

2. Method

2.1. Participants and procedures

Study participants were recruited through bulletins and announcements made at Koreanchurches in the Denver and Memphis communities, and with the help of Korean studentorganizations at universities in Mississippi and Colorado. The announcement explainedthat participation was limited to Korean/English bilinguals, would require two consecutivesessions, 1 week apart, and that each participant would be paid $20 for full participation.One hundred thirty-four people volunteered to participate, and each was eitheradministered either the Korean version or the American version of the MMPI-2, theversion chosen randomly. One week later, available participants were administered theother version of the MMPI-2. This design yielded 107 completed Korean MMPI-2protocols and 82 American MMPI-2 protocols.

Protocols were eliminated from the study if they possessed one or more of the followingfeatures1: Cannot say score X15; F (Infrequency) X20; Fb (Back F; Fb consists of itemswith extremely low endorsement frequency that appear later in the test booklet thanF items) X20; VRIN (Variable Response Inconsistency) X13; TRIN (True Response

1Cheung used somewhat liberal exclusion criteria for screening invalid cases for the Chinese version of the

MMPI (Cheung, Song, & Butcher, 1991) and MMPI-2 (Cheung, Song, & Zhang, 1996) because use of American

standards for F and Fb were likely to overestimate invalidity in normal Chinese samples. For our study, we

decided to use the exclusion criteria that were set by American standards (Butcher et al., 1989) due to a lack of

studies demonstrating that American criteria were problematic for Asian Americans or Asians who lived in the

US for more than 8 years. We expect that use of more liberal F and Fb criteria would slightly elevate overall mean

level, in particular for scales Pa and Sc because F and Fb share a substantial number of items with those scales. It

is also plausible to expect that including cases with high F or Fb might increase cross-language correlations on

scales Pa and Sc because of the resulting increase in variability; still, such an increase would probably be minimal

at best.


Inconsistency) p4; or TRINX13. This screening reduced the sample to 89 valid Koreancases (83% of completed protocols), and 69 valid American cases (84% of completedprotocols), with 53 participants (23 men and 30 women) producing both valid Americanand Korean MMPI-2 protocols. The mean age of this sample was 29.6 years, and meanlength of stay in the US was 11.4 years. Eighteen participants (34%) reported that theypreferred to speak English, while 35 (66%) preferred Korean. Forty participants (75%)were working adults, 12 (23%) were college students, and one (2%) was in high school.

2.2. Instruments

The MMPI was published in 1943 by Hathaway and McKinley who were working in theUniversity of Minnesota Hospitals. The authors used an empirical keying approach in theconstruction of the original ‘‘clinical scales’’ so that scale items were those that bestdistinguished between patients with discrete psychopathological conditions and a group ofexaminees who were not hospitalized. The original MMPI contained 566 self-referencedtrue/false statements (Graham, 2000). In 1989 the updated and restandardized MMPI-2was published by Butcher, Dahlstrom, Graham, Tellegen, and Kaemmer (1989). Becausethe original MMPI was so widely used, great care was taken to make improvements (e.g.,updating language, including more representative norms, applying uniform T-scoreconversions, developing new scales) while maintaining its continuity with the originalinstrument (preserving the clinical scales and their empirical meaning). The MMPI-2consists of 567 items, and contains numerous scales to assess both normal rangepersonality constructs and psychopathological symptomatology. In the present study wefocus on just the most commonly used scales of the MMPI-2: 3 ‘‘validity scales,’’ 10‘‘clinical scales,’’ and 15 ‘‘content scales.’’The three validity scales, L (Lie), F (Infrequency), and K (Correction), were developed

to measure: a deliberate and rather unsophisticated attempt to present oneself in afavorable manner, deviant or atypical ways of responding to test items, and a more subtleand sophisticated attempt to present oneself in a favorable light, respectively (Butcheret al., 1989; Graham, 2000).The 10 MMPI-2 clinical scales were designed to measure: an excess concern about

one’s health (Hs, Hypochondriasis), various symptoms associated with depression(D, Depression), hysterical syndromes associated with involuntary psychogenic loss ordisorder of function (Hy, Hysteria), psychopathic or sociopathic characteristics, includingdelinquent acts, sexual problems, family problems, and difficulties with authorities(Pd, Psychopathic Deviate), gender-role divergence, including interests or hobbies thatwere opposite to the stereotypical gender role (Mf, Masculininity–Femininity), paranoidsymptoms, interpersonal sensitivities, and a tendency to misinterpret the motives andintentions of others (Pa, Paranoia), generalized anxiety and distress, unreasonable fears,compulsions, and obsessions (Pt, Psychasthenia), psychotic symptoms, such as bizarrementation, peculiarities of perception, and halluciations, social alienation, and poor familyrelationships (Sc, Schizophrenia), hypomanic symptoms, including elevated mood,accelerated speech and motor activity, irritability, and flights of ideas (Ma, Hypomania),and social withdrawal and self-deprecation (Si, Social Introversion) (Butcher et al., 1989;Graham, 2000).The 15 content scales were developed using a more modern rational-deductive approach

to scale construction, and cover a wide range of clinical and normal-range concerns. They


include: Anxiety (ANX; tension, worry, fears, lack of confidence, and somatic indicationsof anxiety), Fears (FRS; specific fears such as high places, snakes, spiders, fires, andstorms), Obsessiveness (OBS; rumination about decisions and problems, and compulsionssuch as counting and saving unimportant things), Depression (DEP; brooding, cryingeasily, pessimism, suicidal ideation, and guilt), Health Concerns (HEA; gastrointestinalsymptoms, neurological symptoms, dermatological problems, and pain), Bizarre Menta-tion (BIZ; paranoid ideation, ideas of reference, delusional thinking, and hallucinations),Anger (ANG; fear of losing self-control over aggressive impulses, irritability, impatience,stubbornness, physical and/or verbal abusiveness, and explosivity), Cynicism (CYN;hostility, suspicion, misanthrope, distrust, and selfishness), Antisocial Practices (ASP; anti-authority ideation, rationalization and identification with criminal behavior, admission ofantisocial or unlawful behaviors), Type A (TPA; hard driving, fast paced, task-orientation,competitiveness, and workaholism), Low Self-Esteem (LSE; a lack of self-esteem, feelingsof unattractiveness and uselessness), Social Discomfort (SOD; introversion, socialavoidance, dislike of crowds, parties, or group activities), Family Problems (FAM;general problems with family), Work Interference (WRK; difficulties concentrating,anxiety, tension, lack of self-confidence, and indecisiveness about career choices), NegativeTreatment Indicators (TRT; negative attitudes toward health care providers andtreatment, pessimism about individuals being understanding or helpful) (Butcher et al.,1989; Butcher, Graham, Williams, & Ben-Porath, 1990; Graham, 2000).

All participants completed both the English version of the MMPI-2 (Butcher et al.,1989) and the Korean version of the MMPI-2 (Han, 1993, 1996, 20052; Kim et al., 2005(see footnote 2)), each of which comprises 567 true/false self-referenced items.

3. Results

Table 1 displays descriptive statistics and effect sizes comparing scores on the Koreanand English MMPI-2. Raw scores of the 13 basic and 15 content scales were converted toK-corrected T-scores using the American adult norms. For both versions, the highest meanelevation among the basic scales was on scale F (62.9 for the Korean version; 63.2 for theAmerican version), and the next highest mean scale elevations were on scales Sc (62.8 forthe Korean version; 62.9 for the American version) and D (62.0 for the Korean version;60.0 for the American version). For both versions, the lowest scores were on scale K, withmeans of 50 for the Korean version and 49.6 for the American version. Therefore, meanprofiles were similar for the two versions, both in terms of overall elevation and in terms ofspecific scales elevations; except for scale Pa, all basic scale means fell within five T-score

2Since the completion of the present study, the Korean MMPI-2 has undergone additional revision. In 2002,

before setting out on a Korean MMPI-2 standardization project, the committee members reviewed Han’s

translation and believed that some items needed to be retranslated because it has been about 10 years since Han

translated the items, during which Korea has gone through rapid political and social changes, some of which are

reflected in Koreans’ use of language. Through multistage revision procedures, the committee and Han revised the

179 items. Most of the item changes were very minor, and would not be expected to affect endorsement patterns

significantly. The item changes include: (a) location a word within a sentence, (b) contemporary wording, (c)

grammatical appropriateness and natural word choice. Of the 179 items, 13 were judged by the committee and

Dr. Han to be somewhat different from the original translation, and were, therefore, submitted to a

backtranslation into English. After two rounds of backtranslation, all 13 item pairs were rated equivalent.

ARTICLE IN PRESS

Table 1

Means, standard deviations, and standard differences on basic and content scales between Korean and American

versions of the MMPI-2 (n ¼ 53)

Korean version American version

M SD M SD d

Basic scales

L 56.3 10.3 56.8 11.7 �.05

F 62.9 13.5 63.2 14.5 �.02

K 50.0 11.7 49.6 9.6 .04

Hs 60.0 11.7 59.2 10.1 .07

D 62.0 12.5 60.0 12.5 .16

Hy 54.5 11.6 55.5 9.9 �.09

Pd 56.6 10.6 59.0 9.5 �.24

Mf 53.8 8.1 54.6 10.1 �.09

Pa 51.4 8.8 57.7 12.9 �.58

Pt 58.3 12.2 59.4 10.0 �.10

Sc 62.8 10.6 62.9 11.2 �.01

Ma 56.7 11.8 56.9 11.9 �.02

Si 54.6 10.0 56.2 10.3 �.16

Content scales

ANX 54.3 11.7 54.7 10.2 �.04

FRS 59.7 12.3 55.8 11.5 .33

OBS 56.2 13.2 55.1 11.9 .09

DEP 55.7 10.2 57.3 8.7 �.17

HEA 57.8 11.1 58.3 9.5 �.05

BIZ 57.5 8.9 59.6 10.7 �.22

ANG 53.0 11.5 50.8 10.1 .21

CYN 51.4 8.7 52.8 7.8 �.17

ASP 54.8 8.9 55.2 8.5 �.05

TPA 54.6 11.7 50.8 8.6 .37

LSE 52.6 9.6 53.4 9.0 �.09

SOD 54.0 10.2 52.3 9.9 .17

FAM 51.5 10.1 51.4 8.9 .01

WRK 58.4 11.5 58.1 10.6 .03

TRT 57.9 11.7 59.2 9.8 �.12

Note: L: Lie; F: Infrequency; K: Correction; Hs: Hypochondriasis; D: Depression; Hy: Hysteria; Pd: Psychopathic

Deviant; Mf: Masculininity–Femininity; Pa: Paranoia; Pt: Psychasthenia; Sc: Schizophrenia; Ma: Hypomania; Si:

Social Introversion; ANX: Anxiety; FRS: Fears; OBS: Obsessiveness; DEP: Depression ; HEA: Health Concerns ;

BIZ: Bizarre Mentation; ANG: Anger; CYN: Cynicism; ASP: Antisocial Practices; TPA: Type A; LSE: Low Self-

Esteem; SOD: Social Discomfort; FAM: Family Problems; WRK: Work Interference; TRT: Negative Treatment

Indicators.

J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531–543536

points of one another. All Cohen’s d values were small with the exception of that for Pa,which showed moderately sized (d ¼ :58) mean differences between the two versions.For the content scales, all mean scores fell within four T score points across versions

with d values ranging from .01 to .37 (Mdn ¼ :12). Unlike the basic scales, no content scaleelevations were one standard deviation away from the normative means. The highestelevations on the Korean version were FRS (59.7), WRK (58.4), TRT (57.9), HEA (57.8),and BIZ (57.5). Similarly, the American version produced the highest elevations at BIZ(59.6), TRT (59.2), HEA (58.3), WRK (58.1), and DEP (57.3). No scale means fell below


the US normative means. In sum, for both versions, all clinical and content scale meanscores were elevated when profiles were plotted against American adult norms. Inparticular, scales F, D, and Sc showed the highest elevations for both versions. Mean T

scores for all scales are close across versions, with all scale means except for Pa fallingwithin five T-score points of one another across versions.

Cross-language correlations for the basic and content scales are presented in Table 2,using the established Korea test–retest and American test–retest reliabilities reported by

Table 2

Cross-language correlations and test–retest correlations for MMPI-2 basic and content scales

Cross-language correlations Within-language test–retest correlations

Total bilinguals

(n ¼ 53)

Selected bilinguals

(n ¼ 12)

Korean men

(n ¼ 78)

Korean women

(n ¼ 76)

American

men (n ¼ 82)

American women

(n ¼ 111)

Basic scales

L .21 .36 .78 .77 .86 .81

F .42 .41 .85 .81 .74 .72

K .62 .89 .74 .83 .80 .80

Hs .51 .81 .77 .75 .76 .75

D .50 .62 .86 .73 .79 .80

Hy .38 .57 .79 .75 .70 .74

Pd .39 .69 .68 .73 .79 .69

Mf .27 .54 .63 .65 .83 .74

Pa .37 .24 .86 .69 .67 .56

Pt .49 .69 .79 .83 .72 .68

Sc .62 .73 .82 .75 .72 .54

Ma .52 .82 .75 .76 .80 .65

Si .66 .77 .85 .82 .93 .92

Median .49 .69 .79 .79 .75 .74

Mean .47 .66 .79 .79 .77 .74

Content scales

ANX .70 .77 .87 .81 .89 .88

FRS .63 .67 .72 .75 .82 .87

OBS .63 .85 .77 .80 .84 .84

DEP .41 .65 .87 .85 .84 .88

HEA .58 .46 .84 .83 .80 .86

BIZ .53 .84 .78 .71 .77 .78

ANG .59 .77 .84 .77 .87 .82

CYN .57 .76 .68 .84 .81 .88

ASP .47 .73 .77 .78 .82 .86

TPA .55 .49 .76 .81 .81 .78

LSE .48 .60 .84 .86 .84 .86

SOD .70 .75 .84 .90 .91 .91

FAM .46 .40 .76 .81 .84 .83

WRK .47 .73 .83 .87 .90 .90

TRT .46 .65 .81 .87 .79 .88

Median .55 .73 .81 .81 .82 .81

Mean .56 .70 .81 .81 .81 .81

Notes. American test–retest correlations from Butcher et al. (2001); Korean test–retest correlations from Kim

et al. (2005).


Butcher et al. (2001) and Kim et al. (2005) as comparisons. Cross-language test–retestcorrelations were obtained by correlating MMPI-2 scale scores from the Korean versionwith the corresponding scores from the American version. A correlation coefficient wascomputed for each scale, indexing the degree to which individual scale performance wasthe same across versions. All cross-language correlations were lower than the Korean andAmerican test–retest comparison groups. As in the American test–retest samples, thehighest cross-language correlation was with Scale Si (.66). The next two highest cross-language correlations were on K (.62) and Sc (.62), whereas the lowest cross-languagecorrelations were on scales L (.21) and Mf (.27). Cross-language correlations for thecontent scales (Mdn ¼ :55) were larger than those for the basic scales (Mdn ¼ :49), butwere still lower than the Korean and American test–retest comparison groups. The highestcorrelations were on ANX (.70) and SOD (.70). Low cross language correlations suggestthat the Korean MMPI-2 is not measuring the same dimensions as the American MMPI-2as effectively as one would hope, and not so precisely as to represent a test–retest. Thisfinding also suggests the possibility that the participants were not proficient in bothlanguages.To investigate the latter possibility, a ‘‘clearly bilingual’’ subsample was selected among

the full sample based on three criteria: (a) reporting a preference for speaking Korean, (b)living in the US for more than 8 years, and (c) having at least 1 year of college education.We reasoned that Korean-born Americans who had immigrated late enough in life toretain their native Korean language skills would be the most bilingual. Korean-bornAmericans who immigrated early in childhood might not speak Korean proficientlyenough to be truly bilingual, whereas Korean students studying in the US might not speakEnglish well enough. Finally, Korean blue-collar immigrants to the US who associateprimarily with other Korean immigrants might not master the English language despitemany years in the US. Factors that help distinguish between these groups were soughtamong the limited survey questions asked of the participants. One of the survey questions,‘‘Which language do you prefer?’’ was used to screen out Korean-born Americans whoimmigrated in early childhood. Another survey question, ‘‘How many years have you beenin the US?’’ was used to screen out exchange students, with a cutoff of 8 years to accountfor a lengthy university education. Finally, ‘‘at least 1 year of college education’’ was usedto screen out blue-collar immigrants. Of course, these conservative criteria may havesacrificed true bilinguals who preferred English over Korean, and blue-collar workers whowere truly bilingual. However, the stringency of our criteria gave us confidence that thesmall subsample identified was truly bilingual.Responses from 12 individuals who met these criteria were evaluated in this post hoc

analysis. For the basic scales, the correlations in this subsample were higher (Mdn ¼ :69),and in the cases of K and Ma, exceeded those of the comparison test–retest groups. Thecorrelation for the Pa scale was lower in the select sample. For the content scales,correlations were also higher (Mdn ¼ :73), approaching the established test–retestreliabilities. The cross-language correlation for scale BIZ exceeded the test–retest values.Other high correlations were found for ANX (.77), ANG (.77), CYN (.76), ASP (.73), SOD(.75), and WRK (.73). The lowest correlation was on FAM (.40). In all, these resultssupport the hypothesis that the low correlations found in the full sample were due in partto poor bilingual proficiency, rather than to problems with the Korean MMPI-2.An item analysis was conducted by correlating each item response on the Korean

MMPI-2 with the corresponding item on the American MMPI-2, using Pearson

ARTICLE IN PRESS

Table 3

Items showing strongest cross-language concordance and discrepancy and item test–retest correlations

Item

number

Cross-language r

(n ¼ 53)

Within-language test–retest r True%a

Americanb

(n ¼ 199)

Koreanc

(n ¼ 154)

American

version (n ¼ 53)

Korean version

(n ¼ 53)

Items showing strongest concordance

90 .90 .69 .61 88.7 90.6

412 .82 .69 .67 20.8 28.3

199 .77 .78 .69 58.5 54.7

35 .74 .78 .77 49.1 50.9

1 .74 .93 .70 28.3 32.1

2 .74 .46 .84 83.0 78.4

343 .73 .67 .72 81.1 75.5

80 .72 .79 .42 13.2 22.3

467 .71 .74 .58 58.5 50.9

282 .70 .62 .70 3.8 1.9

Median .72 .70

Mean .74 .68

Items showing strongest discrepancies

184 �.17 .66 .55 58.5 35.8

183 �.17 .44 .63 60.4 88.7

142 �.16 .55 .64 56.6 84.9

206 �.15 .38 .50 67.9 79.2

194 �.15 .55 .66 49.1 30.2

51 �.15 .56 .66 84.9 88.7

40 �.14 .63 .76 18.9 7.5

454 �.14 .52 .62 18.9 7.5

440 �.14 .32 .47 66.0 69.8

231 �.13 .68 .58 69.8 9.62

Median .55 .62

Mean .54 .61

Note: Unfortunately, we cannot reproduce item content because of University of Minnesota Press copyright

policy.aPercent of ‘‘true’’ item endorsement.bButcher et al. (2001).cKim et al. (2005).

J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531–543 539

correlation coefficient (F coefficient). Across the 567 items, correlations ranged from �.17to .90, a mean and standard deviation of .27 and .20, respectively. Table 3 presents the top10 items showing strongest endorsement concordance and discrepancy between versions.Items with simple and direct translations possessed high cross-language correlations, whileitems that contained complicated sentences, idioms, or double negatives had lowcorrelations. For both American and Korean samples, the items showing greatest cross-language concordance also had highest within-language stability, indicating that cross-language concordance is partly explained by item stability within language. Although mostitems correlated very well across languages, 60 items correlated negatively. Of these, manyitems are negatively phrased. It has been observed that the Korean language does notregularly use double negatives (Han, 1993), and their frequent use on the MMPI-2 may


contribute to responding a different way on the Korean version than an examinee might onthe American version. The last columns in Table 3 present the percentage of cases whoendorsed these items as ‘‘true.’’ As expected, items showing strongest cross-languageconcordance had little item endorsement differences across versions; those with strongestcross-language discrepancies had larger item endorsement differences.

4. Discussion

Bilingual participants who took both the English and the Korean MMPI-2 producedmean profiles in both languages that were strikingly similar to the pattern initially foundby Han (1993, 1996) and Han, Lim, Lee, Min, and Moon (2005); namely, elevated scoreson scales F, Sc, and D. Furthermore, profiles did not vary substantially across versions ofthe test. This finding has two major implications. First, it suggests that any problems withthe equivalence of the Korean translation do not manifest in systematic bias in overallscale elevation. This means that observed mean differences in the MMPI-2 betweenKoreans and Americans cannot be explained away as the effect of faulty translation.Second, the similarity of the profiles found here with Han’s (1996) Korean profiles lendssupport to her contention that culture-specific norms are necessary so that practitionerscan accurately identify within-culture variability. Use of American norms with Koreanswill make ‘‘normal Koreans’’ appear abnormal. That Han’s pattern of elevations appearedin this variously American-acculturated sample speaks to the robustness of whatevercultural phenomenon (e.g., true cultural differences in psychopathology, differences inconstruct meaning, differences in self-report strategies) is causing it.Although cross-language correlations tended to yield coefficients in the mid .50–.60,

considered sizable in many contexts, these values are unacceptably low in comparison topublished Korean test–retest reliabilities, which range in the high .70–.80. Clearly,participants were not responding as similarly across versions as they do within versions.Two possibilities might account for this finding. A first possibility is that many of the itemson Korean MMPI-2 are not equivalent to those on the American MMPI-2, either throughmistranslation or through inherent differences in the nature of the two languages. Becausemean profiles were so similar, these item problems would not be systematic in terms ofdirection or type of psychopathology indicated. A second possibility is that the items areequivalent, but that participants lacked the bilingual proficiently in one of the twolanguages to provide a reliable response. For example, exchange students from Korea maynot have been able to understand the subtleties of the English language; Americans whostudied Korean may not have been sufficiently fluent in their second language.To test this latter hypothesis, we selected a subsample that we believed would comp-

rise a more likely proficient bilingual group. We selected only individuals who reportedthat they preferred Korean over English, had been in the US for more than 8 years, andhad at least 1 year of college. Admittedly post hoc, these criteria were guided by ourinformal observations about the nature of our sample, and by our concern for samplesize, but were constructed pre-analysis. Consistent with the hypothesis, the selectgroup produced much higher cross-version correlations. Most of the 13 basic scalesapproached the published test–retest reliabilities, and two of the scales (K and Ma)slightly exceeded the published reliabilities. A majority of the 15 content scales approachedthe published test–retest reliabilities, and one of the scales (BIZ) slightly exceeded thepublished test–retest reliabilities. Interestingly, the correlation for the Pa scale was lower in


the select sample. This suggests, along with the finding of a substantial group meandifference on Pa, that the Korean Pa scale is not equivalent to the American Pa scale.Studies of the psychometric properties of Pa indicate that it is also somewhat of an outliereven with American samples (e.g., in terms of low reliability), so the meaning of thisfinding is unclear. Overall, however, these post hoc analyses suggest that a good portion ofthe cross-version unreliability can be accounted for by sample characteristics, namelyinsufficient bilinguality. It may, in fact, be the case that true bilingualism, sufficient topermit fair comparison to test–retest reliabilities, is very rare.

Analysis of the items answered most and least similarly across translations revealed apotential source of the cross-translation unreliability observed in the full sample. Althoughmost items correlated very well across languages, 60 items correlated negatively. Of these,many of these items are phrased negatively. It has been observed that the Korean languagedoes not regularly use double negatives, and their frequent use on the MMPI-2 maycontribute to responding a different way on the Korean version than an examinee might onthe American version.

This study demonstrated both the substantial strengths and weaknesses of this bilingualresearch design for studying test equivalence. Strengths include the ability to observe how asingle examinee responds to two different language versions of a test, thereby detectingsubtle changes in meaning that an item may take when it is translated into anotherlanguage. The weakness of this design is the reliance on truly bilingual participants, adifficult sample to obtain, which explains why most of the studies using this design (e.g.,Deinard et al., 1996; Tran, 1996; Velasquez et al., 2000) have had very small sample sizes.Future investigations using this design would benefit from a screening procedure that testsparticipants formally on their proficiency in both languages. Along with languageproficiency, acculturation level also needs to be considered in future research.Previous studies have shown acculturation to be associated with general adjustment level,reflected on performance on the MMPI/MMPI-2 (Dong & Church, 2003; Leung, 1986;Tsai & Pike, 2000). Study design should consider not only degree of bilinguality, but alsoacculturation.

Although not directly addressed by this study, the potential utility of the KoreanMMPI-2 for Koreans and Korea Americans in the US is noteworthy. Koreans are one ofthe most rapidly increasing immigrant groups in the US. Koreans in the US numbered 1.2million in 2000, representing the 5th largest Asian ethnic group in the US (United StatesCensus, 2000). Understanding Koreans, Korean–Americans, and the unique patterns ofpersonality and psychopathology that arise within these populations will becomeincreasingly important, especially within the context of Western medical, psychiatric,and psychological treatment. For instance, acculturation stress has been found to beassociated with a number of physical or psychological problems, including anxiety,depression, physical illness, and alcohol and drug consumption (Caetano & Clark, 2002;Myers & Rodriguez, 2002; Organista, Organista, & Kurasaki, 2002). For measurement ofthese problems, a well-developed standardized assessment scale is essential. The MMPI-2is a particularly good candidate because it contains a broad item pool that addresses a widevariety of aspects of both normal personality and psychopathology.

Very recently, the Korean MMPI-2 has been standardized with a normative sample of651 men and 701 women, and demographics very similar to the 2000 Korean Census data.Initial findings were encouraging with regard to the cross-cultural equivalence of thistranslation (Han et al., 2005, 200b; Kim et al., 2005). This instrument has promise as a tool


useful for measuring the personality and psychopathology of Koreans in the US as well asin Korea, particularly when accompanied by assessment of acculturation.To summarize, the results of the current study support the equivalence of the Korean

MMPI-2. Mean profiles were similar across samples, scale score correlations acrossversions were substantial, and in a subsample of the most likely bilingual, comparable inmagnitude to test–retest correlations. An item analysis revealed features of items that maycontribute to unreliability.

Acknowledgements

This research is based on the Master’s thesis of Junmo J. Chung, now at the Universityof Arkansas Medical School. We gratefully acknowledge the contributions of TomLombardo, Karen Christoff, and Ken McGraw as members of his thesis committee. Theauthors also thank Rev. Seung Bin Park of the Korean Baptist Church in Memphis, Rev.Randy Ju of the Korean Christian Church of Denver, and Rev. Kevin Cho of the Christthe Savior Lutheran Church of Denver for their help in recruiting participants.Correspondence concerning this article should be addressed to Kyunghee Han, SloanHall, Department of Psychology, Central Michigan University, Mt. Pleasant, MI 48859.Electronic mail may be sent via the Internet to [email protected].

References

Butcher, J. N. (1996). International adaptations of the MMPI-2. Minneapolis, MN: University of Minnesota Press.

Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. (1989). Manual for the

restandardized Minnesota Multiphasic Personality Inventory: MMPI-2. An administrative and interpretive guide.

Minneapolis: University of Minnesota Press.

Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., Dahlstrom, W. G., & Kaemmer, B. (2001). MMPI-2

manual for administration, scoring, and interpretation (rev ed.). Minneapolis, MN: University of Minnesota Press.

Butcher, J. N., Graham, J. R., Williams, C. L., & Ben-Porath, Y. S. (1990). Development and use of the MMPI-2

content scales. Minneapolis, MN: University of Minnesota Press.

Butcher, J. N., & Gur, R. (1974). A Hebrew translation of the MMPI: An assessment of translation adequacy and

preliminary validation. Journal of Cross-Cultural Psychology, 1, 185–216.

Butcher, J. N., & Pancheri, P. (1976). Handbook of cross-national MMPI research. Minneapolis, MN: University

of Minnesota Press.

Caetano, R., & Clark, C. L. (2002). Acculturation, alcohol consumption, smoking, and drug use among

Hispanics. In K. N. Chun, P. B. Organista, & M. G. Marin (Eds.), Acculturation: Advances in theory,

measurement, and applied research (pp. 223–239). Washington, DC: APA.

Cheung, B., Lee, C., & Jin, W. (1963). MMPI manual. Seoul: Korean Testing Center.

Cheung, F. M., Song, W. Z., & Butcher, J. N. (1991). An infrequency scale for the Chinese MMPI. Psychological

Assessment, 3, 230–237.

Cheung, F. M., Song, W., & Zhang, J. (1996). The Chinese MMPI-2: Research and applications in Hong Kong

and the People’s Republic of China. In J. N. Butcher (Ed.), International adaptations of the MMPI-2:

A handbook of research and clinical applications (pp. 137–191). Minneapolis, MN: University of Minnesota Press.

Deinard, A. S., Butcher, J. N., Thao, U. D., Vang, S. H. M., & Hang, K. (1996). In J. N. Butcher (Ed.),

International adaptations of the MMPI-2 (pp. 194–205). Minneapolis, MN: University of Minnesota Press.

Dong, Y. T., & Church, A. T. (2003). Cross-cultural equivalence and validity of the Vietnamese MMPI-2:

Assessing psychological adjustment of Vietnamese refugees. Psychological Assessment, 15, 370–377.

Graham, J. R. (2000). MMPI-2: Assessing personality and psychopathology (3rd ed.). New York: Oxford.

Han, K. (1990). International applications of the MMPI-2: Translation issues and procedures. In: J. N. Butcher

(Chair), Korean Translation of the MMPI-2. Paper presented at the 25th Annual Symposium on Recent

Developments in the Use of the MMPI (MMPI-2), Minneapolis, MN.

mailto:[email protected]


Han, K. (1993). The use of the MMPI-2 in Korea: Inventory adaptation, equivalence evaluation, and initial

validation. Unpublished Doctoral Dissertation, University of Minnesota, Minneapolis.

Han, K. (1996). The Korean MMPI-2. In J. N. Butcher (Ed.), International adaptations of the MMPI-2:

A handbook of research and clinical applications (pp. 88–136). Minneapolis, Minnesota: University of

Minnesota Press.

Han, K. (2005). The MMPI-II: (Instrument). Seoul, Korea: Maumsarang Inc.

Han, K., Lim, J., Lee, J., Min, B., & Moon, K. (2005). Validation of the Korean MMPI-2 using the Korean

Normative Sample. Paper presented at the 40th Annual Symposium on Recent Developments in the Use of the

MMPI-2/MMPI-A, Fort Lauderdale, FL.

Kim, Y., Kim, J., Kim, J., Noh, M., Shin, D., Yum, T., & Oh, S. (1989). (MMPI

manual). Seoul: Korea Guidance.

Kim, Z., Han, K., Lim, J., Lee, J., Min, B., Moon, K., & Lee, J. (2005). The MMPI-II: (User

manual). Seoul, Korea: Maumsarang Inc.

Konraos, S. (1996). The Icelandic translation of the MMPI-2: Adaptation and validation. In J. N. Butcher (Ed.),

International adaptations of the MMPI-2 (pp. 368–384). Minneapolis, MN: University of Minnesota Press.

Leung, R. (1986). MMPI scoring as a function of ethnicity and acculturation: A comparison of Asian, Hispanic,

and Caucasian gifted high school students. Dissertation Abstracts International, 47(6B), 2622.

Myers, H. F., & Rodriguez, N. (2002). Acculturation and physical health in racial and ethnic minorities. In

K. N. Chun, P. B. Organista, & M. G. Marin (Eds.), Acculturation: Advances in theory, measurement, and

applied research (pp. 163–185). Washington, DC: APA.

Organista, P. B., Organista, K. C., & Kurasaki, K. (2002). The relationship between acculturation and ethnic

minority mental health. In K. N. Chun, P. B. Organista, & M. G. Marin (Eds.), Acculturation: Advances in

theory, measurement, and applied research (pp. 139–161). Washington, DC: APA.

Sireci, S. G., & Berberoglu, G. (2000). Using bilingual respondents to evaluate translated-adapted items. Applied

Measurement in Education, 13, 229–248.

Tran, B. N. (1996). Vietnamese translation and adaptation of the MMPI-2. In J. N. Butcher (Ed.), International

adaptations of the MMPI-2 (pp. 175–193). Minneapolis, MN: University of Minnesota Press.

Tsai, D. C., & Pike, P. L. (2000). Effects of Acculturation on the MMPI-2: Scores of Asian American Students.

Journal of Personality Assessment, 74, 216–230.

United States Census (2000). http://www.census.gov/population/www/socdemo/race.html US Census Bureau.

January 25, 2002.

Velasquez, R. I., Chavira, D. A., Karle, H. R., Callahan, W. J., Garcia, J. A., & Castellanos, J. (2000). Assessing

bilingual and monolingual Latino students with translations of the MMPI-2: Initial data. Cultural Diversity

and Ethnic Minority Psychology, 6, 65–72.

http://www.census.gov/population/www/socdemo/race.html

mmpi teona 2

Documents

Transcript of mmpi teona 2