ARTICLE IN PRESS

International Journal of Intercultural Relations 30 (2006) 531543 www.elsevier.com/locate/ijintrel

Evaluating cross-cultural equivalence of the Korean MMPI-2 via bilingual testretest

Junmo J. Chunga, Nathan C. Weedb, Kyunghee Hanb,
b a Department of Psychology, University of Mississippi, USA Department of Psychology, Central Michigan University, 103 Sloan Hall, Mt. Pleasant MI 48859, USA

Received 13 May 2005; received in revised form 16 August 2005; accepted 23 August 2005

Abstract Previous efforts to establish the cross-cultural equivalence of the Korean MMPI-2 have focused on analysis of internal structure and peer behavioral correlates using only a Korean college sample. The intent of this study was to provide further evidence of the cross-cultural equivalence of the Korean MMPI-2 using a bilingual testretest method. Fifty-three self-reported bilinguals living in the US completed both the MMPI-2 in Korean and English within a 1-week interval. Their proles were compared to previously established testretest reliabilities. Cross-language correlations were sizable, but substantially lower than the published within-language testretest correlations. However, when a procient bilingual subsample was selected from the original sample post hoc, cross-language correlations were more similar to testretest reliabilities. Cross-language item analysis revealed that items with simple and direct grammar had the highest item agreement. Complicated sentences, such as those containing double negatives or unique American idioms, had the least concordance. r 2005 Elsevier Ltd. All rights reserved.

1. Introduction Translated into numerous languages and adapted for a wide variety of cultures (Butcher, 1996; Butcher & Pancheri, 1976), the MMPI (Minnesota Multiphasic Personality Inventory) is the most widely used personality instrument in the world. The MMPI has been used in Korean populations as a clinical assessment and research instrument since its
Corresponding author. Tel.: +1 989 774 6496; fax: +1 989 774 2553.

E-mail address: han1k@cmich.edu (K. Han). 0147-1767/$ - see front matter r 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.ijintrel.2005.08.009

ARTICLE IN PRESS
532 J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531543

translation in 1963 (Cheung, Lee, & Jin, 1963). Despite wide use and acceptance by the Korean psychological community the validity and reliability of this original version remained largely untested, and serious translation problems (including miskeying) were identied (Han, 1993). In 1989, the Korean MMPI was retranslated and standardized with a large normative sample (Kim et al., 1989). Although all items were properly keyed in the same direction as the original English version, the new translation appeared not to correct many distortions present in the original translation, and added others. Further, the simultaneous release of the English MMPI-2 (Minnesota multiphasic personality inventory-revised) made this version obsolete as it was released. Shortly after the MMPI-2 was published, the MMPI-2 was translated into Korean by Han (1990). Han and one bilingual student independently translated the MMPI-2 from English into Korean and then compared the two versions, solving discrepancies between them by mutual consensus. The Korean items were then submitted to a different bilingual student for back translation into English. The two English versions, the original English items and the back translated English items, were then examined for discrepancies by an American psychologist who was an expert in MMPI cross-cultural work. Discrepancies between the two versions guided a revision of the affected Korean items. Out of the total MMPI-2 item pool, 20 items were judged as non-equivalent in the two English versions. The 20 inadequate items were reviewed and re-translated. The corrected version of these items was presented to the same back translator, who then produced a back-translation judged to be equivalent to the original English version (Han, 1993). To measure its psychometric properties, the Korean MMPI-2 was administered to 726 Korean college students from eight universities across Korea, and validation efforts consisted of equivalence data in the form of factor structure, 1-week testretest, and peer behavior correlates (Han, 1993). Although these efforts produced the best data to date in support of the cross-cultural equivalence of the Korean MMPI-2, additional data are needed to support these early efforts, particularly involving: (a) samples other than Korean college students, and (b) additional analytic strategies. The present study attempts to provide additional validation support for the Korean MMPI-2 by examining responses from KoreanEnglish bilinguals on both translations of the MMPI-2. The bilingual retest technique is considered an important method for evaluating the accuracy and adequacy of a translation of a test (Butcher, 1996). In this method, both the original language form and the translated version are administered to a selected group of bilinguals who are familiar with both cultures. Then, scale mean differences or item endorsement frequencies across the two versions are compared, or cross-language correlations are computed. Recently more rigorous statistical techniques, such as those based on IRT (item response theory) have been developed to evaluate the equivalence of two versions using samples of bilingual respondents (e.g., Sireci & Berberoglu, 2000), but they have not been widely used due to strict assumptions and requirement of a large sample size. A number of studies have used the bilingual technique for checking translation adequacy of the MMPI and MMPI-2 items (e.g., Butcher & Gur, 1974; Tran, 1996; Velasquez et al., 2000). Butcher and Gur (1974) administered the English-language and Hebrew translation of the MMPI to 28 bilinguals. The cross-language correlations for the 13 basic scales ranged from .51 to .91, with a mean of .74. In a sample of 32 college students or recent university graduates, Tran (1996) found that the mean correlation coefcient between scores on the English MMPI-2 and those on the Vietnamese translation of the MMPI-2 was .72, with coefcients ranging from a low of .35 for the Hy scale and a high of .88 for the Mf scale. In

ARTICLE IN PRESS
J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531543 533

a Hmong adaptation study involving 30 bilinguals, Deinard, Butcher, Thao, Vang, and Hang (1996) reported that the mean cross-language correlation was .59, with scales Ma and D showing the lowest (.38) and highest (.80) correlations, respectively. The latter was somewhat of an unexpected nding because the authors used very rigorous criteria to evaluate uency in both languages. In contrast, exceptionally high cross-language ` correlations (M :79) were found in a sample of 148 bilingual Icelandic adults (Konraos, 1996), a nding that may owe to the linguistic commonalities between English and Icelandic languages. In Velasquez et al. (2000), 27 bilingual participants were administered both the English version and the Spanish translation of the MMPI-2. No signicant mean differences on the basic scales were found, and the mean cross-language correlation was .71, with coefcients ranging from .60 (Hy) to .77 (D, Pa, Pt, Sc, and Ma). The present study attempts to provide additional validation support for the Korean MMPI-2 by examining the responses from KoreanEnglish bilinguals on both translations of the MMPI-2. If the Korean translation of the MMPI-2 is accurate, bilinguals should respond similarly to the two versions. Consequently, we expect to nd that a sample of KoreanEnglish bilinguals will produce a similar pattern of mean scores on MMPI-2 scales regardless of translation and demonstrate high correlations between their individual scores on the Korean and English versions of MMPI-2 scales. 2. Method 2.1. Participants and procedures Study participants were recruited through bulletins and announcements made at Korean churches in the Denver and Memphis communities, and with the help of Korean student organizations at universities in Mississippi and Colorado. The announcement explained that participation was limited to Korean/English bilinguals, would require two consecutive sessions, 1 week apart, and that each participant would be paid $20 for full participation. One hundred thirty-four people volunteered to participate, and each was either administered either the Korean version or the American version of the MMPI-2, the version chosen randomly. One week later, available participants were administered the other version of the MMPI-2. This design yielded 107 completed Korean MMPI-2 protocols and 82 American MMPI-2 protocols. Protocols were eliminated from the study if they possessed one or more of the following features1: Cannot say score X15; F (Infrequency) X20; Fb (Back F; Fb consists of items with extremely low endorsement frequency that appear later in the test booklet than F items) X20; VRIN (Variable Response Inconsistency) X13; TRIN (True Response
Cheung used somewhat liberal exclusion criteria for screening invalid cases for the Chinese version of the MMPI (Cheung, Song, & Butcher, 1991) and MMPI-2 (Cheung, Song, & Zhang, 1996) because use of American standards for F and Fb were likely to overestimate invalidity in normal Chinese samples. For our study, we decided to use the exclusion criteria that were set by American standards (Butcher et al., 1989) due to a lack of studies demonstrating that American criteria were problematic for Asian Americans or Asians who lived in the US for more than 8 years. We expect that use of more liberal F and Fb criteria would slightly elevate overall mean level, in particular for scales Pa and Sc because F and Fb share a substantial number of items with those scales. It is also plausible to expect that including cases with high F or Fb might increase cross-language correlations on scales Pa and Sc because of the resulting increase in variability; still, such an increase would probably be minimal at best.
1

ARTICLE IN PRESS
534 J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531543

Inconsistency) p4; or TRINX13. This screening reduced the sample to 89 valid Korean cases (83% of completed protocols), and 69 valid American cases (84% of completed protocols), with 53 participants (23 men and 30 women) producing both valid American and Korean MMPI-2 protocols. The mean age of this sample was 29.6 years, and mean length of stay in the US was 11.4 years. Eighteen participants (34%) reported that they preferred to speak English, while 35 (66%) preferred Korean. Forty participants (75%) were working adults, 12 (23%) were college students, and one (2%) was in high school. 2.2. Instruments The MMPI was published in 1943 by Hathaway and McKinley who were working in the University of Minnesota Hospitals. The authors used an empirical keying approach in the construction of the original clinical scales so that scale items were those that best distinguished between patients with discrete psychopathological conditions and a group of examinees who were not hospitalized. The original MMPI contained 566 self-referenced true/false statements (Graham, 2000). In 1989 the updated and restandardized MMPI-2 was published by Butcher, Dahlstrom, Graham, Tellegen, and Kaemmer (1989). Because the original MMPI was so widely used, great care was taken to make improvements (e.g., updating language, including more representative norms, applying uniform T-score conversions, developing new scales) while maintaining its continuity with the original instrument (preserving the clinical scales and their empirical meaning). The MMPI-2 consists of 567 items, and contains numerous scales to assess both normal range personality constructs and psychopathological symptomatology. In the present study we focus on just the most commonly used scales of the MMPI-2: 3 validity scales, 10 clinical scales, and 15 content scales. The three validity scales, L (Lie), F (Infrequency), and K (Correction), were developed to measure: a deliberate and rather unsophisticated attempt to present oneself in a favorable manner, deviant or atypical ways of responding to test items, and a more subtle and sophisticated attempt to present oneself in a favorable light, respectively (Butcher et al., 1989; Graham, 2000). The 10 MMPI-2 clinical scales were designed to measure: an excess concern about ones health (Hs, Hypochondriasis), various symptoms associated with depression (D, Depression), hysterical syndromes associated with involuntary psychogenic loss or disorder of function (Hy, Hysteria), psychopathic or sociopathic characteristics, including delinquent acts, sexual problems, family problems, and difculties with authorities (Pd, Psychopathic Deviate), gender-role divergence, including interests or hobbies that were opposite to the stereotypical gender role (Mf, MasculininityFemininity), paranoid symptoms, interpersonal sensitivities, and a tendency to misinterpret the motives and intentions of others (Pa, Paranoia), generalized anxiety and distress, unreasonable fears, compulsions, and obsessions (Pt, Psychasthenia), psychotic symptoms, such as bizarre mentation, peculiarities of perception, and halluciations, social alienation, and poor family relationships (Sc, Schizophrenia), hypomanic symptoms, including elevated mood, accelerated speech and motor activity, irritability, and ights of ideas (Ma, Hypomania), and social withdrawal and self-deprecation (Si, Social Introversion) (Butcher et al., 1989; Graham, 2000). The 15 content scales were developed using a more modern rational-deductive approach to scale construction, and cover a wide range of clinical and normal-range concerns. They

ARTICLE IN PRESS
J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531543 535

include: Anxiety (ANX; tension, worry, fears, lack of condence, and somatic indications of anxiety), Fears (FRS; specic fears such as high places, snakes, spiders, res, and storms), Obsessiveness (OBS; rumination about decisions and problems, and compulsions such as counting and saving unimportant things), Depression (DEP; brooding, crying easily, pessimism, suicidal ideation, and guilt), Health Concerns (HEA; gastrointestinal symptoms, neurological symptoms, dermatological problems, and pain), Bizarre Mentation (BIZ; paranoid ideation, ideas of reference, delusional thinking, and hallucinations), Anger (ANG; fear of losing self-control over aggressive impulses, irritability, impatience, stubbornness, physical and/or verbal abusiveness, and explosivity), Cynicism (CYN; hostility, suspicion, misanthrope, distrust, and selshness), Antisocial Practices (ASP; antiauthority ideation, rationalization and identication with criminal behavior, admission of antisocial or unlawful behaviors), Type A (TPA; hard driving, fast paced, task-orientation, competitiveness, and workaholism), Low Self-Esteem (LSE; a lack of self-esteem, feelings of unattractiveness and uselessness), Social Discomfort (SOD; introversion, social avoidance, dislike of crowds, parties, or group activities), Family Problems (FAM; general problems with family), Work Interference (WRK; difculties concentrating, anxiety, tension, lack of self-condence, and indecisiveness about career choices), Negative Treatment Indicators (TRT; negative attitudes toward health care providers and treatment, pessimism about individuals being understanding or helpful) (Butcher et al., 1989; Butcher, Graham, Williams, & Ben-Porath, 1990; Graham, 2000). All participants completed both the English version of the MMPI-2 (Butcher et al., 1989) and the Korean version of the MMPI-2 (Han, 1993, 1996, 20052; Kim et al., 2005 (see footnote 2)), each of which comprises 567 true/false self-referenced items.

3. Results Table 1 displays descriptive statistics and effect sizes comparing scores on the Korean and English MMPI-2. Raw scores of the 13 basic and 15 content scales were converted to K-corrected T-scores using the American adult norms. For both versions, the highest mean elevation among the basic scales was on scale F (62.9 for the Korean version; 63.2 for the American version), and the next highest mean scale elevations were on scales Sc (62.8 for the Korean version; 62.9 for the American version) and D (62.0 for the Korean version; 60.0 for the American version). For both versions, the lowest scores were on scale K, with means of 50 for the Korean version and 49.6 for the American version. Therefore, mean proles were similar for the two versions, both in terms of overall elevation and in terms of specic scales elevations; except for scale Pa, all basic scale means fell within ve T-score
Since the completion of the present study, the Korean MMPI-2 has undergone additional revision. In 2002, before setting out on a Korean MMPI-2 standardization project, the committee members reviewed Hans translation and believed that some items needed to be retranslated because it has been about 10 years since Han translated the items, during which Korea has gone through rapid political and social changes, some of which are reected in Koreans use of language. Through multistage revision procedures, the committee and Han revised the 179 items. Most of the item changes were very minor, and would not be expected to affect endorsement patterns signicantly. The item changes include: (a) location a word within a sentence, (b) contemporary wording, (c) grammatical appropriateness and natural word choice. Of the 179 items, 13 were judged by the committee and Dr. Han to be somewhat different from the original translation, and were, therefore, submitted to a backtranslation into English. After two rounds of backtranslation, all 13 item pairs were rated equivalent.
2

ARTICLE IN PRESS
536 J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531543 Table 1 Means, standard deviations, and standard differences on basic and content scales between Korean and American versions of the MMPI-2 (n 53) Korean version M Basic scales L F K Hs D Hy Pd Mf Pa Pt Sc Ma Si Content scales ANX FRS OBS DEP HEA BIZ ANG CYN ASP TPA LSE SOD FAM WRK TRT SD American version M SD d

56.3 62.9 50.0 60.0 62.0 54.5 56.6 53.8 51.4 58.3 62.8 56.7 54.6 54.3 59.7 56.2 55.7 57.8 57.5 53.0 51.4 54.8 54.6 52.6 54.0 51.5 58.4 57.9

10.3 13.5 11.7 11.7 12.5 11.6 10.6 8.1 8.8 12.2 10.6 11.8 10.0 11.7 12.3 13.2 10.2 11.1 8.9 11.5 8.7 8.9 11.7 9.6 10.2 10.1 11.5 11.7

56.8 63.2 49.6 59.2 60.0 55.5 59.0 54.6 57.7 59.4 62.9 56.9 56.2 54.7 55.8 55.1 57.3 58.3 59.6 50.8 52.8 55.2 50.8 53.4 52.3 51.4 58.1 59.2

11.7 14.5 9.6 10.1 12.5 9.9 9.5 10.1 12.9 10.0 11.2 11.9 10.3 10.2 11.5 11.9 8.7 9.5 10.7 10.1 7.8 8.5 8.6 9.0 9.9 8.9 10.6 9.8

.05 .02 .04 .07 .16 .09 .24 .09 .58 .10 .01 .02 .16 .04 .33 .09 .17 .05 .22 .21 .17 .05 .37 .09 .17 .01 .03 .12

Note: L: Lie; F: Infrequency; K: Correction; Hs: Hypochondriasis; D: Depression; Hy: Hysteria; Pd: Psychopathic Deviant; Mf: MasculininityFemininity; Pa: Paranoia; Pt: Psychasthenia; Sc: Schizophrenia; Ma: Hypomania; Si: Social Introversion; ANX: Anxiety; FRS: Fears; OBS: Obsessiveness; DEP: Depression ; HEA: Health Concerns ; BIZ: Bizarre Mentation; ANG: Anger; CYN: Cynicism; ASP: Antisocial Practices; TPA: Type A; LSE: Low SelfEsteem; SOD: Social Discomfort; FAM: Family Problems; WRK: Work Interference; TRT: Negative Treatment Indicators.

points of one another. All Cohens d values were small with the exception of that for Pa, which showed moderately sized (d :58) mean differences between the two versions. For the content scales, all mean scores fell within four T score points across versions with d values ranging from .01 to .37 (Mdn :12). Unlike the basic scales, no content scale elevations were one standard deviation away from the normative means. The highest elevations on the Korean version were FRS (59.7), WRK (58.4), TRT (57.9), HEA (57.8), and BIZ (57.5). Similarly, the American version produced the highest elevations at BIZ (59.6), TRT (59.2), HEA (58.3), WRK (58.1), and DEP (57.3). No scale means fell below

ARTICLE IN PRESS
J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531543 537

the US normative means. In sum, for both versions, all clinical and content scale mean scores were elevated when proles were plotted against American adult norms. In particular, scales F, D, and Sc showed the highest elevations for both versions. Mean T scores for all scales are close across versions, with all scale means except for Pa falling within ve T-score points of one another across versions. Cross-language correlations for the basic and content scales are presented in Table 2, using the established Korea testretest and American testretest reliabilities reported by
Table 2 Cross-language correlations and testretest correlations for MMPI-2 basic and content scales Cross-language correlations Within-language testretest correlations Korean women American American women (n 76) men (n 82) (n 111)

Total bilinguals Selected bilinguals Korean men (n 53) (n 12) (n 78) Basic scales L .21 F .42 K .62 Hs .51 D .50 Hy .38 Pd .39 Mf .27 Pa .37 Pt .49 Sc .62 Ma .52 Si .66 Median Mean .49 .47

.36 .41 .89 .81 .62 .57 .69 .54 .24 .69 .73 .82 .77 .69 .66 .77 .67 .85 .65 .46 .84 .77 .76 .73 .49 .60 .75 .40 .73 .65 .73 .70

.78 .85 .74 .77 .86 .79 .68 .63 .86 .79 .82 .75 .85 .79 .79 .87 .72 .77 .87 .84 .78 .84 .68 .77 .76 .84 .84 .76 .83 .81 .81 .81

.77 .81 .83 .75 .73 .75 .73 .65 .69 .83 .75 .76 .82 .79 .79 .81 .75 .80 .85 .83 .71 .77 .84 .78 .81 .86 .90 .81 .87 .87 .81 .81

.86 .74 .80 .76 .79 .70 .79 .83 .67 .72 .72 .80 .93 .75 .77 .89 .82 .84 .84 .80 .77 .87 .81 .82 .81 .84 .91 .84 .90 .79 .82 .81

.81 .72 .80 .75 .80 .74 .69 .74 .56 .68 .54 .65 .92 .74 .74 .88 .87 .84 .88 .86 .78 .82 .88 .86 .78 .86 .91 .83 .90 .88 .81 .81

Content scales ANX .70 FRS .63 OBS .63 DEP .41 HEA .58 BIZ .53 ANG .59 CYN .57 ASP .47 TPA .55 LSE .48 SOD .70 FAM .46 WRK .47 TRT .46 Median Mean .55 .56

Notes. American testretest correlations from Butcher et al. (2001); Korean testretest correlations from Kim et al. (2005).

ARTICLE IN PRESS
538 J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531543

Butcher et al. (2001) and Kim et al. (2005) as comparisons. Cross-language testretest correlations were obtained by correlating MMPI-2 scale scores from the Korean version with the corresponding scores from the American version. A correlation coefcient was computed for each scale, indexing the degree to which individual scale performance was the same across versions. All cross-language correlations were lower than the Korean and American testretest comparison groups. As in the American testretest samples, the highest cross-language correlation was with Scale Si (.66). The next two highest crosslanguage correlations were on K (.62) and Sc (.62), whereas the lowest cross-language correlations were on scales L (.21) and Mf (.27). Cross-language correlations for the content scales (Mdn :55) were larger than those for the basic scales (Mdn :49), but were still lower than the Korean and American testretest comparison groups. The highest correlations were on ANX (.70) and SOD (.70). Low cross language correlations suggest that the Korean MMPI-2 is not measuring the same dimensions as the American MMPI-2 as effectively as one would hope, and not so precisely as to represent a testretest. This nding also suggests the possibility that the participants were not procient in both languages. To investigate the latter possibility, a clearly bilingual subsample was selected among the full sample based on three criteria: (a) reporting a preference for speaking Korean, (b) living in the US for more than 8 years, and (c) having at least 1 year of college education. We reasoned that Korean-born Americans who had immigrated late enough in life to retain their native Korean language skills would be the most bilingual. Korean-born Americans who immigrated early in childhood might not speak Korean prociently enough to be truly bilingual, whereas Korean students studying in the US might not speak English well enough. Finally, Korean blue-collar immigrants to the US who associate primarily with other Korean immigrants might not master the English language despite many years in the US. Factors that help distinguish between these groups were sought among the limited survey questions asked of the participants. One of the survey questions, Which language do you prefer? was used to screen out Korean-born Americans who immigrated in early childhood. Another survey question, How many years have you been in the US? was used to screen out exchange students, with a cutoff of 8 years to account for a lengthy university education. Finally, at least 1 year of college education was used to screen out blue-collar immigrants. Of course, these conservative criteria may have sacriced true bilinguals who preferred English over Korean, and blue-collar workers who were truly bilingual. However, the stringency of our criteria gave us condence that the small subsample identied was truly bilingual. Responses from 12 individuals who met these criteria were evaluated in this post hoc analysis. For the basic scales, the correlations in this subsample were higher (Mdn :69), and in the cases of K and Ma, exceeded those of the comparison testretest groups. The correlation for the Pa scale was lower in the select sample. For the content scales, correlations were also higher (Mdn :73), approaching the established testretest reliabilities. The cross-language correlation for scale BIZ exceeded the testretest values. Other high correlations were found for ANX (.77), ANG (.77), CYN (.76), ASP (.73), SOD (.75), and WRK (.73). The lowest correlation was on FAM (.40). In all, these results support the hypothesis that the low correlations found in the full sample were due in part to poor bilingual prociency, rather than to problems with the Korean MMPI-2. An item analysis was conducted by correlating each item response on the Korean MMPI-2 with the corresponding item on the American MMPI-2, using Pearson

ARTICLE IN PRESS
J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531543 Table 3 Items showing strongest cross-language concordance and discrepancy and item testretest correlations Item number Cross-language r (n 53) Within-language testretest r Americanb (n 199) Koreanc (n 154) True%a American Korean version version (n 53) (n 53) 539

Items showing strongest concordance 90 .90 412 .82 199 .77 35 .74 1 .74 2 .74 343 .73 80 .72 467 .71 282 .70 Median Mean Items showing strongest discrepancies 184 .17 183 .17 142 .16 206 .15 194 .15 51 .15 40 .14 454 .14 440 .14 231 .13 Median Mean

.69 .69 .78 .78 .93 .46 .67 .79 .74 .62 .72 .74 .66 .44 .55 .38 .55 .56 .63 .52 .32 .68 .55 .54

.61 .67 .69 .77 .70 .84 .72 .42 .58 .70 .70 .68 .55 .63 .64 .50 .66 .66 .76 .62 .47 .58 .62 .61

88.7 20.8 58.5 49.1 28.3 83.0 81.1 13.2 58.5 3.8

90.6 28.3 54.7 50.9 32.1 78.4 75.5 22.3 50.9 1.9

58.5 60.4 56.6 67.9 49.1 84.9 18.9 18.9 66.0 69.8

35.8 88.7 84.9 79.2 30.2 88.7 7.5 7.5 69.8 9.62

Note: Unfortunately, we cannot reproduce item content because of University of Minnesota Press copyright policy. a Percent of true item endorsement. b Butcher et al. (2001). c Kim et al. (2005).

correlation coefcient (F coefcient). Across the 567 items, correlations ranged from .17 to .90, a mean and standard deviation of .27 and .20, respectively. Table 3 presents the top 10 items showing strongest endorsement concordance and discrepancy between versions. Items with simple and direct translations possessed high cross-language correlations, while items that contained complicated sentences, idioms, or double negatives had low correlations. For both American and Korean samples, the items showing greatest crosslanguage concordance also had highest within-language stability, indicating that crosslanguage concordance is partly explained by item stability within language. Although most items correlated very well across languages, 60 items correlated negatively. Of these, many items are negatively phrased. It has been observed that the Korean language does not regularly use double negatives (Han, 1993), and their frequent use on the MMPI-2 may

ARTICLE IN PRESS
540 J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531543

contribute to responding a different way on the Korean version than an examinee might on the American version. The last columns in Table 3 present the percentage of cases who endorsed these items as true. As expected, items showing strongest cross-language concordance had little item endorsement differences across versions; those with strongest cross-language discrepancies had larger item endorsement differences. 4. Discussion Bilingual participants who took both the English and the Korean MMPI-2 produced mean proles in both languages that were strikingly similar to the pattern initially found by Han (1993, 1996) and Han, Lim, Lee, Min, and Moon (2005); namely, elevated scores on scales F, Sc, and D. Furthermore, proles did not vary substantially across versions of the test. This nding has two major implications. First, it suggests that any problems with the equivalence of the Korean translation do not manifest in systematic bias in overall scale elevation. This means that observed mean differences in the MMPI-2 between Koreans and Americans cannot be explained away as the effect of faulty translation. Second, the similarity of the proles found here with Hans (1996) Korean proles lends support to her contention that culture-specic norms are necessary so that practitioners can accurately identify within-culture variability. Use of American norms with Koreans will make normal Koreans appear abnormal. That Hans pattern of elevations appeared in this variously American-acculturated sample speaks to the robustness of whatever cultural phenomenon (e.g., true cultural differences in psychopathology, differences in construct meaning, differences in self-report strategies) is causing it. Although cross-language correlations tended to yield coefcients in the mid .50.60, considered sizable in many contexts, these values are unacceptably low in comparison to published Korean testretest reliabilities, which range in the high .70.80. Clearly, participants were not responding as similarly across versions as they do within versions. Two possibilities might account for this nding. A rst possibility is that many of the items on Korean MMPI-2 are not equivalent to those on the American MMPI-2, either through mistranslation or through inherent differences in the nature of the two languages. Because mean proles were so similar, these item problems would not be systematic in terms of direction or type of psychopathology indicated. A second possibility is that the items are equivalent, but that participants lacked the bilingual prociently in one of the two languages to provide a reliable response. For example, exchange students from Korea may not have been able to understand the subtleties of the English language; Americans who studied Korean may not have been sufciently uent in their second language. To test this latter hypothesis, we selected a subsample that we believed would comprise a more likely procient bilingual group. We selected only individuals who reported that they preferred Korean over English, had been in the US for more than 8 years, and had at least 1 year of college. Admittedly post hoc, these criteria were guided by our informal observations about the nature of our sample, and by our concern for sample size, but were constructed pre-analysis. Consistent with the hypothesis, the select group produced much higher cross-version correlations. Most of the 13 basic scales approached the published testretest reliabilities, and two of the scales (K and Ma) slightly exceeded the published reliabilities. A majority of the 15 content scales approached the published testretest reliabilities, and one of the scales (BIZ) slightly exceeded the published testretest reliabilities. Interestingly, the correlation for the Pa scale was lower in

ARTICLE IN PRESS
J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531543 541

the select sample. This suggests, along with the nding of a substantial group mean difference on Pa, that the Korean Pa scale is not equivalent to the American Pa scale. Studies of the psychometric properties of Pa indicate that it is also somewhat of an outlier even with American samples (e.g., in terms of low reliability), so the meaning of this nding is unclear. Overall, however, these post hoc analyses suggest that a good portion of the cross-version unreliability can be accounted for by sample characteristics, namely insufcient bilinguality. It may, in fact, be the case that true bilingualism, sufcient to permit fair comparison to testretest reliabilities, is very rare. Analysis of the items answered most and least similarly across translations revealed a potential source of the cross-translation unreliability observed in the full sample. Although most items correlated very well across languages, 60 items correlated negatively. Of these, many of these items are phrased negatively. It has been observed that the Korean language does not regularly use double negatives, and their frequent use on the MMPI-2 may contribute to responding a different way on the Korean version than an examinee might on the American version. This study demonstrated both the substantial strengths and weaknesses of this bilingual research design for studying test equivalence. Strengths include the ability to observe how a single examinee responds to two different language versions of a test, thereby detecting subtle changes in meaning that an item may take when it is translated into another language. The weakness of this design is the reliance on truly bilingual participants, a difcult sample to obtain, which explains why most of the studies using this design (e.g., Deinard et al., 1996; Tran, 1996; Velasquez et al., 2000) have had very small sample sizes. Future investigations using this design would benet from a screening procedure that tests participants formally on their prociency in both languages. Along with language prociency, acculturation level also needs to be considered in future research. Previous studies have shown acculturation to be associated with general adjustment level, reected on performance on the MMPI/MMPI-2 (Dong & Church, 2003; Leung, 1986; Tsai & Pike, 2000). Study design should consider not only degree of bilinguality, but also acculturation. Although not directly addressed by this study, the potential utility of the Korean MMPI-2 for Koreans and Korea Americans in the US is noteworthy. Koreans are one of the most rapidly increasing immigrant groups in the US. Koreans in the US numbered 1.2 million in 2000, representing the 5th largest Asian ethnic group in the US (United States Census, 2000). Understanding Koreans, KoreanAmericans, and the unique patterns of personality and psychopathology that arise within these populations will become increasingly important, especially within the context of Western medical, psychiatric, and psychological treatment. For instance, acculturation stress has been found to be associated with a number of physical or psychological problems, including anxiety, depression, physical illness, and alcohol and drug consumption (Caetano & Clark, 2002; Myers & Rodriguez, 2002; Organista, Organista, & Kurasaki, 2002). For measurement of these problems, a well-developed standardized assessment scale is essential. The MMPI-2 is a particularly good candidate because it contains a broad item pool that addresses a wide variety of aspects of both normal personality and psychopathology. Very recently, the Korean MMPI-2 has been standardized with a normative sample of 651 men and 701 women, and demographics very similar to the 2000 Korean Census data. Initial ndings were encouraging with regard to the cross-cultural equivalence of this translation (Han et al., 2005, 200b; Kim et al., 2005). This instrument has promise as a tool

ARTICLE IN PRESS
542 J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531543

useful for measuring the personality and psychopathology of Koreans in the US as well as in Korea, particularly when accompanied by assessment of acculturation. To summarize, the results of the current study support the equivalence of the Korean MMPI-2. Mean proles were similar across samples, scale score correlations across versions were substantial, and in a subsample of the most likely bilingual, comparable in magnitude to testretest correlations. An item analysis revealed features of items that may contribute to unreliability. Acknowledgements This research is based on the Masters thesis of Junmo J. Chung, now at the University of Arkansas Medical School. We gratefully acknowledge the contributions of Tom Lombardo, Karen Christoff, and Ken McGraw as members of his thesis committee. The authors also thank Rev. Seung Bin Park of the Korean Baptist Church in Memphis, Rev. Randy Ju of the Korean Christian Church of Denver, and Rev. Kevin Cho of the Christ the Savior Lutheran Church of Denver for their help in recruiting participants. Correspondence concerning this article should be addressed to Kyunghee Han, Sloan Hall, Department of Psychology, Central Michigan University, Mt. Pleasant, MI 48859. Electronic mail may be sent via the Internet to han1k@cmich.edu. References
Butcher, J. N. (1996). International adaptations of the MMPI-2. Minneapolis, MN: University of Minnesota Press. Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. (1989). Manual for the restandardized Minnesota Multiphasic Personality Inventory: MMPI-2. An administrative and interpretive guide. Minneapolis: University of Minnesota Press. Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., Dahlstrom, W. G., & Kaemmer, B. (2001). MMPI-2 manual for administration, scoring, and interpretation (rev ed.). Minneapolis, MN: University of Minnesota Press. Butcher, J. N., Graham, J. R., Williams, C. L., & Ben-Porath, Y. S. (1990). Development and use of the MMPI-2 content scales. Minneapolis, MN: University of Minnesota Press. Butcher, J. N., & Gur, R. (1974). A Hebrew translation of the MMPI: An assessment of translation adequacy and preliminary validation. Journal of Cross-Cultural Psychology, 1, 185216. Butcher, J. N., & Pancheri, P. (1976). Handbook of cross-national MMPI research. Minneapolis, MN: University of Minnesota Press. Caetano, R., & Clark, C. L. (2002). Acculturation, alcohol consumption, smoking, and drug use among Hispanics. In K. N. Chun, P. B. Organista, & M. G. Marin (Eds.), Acculturation: Advances in theory, measurement, and applied research (pp. 223239). Washington, DC: APA. Cheung, B., Lee, C., & Jin, W. (1963). MMPI manual. Seoul: Korean Testing Center. Cheung, F. M., Song, W. Z., & Butcher, J. N. (1991). An infrequency scale for the Chinese MMPI. Psychological Assessment, 3, 230237. Cheung, F. M., Song, W., & Zhang, J. (1996). The Chinese MMPI-2: Research and applications in Hong Kong and the Peoples Republic of China. In J. N. Butcher (Ed.), International adaptations of the MMPI-2: A handbook of research and clinical applications (pp. 137191). Minneapolis, MN: University of Minnesota Press. Deinard, A. S., Butcher, J. N., Thao, U. D., Vang, S. H. M., & Hang, K. (1996). In J. N. Butcher (Ed.), International adaptations of the MMPI-2 (pp. 194205). Minneapolis, MN: University of Minnesota Press. Dong, Y. T., & Church, A. T. (2003). Cross-cultural equivalence and validity of the Vietnamese MMPI-2: Assessing psychological adjustment of Vietnamese refugees. Psychological Assessment, 15, 370377. Graham, J. R. (2000). MMPI-2: Assessing personality and psychopathology (3rd ed.). New York: Oxford. Han, K. (1990). International applications of the MMPI-2: Translation issues and procedures. In: J. N. Butcher (Chair), Korean Translation of the MMPI-2. Paper presented at the 25th Annual Symposium on Recent Developments in the Use of the MMPI (MMPI-2), Minneapolis, MN.

ARTICLE IN PRESS
J.J. Chung et al. / International Journal of Intercultural Relations 30 (2006) 531543 543 Han, K. (1993). The use of the MMPI-2 in Korea: Inventory adaptation, equivalence evaluation, and initial validation. Unpublished Doctoral Dissertation, University of Minnesota, Minneapolis. Han, K. (1996). The Korean MMPI-2. In J. N. Butcher (Ed.), International adaptations of the MMPI-2: A handbook of research and clinical applications (pp. 88136). Minneapolis, Minnesota: University of Minnesota Press. Han, K. (2005). The MMPI-II: (Instrument). Seoul, Korea: Maumsarang Inc. Han, K., Lim, J., Lee, J., Min, B., & Moon, K. (2005). Validation of the Korean MMPI-2 using the Korean Normative Sample. Paper presented at the 40th Annual Symposium on Recent Developments in the Use of the MMPI-2/MMPI-A, Fort Lauderdale, FL. Kim, Y., Kim, J., Kim, J., Noh, M., Shin, D., Yum, T., & Oh, S. (1989). (MMPI manual). Seoul: Korea Guidance. Kim, Z., Han, K., Lim, J., Lee, J., Min, B., Moon, K., & Lee, J. (2005). The MMPI-II: (User manual). Seoul, Korea: Maumsarang Inc. ` Konraos, S. (1996). The Icelandic translation of the MMPI-2: Adaptation and validation. In J. N. Butcher (Ed.), International adaptations of the MMPI-2 (pp. 368384). Minneapolis, MN: University of Minnesota Press. Leung, R. (1986). MMPI scoring as a function of ethnicity and acculturation: A comparison of Asian, Hispanic, and Caucasian gifted high school students. Dissertation Abstracts International, 47(6B), 2622. Myers, H. F., & Rodriguez, N. (2002). Acculturation and physical health in racial and ethnic minorities. In K. N. Chun, P. B. Organista, & M. G. Marin (Eds.), Acculturation: Advances in theory, measurement, and applied research (pp. 163185). Washington, DC: APA. Organista, P. B., Organista, K. C., & Kurasaki, K. (2002). The relationship between acculturation and ethnic minority mental health. In K. N. Chun, P. B. Organista, & M. G. Marin (Eds.), Acculturation: Advances in theory, measurement, and applied research (pp. 139161). Washington, DC: APA. Sireci, S. G., & Berberoglu, G. (2000). Using bilingual respondents to evaluate translated-adapted items. Applied Measurement in Education, 13, 229248. Tran, B. N. (1996). Vietnamese translation and adaptation of the MMPI-2. In J. N. Butcher (Ed.), International adaptations of the MMPI-2 (pp. 175193). Minneapolis, MN: University of Minnesota Press. Tsai, D. C., & Pike, P. L. (2000). Effects of Acculturation on the MMPI-2: Scores of Asian American Students. Journal of Personality Assessment, 74, 216230. United States Census (2000). http://www.census.gov/population/www/socdemo/race.html US Census Bureau. January 25, 2002. Velasquez, R. I., Chavira, D. A., Karle, H. R., Callahan, W. J., Garcia, J. A., & Castellanos, J. (2000). Assessing bilingual and monolingual Latino students with translations of the MMPI-2: Initial data. Cultural Diversity and Ethnic Minority Psychology, 6, 6572.