J Psychopathol Behav Assess (2010) 32:246254 DOI 10.

1007/s10862-009-9141-5

Comparison of Eleven Short Versions of the Symptom Checklist 90-Revised (SCL-90-R) for Use in the Assessment of General Psychopathology
Jrg M. Mller & Christian Postert & Thomas Beyer & Tilman Furniss & Sandra Achtergarde

Published online: 9 June 2009 # Springer Science + Business Media, LLC 2009

Abstract Eleven short versions of the Symptom Checklist (SCL-90-R) assessing general psychopathology, containing 5 to 53 items, were compared on the basis of data from a sample of one hundred mothers of 0-to-6-year-old children referred for treatment at a Child Psychiatric Family Day Hospital in Mnster, Germany. The SCL short versions were compared with regard to internal consistency, sensitivity and specificity, ability to distinguish between subjects by a new test index (PDTS), and association with indicators of validity (SCL-90-R Global Severity Index, BDI scores). All short versions showed almost equally high internal consistency, sensitivity and specificity, and high correlations with validity indices. The PDTS test index describes a good ability of the original SCL-90-R to differentiate between subjects, a moderate performance for the BSI, the HSCL-25 and the SCL-27, and a poor performance of the very short formsaccording to the standards of interpreting PDTS scores. The SCL-10S is recommended for screening purposes because this scale represented the best compromise between economy and accuracy. However, for other research and clinical purposes, the use of one of the longer short versions (BSI, HSCL-25, or SCL-27) is recommended because of their superior discriminative ability. Keywords SCL-90-R . Symptom checklist . Short version . Internal consistency . Validity . PDTS . Screening

The Symptom-Checklist 90-Revised (SCL-90-R; Derogatis 1992; German version: Franke 2002) is a widely used measure designed for screening and assessment of psychopathology, symptom burden and treatment effectiveness (Hardt and Brhler 2007). Internal consistency, retest reliability and validity of the questionnaire have been proven in many studies (Franke 2002). However, the postulated nine-factor-structure of the instrument could not be replicated (Hessel et al. 2001). Instead, one global factor appeared which indicates general symptom stress. This factor may best be represented by the global scores, especially the Global Severity Index (GSI), which is calculated as the mean of all items (Hardt and Brhler 2007). Information about global symptom distress could be obtained with a much more economical instrument than the SCL-90-R (Hessel et al. 2001). In order to assess symptom burden more economically, several short versions of the SCL-90-R have been developed during the last decades. These short versions comprise between 5 (SCL-5; Strand et al. 2003) and 53 items (Brief Symptom Inventory; Derogatis 1993).

Short Versions of the SCL: Characteristics and Development In the following, we will describe the characteristic features and the development of eleven selected short versions in more detail: the HSCL-25 (Derogatis et al. 1974), the BSI (Derogatis 1993), the SCL-10N1 (Nguyen et al. 1983), the
1 In this paper, the letters N and S were added to the name SCL10 to distinguish the scales by Nguyen et al. (1983) and Strand et al. (2003). The letter R in SCL-10R (Rosen et al. 2000) means revised.

J. M. Mller (*) : C. Postert : T. Beyer : T. Furniss : S. Achtergarde Department of Child and Adolescent Psychiatry, University Hospital Mnster, Schmeddingstrasse 50, 48149 Mnster, Germany e-mail: joergmichael.mueller@ukmuenster.de

J Psychopathol Behav Assess (2010) 32:246254

247

BSI-18 (Derogatis 2000), the SCL-10R and the SCL-6 (Rosen et al. 2000), the SCL-K-9 (Klaghofer and Brhler 2001), the SCL-27 (Hardt and Gerbershagen 2001), the SCL-5 and the SCL-10S (Strand et al. 2003), and the SCL11 (Lutz et al. 2006). Note that some SCL-90-R short versions were derived from already existing short versions or precursor versions of the SCL-90-R. In fact, the development of the SCL-90-R began as early as the 1950s (Schauenburg and Strack 1999). Specifically, the SCL-10N, the SCL-10R and the SCL-6 were developed on the basis of the SCL-90 (Derogatis et al. 1973), which is the direct precursor of the SCL-90-R with almost identical items. Originally, the SCL-90 was derived from the Hopkins Symptom Checklist58 (Derogatis et al. 1974), which is also the origin of the Hopkins Symptom Checklist-25 (HSCL-25, Derogatis et al. 1974; Hesbacher et al. 1980). Therefore, it is justifiable to regard all of these scalesthe HSCL-25, the SCL-10N, the SCL-10R and the SCL-6as short versions of the SCL-90R (Hardt and Brhler 2007). The short versions to be investigated in this study will be introduced in order of their year of publication. We use the term scales when we refer to the dimensions of the questionnaires which were originally intended by the test authors. We use the term factor when we refer to empirically derived dimensions, which rely on a factor analysis. The factors are not necessarily identical to the scales, and therefore may be labelled somewhat differently. HSCL-25 The Hopkins Symptom Checklist-25 (HSCL-25; Derogatis et al. 1974; Hesbacher et al. 1980) covers symptoms from the Anxiety, Depression and Somatic dimension (Hardt and Brhler 2007). Cut-off-scores for screening purposes were devised by several authors (see e.g. Winokur et al. 1984). The scale shows satisfactory to good internal consistency, interrater reliability, test-retest reliability, and validity (Lee et al. 2008). SCL-10N The SCL-10N was created as a short form of the SCL-90 (Nguyen et al. 1983). Ten items were selected for their factor loadings on the three most important factors identifiable in a psychiatric population; Depression, Somatization and Phobic Anxiety (Hoffmann and Overall 1978). BSI The Brief Symptom Inventory (BSI; Derogatis 1993) was designed as a short, but multi-dimensional questionnaire with the nine original dimensions of the SCL-90-R (Somatization, Obsessive-Compulsive Behaviour, Interpersonal Sensitivity, Depression, Anxiety, Hostility, Phobic Anxiety, Paranoid Ideation, and Psychoticism). Containing 53 items, the BSI is more comprehensive than most other short versions. Internal consistency and test-retestreliability coefficients are satisfactory to good and several

studies favor a one-factor-solution (Benishek et al. 1998; Loutsiou-Ladd et al. 2008). BSI-18 The Brief Symptoms Inventory-18 (BSI-18; Derogatis 2000) was developed on the basis of the BSI (Derogatis 1993) and the SCL-90-R (Derogatis 1992), in order to create a leaner self-report measure of psychological distress. In contrast to the BSI, which includes 53 items on nine dimensions, the BSI-18 includes 18 items on three dimensions: Depression, Anxiety, and Somatization (Recklitis et al. 2006). However, similar to other short versions, there is empirical evidence for unidimensionality of the BSI-18 (Asner-Self et al. 2006; Prelow et al. 2005). Reliability and validity of the questionnaire were demonstrated in several studies (Prelow et al. 2005). SCL-10R In their review of studies on the factorial structure of the SCL-90, Rosen et al. (2000) came to the conclusion that there was always a predominant first factor related to general distress, and sometimes a secondary factor which was related to the Somatization, Social insecurity, Hostility and Paranoid thinking scales. On the basis of these findings, Rosen and colleagues developed the SCL-10R as an alternative to the SCL-10N. In contrast to the SCL-10N, the SCL-10R includes items from all nine of the original SCL-90 subscales, in order to more broadly represent both the primary and the secondary factor. Six items representing the primary factor were selected (a) on the basis of factor loadings which were obtained in previous studies, and (b) with the aim of including as many of the original SCL-90 subscales as possible. Additionally, four items representing the secondary factor were selected on the basis of factor loadings which were obtained in previous studies. These items belonged to the original SCL-90 Somatization, Avoidance, Hostility and Paranoia subscale. SCL-6 In addition to the SCL-10R, Rosen et al. (2000) created the SCL-6 as a short index which was aimed at representing only the primary distress factor of the SCL. For this purpose, the authors selected two items each from the SCL-90 Depression, Anxiety, and Psychoticism subscales on the basis of (a) the number of studies in which they loaded on the primary factor and (b) their average factor loading on the primary factor. The resulting 6-itemscale is considered unidimensional (Hardt and Brhler 2007). SCL-K-9 The SCL-K-9 by Klaghofer and Brhler (2001) includes items from all of the nine original subscales of the SCL-90-R. The authors chose one item out of each subscale which correlated most highly with the GSI, in order to achieve concordance with the SCL-90-R. In this way, they created a broad, but unidimensional index including

248

J Psychopathol Behav Assess (2010) 32:246254

different symptom categories with high internal consistency (=.87) and a strong association to the original questionnaire (r=.93). SCL-27 The SCL-27 by Hardt and Gerbershagen (2001) was designed as a short screening instrument for psychopathology in chronic pain patients. The authors selected 27 items out of the SCL-90-R which represent six dimensions specified on the basis of an exploratory factor analysis. Items were selected according to high convergent and low discriminant correlations. Additionally, the authors excluded all items which are contained in the SCL-K-9 (Klaghofer and Brhler 2001), in order to retain the possibility of assessing patients independently with both measures (Hardt and Gerbershagen 2001). The six dimensions cover despressive, dysthymic, vegetative, agoraphobic, mistrust and social phobia symptoms. The authors report high reliability of the subscales as well as a high correlation with the GSI of the 90-item-version. Cut-off-points for screening purposes are delivered (Hardt and Gerbershagen 2001). SCL-5 and SCL-10S The SCL-5 and the SCL-10S (Strand et al. 2003) were developed for the conduction of population surveys in Norway (Tambs and Moum 1993). The scales were not directly derived from the SCL-90-R, but from the HSCL-25. The items of the SCL-5 and SCL10S were selected on the basis of high correlation with the HSCL-25 global score. SCL-11 The SCL-11 was developed by Lutz et al. (2006) with the intention of creating a short, but multidimensional and change-sensitive outcome measure for the evaluation of therapeutic progress in psychotherapy and psychiatry. The authors chose a stepwise item selection procedure on the basis of the BSI (Derogatis 1993). In several steps, 11 items were selected from the BSI according to content and convergent validity (correlation with BDI and SCL-90-R scales), with regard to content validity (depression and anxiety according to DSM-IV and ICD-10-criteria), retestreliability, and change-sensitivity. Research on the SCL-90-R short versions has generally shown high correlations between the original version and the abbreviated versions. This is particularly relevant as some of the short versions include items from the SCL-90, the precursor of the SCL-90-R. Items from the SCL-90 may differ slightly in wording from SCL-90-R items. However, the high correlations suggest that these differences in wording are of minor importance and do not limit the comparability of the shortened versions. All items included in the aforementioned shortened versions belong to certain scales from the original SCL-90-R (Table 1), but not all scales from the original SCL-90-R were

included in all short forms. Importantly, items from the original Depression and the Anxiety scale were considered in all shortened forms. The Somatization scale was considered in all abbreviated versions except the SCL-5, the SCL-6 and the SCL-11. Items from the Phobic fear scale were included in the SCL-10N, the BSI, the SCL-10R, the SCL-K-9 and the SCL-27. Items of the Obsessive-compulsive scale were included in the BSI, the SCL-10R, the SCL-K and the SCL27. The Psychoticism scale was considered in the SCL-10N, the BSI, the SCL-10-R, the SCL-6 and the SCL-K-9. Social insecurity and Paranoid thinking were represented in the BSI, the SCL-10R, the SCL-K-9 and the SCL-27. Hostility items were included only in the BSI, the SCL-10R and the SCL-K-9. Consequently, although the short versions differ in their specific content, they all include some items from the original Depression and Anxiety scales of the SCL-90-R. This characteristic justifies the use of a depression measure, for example the BDI, as a validation criterion for the short versions.

Objectives of the Present Paper In the previous section we have shown that the abbreviated versions of the Symptom Checklist were developed with different purposes in mind and on the basis of different procedures. Therefore we do not know which of the scales performs best in a clinical population. Although there is a plentitude of literature on the psychometric quality of certain short versions, to our knowledge there has to date been no study which has compared all available short versions of the SCL-90-R simultaneously. Moreover, in this study the scales will be compared by using a newly developed test index PDTS2 (Probability of Distinct Test Scores; Mller 2006a), which is especially useful in describing differences between short questionnaires. The descriptive test index PDTS is defined as the ratio of the number of statistically different test scores, to the total number of all test score comparisons, and reports the probability of obtaining statistically different test scores (PDTS). The index is sensitive to skewed score distribution (more frequently observed for short tests) and to item number, both of which lower the chances of distinguishing between test scores (Mller 2006a). The index reflects the practical limitations which result from the shortening of questionnaires better than traditional test indices do. The PDTS also works well with small sample sizes, which has been demonstrated in the simulation study of Mller
2 The index is fully described by a subscript for the underlying test model, which is here the classical test theory and abbreviated with PDTSCTT.

J Psychopathol Behav Assess (2010) 32:246254 Table 1 Scales of the short forms of the SCL-90-R SCL-90-R scales HSCL25 X X X SCL10N X X X X BSI BSI18 X X X SCL10R X X X X X X X X X SCL6 X X SCL-K9 X X X X X X X X X X X SCL27 X X X X X SCL5 X X SCL10S X X X

249

SCL11 X X

Depression Anxiety Somatization Phobic fear Obsessivecompulsive Psychoticism Social insecurity Paranoid thinking Hostility

X X X X X X X X X

(2006a). Sample sizes of n>50 show satisfactory psychometric properties (standard error is below 1% of a PDTS score). A further advantage is that the index provides an easy-to-understand summary for a test user who may not be familiar with psychometric theory. The traditional psychometric criteria (internal consistency, validity) are also considered, but they cannot appropriately reflect the diagnostic limitations of very short tests. Finally, it would appear opportune to define what a good PDTS score is and thus broaden the technical standards proposed in Mller (2006a). We propose five categories to label a PDTS score, from very poor to excellent. A very poor PDTS is below 30%, poor between 30% and 45%, moderately for 45% up to 60%; good for 60% up to 75%; very good for 75% up to 90% and higher values than 90% as excellent. Nevertheless, it should be noted that an acceptable lower boundary depends on the diagnostic question posed. In this study, our aim is to evaluate the eleven short versions of the SCL-90-R with regard to their psychometric qualities for the assessment of general psychopathology/emotional distress. The short versions will be compared on the basis of data from mothers of young children who were referred for treatment at an infant and preschool child psychiatric family day hospital. We have chosen this specific population because mothers of young children with mental health symptoms tend to exhibit various psychological and psychosomatic health symptoms (von Hofacker and Papousek 1998) which are indicative of general pathology. Our evaluative questions are as follows: & & Do the short versions show acceptable internal consistency when applied in our sample (homogeneity)? Do the short versions perform in a comparable way related to their ability to discriminate between subjects, who show clinical versus non-clinical levels

& & &

of distress, by their test scores (new test criterion PDTS)? Do the short versions correlate in a convergent way with the original SCL-90-R score (internal validity)? Do the short versions correlate with the BDI, a standardized questionnaire assessing the degree of depression (concurrent validity)? Do the short versions classify subjects as distressed or not distressed as well as the SCL-90-R (sensitivity, specificity)?

On the basis of these questions, we will try to evaluate whether the short versions of the SCL-90-R reach a psychometric quality comparable to the original version. In summarizing, we will suggest which of the scales appear particularly suitable for application in screening and research contexts.

Method Procedure This study is part of a broader study of children and their families who attended the Preschool Child Psychiatric Family Day Hospital Mnster between 2002 and 2007. Every child was accompanied by a parent, in most cases (98.5%) the mother. The accompanying parents underwent a comprehensive psychiatric assessment on admission, as parental psychopathology may be an important contributing or sustaining factor of child mental health symptoms (Ramchandani et al. 2005). The self-rating questionnaires SCL-90-R and BDI were completed by accompanying parents as part of the standard assessment procedure. Additional clinical diagnostic assessment of parents was conducted by a consultant psychiatrist.

250

J Psychopathol Behav Assess (2010) 32:246254

During the period of data collection, 2002 through 2007, a total of 123 parents were admitted to the Family Day Hospital and took part in the standard assessment procedure. 20 mothers and one father did not complete the SCL-90-R. We did not include two completed questionnaires from fathers to further sample homogeneity. There were no additional exclusion criteria. Finally, n =100 mothers were included in the analysis. Mothers who participated in the study completed the full length version of the SCL-90-R. The short versions which were selected for comparison were subsequently compiled from the total of 90 items. Sample Demographic Information The mean age of the mothers was M=33.12 years (SD= 5.73) with a range between 19.0 and 46.7 years. Information on education was available in n=70 mothers. Of those, 2.9% had no certificate of secondary level educational attainment, 30.0% had a secondary general school certificate (9 years of formal education), 38.6% had a secondary school level I certificate (10 years of formal education), 4.3% had an advanced technical college entrance qualification (12 years of formal education) and 24.3% had a general university entrance qualification (13 years of education). Nationality was known in n=92 mothers. Of those, 98.9% were from Germany and 1.1% from other European countries. Measures SCL-90-R The well-established self-report inventory by Derogatis (1992) consists of 90 items which measure psychological and psychosomatic symptoms with a time frame of one week. Subjects rate each item on a 5-point Likert scale (0 = no problem to 4 = very serious). For this study only the global score GSI was computed. BDI The Beck Depression Inventory (Beck et al. 1988; German version: Hautzinger et al. 1995) assesses the severity of depression in adult subjects and consists of 21 groups of items. Items are selected following DSM diagnostic criteria for major depression. Subjects are asked to choose the one item out of a group which describes best how they have felt during the last seven days. Reliability and validity of the instrument have been proven in numerous studies (review of the German version see Richter et al. 1998). In this study the BDI was chosen as a criterion for comparison of the SCL-90-R short versions because all short versions include items from the original Depression scale of the SCL-90-R.

Statistical Analyses Cronbachs alpha was estimated to evaluate the internal consistency of each scale. A second aspect of test equivalence refers to change in dimensionality assessed by the correlation between the sum score of short versions and the GSI of the complete questionnaire. In order to create an internal validity criterion, mothers were classified as distressed or not distressed on the basis of the published SCL-90-R cut-off score (GSI .57; Schauenburg and Strack 1998). This classification was used as the standard against which the classifications of the short versions were compared with regard to their sensitivity, specificity, and positive and negative predictive value. Sensitivity (SE) refers to a tests ability to produce a positive screening result for subjects who actually have the tested condition, while specificity (SP) refers to a tests ability to produce negative results for subjects who do not have the condition. Therefore, SE and SP are quantitative measures of screening accuracy. The positive predictive value (PPV) informs about the proportion of correct results among all positive screening results, while the negative predictive value (NPV) quantifies the proportion of correct results among all negative screening results. Therefore, PPV and NPV reflect the efficiency of screening. In order to check for deviation from the known association of GSI to BDI, all short versions were analyzed to establish their correspondence with the BDI. All statistical analyses were conducted with the Statistical Package for the Social Sciences (SPSS) 15.0. The PDTSCTT was calculated by a SAS program (Mller 2006b). Data were complete in all analyses which dealt exclusively with the SCL-90-R and any of the short forms (n=100). BDI data were available in n=94 cases (94%).

Results Descriptive Results In Table 2, the number of items per scale, mothers mean scores and standard deviation on the SCL-90-R and on the short scales are reported. Results show that mothers scored highly on the SCL-90-R (M=71.12, SD=43.47). This corresponds to a GSI of .76, which exceeds the published clinical cut-off score (GSI=.57). Accordingly, when applying the GSI score as criterion, a substantial percentage of mothers (63%) were identified as severely distressed. Internal Consistency The internal consistency of the complete SCL-90-R was high (Cronbachs alpha = .96). Cronbachs alpha scores of

J Psychopathol Behav Assess (2010) 32:246254

251

the short versions were, as expected, in a satisfactory range between .77 and .94, with higher internal consistency in the longer scales. Consequently, according to Cronbachs alpha, there are no major concerns about the psychometric properties for the short forms. The BSI with 53 items showed the highest internal consistency among the short scales (Cronbachs alpha = .94). Probability of Obtaining Two Statistically Different Test Scores In contrast to internal consistency, scales differed remarkably with regard to their probability of achieving two statistically different test scores (PDTS; Table 2). The PDTSCTT scores of the short scales ranged between 34.89% (SCL-6) and 62.69% (BSI). The complete SCL-90-R performs good in distinguishing test results with a PDTSCTT of 69.70%. This means that in approximately 70% of comparisons of two arbitrarily selected subjects, the SCL-90-R detects a significant difference. This quality is diminished when the SCl-90-R is shortened. Association with Validity Criteria Shortening a long scale also affects its internal validity, here represented by the correlation with the GSI, and its concurrent validity, here represented by the correlation with the BDI. All short versions correlated strongly and significantly with the GSI of the SCL-90-R (see Table 2). Correlations ranged between .83 (SCL-5) and

.98 (BSI) and do not suggest a decimation of validity. Accordingly, all short versions correlated strongly and significantly with the BDI sum score, however not as strongly as with the GSI. Correlations ranged from r=.58 (SCL-5) up to r=.72 (BSI). Sensitivity, Specificity, Positive and Negative Predictive Value In order to evaluate the screening accuracy and effectiveness of the diverse SCL short forms, we have considered test sensitivity (SE), specificity (SP), positive (PPV) and negative predictive value (NPV; see Table 3). The published cut-off score of the SCL-90-R (GSI .57) served as the criterion for clinical distress. According to the high base rate of severely distressed persons in our sample (63%; n= 37 mothers obtained a GSI score below .57), we chose the 37th percentile as the theoretically optimal cut-off score for all short scales. As can be seen in Table 3, the selected cutoff score resulted in good sensitivity, specificity, positive and negative predictive values for all short scales. Most scores exceeded .90 (SE, PPV) or .80 (SP, NPV), respectively, with the exception of the SCL-5, which showed slightly lower accuracy and effectiveness values. Again, the best values were obtained by the two longest scales, SCL-27 and BSI, with a sensitivity index above 98%, a specificity index above .90, and comparably good effectiveness scores (PPV > .98; NPV > .94). Yet the SCL11, the SCL-10N and the SCL-10S also obtained good values despite their brevity (see Table 3).

Table 2 Basic descriptive indices, internal consistencies, and PDTS scores of the short versions, and correlations with the original SCL-90-R and with the BDI Versionsa Items Mean SD (n=100) Cronbachs alpha SCL-90-R GSI BSI SCL-27 HSCL-25 BSI-18 SCL-11 SCL-10N SCL-10R SCL-10S SCL-K-9 SCL-6 SCL-5
a b

PDTSCTTb

SCL-90-R GSI Correlations (n=94) rtc

BDI

in % .96 .94 .88 .90 .86 .82 .77 .78 .80 .80 .77 .79 69.70 62.69 48.51 53.10 43.06 37.91 35.60 39.49 41.04 39.88 34.89 37.04

rtc .75** .72** .71** .70** .66** .65** .68** .66** .69** .66** .61** .58**

90 53 27 25 18 11 10 10 10 9 6 5

71.12 40.79 18.67 23.98 12.62 9.88 7.84 9.38 9.96 10.70 5.40 5.85

43.47 27.43 13.49 15.03 10.13 7.63 6.36 6.65 6.97 6.90 2.84 4.35

.98** .95** .95** .93** .92** .91** .91** .94** .90** .87** .83**

For reference of short versions see text

CTT

Probability of obtaining two statistically different test scores (Mller 2006a). The subscript which is here the classical test theory

indicates the underlying test theoretical model,

252 Table 3 Sensitivity (SE), specificity (SP), positive (PPV) and negative predictive value (NPV) of the short versions of the Symptom Checklist (SCL-90-R) Short versionsb (n=100) BSI SCL-27 HSCL-25 BSI 18 SCL-11 SCL-10N SCL-10R SCL-10S SCL-K-9 SCL-6 SCL-5 Cut-off 23.00 10.00 14.00 6.33 5.00 4.00 5.00 5.33 7.00 2.00 3.00 SE

J Psychopathol Behav Assess (2010) 32:246254 SP 91.89 94.59 86.49 83.78 89.19 94.59 81.08 83.78 83.78 89.19 72.97 PPV 98.41 98.39 93.55 96.55 93.55 96.72 91.38 96.67 90.48 93.44 91.84 NPV 97.30 94.74 86.84 83.33 86.84 89.78 76.19 87.50 83.78 84.62 64.71

Sensitivity (SE), specificity (SP), positive (PPV) and negative predictive value (NPV) were calculated for the theoretically optimal cut-off score (c= 37th percentile); caseness criterion: SCL-90-GSI .57

b For references of the short versions see text

98.41 98.41 95.24 96.83 98.41 93.65 92.06 96.83 90.48 90.48 85.71

Discussion In the present paper, eleven short versions of the Symptom Checklist-90-Revised (SCL-90-R) were compared with regard to psychometric properties on the basis of a clinical sample; in order to help test users to select the most efficient and informative short form of the SCL-90-R. Among the evaluation criteria were internal consistency, a recently proposed test criterion PDTS, validity indices, and screening accuracy and efficiency. As expected, the internal consistencies of all short versions appear satisfactory ( between .77 and .96), and thus for most test users there would be no limitations in test use. Furthermore, limitations by the number of possible test results (which are limited by very short tests) and test score distribution are not displayed by internal consistencies. Therefore the internal consistencies cannot indicate clearly which short forms should be preferred. For such comparison, the PDTS score was suggested (Mller 2006a). This index describes the ability of a test to separate subjects by their test results. The PDTS is based in this study on the classical test theory and is therefore indicated by PDTSCTT. The PDTSCTT scores show a much broader range from 34.89% up to 69.70%. With this score, differences between the short forms are also numerically displayed. Differences greater than 2% are considered to be significant according to the standard error of the PDTSCTT, which is below 1% for even smaller samples (Mller 2006a). Moreover, we have defined ranges on the PDTS scale from very poor to excellent in order to help interpretation of PDTS scores. As expected, the original SCL-90-R performed best in separating subjects by their test score (PDTSCTT =69.70%), which is, expressed in standardized terms, a good performance. Practically, this means that any two persons who were assessed with the SCL-90R would have a 70% chance of differing significantly in their test scores. The BSI, the HSCL-25 and the SCL-27 (with a PDTSCTT of 62.69%, 53.10% and 48.51%, respectively) are in a

moderate range, and performed numerically clearly below the original SCL-90-R. The very short-forms show a PDTSCTT in a poor range with scores between 34.89% and 43.06%. A different aspect of test quality concerns validity. We have shown that all short versions correlated strongly and significantly with the GSI of the SCL-90R. The highest correlation with the GSI was achieved by the BSI (r=.98), while the lowest correlations with the GSI were found in the very short scales SCL-5 (r=.83) and SCL-6 (r=.87). Obviously, even these correlations are remarkably high, especially in view of the brevity of the two scales. These findings strongly imply that all short versions do indeed embody the main contents of the fulllength SCL-90-R. Similar results were found for the correlation of the short versions with the BDI. We found moderate to strong correlations between the BDI and all short scales. Again, the BSI showed the highest correlation and the shortest versions showed the lowest, which were still about r=.60. These results underline the good validity of the SCL short scales as measures of psychological distress, with a focus on depressive symptoms which are the shared content basis of all SCL-90-R short versions. Finally, the accuracy and effectiveness of the short scales as screening instruments were examined, considering sensitivity, specificity, and positive and negative predictive value. In order to conduct an adequate comparison, we used a cut-off score at the 37th percentile, which is justified by a base rate of 63% cases. Results showed good screening accuracy and effectiveness for all scales, except the very short SCL-5. In accordance with our previous results, the best sensitivity, specificity, and positive and negative predictive values were obtained in the longest scales from all short versions (BSI, SCl-27). Note that of all versions the full SCL-90-R shows the highest psychometric quality and should be applied whenever there is sufficient time.

J Psychopathol Behav Assess (2010) 32:246254

253

Conclusion We see two possible areas of application for the more economic short forms: as screening instruments in order to build the groups distressed and non-distressed, and as more dimensional instruments for assessing distress. The latter scenario may occur more often in research studies, where the best possible compromise between economic assessment and tapping the full potential of information is sought. For both scenarios, we have different suggestions. In the context of screening (screening instruments here are defined as questionnaires not longer than 15 items) the performance of the SCL-10S was slightly superior over all indices, and the questionnaire showed the best compromise between economy and accuracy. However, it depends on the diagnostic question posed whether the type I or the type II error should be minimized, and the decision for a specific questionnaire should therefore be based on the sensitivity or the specificity, respectively (see Table 3). Note that all questionnaires with less than 15 items performed poorly according to the PDTSCTT criterion, which indicates that short questionnaires should only be applied for dichotomous decisions and not as a continuous measure. For research purposes, the PDTSCTT criterion uncovers the more subtle quality differences between the short scales. This is important when the objective of assessment is to depict differences or slight changes in symptom severity (e. g. in the evaluation of treatment outcome). Considering the PDTSCTT scores obtained in this study, the SCL-90-R should not be shortened more than was done in the BSI (with a good PDTSCTT) with 53 items, the HSCL-25 (moderate PDTSCTT) or the SCL-27 (moderate PDTSCTT). In this study, we used the BDI as validity criterion, as depressive symptoms are very common and a central aspect of general psychopathology. In order to broaden the generalizability of the results to other aspects of general psychopathology, the external validity of the short versions recommended here should be examined in further research. This research should rely on additional validity criteria, such as the general psychopathology index of the clinical scales of the MMPI (Butcher et al. 1989), the Basic Personality Inventory (Jackson 1989), the Depression Anxiety Stress Scales (Lovibond and Lovibond 1995), or the Mental Health Questionnaire (Ware and Sherbourne 1992). Finally, note that the conclusions drawn from this study are limited to the specific and relatively small population of mothers who were examined here. As all participants were female, predominately German, and exclusively European, the results may not be generalized to males and to people from other nations, ethnic or racial groupings. The applicability and psychometric properties of the short

versions in other clinical and non-clinical samples still have to be examined in further studies with larger and more heterogeneous samples.
Expertise Statistical methods; Analyses of questionnaires; Research consultant Projects Funding of the project Development, Application and Meta-Analysis of test index of psychological assessment

References
Asner-Self, K. K., Schreiber, J. B., & Marotta, S. A. (2006). A crosscultural analysis of the brief symptom inventory-18. Cultural Diversity and Ethnic Minority Psychology, 12, 367375. Beck, A. T., Steer, R. A., & Garbin, M. G. (1988). Psychometric properties of the Beck depression inventory: twenty-five years of evaluation. Clinical Psychology Review, 8, 77100. Benishek, L. A., Hayes, C. M., Bieschke, K. J., & Stoffelmayr, B. E. (1998). Exploratory and confirmatory factor analyses of the brief symptom inventory among substance abusers. Journal of Substance Abuse, 10, 103114. Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. (1989). Minnesota multiphasic personality inventory (MMPI-2). Manual for administration and scoring. Minneapolis: University of Minneapolis Press. Derogatis, L. R. (1992). SCL-90-R, administration, scoring & procedures manual-II for the R(evised) version and other instruments of the psychopathology rating scale series. Townson: Clinical Psychometric Research, Inc. Derogatis, L. R. (1993). Brief symptom inventory (BSI): Administration, scoring, and procedures manual (3rd ed.). Minneapolis, MN: National Computer Systems. Derogatis, L. R. (2000). The brief symptom inventory-18 (BSI-18): Administration, scoring and procedures manual. Minneapolis, MN: National Computer Systems. Derogatis, L. R., Lipman, R. S., & Covi, L. (1973). SCL-90: an outpatient psychiatric rating scalepreliminary report. Psychopharmacology Bulletin, 9, 1328. Derogatis, L. R., Lipman, R. S., Rickels, K., Uhlenhuth, E. H., & Covi, L. (1974). The Hopkins Symptom Checklist (HSCL): a self-report symptom inventory. Behavior Sciences, 19, 115. Franke, G. H. (2002). Die Symptom-Checkliste von Derogatis SCL-90R. (2. Aufl.). Gttingen: Beltz. Hardt, J., & Brhler, E. (2007). Symptom checklists in patients with chronic pain [German]. Schmerz, 21, 714. Hardt, J., & Gerbershagen, H. U. (2001). Cross-validation of the SCL27: a short psychometric screening instrument for chronic pain patients. European Journal of Pain, 5, 187197. Hautzinger, M., Bailer, M., Worall, H., & Keller, F. (1995). Beckdepressions-inventar (BDI). Testhandbuch. Bern: Hans Huber. Hesbacher, P. T., Rickels, K., Morris, R. J., Newman, H., & Rosenfeld, H. (1980). Psychiatric illness in family practice. Journal of Clinical Psychiatry, 41, 610. Hessel, A., Schumacher, J., Geyer, M., & Brhler, E. (2001). Symptom-Checkliste SCL-90-R: Testtheoretische berprfung und Normierung an einer bevlkerungsreprsentativen Stichprobe. Diagnostica, 47, 2739. Hoffmann, N. G., & Overall, P. B. (1978). Factor structure of the SCL-90 in a psychatric population. Journal of Consulting and Clinical Psychology, 46, 11871191.

254 Jackson, D. N. (1989). Basic personality inventory. Port Huron, MI: Research Psychologists. Klaghofer, R., & Brhler, E. (2001). Konstruktion und Teststatistische Prfung einer Kurzform der SCL-90-R. Zeitschrift fr Klinische Psychologie, Psychiatrie und Psychotherapie, 49, 115124. Lee, B., Kaaya, S. F., Mbwambo, J. K., Smith-Fawzi, M. C., & Leshabari, M. C. (2008). Detecting depressive disorder with the Hopkins Symptom Checklist-25 in Tanzania. International Journal of Social Psychiatry, 54, 720. Loutsiou-Ladd, A., Panayiotou, G., & Kokkinos, C. M. (2008). A review of the factorial structure of the brief symptom inventory (BSI): Greek evidence. International Journal of Testing, 8, 90110. Lovibond, S. H., & Lovibond, P. F. (1995). Manual for the depression anxiety stress scales (DASS). Sydney: Psychology Foundation. Lutz, W., Tholen, S., Schrch, E., & Berking, M. (2006). Die Entwicklung, Validierung und Reliabilitt von Kurzformen gngiger psychometrischer Instrumente zur Evaluation des therapeutischen Fortschritts in Psychotherapie und Psychiatrie. Diagnostica, 52, 1125. Mller, J. M. (2006a). The probability of obtaining two statistically different test scores as a test index. Educational and Psychological Measurement, 66, 601611. Mller, J. M. (2006b). SAS macros to compute the descriptive test index PDTS. Applied Psychological Measurement, 30, 345 346. Nguyen, T. D., Attkisson, C. C., & Stegner, B. L. (1983). Assessment of patient satisfaction: development and refinement of a service evaluation questionnaire. Evaluation and Program Planning, 6, 299314. Prelow, H. M., Weaver, S. R., Swenson, R. R., & Bowman, M. A. (2005). Preliminary investigation of the validity and reliability of the brief-symptom inventory-18 in economically disadvantaged Latina American mothers. Journal of Community Psychology, 33, 139155. Ramchandani, P., Stein, A., Evans, J., & OConnor, T. G. (2005). Paternal depression in the postnatal period and child development: a prospective population study. The Lancet, 365, 22012205.

J Psychopathol Behav Assess (2010) 32:246254 Recklitis, C. J., Parsons, D. K., Shih, M.-C., Mertens, A., Robison, L. L., & Zeltzer, L. (2006). Factor structure of the brief symptom Inventory-18 in adult survivors of childhood cancer: results from the childhood cancer survivor study. Psychological Assessment, 18, 2232. Richter, P., Werner, J., Heerlein, A., Kraus, A., & Sauer, H. (1998). On the validity of the Beck depression inventory. Psychopathology, 31, 160168. Rosen, C. S., Drescher, K. D., Moos, R. H., Finney, J. W., Murphy, R. T., & Gusman, F. (2000). Six- and ten-item indexes of psychological distress based on the Symptom-Checklist 90. Assessment, 7, 103111. Schauenburg, H., & Strack, M. (1998). Die Symptom-Checklist-90-R zur Darstellung von statistischen und klinisch signifikanten psychotherapie-ergebnissen. Psychotherapie, Psychosomatik, Medizinische Psychologie, 48, 257264. Schauenburg, H., & Strack, M. (1999). Measuring psychotherapeutic change with the Symptom Checklist-90-R. Psychotherapy and Psychosomatics, 68, 199206. Strand, B. H., Dalgard, O. S., Tambs, K., & Rognerund, M. (2003). Measuring the mental health status of the Norwegian population: a comparison of the instruments SCL-25, SCL-10, SCL-5 and MHI-5 (SF-36). Nordic Journal of Psychiatry, 57, 113118. Tambs, K., & Moum, T. (1993). How well can a few questionnaire items indicate anxiety and depression? Acta Psychiatrica Scandinavica, 87, 364367. Von Hofacker, N., & Papousek, M. (1998). Disorders of excessive crying, feeding, and sleeping: the Munich interdisciplinary research and intervention program. Infant Mental Health Journal, 19, 180201. Ware, J. E., Jr., & Sherbourne, C. D. (1992). The MOS 36-item shortform health survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30, 473483. Winokur, A., Winokur, D. F., Rickels, K., & Cox, D. S. (1984). Symptoms of emotional distress in a family planning service: stability over a four-week period. British Journal of Psychiatry, 144, 395399.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.