Artu Psychiulr Scand IYYY: 100: Y 8 - I 0 1

Pritited

iti

UK. All righls resrrved

-~~ -~ A C TA P S Y C H I A T R I ( A
~

Cupyrrght

Munhjguurd I Y Y Y

SCANDINAVJCA I S S N 0902-4441

Childhood behavioural disorders in Ambo district, western Ethiopia. 11. Validation of the RQC
Tadesse B, Kebede D, Tegegne T, Alem A. Childhood behavioural disorders in Ambo district, western Ethiopia. 11. Validation of the RQC. 0 Munksgaard 1999 Acta Psychiatrica Scand 1999: 100: 98-101. ( We report here on a study conducted to measure the validity of an Amharic version of the Reporting Questionnaire for Children (RQC), that was used in a survey of childhood behavioural disorders in a predominantly rural district in western Ethiopia. Mothers of 196 children aged 5-15 years, who were initially interviewed by the RQC were re-interviewed by a psychiatrist who was unaware of the RQC status of these children. The re-interview was conducted using a DSM IV checklist. The study showed that a cut-off point of one or more positive responses to any of the 10 questions on the RQC maximized sensitivity (87.5%)and specificity (65%). The discriminatory power of each item was also computed, and the item dealing with wetting/soiling oneself was found to have the highest ability to identify cases from non-cases. The item on abnormal speech was found to have the least discriminating power.

6. Tadesse', D. Kebede', T. Tegegne3, A. Alem3

'Oromia Health Bureau, Addis Ababa, *Department of Community Health, Faculty of Medicine, University of Addis Ababa, and 3Amanuel Psychiatric Hospital. Addis Ababa. Ethiopia

Key words childhood mental disorders: self report questionnaire; prevalence; Ethiopia Belayneh Tadesse, Oromia Health Bureau, P.O. Box
22174, Addis Ababa. Ethiopia

Introduction

Major advances in studying mental health in children in developing countries have not been observed despite the fact that children under 15 years of age constitute 40-50% of their population (1). In addition, lack of accepted definitions for child mental disorders has resulted in a divergence of research strategies in child psychiatry (2). In order to address mental health needs through the primary health care system, information is needed on the frequency and types of mental disorders occurring in a locality. In the past, lack of agreed methods of evaluating and differentiating between the varieties of mental health problems has delayed progress in integrating mental health care into the framework of primary health work. However, considerable improvement occurred after a WHO coordinated team developed a system of classification for use by health workers (3). Experts in seven developing countries, under the sponsorship of the World Health Organization, carried out collaborative operational research on providing mental health care through primary health care
98

services. New, internationally reliable techniques identifying mental disorders in children and adults in a primary health care setting or when using surveys have been developed and tested. The techniques usually employ a two-stage process. During the first stage, screening allows a patient to report the presence or absence of clearly defined symptoms. This first-stage screening is designed to identify 'potential cases' that may then be followed up by the much more time-consuming standardized 4 ) . psychiatric interviews to confirm the diagnosis ( One such screening instrument developed by WHO is the Reporting Questionnaire for Children (RQC). It is a 10-item questionnaire for children between the ages of 5 and 15 and is designed to identify moderate to severe mental retardation, significant degrees of emotional or behavioural disorder (those which adversely affect schooling or socialization for more than a limited period of time), and psychotic disorders. The RQC questions were originally selected by first examining the relevant (but limited) literature from studies conducted in Ethiopia and Sudan, and

Validation of the RQC

then discussing the relevance of items in primary care settings. These questions are unambiguous, easy to answer, and related to recognizable behaviours or experiences of the child. A score of a single positive item was generally adequate for screening cases and perhaps for pointing towards a particular disorder (5).Validation data are available from four countries, and the discriminating power of each item was also studied (6, 7). We have reported earlier on a community-based survey of children in rural western Ethiopia using the RQC (8). We report here on a sub-study of that survey, which was designed to measure the validity of the RQC using the ROC curve.

Material and methods

The study was conducted between September 1994 and May 1995 in Ambo district, western Ethiopia. A description of the study area and methods employed in the survey have been detailed elsewhere (8). Briefly, the study was done in Ambo district which is one of the 23 districts of Western Showa administrative zone. The district is located about 120 km to the west of Addis Ababa, the capital of Ethiopia. Ambo district has 136 sub-districts, of which eight are urban sub-districts and the rest are peasant associations. The population of the district is estimated to be 198 461, of which 17.3% live in the urban area and 82.7% live in rural areas. The English version of the RQC was translated into the Amharic language by three psychiatrists independently. Back-translation into English was done by another psychiatrist who did not know the original version. We employed interviewers who had completed high school and who spoke Oromifa (the local language) and Amharic (the national language). Fifteen male and 15 female interviewers were recruited. Two male supervisors were recruited for supervising the interviewers. Supervisors and interviewers were trained for three days. To pretest the questionnaire, 10 parents or caretakers were interviewed from one of the subdistricts in a district not chosen for the main study. Ten households were selected by choosing every Nth household ( N = total households in the sub-district divided by 10). From each household, a parent or caretaker was interviewed. In cases where both father and mother lived together, both the father and the mother were interviewed separately. When a refusal or a household with no child between 5 and 15 years was encountered, the next alternate house number was chosen in the order of +1, -1, +2 or -2 from the original house number chosen. After

assessing the results of the pre-test, the questionnaire was appropriately modified. The population from which the study sample was drawn included all parents or child caretakers and all children between 5 and 15 years living in the accessible sub-districts. The sampling units were the households in the district. Methods of' validation o f the RQC. Every parent or caretaker with at least one positive answers to the RQC items and parents of the next household with negative answer to the RQC were sent to Ambo hospital to be seen by a psychiatrist on any convenient working day. A total of 196 children (77 negative respondents and 119 positive respondents) were seen by the psychiatrist in 10 days. The psychiatrist used a multi-axial checklist from the DSM IV (Diagnostic and Statistical Manual for mental disorders). The psychiatrist did not have prior knowledge of the RQC status of the 196 children. Those children who needed treatment for their psychiatric illness were given appropriate medications. Data processing was done using the EPI-INFO program on a personal computer. Cross-tabulations and ROC curve analysis were done to evaluate the validity of the RQC.

Results

A comparable number of male (47%) and female (53%) respondents were included in the validation sub-study. More boys than girls in the 5-7 year age group were brought to the psychiatrist. In the other age groups, a similar proportion of boys and girls were seen by the psychiatrist. As shown in Table 1, 196 children were examined by the psychiatrist. A positive response to at least one item out of the 10 RQC items was used as a cutoff point to classify children as having behavioural disorders. This mode of classification was compared with the psychiatrist's diagnosis. Of the 119 children detected as cases by the screening instrument, 84 children were confirmed as cases (PPV=70.6%); out of the 77 classified as negative by the RQC, 65 were confirmed as non-cases (NPV=84.4%).
Table 1. Comparison of RQC classification of children with behavioural disorders (using cut-off level of at least one positive response) versus a psychiatrist's diagnosis using a DSM-IV checklist. Ambo district, western Ethiopia, 1995 Psychiatrist's diagnosis Positive RQC classification Positive Negative Total Negative Total

84 12 96

35 65 100

119 77 196

99

Tadesse et al.
Table 2 Diagnostic sensitivity and specificity of RQC a t different cut-off points Cut-off Sensitivity 87 5 51 0 29 1 12 5 10 4 83 31 21 10

Discussion

(YO)

Specificity

(YO)

65 0 85 0 89 0 90 0 97 0 100 0
100 0 100 0 100 0 100 0

Of the 96 children classified as cases by the psychiatrist, 84 were identified as such by the RQC (sensitivity=87.5%). Likewise, the specificity of the screening instrument was 65%. As shown in Table 2, as the cut-off point increases, the specificity of the screening instrument increases at the expense of the sensitivity; the maximum specificity (1OO'Xn) and least sensitivity (8.3%) is found when the cut-off point is set at six or more positive responses. Conversely, the maximum sensitivity (87.5%) and the least specificity (65%) occur when the cut-off is set at one or more positive responses out of the 10 RQC items. As shown in Table 3 , the value of the discriminating power of an item is computed as the percentage of psychiatrically ill subjects with a positive response divided by the percentage of normal children with a positive response. It is a measure of the tendency for a given item to be responded to positively by children with a behavioural disorder and negatively by children without the disorder. Thus, the higher the discriminating power for an item, the better the item is in discriminating cases and non-cases. It is shown that wetting/soiling oneself and not playing with others are the items with highest discriminating power.
Table 3. Distribution of 196 children according to their responses to a 10-item RQC in Ambo district, western Ethiopia, 1994/95 Psvchiatrist diaanosis Positive RQC items 1 Abnormal speech 2 Sleeping badly 3 Fits or falling 4 Frequent headache 5 Running away 6 Stealing things 7 Nervousness 8 Backwardness 9 Not playing with others 10 Wetting/soiling oneself D=positive/negative.

(YO)

Negative

(YO)

D"

10 (10.4) 15 (15.6) 29 (30.2) 44 (45.8) 15 (15.6) 3 (3 1) 26 (27 1) 12 (12 5) 11 (11.5) 33 (34.4)

10 7 6 23

5 0 8 6
2 4

1.o 2.2 5.0 1.9 3.1

-

3.4
2.1

5.8 8.6

We have shown that the RQC cut-off point of at least one positive response out of the 10 items used to identify children with behavioural disorders has an acceptable level of validity. We have followed procedures to ensure that it is unlikely that our results are due to bias. In order to minimize possible bias, during the validation of the RQC all children in the community who had at least one yes answer to the RQC were invited to be interviewed. For every positive case, the next household with a negative child was also invited to be seen by the psychiatrist. Both the investigators and the psychiatrist were unaware of the RQC or the diagnostic status of a child until the end of the study period. Because the psychiatrist used a DSM IV checklist for diagnosis, the possibility of subjectivity bias by the psychiatrist was minimized. Data collection was carried out by persons who were not health care professionals but who were given appropriate training, so as to minimize the introduction of possible bias during data collection. Mothers were the respondents of child mental status in all households so that there was a consistency of respondents at least by sex pattern. Because the items were unambiguous and described recognizable behavioural and developmental problems, it was not necessary that the parent be literate in order to respond. The results of the evaluation of the validity of the RQC are comparable to other studies in developing countries. A WHO collaborative team that completed a study in four developing countries, has reported a cut-off level of at least one positive RQC item as optimal for identifying cases. They also found a sensitivity ranging from 89.7% in Sudan to 100% in the Philippines (5). The instrument was also found to have a specificity of 69.7% in the Philippines and of 62.7% in Colombia (6). The enuresis and 'never play with others' items also ranked highest in Sudan and the Philippines (5). At least one of the items, 'never play with others' also ranked highest in Senegal in discriminating cases from non-cases (7). Thus, our findings are comparable to these other studies. One potentially important limitation of this validation study is the fact that no weighing was done when indices of validity were calculated. This may be important because the proportion of children with a positive RQC included in the validation sample is higher than that of the corresponding proportion of children with negative RQC responses included in the sample. In such instances, this weighing procedure will usually decrease the sensitivity and increase the specificity obtained without weighing.

100

Validation of the RQC

Acknowledgements
The research was conducted as a partial requirement for the Degree of Master of Public Health by Dr. Belayneh Tadesse at Addis Ababa University. Financial and material support for the study was obtained from the International Development Research Centre of Canada (IDRC) and the Department of Community Health (DCH). We thank Professor R. Giel for his advice. All interviewers and supervisors, Ato Tewodros Tamru, Ato Moges Mamo, and Ato Mekonnen Tadesse, are also acknowledged for their hard work in field supervision, and, especially, our deepest thanks go to Ato Tewodros. The Western Showa Zonal Council is also acknowledged for its facilitation of the work in spite of difficult circumstances in the Ambo Woreda during the data collection period. We are grateful to Ato Solomon Berhanu and Ato Wondwossen Bekele for the data entry and data cleaning. Caretakers of children who have responded to our interviews are also gratefully acknowledged. 3. World Health Organization. Child mental health and psychological development Report of World Health Organization Expert Committee. Tech. Rep. Series 61 3. Geneva: WHO, 1977. HARDINC TW, CARLOS MB, CLIMMENT E, et al. The WHO collaborative study on strategies for extending mental health care, 11: The development of a new research method. A m J Psychiatry 1983: 140: 1474-1480. GIEL R , HARDINC TW, TEN HORN G H M M , et al. The detection of childhood mental disorder in primary care in some developing countries. In: H E N D ~ R S O AS, N BURROWS G D , eds. Handbook of social psychiatry. Amsterdam: Elsevier, 1988: 233-244. GIEL R, DEARANCO MV, CLIMENT CE, et al. Childhood mental disorder in primary health care: result of observation in four developing countries. A report from the WHO collaborative study on strategies for extending mental health care. Pediatrics 1981: 68: 677-683. DIOP B, COLLIGNON R, GUEYE M , HARDINC TW. Diagnosis and symptoms of mental disorder in a rural Senegal. Afr J Med Sci 1982: 1 I : 95-103. TADESSE B, KEBEDE D, TEGECNE T, ALEM A. Childhood behavioural disorders in Ambo district, western Ethiopia. I. Prevalence estimates. Acta Psychiatr Scand 1999: 100 (Suppl): 92-97.

References
1. NIKAPOTA AD. Child psychiatry in developing countries.

Br J Psychiatry 1991: 158: 743-751. E. The child behaviour profile TI. 2. ACHENBAC TM, BROCKE J Consult Clin Psychology 1979: 47: 223-233.

101