R

E S E A R C H

E P O R T

Are Empirically Supported Treatments

Valid for Ethnic Minorities? Toward An
Alternative Approach for
Treatment Research

GUILLERMO BERNAL
MARA R. SCHARRON-DEL-RO
University of Puerto Rico

The psychological community has given considerable attention to the problem of establishing empirically supported treatments (ESTs). The authors argue that a scientific
practice that discriminates against some approaches to knowledge undermines the ESTs
relevance for communities of color. They examine the EST projects contribution to
knowledge of effective treatments for ethnic minorities by considering both how knowledge is constructed and the limits of research (e.g., external validity). Alternatives on
how to best contribute to treatment research of clinical utility with diverse populations
are articulated. An approach for treatment research, derived from an integration of the
hypothesis-testing and discovery-oriented research approaches, is presented, and recommendations to advance treatment research with ethnic minority communities are offered.
empirically supported treatments ethnic minorities psychotherapy treatment
research

Guillermo Bernal and Mara R. Scharron-del-Ro, Department of Psychology, University of Puerto

Rico, San Juan, Puerto Rico.
At his Presidential Address, Guillermo Bernal presented a preliminary version of this article to
Section VI (The Clinical Psychology of Ethnic Minorities) of the American Psychological Association (APA) Division 12 at the APA 104th Annual Convention in Toronto, Ontario, Canada, in
August 1994. This work was supported in part by National Institute of Mental Health Grant
R24-MH49368, by the Ford Foundation, and by the Funds for Institutional Research from the
Deanship of Graduate Studies and Research at the University of Puerto Rico, Ro Piedras
Campus.
We appreciate the suggestions and valuable comments from Eduardo Cumba, Ann Hohmann,
Axel Santos, Blanca Ortz, and Irma Serrano-Garca.
Correspondence concerning this article should be addressed to Guillermo Bernal, University
Center for Psychological Services and Research, Department of Psychology, University of Puerto
Rico, P.O. Box 23174, San Juan, Puerto Rico 00931-3174. Electronic mail may be sent to
gbernal@upracd.upr.clu.edu.
Cultural Diversity and Ethnic Minority Psychology
Vol. 7, No. 4, 328342

Copyright 2001 by the Educational Publishing Foundation

1099-9809/01/$5.00 DOI: 10.1037//1099-9809.7.4.328
328

ESTS

AND

ETHNIC MINORITIES

The controversy over empirically supported

treatments (ESTs) is well upon us. Since the
publication of the American Psychological
Association (APA) Division 12s report for
the Task Force on Promotion and Dissemination of Psychological Procedures (1995),
there have been over 50 articles and reports
examining this issue. In essence, the notion
of empirically validated or supported treatments is an effort to move the field of psychological practice and training toward using treatments that are supported by
available evidence. However, our review of
this relatively recent literature did not yield
many debates on the implications of the Division 12 Task Force report on clinical practice, training, and research with culturally
diverse populations or ethnic minorities.
There are only three reports (Nagayama
Hall, 2001; Sue, 1998, 1999) that address, in
part, the issue of generalization of research
results to ethnic and cultural minority population. Sues articles focused on science, ethnicity, and bias. Nagayama Halls (2001) article focused on psychotherapy research
with ethnic minorities. In this article, we affirm that having a list of treatments for a
specific disorder considered to be established in efficacy or to be possibly efficacious (Chambless & Hollon, 1998, p. 8)
that were actually based on predominantly
White, middle-class, English-speaking
women is of questionable use for ethnic minorities. Are these treatments appropriate
or even efficacious for communities of
color? Does the EST movement contribute
to psychologists knowledge of efficacious
treatments for ethnic minorities? How can
we, as psychologists, best contribute to the
development of treatment research of clinical utility with ethnic minority populations?
To address these questions, we review
the historical background of ESTs, summarize some of the key issues focusing on the
advantages and disadvantages within the
EST debate as they relate to ethnic minorities, and examine the importance of treatment research of clinical utility with these
populations. Also, an alternative approach
for psychotherapy research with ethnic mi-

329
norities is proposed as a resource in rethinking the current state of knowledge on treatments that work with culturally diverse
populations.

Emergence of EST
The Division 12 Task Force on Promotion
and Dissemination of Psychological Procedures published its report on empirically
validated treatments in 1995, in The Clinical
Psychologist (Task Force on Promotion and
Dissemination of Psychological Procedures,
1995). This report included a list of treatments originally labeled empirically validated
therapies, and which later would be referred
to as empirically supported treatments, or ESTs.
The criteria used for determining efficacy
were adapted from those traditionally and
currently used by the Federal Drug Administration (Beutler, 1998). According to
Chambless (1996), one of the principal objectives of the report was to facilitate clinical
training with interventions that were supported by empirical research. Increasing
awareness of these treatments (through the
dissemination of a list) and facilitating training opportunities were the primary strategies to affect clinical training.
The EST project was designed to address
four major objectives (Elliot, 1998). First,
the Task Force intended to establish criteria
for designating specific effective treatments
for specific disorders or problems. Second,
the Task Force aimed to generate lists of
well-established and probably efficacious
treatments. Third, the project was expected
to disseminate these lists to PhD programs
and predoctoral internships in clinical psychology. Finally, the intent was to incorporate the teaching and training in ESTs into
the guidelines for accrediting doctoral training programs and internships in clinical and
counseling psychology.
There was, and still is, a lot of controversy surrounding this report. One of the
issues that inspires much debate is that originally some believed that the Task Force in-

330
tended to offer the criteria1 it developed to
government, professional organizations,
managed care, or mental health insurers
(Elliot, 1998). Much of the criticism surrounding the project argued that by identifying a small number of treatments as efficacious, the implicit suggestion was that
those treatments not included in the list
were ineffective (Beutler, 1998). Also, there
are potentially serious economic implications to sectors of the mental health community not identified with psychotherapies
on the list, should managed care organizations adopt the Task Force list on ESTs as a
criterion to support psychological treatment.
Beutler (1998) noted that there has been
some debate as to whether it is indeed desirable to identify scientifically supported
treatments. Alternatively, Garfield (1998)
pointed out that such an emphasis on types
of psychotherapy may underscore the important role that patient and therapist characteristics play (such as patient variability
and therapist skill) in attaining positive
therapy outcomes. Other methodological
questions and issues, such as internal and
external validity of such treatments, have
also been raised (Persons & Silberschatz,
1998). If one examines the studies in the
EST list, one can observe that most of the
studies there have few (if any) formal consideration of the cultural, interpretative,
population, ecological, and construct validity of the intervention and the results reported. Washington and McLoyd (1982)
proposed that all these dimensions have to
be considered to ensure external validity.
For a detailed description of each of these
types of validity, refer to Bernal, Bonilla, and
Bellido (1995).
Elliott (1998) summarized the arguments that have been raised both in support
of and against ESTs. He concluded that
there is only one thing that everyone agrees
on: the need for more research. According
to Elliott, the majority of the papers he reviewed contain statements in support of the
list. The following are issues in favor of the
EST project: the quantity of information on
the effectiveness of specific treatments; its
potential for improving patient care and

BERNAL

AND

S C H A R R O N - D E L - R I O

clinical training, for influencing policy makers, and for encouraging more and better
therapy research; and its heuristic value.
There are, however, a number of voices
and arguments against this project (Elliot,
1998). There are those who argue against
the dissemination of the EST list and those
who are against the criteria used to designate the treatments in the list. The voices
against the dissemination of the list consider
that the evidence about most of the specific
treatments (questioning the evidence on efficacy) is insufficient. Another argument is
that the list seems to reflect arbitrary, political processes (including among these the
representation on Division 12 Task Force
and the systematic discrimination against
certain kinds of research, treatments, and
patients). For example, Section 4 (Clinical
Psychology of Women) of Division 12 protested the noninclusion of feminist therapy
in ESTs (L. Comas-Daz, personal communication, January 8, 2000). Similarly, those
who argue against the EST criteria consider
them as invalid or unhelpful.
For ethnic minority populations, the distribution of the list could bring attention to
the dearth of efficacy research with this
population and perhaps stimulate studies
with greater cultural representation. However, there are a number of serious concerns
with the criteria used. For example, the inclusion criteria, at least with Latinos/
Latinas, discriminates against the few existing efficacy studies (Comas-Daz, 1981;
Constantino, Malgady, & Rogler, 1986; Ros-

The criteria for determining a treatment as well established include having at least two between-groups
design experiments or 10 or more single case studies
conducted by at least two different investigators showing the treatment to be superior to a control condition
or to some other treatment or noninferiority to an
efficacious treatment. Determining a treatment as
probably efficacious requires two experiments demonstrating the treatments as superior to a control group
or one or more studies meeting the criteria for wellestablished treatments or at least four single-case design
experiments.

ESTS

AND

ETHNIC MINORITIES

sello & Bernal, 1999; Szapocznik, Kurtines,

Foote, Perez-Vidal, & Hervis, 1983, 1986;
Szapocznik et al., 1988; Szapocznik, PerezVidal, Hervis, Foote, & Kurtines, 1986; Szapocznik, Rio, et al., 1989; Szapocznik, Rio,
Perez-Vidal, Kurtines, & Hervis, 1986; Szapocznik, Santisteban, Hervis, Spencer, &
Kurtines, 1981; Szapocznik, Santisteban,
Kurtines, Perez-Vidal, & Hervis, 1984; Szapocznik, Santisteban, Rio, & Perez-Vidal,
1989) because of their small sample size.
This is an example of how internal validity is
emphasized over external validity. Sue
(1999) described this phenomenon as a selective enforcement of certain scientific
principles. The authors of the EST project
(Chambless et al., 1996) were careful to
point out that in the absence of ESTs for
ethnic minorities, the treatments on the list
should be used with this population. The
problem with this suggestion, while probably well intentioned, is its inherent contradiction. On the one hand, the agenda is to
disseminate the available ESTs on the basis
of their empirical support. On the other
hand, in the absence of data of their generalization, the call is to make a leap of faith
and use these treatments in the belief that
they will work as well with everyone else including ethnic minorities. One has to ask
the following question: If the ESTs were developed with Native Americans, African
Americans, Latinos/Latinas, or Asians,
would researchers be advocating for their
use with the general population? While we
agree that using treatments that lack supporting empirical evidence is probably better than using those without any supporting
evidence, the problem of their unknown external validity remains.
There are many reasons why ethnic and
cultural factors should be considered in the
articulation of psychosocial treatments. Psychotherapy itself is a cultural phenomenon,
and culture plays an important role in treatment process. Also, there is a wealth of literature suggesting that treatment models
consider the role of culture, ethnicity, and
minority issues (Bernal, Bonilla, & Bellido,
1995; Lopez et al., 1989; McGoldrick,
Pearce, & Giordano, 1982; Sue & Zane,

331
1987; Tharp, 1991). There is also evidence
from studies on service utilization (Arroyo,
Westerberg, & Tonigan, 1998; Cheung &
Snowden, 1990; Flaskerud & Liu, 1991; McMiller & Weisz, 1996; Schacht, Tafoya, & Mirabla, 1989), treatment preferences (Aldous, 1994; Constantino, Malgady, &
Rogler, 1994; Flaskerud & Hu, 1994;
Flaskerud & Liu, 1991; Penn, Kar, Kramer,
Skinner, & Zambrana, 1995; Schacht et al.,
1989), and health beliefs (McMiller & Weisz,
1996; Penn et al., 1995) to suggest that ethnic minorities may respond differently to
psychotherapy.
Ethnic and cultural concepts may also
conflict with mainstream values inherent to
traditional psychotherapies. Nagayama Hall
(2001) drew attention to the significance of
interdependence, spirituality, and discrimination in the psychotherapy of ethnic minorities. Mainstream psychotherapeutic approaches promote individualistic values
rather than collectivist or interdependent
values on which minorities are often socialized. Spirituality plays a different intra- and
interpersonal role in and among ethnic minorities. For example, the African American
churches serve as comprehensive community resources, more so than their White
counterparts. Also, the frequency and intensity in which discrimination is experienced
have important implications for treatment.
In addition, socioeconomic status and lack
of access to resources are both part of the
ethnic minority experience. Thus, it is essential to better understand how ethnicity interacts with (or may be a proxy for) cultural
values, discrimination, community resources, and socioeconomic status. These dimensions can be integrated in the treatment
protocols to effectively work with ethnic minority groups. Clearly, concerted efforts are
needed to document the efficacy, effectiveness, and clinical utility of treatments with a
diversity of groups.

The Need for Treatment Research With

Ethnic Minorities
In the United States in 2000, 17% of the
population was minority (U.S. Census Bu-

332
reau, 2001b). By the year 2050, conservative
projections show that ethnic minorities will
comprise over 50% of U.S. society (U.S.
Census Bureau, 2001a). Clearly, there is a
need to have empirically sound psychosocial
treatments for this sector of society. Nevertheless, in the United States, members of
ethnic minority groups neither use nor provide psychotherapy in proportion to their
number (Mays & Albee, 1992; Surgeon General, 1999).
Psychotherapy research on the demographic characteristics of therapy seekers
(Vessey & Howard, 1993) suggests that patients who enter treatment protocols tend to
be White, female, educated, and from the
middle and upper middle class. Few efficacy
studies or randomized clinical trials have
been conducted with ethnic minorities (Bernal, Bonilla, & Santiago, 1995; Miranda,
1996). Even if one considers effectiveness
studies such as the one published in Consumers Reports (Consumer Reports, 1995), the
data do not improve. While in this case the
sample size exceeded 3,000 participants,
little demographic information was available. Seligman (1995) described the sample
as middle class and educated and comprised
of people who choose to go to treatment
for their problems (p. 976). He acknowledged that this probably weighted the
sample toward a problem solvers group or
subset (Seligman, 1995) similar to what Vessey and Howard (1993) called therapy seekers. Herein lies part of the issue. Most
people would prefer to have options in addressing their problems. However, ethnic
minorities cannot afford to choose their
treatment because of their socioeconomic
and minority status. This is now referred to
as disparities in the use and quality of mental health services. At a minimum, epidemiological studies show that prevalence rates for
ethnic minorities are at least as high as the
general population (Sue, Zane, & Young,
1994).
In fact, ethnic minorities experience disproportionately higher poverty and social
stressors associated with psychological and
psychiatric conditions than do Whites (Mays
& Albee, 1992). Moreover, researchers have

BERNAL

AND

S C H A R R O N - D E L - R I O

reported that even when controlling for environment and individual variables such as
health and mental health history, attitudes
toward health-related issues, socioeconomic
status, and educational level, there are significant disparities in the use and quality of
mental health services among and across
ethnic, cultural, and racial communities
(National Institute of Mental Health, 1999).
There is evidence suggesting that ethnic minorities are experiencing major mental
health problems and have less access to
health care, and the health care they receive
is often of poorer quality (Surgeon General,
2000). Psychotherapy research with ethnic
minorities is needed to contribute to models
that consider diversity. As Sue et al. (1994)
noted, psychotherapy research with communities of color is important to advance the
field of psychology and psychotherapy. Additionally, treatment research with ethnic
minorities is important to address the problem of external validity. There is an important and significant relationship among external validity, ecological validity, and
culturally sensitive research (Bernal, Bonilla, & Bellido, 1995). As noted earlier, current treatment efficacy trials have unknown
external validity with respect to the application of the so-called effective interventions
to ethnic minorities as well as other nonmainstream groups. Most effectiveness studies do not include minorities in their
sample. The issue of generalization is not
trivial, and as we later see, may be addressed
by a somewhat different methodological approach. Thus, a list of treatments based on a
literature that has an ethnocentric bias is, at
best, of limited use to ethnic minorities.
We understand that an easily available
list of efficacious and probably efficacious treatments may be an important service to the public. However, it is a disservice
not to clarify the limits of their evaluation
for diverse populations and under different
circumstances. Thus, the mission to disseminate and inform the public should not overstate the ESTs applicability. In the quest for
simplicity, such a list may actually misinform
a significant sector of society. At a minimum,
caveats on the limits of the validity of the

ESTS

AND

ETHNIC MINORITIES

information need to be incorporated into

the list. With the existence of a Web page on
ESTs (http://udel.edu/jreiss/apa_treatments/index.shtml#top), this caveat should
be relatively simple to incorporate (Reiss,
1999). Another risk of disseminating the list
without clarifying the issues involved in the
lack of sound external validity for ESTs is
the de facto imposition of a treatment
model on a group or set of groups for which
the treatment was not developed or tested.
The exportation of these empirically based
treatments is well under way in the U.S.
market and has begun in the international
market. To the extent that a specific theory
of psychotherapy is developed, constructed,
and tested in a particular cultural group,
packaged as empirically sound, and imposed
on another, there may be a new form of
cultural imperialism. The insistence on generalization in the absence of data to support
a particular position is an illusion. Ford and
Urban (1998) eloquently stated this problem: presumptive truth evolves into misplaced belief when people become fervently
committed to their model; it ceases to be
provisional and becomes doctrinaire or
dogma instead. Certainty replaces possibility. The imperialist fallacy occurs when people
insist that others adopt their belief or
model (p. 35).

What Do We Know About the Efficacy of

Treatments for Ethnic Minorities?
From the perspective of the conventional
scientific model, we know very little if anything about the efficacy of treatments for
ethnic minorities. Even assuming a conventional paradigm of science, it is important to
address ethnic and cultural issues in treatment research because there are certain
conditions related to these that are associated to effectiveness. The effectiveness literature suggests that treatments may be affected by factors such as ethnic match for
some groups, culturally responsive forms of
treatment, pretreatment interventions, and

333
the training of therapists to work with culturally diverse groups (Sue et al., 1994).
However, according to the update on
empirical supported therapies published by
the Division 12 Task Force (Chambless et
al., 1996, p. 7), no psychotherapy treatment
research that meets the basic criteria important for demonstrating treatment efficacy for ethnic minority populations is
known. Chambless et al. enumerated basic
criteria for demonstrating treatment efficacy
for ethnic minority populations. According
to them, pre- and posttreatment status
should be assessed for clients from one or
more ethnic group or groups; clients should
be blocked according to their particular ethnic group membership and randomly assigned to different treatments or to treatment and control groups; multiple,
culturally cross-validated assessment instruments should be used; and findings should
be replicated. Congruent with the high demands these criteria impose on researchers,
it is of no surprise that the Division 12 Task
Force did not identify a single efficacy study
on treatments for ethnic minorities. One of
the major limitations in this attempt was that
most researchers either did not specify the
ethnicity of the participants or only used
White participants. Others did not use ethnicity as a variable of interest. Chambless et
al. (1996) concluded that the efficacy of
ESTs has not been established with ethnic
minority populations and that no data exist
on this issue or at least has not been reported (p. 7). These conclusions are further aggravated by the fact that there are not
many researchers working in conducting efficacy studies with ethnic minorities. In fact,
there is evidence to support the thesis that
minorities are not part of the scientific body
of psychological knowledge given the limited inclusion of minority participants in
APA publications (Graham, 1992). Other reviews of the behavioral research literature
(Iwamasa & Smith, 1996) conclude that ethnic minorities have been widely ignored.
Thus, efficacy studies involving ethnic minorities may be found in other venues of
communication.

334

Methodological and Theoretical

Implications of the Current Approach
A serious limitation of the EST project is
that the emphasis on criteria draws attention
away from more important research findings that have been demonstrated to have
effects on outcome, such as general equivalence of most treatments and client and
therapist variables (Elliott, 1998). Although
there is a limited literature on EST for ethnic minority populations, there are many
discussions about cultural and ethnic factors
that may mediate psychotherapy seeking behavior, treatment, satisfaction, and outcome
(Bernal, Bonilla, Padilla-Cotto, & PerezPrado, 1998; Organista, Mun
oz, & Gonzalez,
1994; Takeuchi, Sue, & Yeh, 1995; Yeh, Eastman, & Cheung, 1994).
Another problem is that the list consists
primarily of treatments evaluated with randomized clinical trial (RCT) methodology.
RCTs may not be the only approach to
building knowledge. Some authors suggest
that the RCT methodology is at best limited
(Doneberg, Lyons, & Howard, 1999;
Norquist, Levowitz, & Hyman, 1999) if not
flawed (Elliot, 1998). Seligman (1995)
stated, the efficacy study is the wrong
method for empirically validating psychotherapy as it is actually done, because it
omits too many crucial elements of what is
done in the field (p. 966).
Despite the limitations of the RCT methodology, it is not at all clear why the call was
made to block for ethnicity in new trials.
The Division 12 Task Force Criterion 2 and
3 are (Chambless et al., 1996, p.7) as follows:
A: Clients are blocked according to their particular ethnic group membership and randomly assigned to different treatments or to
treatment and control groups
B: Multiple, culturally cross-validated assessment instruments are employed

RCTs are so difficult to carry out in general that to impose a criterion of blocking
for ethnicity almost ensures that a particular
study will not meet the standard. Here again
we must ask the following question: Do studies with mainstream samples block for eth-

BERNAL

AND

S C H A R R O N - D E L - R I O

nicity or use cross-culturally validated instruments? Then, why use one set of criteria on
one group and not on another? Ironically, if
these criteria were applied to research with
nonminorities, the conclusion would have
to be that we know of no psychotherapy
treatment research that meets basic criteria
important for demonstrating treatment efficacy for nonethnic minority populations.
This is another example of what Sue (1999)
described as selective enforcement of scientific criteria (p. 1073).

Limitations of the Conventional Paradigm

for Treatment Research With Minorities
The ESTs are part of a research enterprise
produced from a perspective that eliminates
from the scientific inquiry the cultural, historical, and other contextual variables. It
may be worthwhile to understand the EST
movement limitations and propose alternatives to them by examining the literature on
hypothesis-testing versus discovery-oriented
research. Most psychotherapy research is designed to test hypotheses.
From an ethnic minority perspective and
in a multicultural context that is increasingly
pluralistic, it is important to examine how
knowledge is constructed, the limits of the
research, and the implications for theory
building. For these reasons, we turn to an
examination of a critical approach to science and the implications for treatment research with ethnic minorities.
The concept of paradigm refers to a conceptual, theoretical, and methodological
perspective or approach (Kuhn, 1970). The
conventional paradigm or approach from
which most of the treatment research is articulated conveys many inescapable limitations due to the assumptions in which classic
science is based (Hohmann, 1999). One of
conventional sciences main assumptions is
universal determinism: that is, that there exist invariant and natural laws that govern the
world, including human behavior, that can
be discovered and articulated (Morin,
1984). Hohmann noted that, if complex be-

ESTS

AND

ETHNIC MINORITIES

havioral or social science perspectives are

used, it becomes clear that no approach, no
model, and no theory can be an exact representation of humans complex realities.
The current approach for efficacy and effectiveness research is well embedded in a conventional (positivist) paradigm of science.
This approach assumes that the health service system is such that it can be represented
in a relatively simple manner, with little concern for the elements outside that system or
details within the system (Hohmann, 1999).
This statement underscores the need to consider diversity, multiculturalism, and the
context of the various populations in the service of parsimony.
These and other basic assumptions2 are
part of the theoretical and methodological
base on which most of the EST research has
been developed. This methodological approach assumes the object of inquiry to be
passive and noninteractive, intending to examine it by reducing its complexity to one
dimension and trying to fit multiple causalities and complex models into single and linear relationships. This position ignores most
of what characterizes and describes ethnic
minorities and their realities, and also, as
many would argue, most of what makes individuals unique and human. In summary,
because the ESTs were developed within the
conventional model of science, they are of
questionable utility for ethnic minorities,
and many of their limitations are a result of
or have been rooted in questionable assumptions of the conventional scientific
model.

335
methods. Discovery-oriented research
(Mahrer, 1988) might provide the tools and
methods to move researchers beyond traditional methodologies and perhaps toward a
consideration of research questions that
some have framed as of a more exploratory
nature. In fact, exploratory methods significantly contribute to the advancement of
clinical research (Howard, Orlinsky, & Lueger, 1994).
The discovery-oriented approach, according to Mahrer (1988), aims to provide a
closer, exploratory look at psychotherapeutic process and phenomena. This approach
aims to examine the connections among
therapeutic conditions, operations, and consequences. Discovery-oriented research is an
alternative strategy with a rationale, aims,
and methods that serve as a counterpoint to
hypothesis-testing research. The findings of
discovery-oriented research are the necessary basis for scientific theory building and
psychotherapeutic conceptualization, advancing treatment practice, opening up new
avenues of psychotherapeutic research, and
blending theory, practice, and research in
an integrative manner.
However, rather than abandoning more
traditional hypothesis-testing research altogether in favor of the discovery-oriented approach, our suggestion is to integrate these
dimensions. Such integration may provide
us, as researchers, with means to understand
where we are in psychotherapy research with

Toward An Integration of Approaches to

Treatment Research With
Ethnic Minorities
To step out of the paradigmatic box of mainstream science, we propose an integration of
hypothesis-testing (contemporary) and discovery-oriented (alternate) research. Discovery-oriented and hypothesis-testing research
are two approaches with different goals and

Other basic assumptions are the elimination of the

irreversibility of time and everything that is historical
and contextual, nonlinear causality, the incapacity to
concede autonomy to the objects of inquiry, and the
almost exclusive application of classical logic (Morin,
1984). Among the factors that are left out of the inquiry
process by the contemporary approach of efficacy research are contextual and interactional factors. These
include the norms and emotional climate of the service
organization and how they affect the clinician; the fit
between the client and the clinicians cultures, goals,
personalities, attitudes, strategies, and preferences; and
the influence of family and peers. Ethnic and cultural
values are either ignored or minimized in importance
by the current approach.

336
ethnic minorities and perhaps where we
should go. Others have suggested that an
integration of methods may increase external validity (Borkovec & Castonguay, 1998),
noting that naturalistic research is important for external validity but is valuable only
if it uses scientifically valid methods to address basic knowledge questions (Borkovec
& Castonguay, 1998, p.136). In fact, an ethnographic approach is a legitimate alternative in multicultural contexts. With anthropological methods such as ethnography, a
more comprehensive understanding of cultural knowledge may be acquired (Agar,
1980; Emerson, Fretz, & Shaw, 1995). It is
clear that neither qualitative nor quantitative data alone will lead the field to a clear
understanding of what treatments or services are effective for whom, under what circumstances, and why (Hohmann, 1999,
p. 88).
Instead of considering the discoveryoriented and hypothesis-testing research approaches as opposites, we view these approaches as complementary dimensions. If
we cross the discovery-oriented and hypothesis-testing research dimensions, a 2 2
table is generated. As Figure 1 shows, there
are four quadrants based on the criteria of
whether or not a study can be considered to
be predominately oriented toward one approach or another. For example, if a study is
primarily hypothesis-testing research and
not aimed at discovery, we have the domain
of contemporary science generally considered as experimental and quasi-experimental research. These types of studies follow a deductive, quantitative orientation that is

Figure 1. Hypothesis-testing and discovery-oriented

research as complementary dimensions.

BERNAL

AND

S C H A R R O N - D E L - R I O

focused on internal validity, usually have a

small sample size, and are focused on manipulating the independent variable, usually
the treatment. Most efficacy and some effectiveness studies fit here. Research that is not
oriented toward the testing of hypothesis
but rather aims to discover is exploratory, phenomenological, often qualitative, ethnographic, or naturalistic. In this type of study,
the independent variable is not manipulated. This is where one will find most of the
ethnic minority research efforts. In this
quadrant, we included some effectiveness
research because, depending on the emphasis, this type of research could overlap with
other quadrants.
There is another body of research that is
not aimed at discovery or at hypothesis testing that may be labeled descriptive or theoretical. These type of studies may be descriptive program evaluation efforts, theoretical
research, and clinical essays.
Finally, research that is both oriented toward hypothesis testing and discovery we
have labeled as methodological pluralism (Polkinghorne, 1983) or diversity. This quadrant
describes research efforts that incorporate
multiple strategies and uses a range of methods to both confirm propositions and discover new knowledge. Sue (1999) called for
diverse and multiple methodologies in ethnic minority research as an indispensable
way to increase researchers knowledge
about these populations.

Conclusions
It is not clear whether the questions posed at
the outset of this article can be answered
fully or whether in fact they have answers.
Does the EST movement contribute to ones
knowledge of treatments that are efficacious
with communities of color? Perhaps, but the
approach to science needs to be more inclusive of a diversity of approaches to inquiry.
Although it is clear that more efficacy, effectiveness, and other treatment outcome studies with communities of color are needed, a

ESTS

AND

ETHNIC MINORITIES

methodological pluralistic approach should

guide these efforts.
Concerning the question of how to best
contribute to the field of treatment research
of clinical utility with minority populations,
we need to step out of the contemporary
paradigmatic box. In this manner, it is possible to question the implicit assumptions of
knowledge construction. Additionally, it is
essential that researchers construct theories
of psychotherapy and evaluate treatments
grounded in the realities and experiences of
ethnic minority populations. We call upon
the research, training, and practice communities to question the current paradigms validity and consider working from more inclusive paradigms to develop and test
theories, as well as evaluate the impact of
treatments. This task becomes critical as
the current paradigm limits researchers
knowledge and efforts. The conventional
paradigm underlies most of the research
production of conventional science since its
earliest conception. It affects the scientific
processes as a filter, determining which articles are published or what kind of research
is funded. This filter also works to undermine decisions on what information is
important and what can be discarded, therefore privileging certain trends and theoretical models over others. The EST project is
no exception.
Scientific inquiry includes many alternatives to approach knowledge. Historically,
some have taken precedence and have been
favored over others. Whereas Sue (1999) described how the quest for internal validity
has led some to favor research despite its
questionable external validity, we present
how hypothesis-testing research has been favored over discovery-oriented research in
the same manner. In both cases, the generalizability of the favored researchs findings is
questionable, and the body of knowledge derived is biased and incomplete. Therefore, it is
important, for the sake of a more equitable
application of scientific principles, to bring
balance to the scientific efforts as a field. This
balance may be obtained by incorporating approaches based on methodological diversity.

337
If the proposed criteria used by the Division 12 Task Force on Dissemination of
Psychological Procedures is accepted as a
given, most if not all the research done by
and with communities of color is filtered out
through criteria elaborated from conventional paradigm assumptions (or an exclusively hypothesis-testing orientation). Thus,
one would have to conclude that there is no
relevant knowledge that can be drawn from
the existing literature on treatments with
communities of color. The inevitable conclusion is that there are few if any psychotherapy treatment research that meets these
criteria well enough to demonstrate treatment efficacy with ethnic minorities. Furthermore, to demonstrate efficacy with ethnic minorities, the bar is higher, and an
extra set of criteria needs to be met. Conversely, if one considers research that is discovery oriented, there is an impressive
wealth of quality research describing treatments and interventions with ethnic minorities that can inform treatment, training, and
practice.
Although this seemingly discriminatory
course of action may not be an intentional
one, it is clearly a reflection of one among
many of the possible consequences of the
limitations that the basic assumptions of the
conventional paradigm impose on the scientific inquiry. Working from the conventional
paradigm means assuming parsimony
(which is analogous to simplicity), regularity, normality, and generalization. Therefore, work with ethnic minorities may be ill
fitted in this approach, if it leaves out individual differences and the contextual, interactional, and process instances (Morin,
1984) that characterize the complexity of
life that defines peoples identities as ethnic
minorities. Clearly, a rethinking and a reconsideration of these basic assumptions are
needed. Here we are not proposing a rejection of science but, as others have suggested,
a true application of scientific criteria and a
clarification of the rules of evidence (Sue,
1999). We need to rethink how science is
practiced and how the rules of evidence
should be more inclusive of a diversity of

338
empirical sources of information and data.
The realities and processes need to be revisited and researched from more complex approaches, from different instances, and
from multiple perspectives. Another approach from which to stand on is needed;
this has been our main motive in this article:
to both question the implicit assumptions of
the contemporary paradigm and propose alternatives for psychotherapy research with
ethnic minorities.
Nagayama Hall (2001) considered an alternative approach comparing ESTs with
culturally sensitive therapies. He suggested
that these two research traditions collaborate to advance the field of psychotherapy
research with ethnic minorities. He recommended several steps to achieve this collaboration beginning with an evaluation of the
ESTs cultural relevance and sensitivity.
To advance the knowledge on treatments and services that are both effective
and relevant to minority communities, we
offer the following recommendations for
those involved in research, training, and
practice:
1. Reconceptualize knowledge in the field
with integrative approaches for psychotherapy and intervention research.
2. Approach the object of study from a
position of methodological diversity.
3. Focus efforts on the treatment of specific ethnic minority groups rather
than on comparative approach. Comparative studies often have weak conceptualizations (i.e., it is not clear why
ethnicity or race matters), lack documentation on the meaning of ethnicity/race (i.e., no unpacking of ethnicity in empirical studies), and often are
based on an underlying deficit model.
The exceptions to this are studies focused on ethnic matching. It is more
important to document that a treatment works with a group, and then
why it works or what makes it work,
rather than ethnic comparisons
aimed at findings differences in outcome. At this point in the fields de-

BERNAL

4.

5.

6.

7.

8.

AND

S C H A R R O N - D E L - R I O

velopment, it is best to focus on specific ethnic groups, unless there is a

clear theoretical basis for a comparative approach.
Invite reviews of discovery-oriented
studies on the role of ethnicity in
therapeutic conditions, operations,
and consequences. For example, it
would also be important to have reviews of discovery-oriented research
on how treatments need to be modified with cultural and ethnically sensitive criteria for a particular ethnic
minority group. Also, it would be of
considerable value to revise studies
based on approaches for adapting
and development of psychosocial
treatments with specific ethnic minority groups (e.g., Bernal, Bonilla, &
Bellido, 1995).
Invite reviews of discovery-oriented
research on how treatments need to
be modified with cultural- and ethnicsensitive criteria for a particular ethnic minority group.
Develop instruments with specific ethnic minority groups on outcome measures of change.
Carry out hypothesis-testing research,
both efficacy and effectiveness studies with specific ethnic minority
populations.
Conduct qualitative and quantitative
studies aimed at an understanding of
what treatments or services are beneficial for whom, under what set of
circumstances, and why these treatments work.

Considering alternative methods increases the likelihood that the object of

study will be approached as a more complex
phenomenon. In this article, we have suggested an approach that may serve as coordinates to organize the diversity of information on the treatment of ethnic minorities.
An approach based on methodological diversity will, perhaps, move us to closer representations of realities, given that inquiry is

ESTS

AND

ETHNIC MINORITIES

based on multiple perspectives making use

of a diversity of methods.
To the extent that a list of treatments are
exported and marketed to other cultural
groups without the adequate evaluations
and testing, we as researchers run the risk of
engaging in false and misleading ways of
thinking on the applicability of our limited
knowledge of ESTs. The external validity of
ESTs is simply not known, and there is no
scientific basis (contemporary or otherwise)
for the application to other groups except
on a faith, perhaps even blind, of a distorted
view of empiricism. At a minimum, efficacy
and effectiveness research is needed with
the primary ethnic minority groups (African
American, Latinos and Latinas, Asian Americans, and Native Americans) and ideally
with every cultural group, including large
heterogeneous samples, to support the
claims of generalization.

References
Agar, M. H. (1980). The professional stranger: An
informal introduction to ethnography (2nd ed.).
New York: Academic Press.
Aldous, J. L. (1994). Cross-cultural counseling
and cross-cultural meanings: An exploration
of Morita psychotherapy. Canadian Journal of
Counseling, 28, 238249.
Arroyo, J. A., Westerberg, V. S., & Tonigan, J. S.
(1998). Comparison of treatment utilization
and outcome for Hispanics and non-Hispanic
Whites. Journal of Studies on Alcohol, 59,
286291.
Bernal, G., Bonilla, J., & Bellido, C. (1995). Ecological validity and cultural sensitivity for outcome research: Issues for the cultural adaptation and development of psychosocial treatments with Hispanics. Journal of Abnormal
Child Psychology, 23, 6782.
Bernal, G., Bonilla, J., Padilla-Cotto, L., & PerezPrado, E. (1998). Factors associated to outcome in psychotherapy: An effectiveness
study in Puerto Rico. Journal of Clinical
Psychology, 54, 329342.
Bernal, G., Bonilla, J., & Santiago, E. (1995). Psychometric properties of the BDI and SCL-36
in a Puerto Rican sample (in Spanish). Re-

339
vista Latinoamericana de Psicolog a, 27,
207230.
Beutler, L. E. (1998). Identifying empirically supported treatments: What if we didnt? Journal
of Consulting and Clinical Psychology, 66,
113120.
Borkovec, T. D., & Castonguay, L. G. (1998).
What is the scientific meaning of empirically
supported therapy? Journal of Consulting and
Clinical Psychology, 66, 136142.
Chambless, D. L. (1996). In defense of dissemination of empirically supported psychological interventions. Clinical Psychology: Science
and Practice, 3, 230235.
Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal
of Consulting and Clinical Psychology, 66, 718.
Chambless, D. L., Sanderson, W. C., Shoham, V.,
Johnson, S. B., Pope, K. S., Crits-Christoph,
P., Baker, M., Johnson, B., Woody, S. R., Sue,
S., Beutler, L., Williams, D. A., & McCurry, S.
(1996). An update on empirically validated
therapies. The Clinical Psychologist, 49, 518.
Cheung, F. K., & Snowden, L. R. (1990). Community mental health and ethnic minority
populations. Community Mental Health Journal,
26, 277291.
Comas-Daz, L. (1981). Effects of cognitive and
behavioral group treatment in the depressive
symptomatology of Puerto Rican women.
Journal of Consulting and Clinical Psychology, 49,
627632.
Constantino, G., Malgady, R. G., & Rogler, L. H.
(1986). Cuento therapy: A culturally sensitive
modality for Puerto Rican children. Journal of
Consulting and Clinical Psychology, 54, 639645.
Constantino, G., Malgady, R. G., & Rogler, L. H.
(1994). Storytelling through pictures: Culturally sensitive psychotherapy for Hispanic children and adolescent. Journal of Clinical Child
Psychology, 23, 1320.
Consumer Reports. (1995, November). Mental
health: Does therapy help? Consumer Reports,
734739.
Doneberg, G. R., Lyons, J. S., & Howard, K. I.
(1999). Clinical trials versus mental health
services research: Contributions and connections. Journal of Clinical Psychology, 55,
11351146.
Elliot, R. (1998). Editors introduction: A guide
to the empirically supported treatments controversy. Psychotherapy Research, 8, 115125.

340
Emerson, R. M., Fretz, R. I., & Shaw, L. L. (1995).
Writing ethnographic fieldnotes. Chicago: University of Chicago Press.
Flaskerud, J. H., & Hu, L. (1994). Participation in
and outcome of treatment for major depression among low income Asian-Americans. Psychiatry Research, 53, 289300.
Flaskerud, J. H., & Liu, P. Y. (1991). Effects of an
Asian clienttherapist language, ethnicity
and gender match on utilization and outcome of therapy. Community Mental Health
Journal, 27, 3142.
Ford, D., & Urban, H. D. (1998). Contemporary
models of psychotherapy: A comparative analysis
(2nd ed.). New York: Wiley.
Garfield, S. L. (1998). Some comments on empirically supported treatments. Journal of Consulting and Clinical Psychology, 66, 121125.
Graham, S. (1992). Most of the subjects were
White and middle class: Trends in published
research on African Americans in selected
APA journals, 19701989. American Psychologist, 47, 629639.
Hohmann, A. A. (1999). A contextual model for
clinical mental health effectiveness research.
Mental Health Services Research, 1, 8392.
Howard, K. I., Orlinsky, D. E., & Lueger, R. J.
(1994). Clinically relevant outcome research
in individual psychotherapy: New models
guide the researcher and clinician. British
Journal of Psychiatry, 165, 48.
Iwamasa, G. Y., & Smith, S. K. (1996). Ethnic diversity in behavioral psychology: A review of
the literature. Behavior Modification, 20,
4559.
Kuhn, T. S. (1970). The structure of scientific revolutions (2nd ed.). Chicago: University of Chicago Press.
Lopez, S. R., Grover, K. P., Holland, D., Johnson,
M. J., Kain, C. D., Kanel, K., Mellins, C. A., &
Culkin, M. (1989). Development of culturally
sensitive psychotherapists. Professional Psychology: Research and Practice, 20, 369376.
Mahrer, A. R. (1988). Discovery-oriented psychotherapy research: Rationale, aims, and methods. American Psychologist, 43, 694702.
Mays, V. M., & Albee, G. W. (1992). Psychotherapy and ethnic minorities. In D. K. Freedheim & H. J. Freudenberger (Eds.), History of
psychotherapy: A century of change (pp.
552570). Washington, DC: American Psychological Association.

BERNAL

AND

S C H A R R O N - D E L - R I O

McGoldrick, N., Pearce, J. K., & Giordano, J.

(1982). Ethnicity and family therapy. New York:
Guildford Press.
McMiller, W. P., & Weisz, J. R. (1996). Helpseeking preceding mental health clinic intake among African-American, Latino, and
Caucasian youths. Journal of the American Academy of Child and Adolescent Psychiatry, 35,
10861094.
Miranda, J. (1996). Introduction to the Special
Section on recruiting and retaining minorities in psychotherapy research. Journal of Consulting and Clinical Psychology, 64, 848850.
Morin, E. (1984). Ciencia con consciencia [Science
with conscience]. Barcelona, Spain:
Anthropos.
Nagayama Hall, G. (2001). Psychotherapy research with ethnic minorities: Empirical, ethical, and conceptual issues. Journal of Consulting and Clinical Psychology, 69, 502510.
National Institute of Mental Health. (1999). Strategic plan on reducing health disparities. Rockville, MD: Author.
Norquist, G., Levowitz, B., & Hyman, S. (1999).
Expanding the frontier of treatment research. Prevention and Treatment, 2, Article
0001a. Retrieved August 12, 2001 from the
World Wide Web: http://journals.apa.org/
prevention/volume2/pre0020001a.html.
Organista, K. C., Mun
oz, R. F., & Gonzalez, G.
(1994). Cognitivebehavioral therapy for depression in low-income and minority medical
outpatients: Description of a program and exploratory analyses. Cognitive Therapy and Research, 18, 241259.
Penn, N. E., Kar, S., Kramer, J., Skinner, J., &
Zambrana, R. (1995). Panel VI: Ethnic minorities, health care systems, and behavior.
Health Psychology, 14, 641646.
Persons, J. B., & Silberschatz, G. (1998). Are results of randomized controlled trials useful to
psychotherapists? Journal of Consulting and
Clinical Psychology, 66, 126135.
Polkinghorne, D. (1983). Methodology for the human sciences. Albany: State University of New
York Press.
Reiss, J. E. (1999, March 26). Finding effective psychotherapy. Dissemination Subcommittee of
the Committee on Science and Practice,
American Psychological Association. Retrieved May 17, 2000 from the World Wide
Web: http://udel.edu/jreiss/apa_treatments/index.shtml#top.

ESTS

AND

ETHNIC MINORITIES

Rossello, J., & Bernal, G. (1999). The efficacy of

cognitivebehavioral and interpersonal treatments for depression in Puerto Rican adolescents. Journal of Consulting and Clinical Psychology, 67, 734745.
Schacht, A. J., Tafoya, N., & Mirabla, K. (1989).
Home-based therapy with American Indian
families. American Indian and Alaska Native
Mental Health Research, 3, 2742.
Seligman, M. P. E. (1995). The effectiveness of
psychotherapy: The Consumer Reports study.
American Psychologist, 50, 965974.
Sue, S. (1998). In search of cultural competence
in psychotherapy. American Psychologist, 53,
440448.
Sue, S. (1999). Science, ethnicity, and bias:
Where have we gone wrong? American
Psychologist, 54, 10701077.
Sue, S., & Zane, N. (1987). The role of culture
and cultural techniques in psychotherapy: A
critique and reformulation. American Psychologist, 42, 3745.
Sue, S., Zane, N., & Young, K. (1994). Research
in psychotherapy with culturally diverse
populations. In A. E. Bergin & S. L. Garfield
(Eds.), Handbook of psychotherapy and behavior
change (4th ed., pp. 783820). New York: Wiley.
Surgeon General. (1999, April 4, 2000). Executive
Summary: A report of the Surgeon General on mental health. Washington, DC: U.S. Public
Health Service. Retrieved April 5, 2000
from the World Wide Web: http://www.
surgeongeneral.gov/library/mentalhealth/
summary.html.
Surgeon General. (2000). Supplement to Mental
health: A report of the Surgeon General. Disparities in mental health care for racial and ethnic
minorities. Washington, DC: U.S. Public
Health Service.
Szapocznik, J., Kurtines, W. M., Foote, F. H.,
Perez-Vidal, A., & Hervis, O. (1983). Conjoint
versus one-person family therapy: Some evidence for the effectiveness of conducting
family therapy through one person. Journal of
Consulting and Clinical Psychology, 51, 889899.
Szapocznik, J., Kurtines, W. M., Foote, F., PerezVidal, A., & Hervis, O. (1986). Conjoint versus one-person family therapy: Further evidence for the effectiveness of conducting
family therapy through one person with drugabusing adolescents. Journal of Consulting and
Clinical Psychology, 54, 395397.

341
Szapocznik, J., Perez-Vidal, A., Brickman, A. L.,
Foote, F. H., Santisteban, D. A., Hervis, O., &
Kurtines, W. M. (1988). Engaging adolescent
drug abusers and their families in treatment:
A strategic structural systems approach. Journal of Consulting and Clinical Psychology, 56,
552557.
Szapocznik, J., Perez-Vidal, A., Hervis, O. E.,
Foote, F. H., & Kurtines, W. (1986). Terapia
de familia a traves de un solo miembro:
FamUno [One-person family therapy:
FamUno]. Cuadernos de Psicologia, 8, 5379.
Szapocznik, J., Rio, A., Murray, E., Cohen, R.,
Scopetta, M., Rivas-Vazquez, A., Hervis, O., &
Posada, V. (1989). Structural family versus
psychodynamic child therapy for problematic
Hispanic boys. Journal of Consulting and Clinical Psychology, 57, 571578.
Szapocznik, J., Rio, A., Perez-Vidal, A., Kurtines,
W., & Hervis, O. (1986). Bicultural effectiveness training (BET): An experimental test of
an intervention modality for families experiencing intergenerational/intercultural conflict. Hispanic Journal of Behavioral Sciences, 8,
303330.
Szapocznik, J., Santisteban, D., Hervis, O., Spencer, F., & Kurtines, W. (1981). Treatment of
depression among Cuban American elders:
Some validational evidence for a life enhancement counseling approach. Journal of
Consulting and Clinical Psychology, 49, 752754.
Szapocznik, J., Santisteban, D., Kurtines, W. M.,
Perez-Vidal, A., & Hervis, O. E. (1984). Bicultural effectiveness training: A treatment intervention for enhancing intercultural adjustment in Cuban American families. Hispanic
Journal of Behavioral Sciences, 6, 317344.
Szapocznik, J., Santisteban, D., Rio, A., & PerezVidal, A. (1989). Family effectiveness training: An intervention to prevent drug abuse
and problem behaviors in Hispanic adolescents. Hispanic Journal of Behavioral Sciences,
11, 427.
Takeuchi, D. T., Sue, S., & Yeh, M. (1995). Return rates and outcomes from ethnicityspecific mental health programs in Los Angeles. American Journal of Public Health, 85,
638643.
Task Force on Promotion and Dissemination of
Psychological Procedures. (1995). Training
in and dissemination of empirically-validated
psychological treatments: Report and recommendations. The Clinical Psychologist, 48, 323.

342
Tharp, R. G. (1991). Cultural diversity and treatment of children. Journal of Consulting and
Clinical Psychology, 11, 101106.
U.S. Census Bureau. (2001a, January 13, 2000).
Projections of the total resident population by 5-year
age groups, race, and Hispanic origin with special
age categories: Middle series, 2050 to 2070. U.S.
Census Bureau, Population Division, Statistical Information Staff. Retrieved August 13,
2001 from the World Wide Web: http://
www.census.gov/population/projections/
nation/summary/np-t4-g.txt.
U.S. Census Bureau. (2001b, January 2, 2001).
Resident population estimates of the United States
by sex, race, and Hispanic origin: April 1, 1990 to
November 1, 1999, with short-term projection to

BERNAL

AND

S C H A R R O N - D E L - R I O

November 1, 2000. U.S. Census Bureau, Population Division, Statistical Information Staff.
Retrieved January 29, 2001 from the World
Wide Web: http://www.census.gov/
population/estimates/nation/intfile3-1.txt.
Vessey, J. T., & Howard, K. I. (1993). Who seeks
psychotherapy? Psychotherapy, 30, 546553.
Washington, E. D., & McLoyd, V. (1982). The
external validity of research involving American minorities. Human Development, 25,
324339.
Yeh, M., Eastman, K., & Cheung, M. K. (1994).
Children and adolescents in community
health centers: Does the ethnicity or the language of the therapist matter? Journal of Community Psychology, 22, 153163.