COMFORT, EFFECTIVENESS, AND SELF-AWARENESS AS

CRITERIA OF IMPROVEMENT IN PSYCHOTHERAPY”2

MORRIS B. PARLOFF, PH. D., HERBERT C. KELMAN, PH. D., AND JEROME D. FRANK, M.D.

I technique. Failures seldom cause the thera-

pist to question the aims or methods of his
There is no more urgent problem facing
form of therapy.
psychiatry today than the evaluation of the
Statistical studies of the results of psycho-
effectiveness of psychotherapy. It is esti-
therapy, however, all show about two-thirds
mated that at least 9,000,000 Americans are
of the patients, plus or minus about 10%, as
suffering from some form of mental illness
improved regardless of the type of therapy
(25, p. 125). Many of these receive psycho-
(r, 16). Moreover, the same improvement
therapy and many more feel that if only they
figure crops up with methods of treatment
could get it they would be relieved. As the
that are considered by their proponents to
approximately io,ooo psychiatrists, clinical
involve little if any psychotherapy, such as
psychologists, and psychiatric social workers
carbon dioxide treatment(7). What does
in this country cannot begin to satisfy the
this mean? The cynic may say that it proba-
demand, public and private agencies are sup-
bly represents the improvement rate that oc-
porting vast training programs requiring
curs over a period of several months as a
large investments of time and money. So-
result of ordinary life experiences or the
ciety has a very large stake in psychotherapy
spontaneous mobilization of the patient’s re-
but can be expected to continue support at
cuperative forces. Those who accept such
present levels only if the results prove to be
an interpretation might conclude that psy-
worth the cost.
chotherapists make their living off the
In the face of this situation the unpleasant
spontaneous remission rate. Yet every psy-
truth is that there is no agreement as to what
chotherapist has had patients whose improve-
kind of psychotherapy is best for different
ment followed so closely upon occurrences
patients or psychiatric disorders, and the
in the therapeutic situation as to make it
value of any type of psychotherapy remains
highly unlikely that this was due to mere
to be conclusively demonstrated. At present
chance.
the type of treatment received by a patient is
In the present state of knowledge interpre-
largely determined by the accident of train-
tation of these statistical findings is impos-
ing of the therapist to whom he happens to
sible. It is not even clear, for example, to
go. Each therapist is confident that he is
helping at least some of his patients. He what extent the consistency of improvement
rates represents actual changes in the pa-
can see them gain an understanding of their
tients and to what extent they reflect a habit
difficulties, lose their symptoms, and become
more effective in their personal relationships. of mind of the judges. We do not know in
His successful patients are sure he has what ways patients said to improve under
helped them and do not hesitate to sing his different therapies resemble or differ from
praise and refer their friends. Therapists each other, or how to compare the techniques
of different therapies and the nature of the
tend to attribute failures to the patient’s not
results obtained.
being ready for therapy, or to a breach of
These and many other questions must be
‘Read at the iioth annual meeting of The Ameri- clarified if psychotherapy is to become a sci-
can Psychiatric Association, St. Louis, Mo., May ence in the sense that different psychothera-
3-7, 1954.
2 research on which this paper is based was
pies can be adequately described, their rela-
supported a grant
by from the National Institute tive effectiveness convincingly demonstrated,
of Mental Health of the National Institutes of and the type of therapy tailored to the per-
Health, U. S. Public Health Service. Dr. Kelman sonality of the patient and the nature of his
was a Public Health Service research fellow of the
National Institute of Mental Health while collabo- disorder. To achieve these goals we must be
rating on this paper. able () to describe therapeutic techniques
343
344 COMFORT, EFFECTIVENESS, AND SELF-AWARENESS [Nov.

in such terms that others can perform them likely false. It is probable that measures of
similarly(4); (2) to describe the person- different criteria of improvement would be
ality of the therapist as it relates to his influ- affected differently depending on such fac-
ence on different types of patients( ii, iv); tors as: (i) The type of therapy. For ex-
() to describe patients in terms relevant for ample, group therapy may tend to increase a
psychotherapy(6, 8); and (4) to describe patient’s social facility more than his under-
improvement in terms permitting comparison standing of the historical origins of his in-
of one type of therapy with another. terpersonal difficulties; certain kinds of indi-
The problem of describing improvement vidual therapy, on the other hand, may tend
has become one of concern to us in connec- to increase the patient’s insight without in-
tion with a current study of attributes which creasing his social facility. (2) The type of
influence the response to therapy of outpa- patient. Different individuals may have dif-
tients in a psychiatric clinic. For this study ferential predispositions to change. Some
it was necessary to devise measures of im- may change readily with respect to symptoms
provement which made possible comparison and slowly as regards insight, while others
of the results of 3 different types of therapy may have opposite tendencies. Also, differ-
with a wide range of patients. This paper ent persons may have differential need for
describes the measures selected, the consid- change. One patient may be “sick” sympto-
erations which led to their choice, and prob- matically, but functioning adequately in his
lems connected with their use. We believe social relations, while another may have his
that it is possible to measure the effects of difficulties in the latter area. Successful
therapy by means of a limited number of therapy for the first patient would involve
criteria, based on the patient’s conscious, re- chiefly change in symptoms; for the second,
portable feelings and overt behavior, which improvement in his social relations. Finally,
can be quantified, thereby allowing com- there may also be individual differences in
parison of the effects of one therapy with the extent to which changes on the different
another, and which at the same time are criteria converge (17). () Time. Measures
relevant to what is generally meant by im- of some criteria may show change early in
provement. The criteria proposed are com- therapy but remain unaffected thereafter.
fort, effectiveness, and self-awareness. Other criteria may show no change, or per-
haps even worsen, during the early period
but begin to change after further treatment.
II
For example, the greater part of sympto-
Most of the accumulated literature on matic relief may occur early in therapy;
therapy is so difficult to evaluate because im- whatever symptoms survive the early treat-
provement is thought of in global terms and ment period may be extremely resistant to
rated only with respect to degree. The fatal change. On the other hand, anxiety may in-
flaw in this procedure, as is now rather gen- crease early in therapy and decline later. The
erally recognized, is that it assumes improve- conditions under which different criteria of
ment to be a unitary phenomenon. All cri- improvement are or are not positively corre-
teria of improvement are assumed to vary lated thus become a matter for empirical
together, so that regardless of the specific study.
criterion selected the results would be the If improvement is not a unitary phenome-
same. This assumption probably does hold non, over-all evaluations have serious draw-
for patients who are markedly improved or backs. Different evaluators may mean differ-
recovered. Therapists of different schools, ent things by improvement and the same
despite differences in their implicit or ex- evaluator may shift his criterion from one
plicit criteria, agree readily as to the im- patient to the next. He may call one patient
provement of these patients, suggesting that improved because his headaches are better
the criteria have changed together. and another improved because he has found
With lesser degrees of improvement, how- a better job. Depending on the patient’s
ever, the assumption that all criteria vary original level, he may consider a given
together is at best questionable and very change, such as loss of a symptom, “slight
 1954] M. B. PARLOFF, H. C. KELMAN AND J. D. FRANK 34

improvement” in one case and “great im- given from data of a preliminary study of
provement” in another. The same change 16 neurotic patients who were evaluated be-
may be considered to represent improvement fore and after 20 weeks of group therapy.
in one patient and worsening in another. In- Certain relationships among 4 of the many
creased anxiety, for example, might be taken measures studied will be considered here.
as a sign of improvement in an obsessional The measures are:
neurotic but a danger signal in an incipient i. A symptom check list, filled out by the
schizophrenic. patient, which contained items pertaining
The problem is most sharply emphasized chiefly to bodily distress and emotional
by the patient who moves in opposite direc- disturbances.
tions at the same time. For example, how is 2. A rating scale of the patient’s discom-
the over-all improvement to be rated in a fort, filled out by the staff and containing
woman who stops having migraine when she such items as sexual conflicts, social uneasi-
leaves her husband and gets it back when she ness, feelings of being exploited by others.
resumes her role as housewife and mother? 3. A measure of self-acceptance obtained
Parenthetically, leaving “improvement” by having a patient describe his behavior in
undefined probably facilitates overestimation the group by means of a Q-sort technique,
of improvement. In the course of several and then, by re-sorting the items, describe
months any patient is apt to change in some what he would like his behavior to be(19).
way whether or not he is receiving therapy, The degree of correlation between the two
and the rater can usually find some positive sorts was taken as a measure of self-accept-
change when he is looking for it and no re- ance.
strictions are put on his choice. The study 4. A measure of the accuracy of the pa-
of Teuber and Powers(23) strikingly illus- tient’s description of his own behavior (self-
trates the danger of relying solely on thera- awareness), obtained with the same Q-sort
pists’ estimates of improvement unchecked by correlating the patient’s description of his
by an objective criterion. Although thera- own group behavior with that made by a
pists of predelinquent boys believed about trained observer. The higher the correlation,
two-thirds of the children had “substan- the more self-aware the patient was con-
tially benefited” from the treatment, no sig- sidered to be.
nificant difference in number of court ap- Certain correlations among these measures
pearances was found between the treated at the start of treatment, and among their
boys and an untreated control group. degrees of change after treatment, are given
In short, global ratings of improvement in Table i. (All omitted correlations did
tell nothing about the nature of the change not differ significantly from zero.)
produced by the therapy in question. They The fact that the staff’s estimate of dis-
therefore contribute little to our understand- comfort and the patient’s own report of his
ing and, above all, do not permit comparison symptoms tended to be positively correlated
of the of one therapy
effects with another. It
means little to conclude that therapy A is TABLE 1
better than therapy B because the results of
CORRELATIONS BETWEEN CERTAIN MEAsURES OF
therapy A show 90% improved and those of IMPROVEMENT
therapy B 6o% improved, when these per-
Correlation Coefficiente (r)
centages really mean that in therapy A 9o% -Th
Anit. of
of the patients lose their somatic symptoms change
Before under
and that after exposure to therapy B 6o% Measures therapy therapy
of the patients get along better with other Patient’s symptoms and
people. We agree with others(17, 20) that staff’s estimate of his dis-
comfort .45 * .44
from a research point of view the effort to Patient’s symptoms and lack
obtain over-all evaluations of improvement of self-acceptance 51 t .14
leads up a blind alley. Patient’s symptoms and lack
of self-awareness -.12 .53
To illustrate the probable nonunitary na-
* p less than .io.
ture of improvement, an example may be t p less than .05.
346 COMFORT, EFFECTIVENESS, AND SELF-AWARENESS [Nov.

to start with and to vary together under patients to which a given criterion of im-
treatment suggests that they were to some provement can be applied depends on how
extent measures of the same thing. This widely held and generally applicable the
finding led us to abandon the former meas- values are on which it is based (12, 17).
ure in subsequent research and to rely solely With respect to the baseline from which
on the patients’ own ratings as a measure of improvement is measured, 3 possibilities are
comfort( i6). the patient’s level at the beginning of treat-
That patients’ symptoms and lack of self- ment, his highest level of functioning prior
acceptance are correlated at the start of to treatment, and the highest level of which
treatment bears out the hypothesis that fail- he is capable. All are legitimate depending
ure to accept oneself contributes to feelings on what the goals of therapy are, but we have
of discomfort( 13). Despite this initial re- no hesitation in selecting the first as being
lationship the two measures do not vary to- the most widely used, as well as the easiest to
gether under the conditions of therapy in handle.
this study. Our criteria of improvement are based on
Finally, that the severity of symptoms is the values of persons most concerned with
not correlated with the patient’s lack of self- the outcome of psychotherapy-the patient
awareness initially is in keeping with the himself, his associates, and his therapist. It
general clinical observation that severity of is safe to say that every patient seeks to gain
symptoms and degree of self-awareness are from therapy more comfort in his daily
not directly related. On the other hand, the functioning. Those with whom he interacts
correlation of decrease of symptoms with in- are apt to be less concerned with how well he
crease in self-awareness is consistent with the feels than his effectiveness as a social being.
supposition that as patients get more com- Comfort and effectiveness have therefore
fortable they can dare to see themselves more been selected as the major criteria of im-
accurately. provement. Since increase in the patient’s
comfort and effectiveness are the goals of all
III the healing arts, we assume that these cri-
teria are also acceptable to all psychothera-
If improvement is not a unitary phenome-
pists. Many schools have an additional
non, the problem at once arises as to how to
therapeutic goal, namely, increase in self-un-
select criteria for measuring the effects of
derstanding. We have therefore included a
therapy. Ideally, in order to determine the
measure of self-awareness as a criterion of
effectiveness of any form of therapy we
improvement. Since self-awareness is of a
should know the conditions giving rise to the
different order than comfort and effective-
illness it purports to treat and then determine
ness it will be considered separately.
the extent to which the therapy modifies or
Before we describe the measures selected
eliminates these conditions. The causes of
psychiatric illness are so varied, complex, for comfort and effectiveness, it must be
and poorly understood that this approach is pointed out that the criteria can be measured
in any number of ways, and changes in dif-
not practicable at present.
ferent measures of the same criterion are
Fortunately the effectiveness of different
not necessarily positively correlated. For ex-
forms of therapy can be compared, regard-
ample, two possible measures of the criterion
less of the means used to bring improvement
“comfort” are the discomfort-relief quotient
about, if a consensus can be reached as to
(5), and a symptom checklist. For some pa-
what the baseline is from which improvement
tients, early in therapy, symptoms may be
is to be measured, and what criteria of im-
provement will be used. Patients may change reduced, yet discomfort in terms of the dis-
in any number of ways while undergoing comfort-relief quotient increases. The two
therapy. Which of these changes shall be measures would thus show contradictory
called improvement depends on value judg- trends though both are reasonable measures
ments as to the goals of therapy and these of comfort.
rest on value judgments as to the nature of The measures we have selected represent
mental health. The range of therapies and only one possible operational definition of
 ‘954] M. B. PARLOFF, H. C. KELMAN AND J. D. FRANK
347
each of the criteria. They seem to have a face more difficulties. As with comfort, it seemed
validity, that is, they are obviously related to wisest to try to measure the degree to which
the criterion they are supposed to define. The the patient fell short of satisfactory func-
extent to which other measures of the same tioning rather than the extent to which he
criteria would yield similar results, however, exceeded it, so that the scale actually meas-
would have to be determined by further ures ineffectiveness. Fifteen categories were
research. selected, the first 7 representing “active”
In choosing measures of comfort, it be- modes of social ineffectiveness, the next
came obvious that since this is a subjective 7 their “passive” counterparts, while the
state no one can judge it but the patient. It fifteenth was sexual adjustment. The 7
seemed wise to construct the chief measure pairs of categories are: overindependent-
of this criterion in terms not of comfort but overdependent; superficially sociable -

its opposite-discomfort-since almost all withdrawn; extrapunitive - intrapunitive;

clinic outpatients seek treatment to be re- officious-irresponsible; impulsive-overcau-
lieved of distress, rather than to be helped tious; hyperreactive-.-constrained; and over-
to move from a satisfactory state of comfort systematic-unsystematic.
to a better one. Accordingly our major meas- On the basis of material obtained from an
ure of this criterion is a check ljst-ef-ite.. interview with the patient and with another
ing symptoms based on th”Cornell Index) informant who knows him well (usually a
Each symptom is rated on 4-point scle4’f close relative) the degree of ineffectiveness
severity. Since this scale doe fpr#{243}ide for in each category is rated on a 5-point scale.
changes in the direction of positive comfort The rating is determined by the degree of in-
(as contrasted to diminution of discomfort) appropriateness of the behavior and its fre-
we are also using one modified from Lorr quency. In arriving at each rating the degree
(15), containing 12 items chiefly concerned of the ineffectiveness of the patient’s behavior
with the patient’s feelings about various as- is first rated separately with respect to vari-
pects of his interpersonal functioning. Each ous significant people in his life. These per-
is rated on a 6-point scale ranging from the sons are grouped under 5 headings-own
healthy, satisfying end of the continuum to family, marital family, occupation, social ac-
the distressing one. For example, one scale tivities, and interviewer. It should be noted
ranges from “nearly always felt adequate” to that this procedure takes account of the fact
“nearly always felt inferior ;“ another from that degree and form of a patient’s ineffec-
“saw associates as almost always friendly” to tiveness are closely related to the situations
“almost always antagonistic.” in which he finds himself. A patient may on
To test the reliability of this measure and this scale obtain fairly high scores on op..
the symptom check list, over one-third of the posed modes of ineffectiveness with different
items were rewritten in alternate form. The significant persons. For example, he may be
average correlation between the original intrapunitive with his mother but extrapuni-
items and their equivalent forms was 0.74. tive with his wife, withdrawn from work as-
Total scores as well as individual items are sociates but superficially sociable with tavern
used to evaluate improvement. The use of a acquaintances, and so on. A final single rat-
total score circumvents the objection of ing for each category is obtained by using
functional equivalence of symptoms which the ratings for each of the 5 headings as a
is urged against the use of symptoms as a guide but taking into consideration also the
measure of improvement; that is, if a pa- relative importance to the patient of the per-
tient gives up one symptom he may develop sons with whom the ineffective behavior was
another to replace it. The total score would shown and the number of persons involved.
show improvement only if there were a net Despite the inferential nature of these rat-
decline in number and intensity of all symp- ings they were found to have adequately high
toms. If a patient replaces distressing head- interjudge reliability. The average intercor-
aches with equally distressing stomach aches, relation of 4 judges was o.69, and the aver-
his total score will remain unchanged. age difference between their ratings was only
The problem of rating effectiveness posed six-tenths of a scale unit.
348 COMFORT, EFFECTIVENESS, AND SELF-AWARENESS [Nov.

IV them more accurate and more comparable.

They do the former by breaking the over-all
Since our measures are based exclusively
judgment down into subcategories which
on rating scales, certain questions and prob-
serve as checks on each other; the latter by
lems connected with the use of this tool must
forcing all the judges to use the same terms
be considered. It may be attacked in prin-
regardless of the particular patient and ther-
ciple on the grounds that results of therapy
apy involved. The use of rating scales does,
are not amenable to measurement because it
indeed, lose nuances of feeling and behavior,
is a highly personal relationship differing for
and a given set of scales may prove to have
each patient and therapist and involving
omitted the very phenomena that most re-
emergent rather than static situations(2).
quire study. If this is the case, however, the
According to this view the infinite variety
answer is not to abandon the method but to
and subtlety of patients’ feelings in therapy
develop rating scales to fill the gaps.
cannot be adequately comprehended by rat-
The scores on our scales represent either
ing scales or any other limited objective
direct judgments by the patient or judgments
measures. It is claimed that only the intui-
by staff members. The latter are based on 3
tive judgment of the therapist, untrammeled
sources of data: what the patient says about
by the chains of scientific method, can prop-
himself, the limited segment of his behavior
erly evaluate his patient’s unique therapeutic
observable in the interview, and a relative’s
experience( ro).
report. The limitations and sources of error
We grant that certain gifted therapists
inherent in such data have been well de-
might be excellent judges of the results of
scribed by Mosak( 18). The patient’s reports
therapy on their patients. Such judgments,
are subject to 2 sources of error which may
however, afford no basis for developing a sci-
be termed attentional and motivational. From
ence of psychotherapy. If the basis for a
the standpoint of attention, with the best will
judgment cannot be made explicit, real ad-
in the world it is difficult to introspect one’s
vance in knowledge is not possible(3). Every
feelings accurately or to give an undistorted
clinical judgment, however intuitive, is im-
report of past experiences, especially for a
plicitly a statistical judgment, because in
making it the clinician is comparing his pres- person inexperienced in doing this. From the
standpoint of motivation, the patient’s re-
ent patient with all the other patients he
happens to remember. That is, he is actually ports are influenced by his fears and hopes
and by his relationship to the interviewer. An
using statistical techniques but in a subjective
example of the former is the patient whose
and uncontrolled way. Actually, some evi-
dence suggests that highly circumscribed sta- toothache disappears while he sits in the den-
tist’s waiting room. Another is the patient
tistical data may be superior to the intuitive
whose fear of heart disease intensifies his
clinical approach in making prognoses, which
distress from cardiac symptoms. The pa-
is analogous to evaluating improvement(21).
Kelley( 14), for example, found that the tient’s relationship with the interviewer may
lead him, for example, to exaggerate his diffi-
Strong Vocational Interest Test consistently
predicted over-all competence in clinical psy- culties in order to convince the interviewer
he needs help, or minimize them, especially
chology better than any other predictive
measure including pooled clinical judgments. after a period of treatment, to please the
We believe that the implicitly statistical interviewer by showing how effective treat-
nature of clinical judgments should be made ment has been. Initially, distrust of the in-
explicit and that they can be objectified and terviewer may lead him to conceal, con-
quantified. Rating scales achieve both these sciously or unconsciously, certain difficulties
ends. Judgments on rating scales are clinical which he reveals as he becomes more secure,
judgments, in the sense that they are ar- so that he may appear sicker when he is
rived at after consideration of many impon- actually improving. Certain patients cannot
derable variables which may differ from case admit the existence of a problem to someone
to case and which cannot be precisely formu- else, or sometimes even to themselves, until
lated. They are an improvement over crude they have at least partly overcome it. So in
clinical judgments in that they tend to make a final interview one may learn for the first
‘954] M. B. PARLOFF, H. C. KELMAN AND J. D. FRANK 349

time of a difficulty that was present from the values based on their theories of person-
beginning. Thus changes in ratings after a ality. We therefore include a criterion de-
period of therapy might be due to a change rived from certain prevalent views of per-
not in how the patient actually felt or saw sonality and have studied its relation to
his behavior but in what he was able or will- changes in comfort and effectiveness. This
ing to report about himself. we term self-awareness.
Raters are vulnerable to the same type of It is often claimed that changes in con-
attentional and motivational errors as the pa- scious distress and overt behavior are super-
tient, usually because of a desire to see im- ficial and of little significance unless accom-
provement. For them, however, the errors panied by changes assumed to be more
are apt to be less severe, both because of fundamental, such as increased self-under-
their training and because they are not as standing, modification of underlying atti-
deeply involved emotionally. tudes, improved integration, making the un-
While these sources of error cannot be conscious conscious, and making the patient’s
completely eliminated certain steps were potential energy available to him(26). While
taken to minimize them. Many different such concepts are needed in thinking about
sources of data-interview material, direct the goals and processes of therapy, they are
observations of behavior, interviews with useful for research only to the extent that
relatives-made it possible to check the find- they can be defined operationally, measured
ings from different sources against each reliably, and clearly related to generally ac-
other. The patient’s description of his be- cepted criteria of improvement.
havior at home was evaluated with respect to Statements that a patient has undergone a
some of its aspects by reference to a rela- reorganization of personality, or their equiva-
tive’s report, as well as by the observation lent, may be defined from an operational
of the patient’s facial expressions, posture, standpoint in at least 3 ways. (i) They may
and tone of voice. The latter also served as mean that the patient has come to express
a check on his description of his feelings. attitudes more highly approved by the thera-
Finally, we believe that the accuracy of the pist than those expressed at the start of treat-
clinical judgment of any one rater can be ment. To take an extreme example, some
heightened by a conference among the differ- therapists say that even if a patient fails to
ent raters. Ratings based on a conference improve symptomatically, his treatment is
may have certain disadvantages, among them worthwhile because it makes him more in-
the danger that a member of the group with sightful or tolerant. Operationally this means
more prestige than the others, say the senior merely that he has become closer to what the
psychiatrist, may unduly influence the final therapist considers a mentally healthy per-
ratings. This danger is greatly reduced if son. Sometimes this implies that the patient
all the conferees are aware of it. It is further has become more like the therapist; probably
minimized by having each member make his more often it means that he has become more
own ratings before coming to the conference. like what the therapist would like to be(22).
Whatever the drawbacks in the conference Basic personality change in this sense is
method, they are more than counterbalanced equivalent to accepting the therapist’s value
by the opportunity given each participant to system, at least in part, and is closely anal-
reevaluate his impressions in the light of new ogous to being converted to a religious faith.
information or differences in emphasis sup- That this may be a powerful therapeutic ex-
plied by the other members. perience is recognized. Questions as to the
conditions under which a patient’s values ap-
V proach those of the therapist and the extent
Comfort and effectiveness are criteria of to which this is related to improvement in
improvement based on values generally held comfort and effectiveness, are beyond the
by patients, the persons with whom they scope of this paper.
customarily interact, and psychotherapists. (2) Statements as to the occurrence of
Psychotherapists are inclined to judge the deeper personality changes may be hypoth-
effects of therapy by reference to additional eses as to the means by which improvement
350 COMFORT, EFFECTIVENESS, AND SELF-AWARENESS [Nov.

in comfort and effectiveness was brought turn “know thyself.” Increased understand-
about. Thus it may be said that a patient ing of oneself and the nature of one’s illness
has become more comfortable and effective is an important objective of treatment in all
because he has gained insight. Two meth- chronic illness, on the assumption that the
odological difficulties with this view are that better the patient understands his condition
there are no unequivocal tests of deeper per- the better he can care for himself. Anala-
sonality changes and the relationship of such gously, all uncovering, exploratory or client-
changes to the goals of therapy remains un- centered methods of psychotherapy, as well
clear. Projective tests(9) and indices of as those based on learning theory, include in
autonomic activity(24), as possible measures their goal gain in self-awareness. The impli-
of personality change, promise to shed much cation is that the more completely and accu-
light on the psychotherapeutic process, but rately a patient knows himself, the more free
present great difficulties in interpretation. he will be of subjection to inner compulsions
Neither is it possible at present to relate and outer pressures and the better able to
surmised personality changes to changes in meet the stresses of life.
comfort or effectiveness. One judge might Accuracy and completeness of self-aware-
conclude that a certain patient had undergone ness may well be a culturally determined
a significant personality reorganization be- goal, limited largely to the intellectually ori-
cause the quality of his work had improved, ented segment of the population(27). Many
another that he had not because his relation- persons make marked strides in comfort and
ship with his wife was no better. effectiveness by forcibly directing their atten-
() Statements as to personality change tion away from themselves to the outside
are prognostic statements about the likeli- world. Others are greatly helped by embrac-
hood that improvement will be maintained ing illusion and submerging themselves in a
after therapy stops. To say that a patient cult. Moreover, it is unlikely that any psy-
has undergone a favorable reorganization of chotherapist would consider increase in self-
attitudes is to imply that he will be able to understanding to be a desirable goal for all
handle future stresses more effectively. Such types of patients. Older persons with depres-
prognoses in principle are amenable to ex- sive tendencies, for example, may need to
perimental attack by means of follow-up strengthen their favorable illusions about
studies, though these involve a formidable themselves rather than learn to appraise
methodological problem-how to compare themselves with cold objectivity. Such an
the severity of situational stresses at the time appraisal may reveal their inadequacies all
of the follow-up with those prior to treat- too clearly at a time of life when not much
ment. The problem is complicated by fre- can be done about them. Nevertheless, the
quent difficulty in determining the extent to concept of self-awareness underlies so much
which changes in situational stresses are in- thinking about the goals and processes of
dependent of treatment or caused by it. psychotherapy that some attempt to measure
Parenthetically, the importance of tempo- changes in this criterion seems worthwhile.
rary improvement should not be underesti- We have fallen back on a very simple device
mated. The fact that a diabetic, brought out which attempts to measure the patient’s abil-
of coma by insulin, will relapse if the insulin ity to appraise accurately his own behavior
is discontinued, does not mean that insulin is in the interview. This is only a small aspect
to be dismissed as affording merely tempo- of self-awareness, to be sure. Whether it
rary relief. Transient symptomatic improve- will prove a significant one, studies now under
ment with shock treatment has saved many a way will tell. An interview with the patient
depressed patient from suicide. is observed through a one-way screen. At
One goal of therapy, formulated in terms the close of the interview both patient and
of underlying personality change, is so widely observer rate the patient’s behavior in terms
held that we are including a measure of one of a check list containing 13 adjectives de-
aspect of this goal and are studying how it scriptive of behavior: alert, shy, stubborn,
varies with comfort and effectiveness. This annoyed, relaxed, friendly, polite, honest, re-
goal may be summed up by the Socratic dic- strained, overtalkative, inconsistent, f right-
 19541 M. B. PARLOFF, H. C. KELMAN AND J. D. FRANK 35’

ened, and cautious. Some of these also refer 8. Freedman, M. B., Leahy, T., et al. The inter-
to feelings. The extent to which each adjec- personal dimension of personality. J. Personality,
20: 143, 1951.
tive applied to the patient in the interview, 9. Haimowitz, Natalie R. Personality changes
ranging from “not like I behaved” to “very in client-centered therapy. In Success in Psycho-
much like I behaved,” is rated on a 4-point therapy, W. Wolff, ed. New York: Grune &
scale. Stratton, 1952.
,o. Hall, S. Barton. Psychotherapy: misappre-
The observer’s rating is accepted as the
hensions and realities. Brit. J. Med. Psychol., 26
accurate one, an assumption which seems 295, 1953.
justified on a priori grounds as well as by Holt, R. R., and Luborsky,
ii. L Research in
the high reliability of independent ratings by the selection of psychiatrists: a second interim re-
2 observers (r= .88). The accuracy of the port. Bull. Menninger Clin., x6: 125, 1952.
12. Hunt, J. McV. The problem of measuring
patient’s perception of his own behavior to-
the results of psychotherapy. Psychol. Serv. Center
wards the interviewer is given by the agree- J., I: 122, 1949.
ment between his ratings and those of the 13. Kauffman, P. E., and Raimy, V. C. Two
observer. methods of assessing therapeutic progress. J. Ab-
norm. Soc. Psychol., 44:379, 1949.
SUMMARY Kelley, E. Lowell.
14. The prediction of success
in clinical psychology. In Hoch, P. H., and Zubin,
Since improvement under psychotherapy
J. Relation of Psychological Tests to Psychiatry.
is not a unitary phenomenon, progress in New York, Grune & Sratton, 1952.
studying the effectiveness of psychotherapy ,5. Lorr, M., Jenkins, R. C., Ct al. The multidi-
depends on the use of carefully defined and mensional scale for rating psychiatric patients
generally agreed upon criteria of improve- (form for outpatient use). Washington, U. S. Vet.
Admin., 1952.
ment. In the present state of knowledge such
i6. Miles, H. W., Barrabee, E. L., and Finesinger,
criteria must be based on value judgments J. E. Evaluation of psychotherapy. Psychosomat.
by the patient, and by persons with whom he Med., 8: 83, 1951.
interacts. It is suggested that the criteria 17. Miller, James G. Objective methods of evalu-
comfort and effectiveness represent these ating process and outcome in psychotherapy. Am.
J. Psychiat., xo8: 258, 5i.
values. Means of measuring them are de-
i8. Mosak, Harold H. Problems in the definition
scribed and problems connected with the
and measurement of success in psychotherapy. In
measures considered. Some implications of Success in Psychotherapy, W. Wolff, ed. New
the viewpoint that improvement must be York: Grune & Stratton, 1952.

evaluated in terms of underlying personality 19. Parloff, Morris B. An analysis of therapeu-

tic relationships in a group therapy setting. Unpub.
changes are discussed. A simple measure of
Ph. D. thesis, Western Reserve University,
self-awareness is described which may be a
20. Sanford, Nevitt. Clinical methods: psycho-
possible indicator of certain such changes. therapy. Annual Rev. Psychol., 4 317, 1953.
2!. Sarbin, T. R. Logic of prediction in psy-
BIBLIOGRAPHY chology. Psychol. Rev., 5! : 210, 1944.
22. Schrier, Harvey. The significance of identi-
i. Appel, K. E., Lhamon, W. T., et al. Long
fication in therapy. Am. J. of Orthopsychiat., 23:
term psychotherapy. Res. Public. Ass. Nerv. Ment.
585, 1953.
Dis., 31:21, 1953.
23. Teuber, H.-L, and Powers, Edwin. Evaluat-
2. Cameron, D. E. Summary of papers XXII to
ing therapy in a delinquency prevention program.
XXV. Res. Public. Ass. Nerv. Ment. Dis., 3! : 301,
Res. Public. Ass. Nerv. Ment. Dis., 3! 138, 1953.
‘953.
3. Cohen, M. R., and Nagel, E. Introduction to . Thetford, William N. An objective measure-
Logic and Scientific Method. New York: Har- ment of frustration tolerance in evaluating psycho-
court, 1934
therapy. In Success in Psychotherapy, W. Wolff,
4. Dollard, J., Auld, F., and White, A. M. Steps ed. New York: Grune & Stratton, 1952.

in Psychotherapy. New York: Macmillan, 1953.

25. U. S. Committee on interstate and foreign
. Dollard, J., and Mowrer, 0. H. A method commerce. Health inquiry. Washington, U. S.
for measuring tension in written documents. J. Abn. Govt. Pr. Off., ‘954.
Soc. Psychol., 42:3, 1947. 26. Whitaker, Carl A., and Malone, Thomas P.
6. Frank, J. D., et al. Behavioral patterns in The Roots of Psychotherapy. New York: Blakis-
early meetings of therapeutic groups. Am. J. Psy- ton, 1953.
chiat., io8: 77!, 1952. 27. Whitehorn, John C. Psychiatry and human
7. Frank, John A. A criticalevaluation of car- values. Samuel W. Hamilton lecture, American
bon dioxide oxygen inhalation therapy in mental Psychopathological Association, New York, June ,

disorders. Am. J. Psychiat., I zo: 93, 1953. 1953.

352 COMFORT, EFFECTIVENESS, AND SELF-AWARENESS [Nov.

DISCUSSION My personal orientation is toward psychoanalytic

psychology, which the authors seem to attribute to
Joan Fleming, M. D., Chicago, 111.-With sev- the group who consider self-understanding as an
eral basic assumptions as stated in the paper, I important measure of improvement. Certainly, this
would agree that improvement is not a unitary phe- is not a goal in itself in spite of the interpretation
nomenon and that the goals of therapy vary with of what Socrates said. Socrates lived so long be-
the patient and with the therapist. However, there fore Freud that he never heard the famous story
are certain assumptions implicit in the paper with of the American who spent 2 years in Europe
which I would not agree. There is a strong undergoing a certain type of psychotherapy. This
tendency to treat “a type of psychotherapy” as if “improved” patient was met at the boat by his
it were an entity [in omitted paragraphs]. A plea friends eager to see the results. “Well, I still
is made for efforts toward adequate descriptions stutter,” he said, “but I know why.”
of “type of psychotherapies” in order to differ- Many questions could be asked regarding minor
entiate one from another for the purpose of testing points in the paper. On the whole, the authors
each separately. There is also the implicit assump- demonstrate an awareness of the complications,
tion that a “type of psychotherapy” is something objections, and sources of error in their work.
“done to” the patient. It is hard for me to believe Reading the paper left me with a feeling of disap-
that the authors believe this. Therefore I would pointment and of hope: disappointment that the
suggest that they pay some attention to this possible presentation omitted so much of the clinical data and
interpretation of their work. For, with this impli- especially that such short shrift was given to “self-
cation, many of their other statements are belied understanding.” There was a disappointment that
and their paper seems to become concerned with so much emphasis was given to evaluation of
an argument for measuring “types of psycho- “types of psychotherapy.”
therapy.” My hope is for continued study and publication
Another result of this implicit assumption is to with emphasis on the process of psychotherapy,
seem to measure the effects of psychotherapy as whatever the procedure, the goals, or the psycho-
separated from the effect of the patient’s own logical systems of the therapist.
restorative powers. The authors quote the figure
that about 3 of patients show improvement regard-
less of the therapeutic procedure. CLOSING DISCUSSION
I am not defending any type of psychotherapy.
In my opinion, psychotherapy of any type acts only Jerome D. Frank, M. D., Baltimore, Md.-We
as a catalytic agent in the natural drive toward the share Dr. Fleming’sopinion that the decisive fac-
restoration of a functioning equilibrium. This equi- tor in producing improvement lies more in the
librium may be more or less healthy in terms of patients’ restorative powers than in the form of
social values but it is an expression of the patient’s psychotherapy used, and have started research proj-
adaptive capacities. ects to evaluate attributes influencing their response
It seems to me the emphasis in this paper on to psychotherapy. But different schools of psycho-
criteria for evaluating different types of psycho- therapy do exist, each of which claims special
therapy is premature when the description of the virtues. Before trying to define their processes in
process of psychotherapy is so inadequately de- more detail we must determine whether their effects
scribed at present. This basic study must deal first really are different. If not, there is little point in
with our language which is extremely deficient in trying to describe them more precisely. A pre-
words to describe, let alone explain, the processes requisite for studying differences in both responsive-
of inter- and intrapersonal communication. The ness of patients and effectiveness of therapies is the
authors, I am sure, felt this deficiency when they development of measures of improvement by which
tried to make up the rating scales used to test their all neurotic patients and all forms of psychotherapy
criteria. can be compared, which is what we have tried to do.