Psychological Bulletin Copyright 1982 by the American Psychological Association, Inc.

1982, Vol. 92, No. 3, 581-604 0033-2909/82/9203-0581S00.75

Meta-Analysis of Comparative Therapy Outcome Studies:

A Replication and Refinement
David A. Shapiro and Diana Shapiro
Medical Research Council/Social Science Research Council Social and Applied Psychology Unit
University of Sheffield, Sheffield, England

The results are reported of a meta-analysis of 143 outcome studies, published

over a 5-year period, in which two or more treatments were compared with a
control group. Consistent with previous reviews, the mean of the 1,828 effect size
measures obtained from the 414 treated groups approached one standard devia-
tion unit, and differences among treatment methods accounted for, at most, 10%
of the variance in effect size. The impact of differences between treatment meth-
ods was outweighed by the combined effects of other variables, such as the nature
of the target problem under treatment, aspects of the measurement methods used
to assess outcome, and features of the experimental design. However, multiple
regression analysis suggested that differences between treatments were largely
independent of these other factors. Direct comparisons between pairs of treat-
ments figuring together in the same subsets of the data suggested some consistent
differences, with cognitive and certain multimodal behavioral methods yielding
favorable results. The practical implications of the conclusions drawn were lim-
ited, however, by the predominantly analogue nature of the research reviewed
and its unrepresentativeness of clinical practice.

Recent reviews of the comparative psy- & Wilson, 1978; Rachman & Hodgson,
chotherapy outcome literature (Frank, 1979; 1980).
Luborsky, Singer, & Luborsky, 1975; Smith Meta-analysis, which was used by Smith
&Glass, 1977; Smith, Glass, & Miller, 1980), and Glass (1977) and Smith et al. (1980), is
together with the results of the widely ac- a quantitative technique in which the results
claimed study by Sloane, Staples, Cristol, for each dependent variable in each study are
Yorkston, and Whipple (1975), have con- expressed as a difference between means for
verged on the conclusion that diverse ther- treated and control groups divided by the
apies are modestly, but equally, effective. On standard deviation of the control group
the other hand, proponents of behavioral scores. These effect size scores are then av-
therapies, such as Eysenck (1978), Kazdin eraged across studies, and the impact on ef-
and Wilson (1978), and Rachman and Wil- fect size of several study characteristics (such
son (1980), have denounced this conclusion as treatment method, client and therapist
as being based on misconceived aggregation variables, and measurement and design fea-
of data from unsound research studies and tures) is determined empirically. Beyond the
contradicted by the results of specific, well- psychotherapy outcome field, this method
conducted studies alleged to favor certain has been applied to such issues as the effect
behavioral methods (Bandura, 1977; Franks of school class size on achievement (Glass
& Smith, 1979); the relation between social
This study was presented at the annual meeting of the class and achievement (White, 1982); the ef-
Society for Psychotherapy Research, Aspen, Colorado, fects of cooperative, competitive, and indi-
June 1981.
Diana Shapiro is now with the North Derbyshire Dis- vidualistic goal structures on achievement
trict Psychology Service. (Johnson, Maruyama, Johnson, Nelson, &
We gratefully acknowledge helpful discussions with Skon, 1981); the validity of Fiedler's contin-
Chris Barker, Gene Glass, Paul Jackson, Mary Lee gency model of leadership effectiveness
Smith, and Peter Warr.
Requests for reprints should be sent to David A. Shap- (Strube & Garcia, 1981); interpersonal ex-
iro, MRC/SSRC Social and Applied Psychology Unit, pectancy or experimenter effects (Rosenthal
The University, Sheffield S10 2TN, England. & Rubin, 1978); self-serving bias in inter-

581
582 DAVID A. SHAPIRO AND DIANA SHAPIRO

personal influence situations (Arkin, Cooper, 1978; Rachman & Wilson, 1980)? Among
& Kolditz, 1980); sex differences in confor- the refinements embodied in the present
mity (Cooper, 1979); and sex differences in study were the following:
influencibility (Eagly & Carli, 1981). Pro- 1. Rachman and Wilson (1980) com-
ponents of meta-analysis, such as Glass (1978) plained that many of the studies included by
and Smith (1980), claim that it is less sub- Smith and Glass (1977) contained no control
jective and more precise than the conven- group. Furthermore, interpretation of several
tional, qualitative literature review or the tal- of the simultaneous comparisons between
lying of positive, neutral, and negative results behavioral and nonbehavioral methods on
in the box-score method used by Luborsky which these authors base their equivalence
et al. (1975). On the other hand, critics of argument depends on the use of control-ref-
meta-analysis argue that its superiority may erencing procedures, whereby hypothetical
be more apparent than real (Barber, 1978; control-group scores are derived from the
Elashoff, 1978; Eysenck, 1978; Gallo, 1978; results of comparable studies in the meta-
Kazdin & Wilson, 1978; Mansfield & Bussey, analysis (Smith et al., 1980). Strictly speak-
1977; Rachman & Wilson, 1980). ing, the absence of an untreated control
The contribution of meta-analysis to lit- group does not invalidate an otherwise well-
erature reviews received a balanced appraisal designed comparison between two active
from Cook and Leviton (1980). These au- treatments. Since our concern was with both
thors applaud meta-analysis for its utility in absolute and relative efficacy of different
reviewing large sets of data, its criticism of treatment methods, it was deemed cost ef-
poor practices in qualitative reviews, its abil- fective to follow the preferences of Rachman
ity to establish dependable generalizations, and Wilson (1980) and consider only studies
and its potential to offer clues to the expla- including untreated or minimally treated
nation of findings. Nonetheless, they caution controls.
that meta-analysis does not obviate the need 2. A related criticism of the Smith and
for "judgements about the definition of the Glass (1977) study was the excessive reliance
area of investigation, the relevance of meth- on the aggregation of data from disparate
odological and substantive characteristics of studies. Although Smith and Glass (1977)
studies, and the appropriate meta-analytic achieved statistical control over discernible
tools to be used" and that it can "lead to an sources of bias via regression analysis and
unwarranted psychological sense of security experimental control via the elimination
if there is a consistent replication of a rela- from a subanalysis of all studies not involving
tionship." Meta-analysis thus requires the direct comparisons, we preferred to restrict
assumption that its estimates are unbiased or our attention to studies making simultaneous
the demonstration via subsidiary analyses comparisons between two or more treat-
that "no bias of importance resulted from the ments and a control group.
principal forces from which bias would be 3. Smith and Glass (1977) omitted several
expected" (Cook & Leviton, 1980). behavioral studies considered important by
The present study was designed to repli- Rachman and Wilson (1980). Rather than
cate the Smith and Glass (1977) comparison attempt an exhaustive analysis, we took as
among treatment methods, using meta-ana- a representative sample all published, con-
lytic methods refined in the light of criticisms trolled comparisons between treatments lo-
advanced by Kazdin and Wilson (1978) and cated via Psychological Abstracts, 1975-1979.
Rachman and Wilson (1980). These criti- Although no search procedure is infallible,
cisms are discussed by Shapiro and Shapiro we are confident that any omissions did not
(1982). Our basic question is this: Do the seriously distort our findings.
results of recent controlled comparisons be- 4. Rachman and Wilson (1980) criticized
tween treatment methods support the asser- the inclusion of unpublished doctoral disser-
tion of equivalent effects, as propounded by tations in the Smith and Glass (1977) anal-
Smith and Glass (1977), or is there evidence ysis. In addition to the possibility that such
for the contrary view that certain behavioral studies may not meet the criteria of accept-
methods are superior (Kazdin & Wilson, ability for publication, their relative inacces-
 META-ANALYSIS OF THERAPY OUTCOMES 583

sibility creates problems for the critic wishing Studies of clients below the age of 16 years were ex-
to consult the source data. We therefore ex- cluded. Analogue research was included. No rational
criterion exists to distinguish absolutely analogue from
cluded dissertations from the analysis. clinical research (Kazdin, 1978; Mathews, 1978; Smith
5. Smith and Glass (1977) were criticized & Glass, 1977), so characteristics contributing to this
by Rachman and Wilson (1980) for their fail- alleged distinction, such as problem severity and dura-
ure to differentiate between different kinds tion of therapy, were entered as variables in the analysis.
of outcome measurement and for their novel Data on process variables bearing no direct relation to
client status with respect to the target problem or general
conclusion that behavioral methods tended well-being were excluded.
to be assessed by more subjective outcome Design requirements. All studies followed the con-
measures. We incorporated refinements in ventional group comparative design, including at least
the categories and dimensions used to char- three groups. In every study, two or more groups re-
ceived different psychological treatments, and another
acterize outcome measurement in an at- group (a no-treatment or, failing that, a minimal-treat-
tempt to overcome these objections. ment group) could be appropriately designated the con-
It should be noted, however, that some of trol group for the calculation of effect sizes. Designs de-
the criticisms of meta-analysis cannot be di- pendent on own-control or within-subjects comparisons
were excluded, except where data from the first treat-
rectly met by refining the methods. In par- ment block permitted between-subjects comparisons.
ticular, the claim that the source studies are Measures lacking data from the control group (such as
simply too disparate to permit generaliza- follow-up data not obtained from these subjects or ob-
tion, of which much is made by critics such tained after these subjects had received treatment) were
as Eysenck( 1978), Kazdin and Wilson (1978), excluded.
Reporting requirements. All included studies were
and Rachman and Wilson (1980), is properly published reports listed in Psychological Abstracts, 1975-
addressed not to the meta-analytic method 1979. Dissertations and other documents with restricted
as such but rather to anyone seeking to gen- circulation were excluded. All studies presented suffi-
eralize from the data in question (Shapiro cient data to permit the calculation or estimation of ef-
fect sizes. The minimum requirement here was the pro-
& Shapiro, 1982). vision of «s and p values. No reference was made to
Our goal in the present study was to con- unpublished supporting documents. Brief reports were
tribute to the debate about generalizing from included, although these had more missing data and re-
existing literature on comparative therapy quired more estimation of effect sizes than most ex-
tended reports.
outcomes by ascertaining the extent to which Search implementation. The large number of therapy
conclusions may be drawn that meet the re- titles encountered precluded a dependable computer
quirements of Cook and Leviton (1980). search (for studies comparing two or more treatments
Whatever the scientist's reservations about with a control). This was therefore undertaken manually
such generalization, it is nonetheless relied by the second author, using Psychological Abstracts in-
dexes. In the initial location of papers, an effort was
on when preferred orientations for therapy made to err in the direction of inclusion. Thus, titles
training and practice are selected on the basis and available abstracts led to the identification of some
of outcome literature (Shapiro, 1980; Shap- 400 studies; on closer examination, however, only 143
iro & Shapiro, 1977). studies, reported in the 140 papers listed in the Appen-
dix, proved suitable for inclusion. In order to avoid bias,
no use was made of reviews or bibliographies. Any errors
Method of omission in the search procedure were thus unlikely
to bias our results. Of the 143 studies, 21(15%) were
Literature Search also included by Smith et al. (1980); the two meta-anal-
Definition of area of study. We followed Smith and yses are largely independent of one another. The two
Glass (1977) in using Meltzoff and Kornreich's (1970) meta-analyses also cover different, although overlapping,
definition of psychotherapy: time periods. Smith et al. (1980) included studies ap-
pearing in any year up to 1977, whereas publication
The informed and planful application of techniques dates of the present source studies ranged from 1974 to
derived from established psychological principles, by 1979.
persons qualified through training and experience to Development and implementation of coding system.
understand these principles and to apply these tech- The coding system was developed from that used by
niques with the intention of assisting individuals to Smith and Glass (1977), of which details were kindly
modify such personal characteristics as feelings, val- supplied by Glass, amended and extended to meet criti-
ues, attitudes and behaviors which are judged by the cisms and possible limitations. All coding was carried
therapist to be maladaptive or maladjustive. (p. 6) out by the two authors, both clinical psychologists. The
first author has 10 years' postdoctoral research and clin-
We interpreted this definition to exclude studies in which ical experience; the second, 5 years' research and 2 years'
no clear target problem or complaint could be identified. clinical experience after a master's-level clinical training.
584 DAVID A. SHAPIRO AND DIANA SHAPIRO

Table 1
Treatment Method and Effect Size
No. of No. of No. of Effect
Method groups studies comparisons size SD SE Advantage

Behavioral 310 134 56 1.06 .84 .05 .32**

Rehearsal, self-control, and
monitoring 38 21 16 1.01 .90 .15 .20
Biofeedback 9 9 9 .91 .48 .16 -.33
Covert behavioral 19 13 10 1.52 1.23 .28 .22
Flooding 18 10 9 1.12 .73 .17 .11
Relaxation 42 31 27 .90 .76 .12 -.14
Systematic desensitization 77 55 50 .97 .65 .07 .04
Reinforcement 28 17 13 .97 .71 .13 .36
Modeling It 8 6 1.43 1.18 .36 .07
Social skills training 14 14 14 .85 .71 .19 .13
Study skills training 4 4 4 .26 3.9 .20 -.75
Cognitive 35 22 20 1.00 .53 .09 .40***
Dynamic/humanistic 20 16 13 .40 .33 .07 -.53**
Mixed (mainly behavioral) 40 28 24 1.42 1.03 .16 .52**
Unclassified (mainly behavioral) 18 14 14 .78 .59 .13 -.23*
Minimal 41 36 36 .71 .73 .11 -.56***

* /; < .05. ** p < .01. *** /; < .001.

Most coding operations were refined after pilot coding but also involved a broader range of target-relevant ac-
of some 20 studies, which were receded with the final tivities than could readily be subsumed within the latter
system. Three codings, however, were made only after category. Cognitive methods included the cognitive and
relevant information had been obtained from almost all cognitive-behavioral methods associated with the names
the studies and categories devised on the basis of this of Ellis (1962), Beck (1976), and Meichenbaum (1977);
information. These codings were of treatment method, packages including these methods alongside conven-
target problem, and measurement technology. In de- tional behavioral methods were classified as cognitive,
veloping the system and in the coding itself we con- since these methods in any case include some elements
stantly took care to avoid reference to outcome data, so of conventional behavior therapy. Dynamic and hu-
as to prevent bias. All effect-size calculations and esti- manistic methods included therapies based on psycho-
mations were performed either subsequent to the coding dynamic, client-centered, or other humanistic ratio-
of a study on other variables or by the author not re- nales, including encounter and gestalt. Mixed treatments
sponsible for coding the study in other respects. were methods that defied classification into any one of
Independent coding of 20 studies by both authors re- the categories because they contained elements of more
sulted in at least 80% agreement on each coding dimen- than one, and unclassified methods were too dissimilar
sion. to the existing categories to belong in any one but too
Treatment and therapist variables. Treatments were infrequently represented to form a category of their own.
classified into 15 categories, as shown in Table 1. Be- Both of the latter groups were predominantly behavioral
havior rehearsal, self-control, and self-monitoring ex- in orientation. 1
plicitly entailed active self-direction by the client, in con- Subsidiary treatment variables. Also coded were the
trast with the allegedly passive conditioning paradigms mode of therapy (individual, group, couples/families, or
of desensitization and reinforcement methods (D'Zurilla mixed), therapist experience (in years, with undergrad-
& Goldfried, 1971; Goldfried, 1971; Kanfer & Karoly, uates coded 0, postgraduate trainees coded 2 unless stage
1972; McFall & Twentyman, 1973). Biofeedback, flood- of training was specified, and doctoral level therapists
ing, systematic desensitization, and modeling were all coded 5 unless length of subsequent experience could
defined in the conventional manner of the behavioral be inferred), and duration of therapy (in hours, with a
literature (Kazdin & Wilson, 1978). Covert behavioral session counted as 1 hour unless less than 40 minutes).
methods included covert sensitization (Cautela, 1967), Finally, the reproducibility of the treatment method was
covert reinforcement, and covert modeling but excluded
flooding and desensitization, even if wholly imaginal.
1
Relaxation included meditation procedures as well as Supplementary documentation, comprising tables
conventional relaxation based on Western rationales. listing the source studies in which each treatment occurs
Reinforcement methods included both positive and aver- together with the frequency distributions, means, and
sive conditioning using overt stimuli. Social skills train- standard errors obtained at each level of each code, is
ing often included elements similar to treatments coded obtainable from JSAS Catalogue of Selected Documents
as behavior rehearsal, self-control, and self-monitoring in Psychology, 1982, 12, 46. (Ms. No. 2506)
 META-ANALYSIS OF THERAPY OUTCOMES 585

coded. The maximum code, 3, required a full description coded 1, psychometric measures and behavioral ratings
given in the text or available from the author or another were coded 2, behavioral counts were coded 3, and real-
cited work, or reference to a manual or direct replication life performance measures and physiological data were
of another author's well-described method. The inter- coded 4.
mediate code of 2 was assigned for brief descriptions of Design quality and internal validity. Aspects of in-
method or reference to adaptation of a method well de- ternal validity included mode of assignment of clients
scribed elsewhere. The lowest code, 1, was assigned (coded on a 4-point scale representing increasing degrees
where the description was minimal and imprecise. . of control: nonrandomized = 1, randomized without
Client variables. The presenting target problem was matching = 2, randomized with matching of groups ^
recorded for every study, and these targets were grouped 3, randomized with matching sets = 4), assignment of
into 5 target classes (see Table 2). Where it could be therapists (categorized as for clients but with further cat-
positively made, psychiatric diagnosis of the client egories involving a single therapist in all groups and
groups was also recorded. Rather than assume that all counterbalancing therapists across all groups), attrition
minor disorders such as specific phobias and perfor- rates in treated and control groups (coded for each mea-
mance anxieties should be classified as neurotic, psy- sure separately), pretreatment equivalence on each out-
chiatric diagnosis was only recorded when offered by the come measure (no significant difference between groups
author of the source study. Although Smith and Glass on any measure = 3, no difference on measure in ques-
(1977) also coded client intelligence and client-therapist tion but other measures indeterminate or showing dif-
similarity, examination of source studies suggested that ferences = 2, differences on measure in question =•• 1),
the only relevant data available in most cases is the and degree of blindness of the person obtaining the out-
clients' educational level (low/mentally retarded, aver- come data to the treatment received by the client (single
age, high, or mixed). Mean client age was also coded. blind = 3, knew group composition = 2, acted as ther-
We also coded two important aspects of the analogue- apist = 1). Other design variables included sample size
clinical continuum. Severity/screening was coded tin a of both treatment and control groups, and number of
3-point scale (Clinical severity = 3, 80th percentile on therapists involved in the treatment group.
target measure or thorough screening = 2, below 80th Effect size calculation. The methods of calculation
percentile or inadequate screening = 1), and the source and estimation followed those applied by Smith et al.
of clients was classified as solicited, committed, or sought (1980, Appendix 7), kindly made available prior to pub-
treatment. lication by Glass. The method used to obtain the effect
Contextual variables. The setting of treatment (lab- size was coded. Wherever possible, mean difference over
oratory, student health, paraprofessional, clinical, or control standard deviation was used. Failing sufficient
home) was recorded, together with available information data for this, error terms were calculated from statistics
on the presence or absence of concurrent medication. such as F, t, Neuman-Keuls g, and so on. Failing even
Outcome measurement. Outcome measures were this amount of information, a conservative estimate was
coded in four respects in place of the two codings applied obtained from supplied ns and p values; these estimates
by Smith and Glass (1977), The Smith and Glass (1977) were more accurate and less conservative if means were
"type of outcome measure" coding was renamed do- supplied. Also coded in this category were a few cases
main, in accordance with its reference to the area of in which effect sizes were calculated by assigning nu-
functioning (fear/anxiety, self-esteem, adjustment, work/ merical values to frequency categories reflecting at least
school achievement, personality traits, social behavior, three levels of outcome. Dichotomous frequency data
emotional/somatic disorders, and physiological stress) were submitted to simple prohibit analysis (Finney,
tapped by the measure. For summary analysis, domain 1972). Also coded was the source of data (posttest scores,
was recorded in terms of the assumed tractibility of the covariance adjusted posttest, prepost difference scores,
functions assessed, with fear/anxiety and self-esteem or other). Finally, we also recorded the number of null
coded 1, and all others coded 0. The Smith and Glass effect sizes for each treatment as a partial check for spu-
(1977) dimension of reactivity was differentiated into rious inflation of effect size by selective reporting. We
two concepts, Relatedness to the goals of treatment, con- recorded the number of measures mentioned by the au-
sidered a virtue by behavioral researchers, was coded thors on which data were not available because they
separately on a 4-point scale of specificity (projective failed to differentiate treated from control groups, and
measures = 1, standardized trait measures = 2, directly for which effect sizes hence were not calculated. Inter-
related measures or physiological measures where target mediate cases were not included in this tally because
is nonphysiological = 3, behavioral tests or physiological although the source paper indicated that treated groups
measures of physiological targets = 4). The remaining were at least marginally superior to controls, effect sizes
procedural aspects of fakeability and liability to the in- could not be estimated.
fluences of demand characteristics were coded on a re- Data reduction. Consideration of each outcome mea-
duced, 4-point scale of reactivity, on which therapist rat- sure and each occasion of measurement yielded varying
ings were coded 4, simple self-reports and behavior in numbers of effect size measures for each treatment
the presence of a nonblind experimenter were coded 3, group. Smith et al. (1980) reduced their data by elimi-
blind ratings and standardized tests were rated 2, and nating redundant measures; that is, measures matching
physiological and achievement measures were coded 1. others in measurement characteristics and effect size or
An aspect of measurement insufficiently examined by subtest scores on multifactorial instruments that could
Smith and Glass (1977) was the type of measurement be averaged without apparent loss of information. We
technology employed. This was coded on a 4-point soft- preferred to retain all data, except that measures per-
hard scale, whereby self-ratings and projective tests were mitting only a relatively imprecise effect size estimate
586 DAVID A. SHAPIRO AND DIANA SHAPIRO

were discarded if the study provided more complete data study, and / tests were used to evaluate the difference
on enough other measures, thus yielding a more accurate from zero of the resultant study-by-study difference
indication of effect size. Analyses confined to parameters scores.
that were invariant across all effect sizes obtained for a Treatment X Client interactions. Cross-tabulation of
given treatment group were conducted, with each treat- effect size data in terms of the broad classes of treatment
ment group represented only once, by the mean of all type and target class was evaluated via two-way analysis
effect sizes obtained for that group. of variance. Interaction effects from such analysis would
serve as evidence of differential effectiveness of treat-
ment approaches with different types of presenting prob-
Analysis lems.
Multiple regression analysis. In order to assess the
Data analysis used Release 8.0 of SPSS (Statistical extent to which treatment methods influence outcome
Package for the Social Sciences; Nie, Hull, Jenkins, independently of other study parameters, sets of vari-
Steinbrenner, & Bent, 1975) and proceeded through the ables representing treatment, client, measurement, and
phases outlined below. Issues relating to the application design variables were entered sequentially in a series of
of inferential statistics to meta-analysis are discussed by regression equations, with effect size as the dependent
Glass, McGaw, and Smith (1981), Shapiro and Shapiro variable. These analyses served to supplement the lim-
(1982), and Strube and Hartmann (1982). ited number of direct comparisons between treatment
Overall effect size. Descriptive statistics were ob- methods that were available by comparing the impact
tained for the complete sample of effect sizes, for com- on effect size of different treatments while controlling
parison with Smith and Glass's (1977; Smith et al., 1980) statistically for the influence of other study parameters,
estimates. without restricting the analysis to direct controlled com-
Description of the data base. The data of the study parisons.
were characterized in terms of distributions on all the
coded parameters. This provided a profile of the com- Results
parative outcome literature undertaken during the pe-
riod covered by the study and thus indicated the likely
generalizability of its findings. Overall Effect Size
Correlates of effect size. Univariate breakdowns and
correlations were obtained to ascertain the relationship The mean value of the 1,828 effect sizes
of each parameter with effect size. The parameters were was .93, and the standard deviation, 1.16.
conceptualized as falling into five groups, comprising The mean value implies that the average
treatment and therapist variables, client variables, con-
textual variables, measurement variables, and design
treated client lies at the 82nd percentile of
variables. untreated clients. If all the 540 null effect
Comparative evaluations of treatment methods. To sizes recorded are assumed to take the value
overcome the confounding influence of other variables zero, then the overall mean effect size is re-
covarying across studies with treatment method, the duced to .72. Omission of the 177 effect sizes
mean effect size obtained for each treatment was com-
pared with the mean effect size of all other treatments (M = .57) associated with minimal or pla-
found in the studies involving that treatment. This com- cebo treatment raises the overall mean effect
parison was performed as follows, for each treatment size from .93 to 1.03. Of the 1,828 effect sizes,
method in turn. First, the data from all studies involving 206 (11.3%) were negative. The 414 mean
the given treatment were selected. Then advantage
scores were obtained from these data by calculating, for effect sizes, of which one represented each
each of the selected studies, a mean effect size for the of the 414 treatment groups in the study,
treatment in question, and a mean effect size for all other yielded a mean of .98 and a standard devia-
treatments combined. The latter mean was subtracted tion of .80.
from the former for each study in turn, and the mean
of these difference scores was evaluated via a / test. These
calculations were performed twice, the second calcula- Description of the Data Base
tion involving the prior elimination of data from min-
imal treatments. Associated with each of the 1,828 sepa-
Controlled comparisons between treatment methods. rately coded effect size measures were scores
A similar method was applied to controlled comparisons
between treatment methods. To ensure dependable re-
on the parameters outlined above. Parame-
sults, comparisons between two methods were only con- ters that were by definition constant for all
sidered if they were performed in at least four source effect size data obtained on a single treatment
studies and yielded at least 10 effect sizes for each of the group were analyzed on a one-score-per-
two treatments. After data from all studies making a group basis over the 414 groups included in
given comparison were selected, differences were eval-
uated following similar methods to those used for com- the study. Statistical analysis of the 1,828
parative evaluation of single methods. A mean effect size effect size measures must be interpreted with
was obtained for each of the two treatments in each caution in view of the nonindependence of
 META-ANALYSIS OF THERAPY OUTCOMES 587

the mean 4.42 measures obtained from each 2. The predominant mode of therapy was
group (see Footnote 1). group (52% of groups), and the average ther-
Noteworthy features of the data base, as apist had some 3 years of experience, the
revealed by the distributions summarized in level of an advanced postgraduate student.
Tables 1, 2, and 3 include the following: Therapy typically lasted for around 7 hours,
1. Table 1 shows that the source studies and most procedures were at least moder-
were primarily of behavioral methods, with ately reproducible.
systematic desensitization the most widely 3. Table 2 shows that the most common
represented method, followed by relaxation target problems were performance anxieties,
and rehearsal/self-control/monitoring. Cog- followed by physical and habit problems, so-
nitive and minimal therapies were less widely cial and sexual problems, and phobias.
represented, and verbal methods figured very 4. Positive psychiatric diagnoses were
little in the data. made for only 6% of treatment groups, and

Table 2
Treatment Targets and Effect Sizes for 414 Groups
Target N Effect size SD SE

Anxiety and depression 30 7 .67 .62 .10

General anxiety 7 .16 .43 .16
Mixed phobias and anxiety 4 .95 .38 .19
Tension 9 1.10 .77 .26
Depression 10 .51 .33 .10
Phobias 76 18 1.28 .88 .10
Agoraphobia 3 2.03 .11 .06
Fear of flying 3 1.05 .90 .52
Snake phobia 40 1.23 .68 .11
Spider phobia 13 1.08 .59 .16
Rat phobia 5 2.94 1.59 .71
Dental phobia 9 .97 .71 .24
Cardiac catheterization 3 .40 .60 .35
Physical & habit problems 106 26 1.10 .85 .08
Hypertension 2 1.35 .98 .65
Headache 12 .90 .44 .12
Obesity 30 1.09 .91 .17
Underweight 2 .44 .89 .63
Heroin abuse 4 .48 .04 .02
Smoking 30 1.24 .99 .18
Nailbiting 10 1.43 .51 .16
Insomnia 12 1.40 .75 .22
Stuttering 4 -.10 .42 .21
Social and Sexual problems 76 18 .95 .75 .09
Assertion 21 1.02 .57 .12
Social inadequacy 15 .69 .58 .15
Class discussion difficulty 2 .25 .16 .11
Dating anxiety 20 1.13 .89 .20
Marital communication 4 .73 .78 .39
Sexual difficulties 14 1.02 .91 .24
Performance anxieties 126 30 .80 .71 .06
Academic underachievement 14 .69 1.48 .40
Text- anxiety 70 .75 .41 .05
Public speaking anxiety 38 .95 .76 .06
Vocational indecision 4 .57 .40 .20
Note. .F targets = 3.44, p < .01; if targets = .21; F target classes = 6.38, p < .01; if target classes = .06.
588 DAVID A. SHAPIRO AND DIANA SHAPIRO

age (Mdn = 20.25 years) and educational effect sizes (M = 1.42), followed by behav-
level reflect a predominance of college stu- ioral and cognitive methods (M = 1.06 and
dents among the clients, in only 17% of 1.00, respectively), unclassified treatments
whom did the target problem reach clinical (also predominantly behavioral; M = .78),
severity, and who were predominantly solic- minimal or placebo treatments (M = .71),
ited for treatment; information was rarely and dynamic and humanistic (verbal) ther-
provided concerning concurrent medication, apies (M = .40). Among the behavioral meth-
and most data were obtained in laboratory ods, it is noteworthy that the most frequently
settings. studied method, systematic desensitization,
5. The most frequent domains of out- is associated with only average outcome
come measurement were fear/anxiety (39%) (M = .97), whereas relaxation alone appears
and social behavior (21%); most measures almost as effective (M = .90). Although fig-
exhibited moderate or high specificity, and uring in only four studies, the poor perfor-
moderately low reactivity. Psychometric in- mance of study skills training is striking.
struments (41%) and self-ratings (27%) were Treatment mode was significantly related to
the predominant measurement techniques. effect size, F(3, 403) = 3.91, p<.01. Al-
6. The typical design involved random though individual therapy appeared the most
assignment of clients (90%, with 32% involv- effective mode (M = 1.12), it was closely fol-
ing matching) with relatively little attrition, lowed by the predominant group mode (M =
and the same therapist or therapists in each .89), and the only striking result was obtained
of the treatments compared (80%). The ma- for the rarely used couple/family mode (M =
jority of the data showed satisfactory pre- .21). Therapist experience, ranging from 0 to
treatment equivalence, and more than half 8 years, was negatively correlated with effect
(53%) of the available data indicated that the size, /-(269) = -0.14, p < .01. The distribu-
measures were single blind. The majority of tion suggests a downward trend in mean ef-
the effect sizes (60%) were obtained via the fect size with increasing therapist experience
preferred formula of mean difference over from up to 1 year (M = 1.17) to 4 years (M
control standard deviation, using posttreat- = .62), which was reversed for the most ex-
ment scores. perienced therapists (M = .87), although their
outcome was still inferior to that obtained by
Correlates of Effect Size neophyte therapists. However, multiple
regression analysis of the data from the 271
In interpreting the relations between coded groups for which therapist experience was
parameters and effect size, we added careful recorded suggested that this was due to a ten-
description of the obtained distributions (see dency for studies involving less experienced
Footnote 1) to summary data in the form of therapists to be concerned with target prob-
the correlations presented in Table 3. Atten- lems yielding higher effect sizes. Entered
tion was focused in particular on the con- alone, therapist experience accounted for 2%
ditions associated with substantial deviation of the variance in effect size, R2 = .020, F(\,
from the overall mean effect size of .93. 269) = 5.61, p < .05; entered after dummy
Treatment and therapist variables. The variables representing 23 target problems,
mean effect sizes for each treatment method which themselves accounted for almost 20%
and for the types into which they were of the variance, R2 = .190, F(23, 247) = 2.52,
grouped are shown in Table 1. The effect due p < .01, therapist experience no longer pre-
to treatment methods was highly significant, dicted effect size, R2 change = .0007, F < 1.
F(14, 339) = 3'.43, j? < .001, and about 10% The duration of therapy also appeared neg-
of the variance between groups was ac- atively related to outcome until the longest
counted for (ij2 = . 11). The means associated durations were reached, although there was
with treatment methods ranged from .26 for no evidence of a linear relation, r (412) = .05,
study skills training to 1.52 for covert be- p < .10. There was no relation between re-
havioral methods. The data for treatment producibility of treatment and effect size,
types show mixed methods (predominantly r(412) = .03, p < .10.
behavioral in orientation) to yield the largest Client variables. The effect size for each
 META-ANALYSIS OF THERAPY OUTCOMES 589

target, and for the five target classes into results discrepant from those obtained for the
which they were grouped, are shown in Table class to which they were assigned included
2. Differences between targets were highly mixed phobias and anxiety (M = .95), and
significant, F(29, 384) = 3.53, p < .001, and tension (M = 1.10), cardiac catheterization
accounted for over 20% of the variance in stress (M = .40), class discussion difficulty
effect sizes, i]2 = ,21. The means indicate (M = .25), and marital communication dif-
large effects for phobias (M = 1.29) and small ficulty (M - .73). The class of physical and
effects for anxiety and depression (M = .67), habit problems yielded divergent results for
with intermediate results for physical and targets within it, but there was no simple
habit problems, (M = 1.10), and perfor- differentiation between effect sizes for the
mance anxieties (M = .80). Although the somatic symptoms and for the addictive or
grouping into target classes was inevitably habit disorders represented. There were no
somewhat arbitrary, it appears reasonably statistically dependable relations with diag-
well justified by the outcomes obtained; dif- nosis, education or source of clients. How-
ferences between the five classes accounted ever, effect size was positively related to age
for 5.9% of the variance of the scores ob- of client, r(412) = .17, p < .01. There was no
tained for the 414 groups. Targets yielding linear relation between severity/screening

Table 3
Means or Proportions on Background Variables and Their Correlations
With Effect Size for 414 Groups
Mean/
Variable proportion SD r n
Therapy
Mode (individual vs. others) .41 .15** 407
Therapist experience (years) 2.91 1.93 -.14* 271
Duration (hours) 6.89 4.26 .05 396
Reproducibility 2.27 .65 .03 414
Client
Severity/screening 1.65 .75 .04 393
Source (solicited vs. others) .89 .05 394
Age (years)" 24.57 8.36 .04 393
Education (high vs. others) .82 .00 344
Contextual
Medication (present vs. absent) ,33 -.29* 48
Setting (lab vs. others) .71 .08 390
Design
Treatment n 11.98 7.12 -.14** 414
Control n 11.88 6.50 -.21** 414
Assignment of clients 2.36 .85 .05 388
N of therapists 2.14 1.31 -.06 310
Attrition of treated group (%) 10.68 13.49 -.06 1,483
Attrition of control group (%) 9.19 13.46 .00 1,486
Blindedness 2.24 .87 -.10** 1,006
Equivalence 1.26 .54 .07* 1,401
Measurement
Domain (tractability) .48 .09** 1,828
Reactivity 2.17 .62 .11** 1,828
Specificity 3.08 .63 .07* 1,828
Measurement technology 2.02 .90 -.11** 1,828
Follow-up (months) .79 2.36 -.01 1,828
a
Median age = 20.25 years.
*p< .05. ** p < .01.
590 DAVID A. SHAPIRO AND DIANA SHAPIRO

and effect size, r(390) = .04, p > .10, with larger effect sizes appeared to be associated
the largest effects (M = 1.13), among mod- with either no attrition (M = .96), or a large
erately screened clients. attrition rate, with means of .91 and 1.22 for
Contextual variables. There was a signif- measures suffering between 30% and 40%
icant effect due to setting, F(4, 385) = 2.53, and over 40% attrition, respectively; smaller
p < .05. Effect sizes for home-based treat- effect sizes were associated with intermediate
ments (M = 1.70) were larger than for other amounts of attrition from the control group,
settings. Although information on concur- with means ranging from .70 to .83. Pretreat-
rent medication was not often provided, ef- ment equivalence was positively correlated
fect sizes were larger without medication with effect size, r (1399) = .07, p < .05. Blind-
(A/= 1.37) than with it (M = .72), ^(1, 46) = edness of the data-gatherer was negatively
4.11, p < .05. correlated with effect size, r(1004) = -.10,
Measurement variables. Each of the mea- p< .01.
surement variables was significantly related Effect size calculation. There was some
to effect size. With respect to domain, F(7, suggestion of a relation between type of effect
1820), p < .001, the most striking feature was size calculation and the effect size obtained,
the weak effect associated with achievement F(3, 1824) = 1.12,/j = .095, and (intendedly
measures (M = .28), and personality traits conservative) estimates from p values yielded
(M = .52). There was a tendency for more relatively low effect sizes (M = .76). The
specific measures to yield more favorable source of the effect size was modestly asso-
outcomes, r(1826) = .07, p < .01, Similarly, ciated with its magnitude, 7^(3, 1824) = 3.60,
more favorable outcomes were associated p< .01, but this effect was largely attributable
with relatively reactive measures, r(1826) = to the four effect sizes coded as other (M =
.11,7? < .001. The results for measurement .66). The conventional calculation of effect
technology, F(6, 1821)= 11.09, p<.001, sizes from posttest data tended, if anything,
suggested relatively favorable outcomes for to be associated with larger effect sizes (M =
self-ratings and behavioral counts, and rela- .98) than covariance-adjusted posttests (M =
tively unfavorable outcomes for hard, non- .75) or pre-post difference scores (M = .80).
psychological data and physiological and Summary of correlations with effect size.
psychometric measures. The significant ef- Table 3 shows the correlations obtained be-
fect due to length of follow-up, F( 11, 1816) = tween 23 variables and effect size. Of the vari-
5.22, p < .001, was not accounted for by any ables presented in Table 3, 11 yielded sig-
linear relation, r(1826) = .006. The distri- nificant correlations with effect size, although
bution suggests a U-function with somewhat the significance of the smallest correlations
weaker effects for intermediate lengths of fol- could be spurious, in view of the noninde-
low-up (M = .81), than for immediate post- pendence of the data obtained on each of the
testing (M = .95), or follow-up after 4 months 414 treatment groups. In sum, these corre-
or more (M = 1.04). lations suggest that effect sizes were larger for
Design variables. Most design variables studies involving individual therapy admin-
bore at least a suggestive relation to effect istered by relatively inexperienced therapists
size. Assignment of clients was unrelated to and with no noncurrent medication, in which
effect size, F(l, 384) = 1.56, p > .10. Simi- the design involved relatively small numbers
larly, assignment of therapists was unrelated of clients, who were well-matched before
to effect size, F(5, 294) = 1.14, p > .25. Effect treatment and were assessed via specific, re-
size decreased with increased number of both active, soft measures of tractable psycholog-
treated clients, r(412) = .14, p < .001, and ical domains that were administered non-
control clients, r(412) = .21, p < .001. A blind. Of course, these correlations are all
positive correlation was obtained between quite small, and zero-order correlations alone
effect size and attrition in the treated group, permit no conclusion as to the independence
r(1482) = .07, p < .01. The relationship be- or otherwise of the effects reported.
tween effect size and attrition in the control Comparative evaluation of treatment
group, F(5, 1480) = 2.77, p < .05, was less methods. The final column of Table 1 shows
readily interpreted, r(1484) = .02, p> .25; the mean of the study-by-study difference
 META-ANALYSIS OF THERAPY OUTCOMES 591

scores between each treatment and the treat- strong effects, we further examined the 13
ments with which it was directly compared, studies comparing dynamic and humanistic
These data indicate significant superiority of therapies with other treatments. In 8 studies,
mixed, /(23) = 2.61, p < .01, and cognitive, the dynamic or humanistic method was pre-
t(l9) = 5.27, p < .001, methods. There was sented as containing no therapeutic elements
a substantial advantage of behavioral meth- unique to itself and could indeed be viewed
ods as a whole (M= .32), t(55) = 3.40, p < as a straw man. In the remaining 5 studies,
.01, and significant negative scores indicated specific therapeutic elements were ascribed
inferior results for minimal treatments, to the dynamic or humanistic therapy. The
/(35) = 6.36, p < .001, dynamic and human- mean advantage score for the former group
istic therapies, Z(12) = 2.13,p < .05, and un- was —.40, and for the latter group, -.77 (ris-
classified methods, £(13) = 2.55, p < .05. ing to-.26 with the exclusion of one study
Elimination of minimal treatments made lit- yielding an extreme score of -3.06). Despite
tie difference to these comparisons, except the applicability of the straw man argument
that the inferiority of unclassified treatments to much of this data, it does not wholly ac-
became rather more marked (M = —.38), count for the modest showing of dynamic
t(\ 3) = 4.47, p < .001. In view of the sugges- and humanistic therapies,
tion by Smith et al. (1980, p. 119) that insight Controlled comparisons between treatment
therapies are often set up as a "straw man" methods. Table 4 shows the results of the
or quasi-control group not expected to yield 21 direct comparisons based on at least 4

Table 4
Controlled Comparisons Between Treatment Methods
Method B

Systematic Social skills

Method A Relaxation desensitization training Mixed Minimal

Rehearsal, self-control, and monitoring • .64***

(6)
Biofeedback -.20 -.72*
(8) (4)
Covert behavioral .54*
(4)
Relaxation -.24 -.59* .29*
(13) (5) (4)
Systematic desensitization -.28** .50****
(7) (15)
Reinforcement .32 .14
(5) (5)
Social skills training .06 . .37**
(5) (4)
Cognitive .53**** .28 .68**
(9) (4) (7)
Dynamic/humanistic .35 —.93
(4) (4)
Unclassified -.16 .02 .46**
, (4) (6) ^ (6)

Nole. All comparisons are Method A - Method B differences, based on 1 difference score per study and N of studies
given in parentheses.
* p < .10. ** p < .05. *** p < .01. **** p < .001.
592 DAVID A. SHAPIRO AND DIANA SHAPIRO

Table 5
Breakdown of Effect Sizes by Treatment Type and Target Class
Treatment type
Dynamic/
Target class Behavioral Cognitive humanistic Minimal Total
Anxiety/depression
ES .74 1.34 .40 .38 .67
n 21 1 5 3 30
Phobias
ES 1.46 .92 .66 1.28
n 56 9 11 76
Physical/habit problems
ES 1.19 .37 .37 1.07 1.10
n 80 5 5 14 104
Social/sexual problems
ES 1.08 1.19 .36 .55 .97
« 51 9 8 6 74
Peformance anxieties
ES .81 .97 .65 .36 .80
n 102 15 2 7 126
Total
ES 1.06 .94 .40 .71 .98
n 310 39 20 41 410

Note. ES = effect size; n = number of groups.

studies yielding a total of at least 10 effect to relaxation,/(4) = 231, p < . 10, and to bio-
sizes for each of the two treatments under feedback, t(3) = 2.45, p< . 10, but these low-
comparison. In assessing comparisons be- power comparisons were only marginally sig-
tween purportedly active treatments and nificant. No other comparisons approached
minimal or placebo methods, a directional, statistical significance. Of particular interest,
one-tailed test was adopted, in view of the because based on a fair number of studies,
a priori expectation that active treatments were the apparent equivalence of biofeed-
would be more effective. Using this criterion, back and relaxation, t(l) = .74; and of
six methods were significantly superior to systematic desensitization and relaxation,
minimal or placebo treatments. These were t(\2) = .94.
rehearsal/self-control, /(5) = 3.54, p < .01;
covert behavioral, ~t(3) = 2.63, p < .05; sys- Treatment X Client Interaction
tematic desensitization, £(14) = 4.71, p <
.001; social skills training, t(3) = 3.20, p < Table 5 shows the breakdown of the mean
.05; cognitive therapy, /(6) = 3.11, p < .05; effect sizes obtained for 410 groups by treat-
and unclassified treatments, t(5) = 2.81, p < ment type and target class. Excluded from
.05. Marginally significant was the superi- this analysis were the 4 groups classified as
ority of relaxation over minimal treatments, of mixed type, because they combined major
Z(3) = 2.32, p< .10. All comparisons between elements of more than one of the coded
active treatments were evaluated via two- types. A regression-model analysis of vari-
tailed tests. Cognitive therapy was superior ance (ANOVA) of these data yielded a main
to systematic desensitization, /(8) = 2.86,p < effect due to treatment type, F(3,391) = 5.51,
.05. Mixed methods also were superior to p < .001. There was no significant effect due
systematic desensitization, t(6) = 2.86, p < to target class (F < 1), or to the two-way in-
.05. Mixed methods also appeared superior teraction, F(ll, 391) = 1.41, p < .10. The
 META-ANALYSIS OF THERAPY OUTCOMES 593

only suggestive evidence of outcomes dis- justment measures (M = .28) and emotional/
crepant from that predicted from the mar- somatic disorder (M = .36); the fair showing
ginal totals occurred in relation to physical of dynamic and humanistic methods with
and habit problems, which appear effectively physiological stress measures (M = .86); and
treated by minimal treatments (M = 1.07) the relatively good results of cognitive ther-
but ineffectively treated by cognitive meth- apy (M = 1.39), and poor results of minimal
ods (M = .37). treatments (M = .06), with achievement
Table 6 shows a similar breakdown by measures. Inevitably, the results of the break-
treatment type and domain of outcome mea- downs of Tables 5 and 6 are dominated by
surement. In addition to main effects corre- the preponderance of behavioral methods in
sponding to those reported earlier, ANOVA the data.
revealed a significant interaction, F(2l,
1786)= 1.77, /7<.02. Although based on Multiple Regression Analysis
relatively few effect sizes, noteworthy cell
means in Table 6 include the relatively poor Dummy variables were created to repre-
performance of cognitive methods with ad- sent 14 treatment methods, 3 treatment

Table 6
Breakdown of Effect Sizes by Treatment Type and Domain of Measurement
Treatment type

Dynamic/
Domain Behavioral Cognitive humanistic Minimal Total

Fear/anxiety
ES 1.14 .93 .45 .72 1.06
n 558 78 11 72 719
Self-esteem
ES 1.19 1.24 .35 .30 .95
« 103 14 31 16 164
Adjustment
ES 1.50 .28 .09 .16 .96
n 39 11 8 8 66
Achievement
ES .26 1.39 .45 -.06 .28
n 118 6 3 12 139
Personality traits
ES .53 1.03 .89 .32 .52
n 23 1 1 5 30
Social behavior
ES .99 1.07 .31 .62 .95
n 264 87 13 31 391
Emotional/somatic disorder
ES .97 -.36 .18 .87 .89
n 170 5 10 20 205
Physiological stress )
ES 1.02 1.63 .86 .44 .97
n 82 5 4 13 104
Total
ES 1.10 .97 .35 .57 .93
n 1357 203 81 177 1818
Note, ES = effect size; n = number of effect sizes.
594 DAVID A. SHAPIRO AND DIANA SHAPIRO

Table 7
Impact of Five Sets of Variables on Effect Size
1,802 measures 407 groups

Alone With all sets Alone With all sets

• Set R2 F A*2 F R2 F A# F
Methods .064 8.67*** .049 7.85*** .112 3.54*** .089 3.67***
Targets .111 7.66*** .076 5.88** .217 3.60*** .091 1.77**
Sample size .046 43.47*** .034 38.11*** .068 14.78*** .049 13.82***
Mode .007 13.36*** .005 11.21*** .021 8.70** .009 5.08*
Measurement variables .043 20.04*** .037 20.74***

*p< .05.** p< .01. ***

types, and 29 target problems. Ideally, a mul- p < .01, when the other sets of variables were
tiple regression analysis would have included taken into account. On the left in Table 7
alongside these dummy variables all other are presented the results of an analysis of the
variables showing evidence of association 1,802 effect size measures, in which four
with effect size. Unfortunately, missing val- measurement variables were also included.
ues arising from incomplete reporting by Overall, only 22% of the variance was ac-
source study authors reduced the number of counted for by the five sets of variables, (R2 =
variables on which sufficient data were avail- .219), F(50, 1751) = 9.79,p< .001. Entered
able to permit a dependable analysis. In ad- alone, the 14 methods accounted for about
dition to these sets of dummy variables, com- 6% of the variance (R2 = .64), F(14, 1787) =
plete data were available for 1,802 effect 8.67, p < .001, which was reduced to 5% (R2
sizes, representing 407 treatment groups, change = .049), ^14, 1751) = 7.85, p < .001,
with respect to treatment mode; treatment when the other four sets of variables were
and control group «s; and the four measure- taken into account. Entered alone, the 29
ment variables, domain, reactivity, specific- targets accounted for 11% of the variance
ity, and measurement technology. (R2 = .10,^(29, 1772) = 7.66,/><.001; this
The rightmost section of Table 7 shows the was reduced to about 8% (R2 change = .76),
variance in mean effect size for each treat- F(29, 1751) = 5.88, p<. 001, when the other
ment group accounted for by four sets of 4 sets of variables were included in the equa-
variables and presents R2 values for each set tion. Overall, the results presented in Table
of variables alone, together with the change 7 indicate that the effects of each set of vari-
in R2 associated with the addition of that set ables were largely, although not wholly, in-
at the last step in the regression equation. dependent of those of the other sets of vari-
Taken together, the four sets of variables ac- ables. Estimates of the impact of treatment
counted for over 35% of the variance (R2 = methods ranged from 5% to 11% of the vari-
.362), 7^(46, 360) = 4.45, p < .001. As shown ance. Each of the four variable sets included
in Table 7, treatment methods accounted for in both analyses accounted for less variance
11 % of the variance when entered alone in the analysis of individual measures, where
(R* = .112), F(14, 392) = 3.54, p < .001, and more variance remained unaccounted for
this, effect was only slightly diminished by (78% as opposed to 64%) despite the intro-
taking account of the other sets of variables, duction of the four measurement variables.
R2 change = .089, F(14, 360) = 3.67, p < Standardized regression coefficients for
.001. In contrast, the initially larger impact each variable in the analysis of 1,802 effect
of the 29 treatment targets (R2 = .217), F(29, size measures are shown in Table 8. On the
377) = 3.60, p < .001, was reduced to a level left in the table are presented the betas ob-
comparable to that of the treatment methods tained for each variable when the set of vari-
(R2 change = .091), F(29, 3 6 0 ) = 1.77, ables to which it was assigned was entered
 META-ANALYSIS OF THERAPY OUTCOMES 595

alone; on the right are the betas obtained ificity and technology were the most pow-
when all variables were in the equation. The erful measurement variables, with betas of
close correspondence between the two col- .103, F(\, 1751)= 14.49, p < .001, and
umns of figures, consistent with the high de- -.121, F(\, 1751)= 17.66, p<. 001, respec-
gree of independence between variable sets tively. Four variables yielding significant cor-
noted in Table 7, indicates that controlling relations with effect size in Table 3 were ex-
for other variable sets has little impact on the cluded from the multiple regression analysis
relation between each variable and effect size. on account of missing values; of the 7 vari-
Among the dummy variables representing ables with such significant correlations that
treatment methods, a possible exception to did appear in the multiple regression equa-
this trend was obtained for flooding, whose tions, 5 yielded significant betas of un-
significant beta of .085,/{I, 1751) = 10.41, changed sign when all variable sets were in-
p < .01, fell to -.005 (F < 1) when the other cluded in the equation. These were specific-
variable sets were entered into the equation. ity, technology, domain Q3 = -.062), F(l,
It is noteworthy that the direction of impact 1751) = 5.03, p < .05, control « (/S = -.335),
of treatment « was reversed by taking ac- F(l, 1751) = 64.85, /x.OOl, and mode
count of control n in the variable set com- (0 = .086), F(l, 1751) = 9.74, p< .01.
prising these 2 variables and that the impact Standardized regression coefficients cor-
of reactivity was reduced from a zero-order responding to those shown in Table 8 were
r of. 11 to a beta of .06 when entered along- also obtained for all but the measurement
side the other 3 measurement variables and variables in the analysis of the mean effect
to a beta of-.013 when all variable sets were sizes obtained for each of 407 treatment
included. In these latter circumstances, spec- groups. Consistent with the multiple R2 data

Table 8
Standardized Regression Coefficients for Treatment, Design, and Measurement Variables
Set alone With all sets

Variable

Rehearsal, self-control, & monitoring .064 4.93* .052 3.13

Biofeedback .021 <1 .000 <1
Covert behavioral .149 29.98*** .112 17.28***
Flooding .085 10.41** -.005 <1
Relaxation .055 3.30 .052 2.77
Systematic desensitization .111 9.37** .126 12.36***
Reinforcement .053 3.88* .052 3.30
Modeling .163 35.62*** .142 24.60***
Social skills training .040 2.14 .065 5.13*
Study skills training -.031 1.64 .011 <1
Cognitive .113 12.94*** .160 26.36***
Dynamic/humanistic -.042 2.39 .001 <1.
Mixed .207 45.17*** .202 45.63***
Unclassified .026 <1 -.010 <1
Behavioral .162 22.46*** .159 22.96***
Verbal -.040 2.16 -.004 <1
Cognitive .110 11.88*** .164 27.08***
Mode .086 13.36*** .086 9.74**
Treatment N .272 50.94*** .322 63.71***
Control N -.355 86.71*** -.335 64.85***
Domain -.125 26.02*** -.062 5.03**
Reactivity -.064 5.70 -.013 <1
Specificity .163 40.67 .103 14.49
Technology -.111 16.24*** -.121 17.66***
Note. Variables representing the 3 therapy types, behavioral, verbal, and cognitive, were entered in place of the 14
methods in a separate analysis.
* p< .05. ** p < .01. *** p < .001.
596 DAVID A. SHAPIRO AND DIANA SHAPIRO

of Table 7, these betas were larger than those On the other hand, some clear exceptions
shown in Table 8. Their relative magnitudes to these general trends are worthy of men-
were generally very similar, however. The tion. The present data revealed a modest but
only exception was obtained for the dummy undeniable superiority of behavioral and
variable representing cognitive therapy, whose cognitive methods and a corresponding rel-
beta rose from a nonsignificant .099, ^(1, ative inferiority of dynamic and humanistic
392) = 2.51, to .216, F(l, 360) = 12.56, p < (verbal) methods, in the results of simulta-
.001, when the other three variable sets were neous controlled comparisons; statistical
included in the equation. This suggests that control of variation in other study charac-
cognitive therapy was typically studied under teristics via multiple regression analysis fur-
somewhat adverse conditions with respect to ther highlighted the apparent superiority of
the other sets of variables, yielding more fa- cognitive therapy and did little to diminish
vorable outcomes with statistical control for the impact of the treatment methods gener-
this adversity. ally.
Specific comparisons between treatment
Discussion methods present together in at least four
studies yielded some quite clearcut results.
The findings of the present study are First, active treatments were superior to min-
broadly in agreement with those of most pre- imal, placebo treatments in the majority of
vious reviews, whether conventional or meta- such comparisons. Of more interest were the
analytic. The evidence from the relatively comparisons among active treatments. In
well-conducted source studies conforming to view of the procedural and theoretical com-
the design in which two or more treatment monalities among relaxation, biofeedback
groups are compared with a control group and systematic desensitization, it is note-
is consistent with the less restrictive but worthy that these three treatments figured
largely independent survey by Smith et al. prominently among those behavioral meth-
(1980) in suggesting a mean effect size ap- ods that come out relatively unfavorably in
proaching one standard deviation. The ef- the data of Table 4. In view of recent con-
fects of different treatment methods were troversy (Ledwidge, 1978, 1979; Mahoney
not, on the whole, impressively different & Kazdin, 1979), the advantage of cognitive
from one another, accounting for less vari- therapy over desensitization is of particular
ance than differences between target prob- note. The nine studies making this compar-
lems under treatment. Simultaneous con- ison were primarily concerned with perfor-
trolled comparisons among the behavioral mance anxiety (6 studies); all were with stu-
methods with which most of our data were dent clients and none involved depression.
concerned suggested that these have broadly None was included in Ledwidge's (1978, Ta-
similar effects. The impact of client, thera- ble 1) review of earlier studies. The authors
pist, design, and measurement variables was of all 9 studies interpreted their outcomes as
generally modest, and although treatment favoring cognitive therapy over desensitiza-
targets and design features appeared to ac- tion, except that in one study a combined
count for a large proportion of the variance treatment (coded by our conventions as cog-
in effect sizes, some of this variance was at- nitive) was superior to either component in
tributable to covariation with other study isolation. Thus, the conclusions of meta-
characteristics, such that the overall percent- analysis were congruent with those drawn by
ages of variance accountable for in our two individual authors.
regression analyses were 36% and 22%. There As with all our findings, these results can-
was little convincing evidence of differential not be interpreted without reference to the
impact of treatment methods with clients nature of the data base. They should not be
with differing problems or with outcome taken to imply that cognitive therapy is su-
measurements in different psychological do- perior to orthodox behavioral methods in the
mains. Thus, the broad thrust of our findings treatment of depression or with clinical pop-
suggests moderate, generally uniform, effects ulations. Their major value is in their dem-
of therapy, with a large proportion of the onstration that meta-analytic methods can
variance in effect sizes unaccounted for. yield clearcut findings within a closely de-
 META-ANALYSIS OF THERAPY OUTCOMES 597

fined subset of a larger data base and that the sulted in an unwarranted inflation of the
overall picture of broadly comparable treat- overall effect size estimate. Thus the gain in
ment outcomes does not rule out the exis- replicability achieved by the elimination of
tence of tangible exceptions to the general dissertation studies, in accordance with the
rule. preference of such critics as Rachman and
The nature of our data base is itself an Wilson (1980), may be offset by the intro-
issue warranting comment. Overwhelmingly, duction of bias, possibly reflecting a prejudice
studies simultaneously comparing two or against the null hypothesis (Greenwald, 1975)
more treatments with a control group are in published work (Shapiro & Shapiro, 1982).
concerned with behavioral methods, applied The sparse representation and poor show-
in what can only be described as an analogue ing of dynamic and humanistic methods in
context, in which students are recruited for our data are striking. The professional atti-
brief treatment of relatively circumscribed tudes and values of dynamically oriented
and minimally disabling target problems, therapists may well militate against their con-
and outcomes are assessed immediately ducting and publishing contrast-group out-
posttherapy. Few indeed are the studies of come studies of the kind reviewed here. Ad-
dynamic therapy; almost as rare is the in- herents of these approaches may consider
volvement of clinically referred clients in them ill-served by studies in which they were
controlled outcome studies of the kind re- predominantly represented by group meth-
viewed here. Similar patterns were reported ods (80% of the total effect sizes for these
by Smith et al. (1980), despite the less ex- therapies) that were administered by thera-
acting design requirements used in the selec- pists with a mean 3.7 years of experience, for
tion of their largely independent data base. a mean duration of 11 hours, with 75% of
No meta-analysis can transcend the limita- the data obtained immediately after treat-
tions of the data gathered by researchers un- ment and with only 33% of the data obtained
dertaking its source studies. The frequency in clinical settings. On the other hand, the
data reported in Tables 1 to 3 show how un- target problems predominantly addressed by
representative of clinical practice the litera- these methods (51% social and sexual prob-
ture reviewed is. This concentration of out- lems, 17% anxiety and depression, and 16%
come research at the analogue end of the physical and habit problems) appear appro-
clinical-analogue continuum places severe priate enough, and the mean age of 27 years
limitations on its generalizability to everyday for clients receiving these treatments indi-
practice and its utility in the formulation of cates that a substantial proportion were not
policy concerning service provision and the college students. The majority of the few in-
orientation of training and professional de- stances of dynamic and humanistic therapy
velopment (Vandenbos, 1980; Shapiro & were interpretable as straw man treatment
Shapiro, in press). groups, not expected to yield good results by
Our overall effect size of .93 was slightly the investigators. Smith et al. (1980) found
larger than the .68 and .85 obtained by Smith that the apparent allegiance of the researcher
and Glass (1977) and by Smith et al. (1980), was a good predictor of the outcome ob-
respectively. A possible explanation of this tained for a given treatment. On the other
slight difference lies in our exclusion of dis- hand, whether or not dynamic or humanistic
sertations. Smith et al. (1980) found a mean treatment groups appearing in the present
effect size for dissertations of .66, whereas meta-analysis were straw men, their apparent
that for journal articles was .87. Glass et al. effects were consistently modest in compar-
(1981, p. 66) present comparisons from other ison with other treatments. Nonetheless, the
meta-analytic investigations indicating sim- volume and representativeness of these data
ilar discrepancies between journal article and are insufficient to warrent their interpretation
dissertation data, with journal article data as supporting an adverse evaluation of the
yielding more favorable results. Since Smith verbal therapies.
et al. (1980) found that the rated internal The major impact of target problem on
validity of dissertation studies was not infe- effect size is consistent with clinicians' rec-
rior to that of journal articles, their omission ognition that some problems are more readily
from the present meta-analysis may have re- overcome than others, irrespective of the
598 DAVID A. SHAPIRO AND DIANA SHAPIRO

treatment approach. Specifically, the large of assessment yielded more precise conclu-
effects for simple phobias, in contrast to the sions concerning their impact on effect size.
more modest effects for anxiety and depres- Although replicating Smith et al.'s (1980)
sion, are consistent with the view that re- finding that reactivity (here differentiated
search focusing on the former targets may into specificity and other aspects of reactiv-
yield overoptimistic results, which may not ity) is a clear correlate of effect size, our data
generalize to the latter targets. Similarly, the suggested that this effect is primarily attrib-
modest superiority of individual over group utable to the specificity component of Smith
therapy accords with clinical lore; in this et al.'s (1980) reactivity dimension. Further-
case, however, the implication is that the em- more, this effect was equaled by a statistically
phasis in outcome research on group meth- independent effect associated with the soft-
ods may underestimate treatment outcomes. ness of the measurement technology used,
Equally consistent with clinical lore is the which was a new variable devised for the
greater impact of therapy on measures of present analysis.
anxiety and self-esteem, and on relatively soft We do not claim that a meta-analysis such
and treatment-specific measures. Our expla- as the one reported here can alone furnish
nation of the apparent superiority of out- definitive answers to the perennially vexing
comes with relatively inexperienced thera- and complex questions concerning the im-
pists in terms of the tendency for such ther- pact of psychological treatments. The inte-
apists to be employed in studies of relatively gration of data from diverse studies is fraught
tractable target problems is also consistent with difficulties, and meta-analysis is no
with the view of therapy research held by more than an attempt to apply to that task
many clinicians. the judgmental, analytic, and computational
Our analysis of design features yielded gen- skills of the empirical researcher. As in all
erally reassuring results. If anything, client empirical research, however, the outcomes
attrition and poor matching of groups were obtained are a function of the assumptions,
associated with weaker rather than stronger resources, and skills applied by the investi-
treatment effects, suggesting that these design gator. Nonetheless, the present study points
deficiencies are unlikely to inflate the esti- clearly to three broad conclusions: First, the
mated impact of treatment. The negative re- average impact of psychological treatments
lation between effect size and sample size in recently published comparative outcome
may well reflect the tendency for studies to research approaches one standard deviation
be written up and published only when sta- unit; second, the modest differences between
tistically significant results are obtained, such treatment methods are largely independent
that publication of data from small samples of other factors influencing outcome; and
necessitates more powerful effects. Other de- third, contemporary outcome research is not
sign features appeared unrelated to effect size representative of clinical practice.
apart from the disconcerting tendency for
larger effects to emerge from nonblind as- References
sessments. This underlines the importance Arkin, R., Cooper, H., & Kolditz, T. A statistical review
of reducing this source of bias in future re- of the literature concerning the self-serving bias in
search. interpersonal influence situations. Journal of Person-
ality, 1980, 48, 435-448.
Our efforts to refine the meta-analytic Bandura, A. Self-efficacy: Toward a unifying theory of
methods applied by Smith et al. (1980) have behavioral change. Psychological Review, 1977, 84,
yielded tangible benefits. First, the focus on 191-215.
studies including two or more active treat- Barber, T. X. Commentary on Rosenthal and Rubin.
Behavioral and Brain Sciences, 1978, 388-390.
ments produced a data base considerably Beck, A. T. Cognitive therapy and the emotional dis-
richer in potential for direct comparisons free orders. New York: International Universities Press,
of bias arising from noncomparable studies 1976.
of the different methods; in consequence, we Cautela, J. Covert sensitization. Psychological Reports,
are able to conclude rather more firmly that 1967, 20,459-468.
Cook, T. D., & Leviton, L. C. Reviewing the literature:
treatment methods make a difference, albeit A comparison of traditional methods with meta-anal-
modest, to effect size. Second, our refinement ysis. Journal of Personality, 1980, 48, 449-472.
of the coding system with respect to methods Cooper, H. M. Statistically combining independent stud-
 META-ANALYSIS OF THERAPY OUTCOMES 599
ies: A meta-analysis of sex differences in conformity modification: Misconceptions and premature evacu-
research. Journal of Personality and Social Psychol- ation. Psychological Bulletin, 1979,56, 1044-1049.
ogy, 1919,37, 131-135. Mansfield, R. S., & Bussey, T. V. Meta-analysis of Re-
D'Zurilla, T. J., & Goldfried, M. R. Problem solving search: A rejoinder to Glass. Educational Researcher,
and behavior modification. Journal of Abnormal Psy- 1977, 6, 3.
chology, 1911, 78, 107-126. Mathews, A. Fear-reduction research and clinical pho-
Eagly, A. H., & Carli, L. L. Sex of researchers and sex- bias. Psychological Bulletin, 1978, 85, 390-404.
typed communications as determinants of sex differ- McFall, R., & Twentyman, C. Four experiments on the
ences in influencibility: A meta-anatysis of social in- relative contribution of rehearsal, modeling and
fluence studies. Psychological Bulletin, 1981, 90, 1- coaching to assertion training. Journal of Abnormal
20. Psychology, 1973, 81, 199-218.
Elashoff, J. D. Commentary on Rosenthal & Rubin. Meichenbaum, D. B. Cognitive behavior modification:
Behavioral and Brain Sciences, 1978, 3, 392. An integralive approach. New York: Plenum, 1977.
Ellis, A. Reason and emotion in psychotherapy. New Meltzoff, J., & Kornreich, M. Research in psychother-
York: Lyle Stuart, 1962. apy. New York: Atherton, 1970.
Eysenck, H. J. An exercise in mega-silliness. American Nie, N. H, Hull, C. H., Jenkins, J. G., Steinbrenner, K.,
Psychologist, 1918, 33, 517. & Bent, D. H. Statistical Package for the Social Sci-
Finney, D. J. Probit analysis (3rd Edn). Cambridge, ences (2nd ed.). New York: McGraw-Hill, 1975.
England: Cambridge University Press, 1972. Rachman, S. J., & Hodgson, R. Obsessions and com-
Frank, J. D. The present status of ourcome studies. Jour- pulsions. Englewood Cliffs, N.J.: Prentice Hall, 1980.
nal of Consulting and Clinical Psychology, 1979, 47, Rachman, S. J., & Wilson, G. T. The effects of psycho-
310-316. logical therapy: Second enlarged edition. New York:
Franks, C. M., & Wilson, G. T. Annual review of be- Pergamon Press, 1980.
havior therapy: Theory and practice (Vol. 6). New Rosenthal, R., & Rubin, D. B. Interpersonal expectancy
York: Brunner/Mazel, 1978. effects: The first 345 studies. Behavioral and Brain
Gallo, P. S. Meta-analysis—A mixed meta-phor. Amer- Sciences, 1978, 3, 377-415.
ican Psychologist, 1978, 33, 515-517. Shapiro, D. A. Science and psychotherapy: The state of
Glass, G. V. Integrating findings: The meta-analysis of the art. British Journal of Medical Psychology, 1980,
research. Review of Research in Education, 1978, 5, 53, 1-10.
351-379. Shapiro, D. A., & Shapiro, D. The 'double standard' in
Glass, G. V., McGaw, B., & Smith, M. L. Mela-analysis evaluation of psychotherapies. Bulletin of the British
in social research. Severely Hills, Calif: Sage Publi- Psychological Society, 1977, 30, 209-210.
cations, 1981. Shapiro, D. A., & Shapiro, D. Comparative therapy out-
Glass, G. V., & Smith, M. L. The effects of class size on come research: Methodological implications of meta-
achievement. Journal of Education and Policy Stud- analysis. Journal of Consulting and Clinical Psychol-
ies, 1979, 1, 2-16. ogy, in press.
Goldfried, M. R. Systematic desensitization as training Shapiro, D. A., & Shapiro, D. Meta-analysis of com-
in self-control. Journal of Consulting and Clinical parative therapy outcome research: A critical ap-
Psychology, 1971, 37, 228-234. praisal. Behavioural Psychotherapy, 1982, 10, 4-25.
Greenwald, A. G. Consequences of prejudice against the Sloane, R. B., Staples, F. R., Cristol, A. H., Yorkston,
null hypothesis. Psychological Bulletin, 1975, 82, 1- N. J., & Whipple, K. Psychotherapy versus behavior
20. therapy. Cambridge, Mass.: Harvard University Press,
Johnson, D. W., Maruyama, G., Johnson, R., Nelson, 1975.
D., & Skon, L. Effects of cooperative, competitive and
Smith, M. L. Sex bias in counseling and psychotherapy.
individualistic goal structures on achievement: A
Psychological Bulletin, 1980, 87, 392-407.
meta-analysis. Psychological Bulletin, 1981, 89, 47-
62. Smith, M. L., & Glass, G. V. Meta-analysis of psycho-
Kanfer, F. H., & Karoly, P. Self-control: A behavioristic therapy outcome studies. American Psychologist, 1977,
excursion into the lion's den. Behavior Therapy, 1972, 32, 752-760.
3, 398-416. Smith, M. L., Glass, G. V., & Miller, T. I. The benefits
Kazdin, A. E. Evaluating the generality of findings in of psychotherapy. Baltimore: Johns Hopkins Univer-
analogue therapy research. Journal of Consulting and sity Press, 1980.
Clinical Psychology, 1978, 46, 673-686. Strube, M. J., & Garcia, J. E. A meta-analytic investi-
Kazdin, A. E., & Wilson, G. T. Evaluation of behavior gation of Fiedler's contingency model of leadership
therapy: Issues, evidence and research strategies. effectiveness. Psychological Bulletin, 198!, 90, 307-
Cambridge, Mass.: Ballinger, 1978. 321.
Ledwidge, B. Cognitive behavior modification: A step Strube, M. J., & Hartmann, D. P. A critical appraisal
in the wrong direction? Psychological Bulletin, 1978, of meta-analysis. British Journal of Clinical Psychol-
55, 353-375. ogy, 1982, 21, 129-139.
Ledwidge, B. Cognitive behavior modification or new
ways to change minds: Reply to Mahoney and Kaz- Vandenbos, G. R. Psychotherapy: Practice, research,
din. Psychological Bulletin, 1979, 86, 1050-1053. policy. Beverly Hills, Calif: Sage Publications, 1980.
Luborsky, L., Singer, B., & Luborsky, L. Comparative White, K. R. The relation between socioeconomic status
studies of psychotherapies. Archives of General Psy- and academic achievement. Psychological Bulletin,
chiatry, 1975, 32, 995-1008. 1982, 91, 461-481.
Mahoney, M. J., & Kazdin, A. E. Cognitive behavior (Appendix follows on next page)
600 DAVID A. SHAPIRO AND DIANA SHAPIRO

Appendix
Studies Included in the Meta-Analysis
Each reference is followed by the number (or numbers) assigned to the study (or studies) obtained
from it as listed in Table A of the supplementary documentation available from JSAS (see Foot-
note 1).
.Adesso, V. J., Vargas, J. M., & Siddall, J. W. The role Chang-Liang, R., & Denney, D. R. Applied relaxation
of awareness in reducing nail-biting behavior. Behav- as training in self-control. Journal of Counseling Psy-
ior Therapy, 1979, 10, 148-154. (1) chology, 1916,23, 183-189. (18)
Aiduk, R., & Karoly, P. Self-regulatory techniques in Chesney, M. A., & Shelton, J. L. A comparison of mus-
the modification of non-assertive behavior. Psycho- cle relaxation and electromyogram biofeedback treat-
logical Reports, 1975, 36, 895-905. (2) ments for muscle contraction headache. Journal of
Alden, L, Safran, J., & Weideman, R. A comparison Behavior Therapy and Experimental Psychiatry, 1976,
of cognitive and skills training strategies in the treat- 7, 221-225. (19)
ment of unassertive clients. Behavior Therapy, 1978, Cox, D. J., Freundlich, A., & Meyer, R. G. Differential
9, 843-846. (3) effectiveness of electromyograph feedback, verbal re-
Anton, W. D. An evaluation of outcome variables in the laxation instructions, and medication placebo with
systematic desensitisation of test anxiety. Behavior tension headaches. Journal of Consulting and Clinical
Research and Therapy, 1976, 14, 217-224. (4) Psychology, 1975, 43, 892-898. (20)
Argyle, M., Bryant, B., & Trower, P. Social skills training Curran, J. P. Social skills training and systematic desen-
and psychotherapy: A comparative study. Psycholog- sitization in reducing dating anxiety. Behavior Re-
ical Medicine, 1974, 4, 435-443. (5) search and Therapy, 1975, 13, 69-77. (21)
Asirdas, S., & Beech, H. R. The behavioral treatment Curran, J. P., & Gilbert, F. S. A test of the relative ef-
of sexual inadequacy. Journal of Psychosomatic Re- fectiveness of a systematic desensitisation program
search, 1975, 19, 345-353. (6) and an interpersonal skills training program with date
Bander, K. W., Steinke, G. V., Allen, G. J., & Mosher, anxious subjects. Behavior Therapy, 1975, 6, 510-
D. L. Evaluation of three dating-specific treatment 521. (22)
approaches for heterosexual dating anxiety. Journal Deffenbacher, J. L., Mathis, H., & Michaels, A. O. Two
of Consulting and Clinical Psychology, 1975, 43, 259- self-control procedures in the reduction of targeted
265. (7) and nontargeted anxieties. Journal of Counseling Psy-
Barbarin, O. A. Comparison of symbolic and overt aver- chology, 1979, 26, 120-127. (23)
sion in the self-control of smoking. Journal of Con- Deffenbacher, J. L., & Parks, D. H. A comparison of
sulting and Clinical Psychology, 1918,46,1569-1571. traditional and self-control desensitisation. Journal of
(8) Consulting and Clinical Psychology, 1979, 26, 93-97.
Barkley, R. A., Hastings, J. E., & Jackson, T. L. The (24)
effects of rapid smoking and hypnosis in the treatment Deffenbacher, J. L., & Payne, D. M. J. Two procedures
of smoking behavior. The International Journal of for relaxation as self-control in the treatment of com-
Clinical and -Experimental Hypnosis, 1977, 25, 7-17. munication apprehension. Journal of Consulting and
(9) Clinical Psychology, 1977, 24, 255-258. (25)
Barrett, C. J. Effectiveness of widows' groups in facili- Denney, D. R. Active, passive and vicarious desensitis-
tating change. Journal of Consulting and Clinical Psy- ation. Journal of Counseling Psychology, 1974, 21,
chology, 1978, 46, 20-31. (10) 369-375. (26)
Bedell, J. R. Systematic desensitisation, relaxation-train- Denney, D. R., & Rupert, P. A. Desensitisation and self-
ing and suggestion in the treatment of test anxiety. control in the treatment of text anxiety. Journal of
Behavior Research and Therapy, 1976, 14, 309-311. Counseling Psychology, 1977, 24, 272-280. (27)
(11) Denney, D. R., & Sullivan, B. J. Desensitisation and
Berlin, J. S., & Dies, R. R. Differential group structure: modeling treatments of spider fear using two types of
The effects on socially isolated college students. Small scenes. Journal of Consulting and Clinical Psychology,
Group Behavior, 1974, 5, 462-472. (12) 1916,44, 573-579. (28)
Blanchard, E. B., Theobald, D. E., Williamson, D. A., Diament, C., & Wilson, T. G. An experimental inves-
Silver, B. V., & Brown, D. A. Temperature biofeed- tigation of the effects of covert sensitization in an an-
back in the treatment of migraine headaches. Archives alogue eating situation. Behavior Therapy, 1975, 4,
of General Psychiatry, 1978, 35, 581-588. (13) 499-509,(29)
Eliott, C. H., & Denney, D. R. Weight control through
Borkovec, T. D. Heart-rate process during systematic covert sensitisation and false feedback. Journal of
desensitisation and implosive therapy for analog anx- Consulting and Clinical Psychology, 1975, 43, 842-
iety. Behavior Therapy, 1974, 5, 636-641. (14) 850. (30)
Branham, L., & Katahn, M. Effectiveness of automated Emmelkamp, P. M. G. Self observation versus flooding
desensitisation with normal volunteers and phobic in the treatment of agoraphobia. Behavior Research
patients. Canadian Journal of Behavioral Science, and Therapy, 1974, 12, 229-237. (31)
1974, 6, 234-245. (15, 16) Epstein, N., & Jackson, E. An outcome study of short
Cautela, J. R., Flannery, R. B., & Hanley, S. Covert term communication training with married couples.
modeling: An experimental test. Behavior Therapy, Journal of Consulting and Clinical Psychology, 1978,
1974, 5, 494-502. (17) 46, 207-212. (32)
 META-ANALYSIS OF THERAPY OUTCOMES 601

Finger, R., & Galassi, J. P. Effects of modifying cognitive Israel, E., & Beiman, I. Live versus recorded relaxation
versus emotionality responses in the treatment of test training: A controlled investigation. Behavior Ther-
anxiety. Journal of Consulting and Clinical Psychol- apy, 1977, 8, 251-254. (50)
ogy, 1977, 45, 280-287. (33) Jaffe, P. G., & Carlson, P. M. Relative efficacy of mod-
Foa, E. B., Blau, J. S., Prout, M., & Latimer, P. Is horror eling and instructions in eliciting social behavior from
a necessary component of flooding (implosion)? Be- chronic psychiatric patients. Journal of Consulting
havior Research and Therapy, 1977, 75, 397-402. (34) and Clinical Psychology, 1976, 44, 200-207. (51)
Fuchs, C, Z., & Rehm, L. P. A self-control behavior Jeffrey, D. B., & Christensen, E. R. Behavior therapy
therapy program for depression. Journal of Consulting versus "will power" in the management of obesity.
and Clinical Psychology, 1977, 45, 206-215. (35) Journal of Psychology, 1975, 90, 303-311. (52)
Fremouw, W. J., & Zitter, R. E. A comparison of skills Kantorowitz, D. A., Walters, J., & Pezdek, K. Positive
training and cognitive restructuring-relaxation for the versus negative self-monitoring in the self-control of
treatment of speech anxiety. Behaviour Therapy, smoking. Journal of Consulting and Clinical Psy-
1978, 9, 248-259. (36) chology, 1978, 46, 1148-1150. (53)
Kaplan, R. M., McCordick, S. M., & Twitchell, M. Is
Frohwirth, R. A., & Foreyt, T. P. Aversive conditioning it the cognitive or the behavioral component which
treatment of overweight. Behavior Therapy, 1978, 9, makes cognitive-behavior modification effective in
861-872. (37) text anxiety? Journal of Counseling Psychology, 1979,
Gatchel, R. J., Hatch, J. P., Watson, P. J., Smith, D., 26, 371-377. (54)
& Gaas, E. Comparative effectiveness of voluntary Kass, W., & Gilner, F. H. Drive level, incentive con-
heart rate control and muscular relaxation as active ditions, and systematic desensitization. Behavior Re-
coping skills for reducing speech anxiety. Journal of search and Therapy, 1974, 12, 99-106. (55)
Consulting and Clinical Psychology, 1971,45, 1093- Katz, R. C., Thomas, S. L., & Williamson, P. Effects of
1100. (38) self-monitoring as a function of its expected benefits
Girodo, M., & Roehl, J. Cognitive preparation and cop- and incompatible response training. Psychological
ing self-talk: Anxiety management during the stress Record, 1976, 26, 533-540. (56)
of flying. Journal of Consulting and Clinical Psychol- Kelly, A. H., & Curran, J. P. Comparison of a self-con-
ogy, 1978, 46, 978-989. (39) trol approach and an emotional coping approach to
Glass, C. R., Gottman, J. M., & Shmurak, S. H. Re- the treatment of obesity. Journal of Consulting and
sponse-acquisition and cognitive self-statement mod- Clinical Psychology, 1976, 44, 683. (57)
ification approaches to eating-skills training. Journal Kendall, P. C., Williams, L., Pechacek, T. F., Graham,
of Counseling Psychology, 1976, 23, 520-526. (40) L. E., Shisslak, C. & Herzoff, N. Cognitive-behavioral
Goldfried, M. R., Lineham, M. M., & Smith, J. L. Re- and patient education interventions in cardiac cath-
duction of test anxiety through cognitive restructur- eterization procedures: The Palo Alto medical psy-
ing. Journal of Consulting and Clinical Psychology, chology project. Journal of Consulting and Clinical
1978, 46, 32-39. (41) Psychology, 1979, 47, 49-58. (58)
Gulanick, N. G., Woodburn, L. T., & Rimm, D. C. Kessler, S. Treatment of overweight. Journal of Coun-
Weight gain through self-control procedures. Journal seling Psychology, 1974, 21, 395-398. (59)
of Consulting and Clinical Psychology, 1975, 43, 536- Kilmann, P. R. Marathon group therapy with female
539. (42) narcotic addicts. Psychotherapy Theory Research and
Hall, S. M., Hall, R. G., Deboer, G., & O'Kulitch, P. Practice, 1974, 11, 339-342. (60)
Self and external management compared with psy- Kipper, D. A., & Giladi, D. Effectiveness of structured
chotherapy in the control of obesity. Behavior Re- psychodrama and systematic desensitization in re-
search and therapy, 1977, 15, 89-95. (43) ducing test anxiety. Journal of Counseling Psychology,
Hannie, T. J., & Adams, H. E. Modification of agitated 1978,25, 499-505. (61)
depression by flooding: A preliminary study. Journal Kirsch, I., & Henry, D. Extinction versus credibility in
of Behavior Therapy and Experimental Psychiatry, the desensitization of speech anxiety. Journal of Con-
1974,5, 161-166. (44) sulting and Clinical Psychology, [911,45, 1052-1059.
Haynes, S. N., Griffin, P. Mooney, D., & Parise, M. (62)
Electromyographic biofeedback and relaxation in- Kirsch, I., & Henry, D. Self-desensitization and medi-
structions in the treatment of muscle contraction tation in the reduction of public speaking anxiety.
headaches. Behavior Therapy, 1975, 6- 672-678. (45) Journal of Consulting and Clinical Psychology, 1979,
Holroyd, K. A. Cognition and desensitization in the 47, 536-541. (63)
group treatment of test anxiety. Journal of Consulting Kockott, G., Dittmar, F., & Nusselt, L. Systematic de-
and Clinical Psychology, 1976, 44, 991-1001. (46) sensitization of erectile impotence: A controlled study.
Holroyd, K. A., & Andrasik, F. Coping and the self-
Archives of Sexual Behavior, 1975, 4, 493-500. (64)
control of chronic tension headache. Journal of Con-
sulting and Clinical Psychology, 1978,46, 1036-1045. Kostka, M. P., & Galassi, J. P. Group systematic desen-
(47) sitization versus covert positive reinforcement in the
Home, A. M., & Matson, J. L. A comparison of mod- reduction of test anxiety. Journal of Counseling Psy-
eling, desensitization, flooding, study skills and con- chology, 1974, 21, 464-468. (65)
trol groups for reducing test anxiety. Behavior Ther- Lamontagne, Y., Gagnon, M. A., Trudel, G., & Boisvert,
apy, 1977, 8, 1-8. (48) J. M. Thought-stopping as a treatment for reducing
Hurley, A. Covert reinforcement: The contribution of cigarette smoking. The International Journal of the
the reinforcing stimulus to treatment outcome. Be- Addictions, 1978, 13, 297-305. (66)
havior Therapy, 1976, 7, 374-378. (49) Lando, H. Self-pacing in eliminating chronic smoking:
602 DAVID A. SHAPIRO AND DIANA SHAPIRO

Serendipity revisited? Behavior Therapy, 1976, 7, for test anxiety. Journal of Consulting and Clinical
634-640.(67) Psychology, 1979, 47, 419-420. (85)
Lent, R. W., & Russell, R. K. Treatment of test anxiety McGlynn, F. D., & McClaren, H. A. Components of
by cue-controlled desensitization and study-skills desensitization in modification of fear among genu-
training. Journal of Counseling Psychology, 1978, 25, inely fearful subjects. Psychological Reports, 1975, 37,
217-224. (68) 959-969. (86)
Levenberg, S. B., & Wagner, M. K. Smoking cessation: McReynolds, W. T., Lutz, R. N., Paulsen, B. K., &
Long-term irrelevance of mode of treatment. Journal Kohrs, M. B. Weight loss resulting from two behavior
of Behavior Therapy and Experimental Psychiatry, modification procedures with nutritionists as thera-
1976, 7, 93-95. (69) pists. Behavior Therapy, 1976, 7, 283-291. (87)
Levitz, L. S., & Stunkard, A. J. A therapeutic coalition Melnick, J., & Russell, R. W. Hypnosis versus systematic
for obesity: Behavior modification and patient self- desensitization in the treatment of test anxiety. Jour-
help. American Journal of Psychiatry, 1974, /j/,423- nal of Counseling Psychology, 1976, 23,291-295. (88)
427. (70) Mendonca, J. D., & Siess, T. F. Counseling for indeci-
Lick, J. Expectancy, false galvanic skin response feed- siveness: Problem solving and anxiety management
back, and systematic desensitization in the modifi- training. Journal of Counseling Psychology, 1976, 23,
cation of phobic behavior. Journal of Consulting and 339-347. (89)
Clinical Psychology, 1975, 43, 557-567. (71) Miller, R. K., & Bornstein, P. H. Thirty-minute relax-
Lick, J. R., & Heffler, D. Relaxation training and atten- ation: A comparison of some methods. Journal of
tion placebo in the treatment of severe insomnia. Behavior Therapy & Experimental Psychiatry, 1977,
Journal of Consulting and Clinical Psychology, 1977, 5,291-294. (90)
45, 153-161. (72) Mitchell, K. R., Hall, R. F., & Piatkowska, O. E. A group
MacDonald, M. L., Lindquist, C. U., Kramer, J. A., program for the treatment of failing college students.
McGrath, R. A., & Rhyne, L. D. Social skills training: Behavior Therapy, 1975, 6, 324-336. (91)
Behavior rehearsal in groups and dating skills. Journal Mitchell, K. R., & Orr, F. E. Note on treatment of het-
of Counseling Psychology, 1975, 22, 224-230. (73) erosexual anxiety using short-term massed sensitiza-
Margolin, G., & Weiss, R. L. Comparative evaluation tion. Psychological Reports, 1974, 35, 1093-1094.
of therapeutic components associated with behavioral (92)
marital treatments. Journal of Consulting and Clinical Moleski, R., & Tosi, D. J. Comparative psychotherapy:
Psychology, 1978, 46, 1476-1486. (74) Rational-emotive therapy versus systematic desensi-
Marshall, W. L., Boutilier, J., & Minnes, P. The mod- tization in the treatment of stuttering. Journal of Con-
ification of phobic behavior by covert reinforcement. sulting and Clinical Psychology, 1976, 44, 309-311.
Behavior Therapy, 1974, 5, 469-480. (75) (93)
Marshall, W. L., Gauthier, J., Christie, M. M., Currie, Monti, P. M., Fink, E., Norman, W., & Curran, J. Effect
D. W., & Gordon, A. Flooding therapy: Effectiveness, of social skills training groups and social skills bib-
stimulus characteristics, and the value of brief in vivo liotheraphy with psychiatric patients. Journal of Con-
exposure. Behavior Research and Therapy, 1977, ./5, sulting and Clinical Psychology, 1979,47, 189-191.
79-87. (76, 77, 78) (94)
Marshall, W. L., Press, L., & Andrews, W. R. A self- Moss, M. K., & Arend, R. A. Self-directed contact de-
administered program for public speaking anxiety. sensitization. Journal of Consulting and Clinical Psy-
Behavior Research and Therapy, 1976, 14, 33-39. chology, 1977, 45, 730-738. (95)
(79) Nemetz, G. H., Craig, K. D., & Reith, G. Treatment of
Marshall, W. L., Stoian, M., & Andrews, W. R. Skills female sexual dysfunction through symbolic model-
training and self-administered desensitization in the ing. Journal of Consulting and Clinical Psychology,
reduction of public speaking anxiety. Behavior Re- 1978, 46, 62-73. (96)
search and Therapy, 1977, 75, 115-117. (80) Nietzel, M. T., Martorano, R. D., & Melnick, J. The
Marzillier, J. S., Lambert, C., & Kellett, J. A controlled effects of covert modelling with and without reply
evaluation of systematic desensitization and social training on the development and generalization of
skills training for socially inadequate psychiatric pa- assertive responses. Behavior Therapy, 1977, 8, 183-
tients. Behavior Research and Therapy, 1976, 14, 192. (97)
225-238. (81) Obler, M. Multivariate approaches to psychotherapy
Mathews, A., Bancroft, J., Whitehead, A., Hackmann, with sexual dysfunctions. The Counseling Psycholo-
A., Julier, D., Bancroft, J., Gath, D., & Shaw, P. The gist, 1975, 5, 55-60. (98)
behavioral treatment of sexual inadequacy: A com- O'Brien, G. T., & Borkovec, T. D. The role of relaxation
parative study. Behavior Research and Therapy, 1976, in systematic desensitization: Revisiting an unre-
14, 427-436. (82) solved issue. Journal of Behavior Therapy and Ex-
Mathews, A., & Rezin, V. Treatment of dental fears by perimental Psychiatry, 1977, 8, 359-364. (99)
imaginal flooding and rehearsal of coping behavior. O'Brien, R. M. Negative practice and desensitization of
Behavior Research and Therapy, 1977, 15, 321-328. anxiety about examinations. Psychological Reports,
(83) 1976,38, 1147-1153. (100)
McConaghy, N. Aversive and positive conditioning O'Brien, R. M., & Dickinson, A. M. Contingency factors
treatments of homosexuality. Behavior Research and in negative practice of smoking. Psychological Re-
Therapy, 1975, 13, 309-319. (84) ports, 1977, 40, 495-505. (101)
McCordick, S. M., Kaplan, R. M., Finn, M. E., & Smith, Odom, J. V., Nelson, R. O., & Wein, K. S. The differ-
S. H. Cognitive behavior modification and modeling ential effectiveness of five treatment procedures of
 META-ANALYSIS OF THERAPY OUTCOMES 603

three response systems in a snake phobia analog study. Saccone, A. J., & Israel, A. C. Effects of experimenter
Behavior Therapy, 1978, 9, 936^942. (102) versus significant other-controlled reinforcement and
Ottens, A. J. The effect of transcendental meditation choice of target behavior on weight loss. Behavior
upon modifying the cigarette smoking habit. Journal Therapy, 1978, 9, 271-278. (118)
of School Health, 1975, 45, 577-583. (103) Shaw, B. F. Comparison of cognitive therapy and be-
Pederson, L. L., Scrimgeour, W. G., & Lefcoe, N. M. havior therapy in the treatment of depression. Journal
Comparison of hypnosis plus counseling, counseling of Consulting and Clinical Psychology, 1977, 45,543-
alone, and hypnosis alone in a community service 551. (119)
smoking withdrawal program. Journal of Consulting Shaw, D. W., & Thoresen, C. E. Effects of modeling and
and Clinical Psychology, 1975, 43, 920. (104) desensitization in reducing dentist phobia. Journal of
Pendleton, L. R., & Tasto, D. L. Effects of metronome- Counseling Psychology, 1974, 21, 415-420. (120)
conditioned relaxation, metronome-induced relaxa- Sherman, A. R., Mulac, A., & McCann, M. J. Synergistic
tion, and progressive muscle relaxation in insomnia. effect of self-relaxation and rehearsal feedback in the
Behavior Research and Therapy, 1976, 14, 165-166. treatment of subjective and behavioral dimensions of
(105) speech anxiety. Journal of Consulting and Clinical
Rakos, R. F., & Schroeder, H. E. Fear reduction in help- Psychology, 1974, 42, 819-827. (121)
givers as a function of helping. Journal of Counseling Shoemaker, J. E., & Tasto, D. L. The effects of muscle
Psychology, 1976, 23, 428-435. (106) relaxation on blood pressure of essential hyperten-
Reinking, R. H., & Kohl, M. L. Effects of various forms sives. Behavior Research and Therapy, 1975, 13, 29-
of relaxation training on physiological and self-report 43. (122)
measures of relaxation. Journal of Consulting and Sipich, J. F., Russell, R. K., & Tobias, L. L. A com-
Clinical Psychology, 1975, 43, 595-600. (107) parison of covert densitization and "nonspecific"
Reister, B. W., Stockton, R. A., & Maultsby, M. C. treatment in the modification of smoking behavior.
Counseling the test anxious: An alternative. Journal Journal of Behavior Therapy and Experimental Psy-
of College Student Personnel, 1977, 75, 506-510. chiatry, 1974, 5, 201-203. (123)
(108) Sloane, R. B., Staples, F. R., Cristol, A. J., Yorkston,
Richards, C. S., & Perri, M. G. Do self-control treat- N. D., & Whipple, K. Short term analytically oriented
ments last? An evaluation of behavioral problem solv- psychotherapy versus behavior therapy. American
ing and faded counselor contact as treatment mainte- Journal of Psychiatry, 1975, 132, 373-377. (124)
nance strategies. Journal of Counseling Psychology, Smith, J. C. Psychotherapeutic effects of transcendental
1978,25, 376-383. (109) meditation with control for expectation of relief and
Richardson, F. C., & Suinn, R. M. Effects of two short- daily sitting. Journal of Consulting and Clinical Psy-
term desensitization methods in the treatment of test chology, 1976, 44, 630-637. (125)
anxiety. Journal of Counseling Psychology, 1974, 21, Snowden, L. R. Personality tailored covert sensitization
457-458. (110) of heroin abuse. Addictive Behaviors, 1978, 3, 43-49.
Romano, J. L., & Cabianca, W. A. EMG biofeedback (126)
training versus systematic desensitization for test anx- Snyder, A. L., & Deffenbacher, J. L. Comparison of re-
iety reduction. Journal of Counseling Psychology, laxation as self-control and systematic desensitization
1978, 25,8-13. ( I l l ) in the treatment of text anxiety. Journal of Consulting
Rosen, G. M., Glasgow, R. E., & Barrera, M. A con- and Clinical Psychology, 1911,45, 1202-1203. (127)
trolled study to assess the clinical efficacy of totally Spiegler, M. D., Cooley, E. J., Marshall, G. J., Prince,
self-administered systematic desensitization. Journal H. T., & Puckett, S. P. A self-control versus a coun-
of Consulting and Clinical Psychology, 1976, 44, 208- terconditioning paradigm for systematic desensitiza-
217. (112) tion: An experimental comparison. Journal of Coun-
Rozensky, R. H., & Bellack, A. S, Individual differences seling Psychology, 1976, 23, 83-86. (128)
in self-reinforcement style and performance in self-
Thorpe, G. L. Desensitization, behavior rehearsal, self-
and therapist-controlled weight reduction programs.
instructional training and placebo effects on assertive-
Behavior Research and Therapy, 1976, 14, 357-364.
refusal behavior. European Journal of Behavioral
(113)
Analysis and Modification, 1975, 1, 30-44. (129)
Russell, M. A. H., Armstrong, E., & Patel, U. A. Tem-
poral contiguity in electric aversion therapy for ciga- Tiegerman, S., & Kassinove, H. Effects of assertive train-
rette smoking. Behavior Research and Therapy, 1976, ing and cognitive components of rational therapy on
14, 103-123. (114) assertive behaviors and interpersonal anxiety. Psy-
Russell, R. K., Miller, D. E., & June, L. N. A comparison chological Reports, 1977, 40, 535-542. (130)
between group systematic desensitization and cue- Turner, R. M., & Ascher, L. M. Controlled comparison
controlled relaxation in the treatment of test anxiety. of progressive relaxation, stimulus control, and par-
Behavior Therapy, 1975, 6, 172-177. (115) adoxical intention therapies for insomnia. Journal of
Russell, R. K., & Wise, F. Treatment of speech anxiety Consulting and Clinical Psychology, 1979, 47, 500-
by cue-controlled relaxation and desensitization with 508. (131)
professional and paraprofessional counselors. Journal Turner, S. M., & Adams, H. Effects of assertive training
of Counseling Psychology, 1976, 23, 583-586, (116) on three dimensions of assertiveness. Behavior Re-
Russell, R. K., Wise, F., & Stratoudakis, J. P. Treatment search and Therapy, 1977, 75, 475-483. (132)
of test anxiety by cue-controlled relaxation and sys- Vargas, J. M., & Adesso, V. J. A comparison of aversion
tematic desensitization. Journal of Counseling Psy- therapies for nailbiting behavior. Behavior Therapy,
chology, 1976, 23, 563-566. (117) 1976, 7, 322-329. (133)
604 DAVID A. SHAPIRO AND DIANA SHAPIRO

Weissberg, M. A comparison of direct and vicarious Wright, J. C. A comparison of systematic desensitization

treatments of speech anxiety: Desensitization with and social skill acquisition in the modification of a
coping imagery and cognitive modification. Behavior social fear. Behavior Therapy, 1976, 7,205-210. (139)
Therapy, 1977, 8, 606-620. (134) Wright, W., Morris, K. T., & Fettig, B. Comparative
Weissberg, M., & Lamb, D. Comparative effects of cog- effects of social skill development. Small Group Be-
nitive modification, systematic desensitisation and havior, 1974, 5, 211-221. (140)
speech preparation in the reduction of speech and Wroblewski, P. F., Jacob, T., & Rehm, L. The contri-
general anxiety. Communication Monographs, 1977, bution of relaxation to symbolic modeling in the
44,21-36. (135) modification of dental fears. Behavior Research and
Wincze, J. P., & Caird, W. K. The effects of systematic Therapy, 1977, 15, 113-117. (141)
desensitization and video desensitization in the treat- Zaichkowsky, L. D., & Kamen, R. Biofeedback and
ment of essential sexual dysfunction in women. Be- meditation: Effects on muscle tension and locus of
havior Therapy, 1976, 7, 335-342. (136) control. Perceptual and Motor Skills, 1978, 46, 955-
Wolfe, J., & Fodor, I. Modifying assertive behavior in 958. (142)
women: A comparison of 3 approaches. Behavior Ziesat, H. A., Rosenthal, T. L., & White, G. M. Behav-
Therapy, 1977, 8, 567-574. (137) ioral self-control in treating procrastination of study-
Woolfork, R. L., Carr-Kaffashan, L., McNulty, T. F., ing. Psychological Reports, 1978, 42, 56-69. (143)
& Lehrec, P. M. Meditation training as a treatment
for insomnia. Behavior Therapy, 1976, 7, 359-365.
(138) Received August 17, 1981