izations has become part of mainstream McKay, D., & Storch, E. (Eds.). (2009).

Cogni-
psychological thinking, the empirical sup- tive behavior therapy for children: Treating Table 1
complex and refractory cases. New York,
port for it is virtually nonexistent (Tryon,
NY: Springer.
Pre–Post Effect Sizes From Five
2008). Until psychodynamic researchers
Shedler, J. (2010). The efficacy of psychody- Imaginary Studies
identify mechanisms associated with psy- namic psychotherapy. American Psychologist,
chopathology on the basis of the specific Treatment pre–post Control pre–post
65, 98 –109. doi:10.1037/a0018378 Study effect size effect size
theory guiding treatment, the ability to ap- Strube, M. J., Gardner, W., & Hartmann, D. P.
proach treatment will continue to be less a (1985). Limitations, liabilities, and obstacles 1 1.00 0.90
scientific enterprise and more an art form. in reviews of the literature: The current status 2 1.00 0.90
Shedler (2010) decried the lack of of meta-analysis. Clinical Psychology Review,
3 1.00 0.90
informed scientific knowledge among prac- 5, 63–78. doi:10.1016/0272-7358(85)90030-3
Tryon, W. W. (2008). Whatever happened to 4 1.00 0.90
ticing psychodynamic therapists and theo- 5 1.00 0.91
symptom substitution? Clinical Psychology
rists when he noted that many are unfamil-
Review, 28, 963–968. doi:10.1016/j.cpr.2008.
iar with the research cited in his article. On Note. Standardized effect size ⫽ 34.6 based on
02.003 methods described by Leichsenring and Rabung
the other hand, Westen (1998) described a Wachtel, P. L. (1997). Psychoanalysis, behavior (2008).
wide range of ways that basic science in therapy, and the relational world. Washing-
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

unconscious processes had advanced since ton, DC: American Psychological Associa-
This document is copyrighted by the American Psychological Association or one of its allied publishers.

Freud’s original theorizing. Many of these tion. doi:10.1037/10383-000

areas likewise are unknown to practitioners Westen, D. (1998). The scientific legacy of Sig-
and theorists; otherwise there would be a mund Freud: Toward a psychodynamically in-
formed psychological science. Psychological fessions for past psychoanalytic arrogance
marked shift in research emphasis to reli-
Bulletin, 124, 333–371. doi:10.1037/0033- and authority” related to a “hierarchical
ance on these updated constructs in treat- 2909.124.3.333 medical establishment that denied training
ment outcome. As a further indicator of the Westen, D., Novotny, C. M., & Thompson- to non-MDs and adopted a dismissive
degree to which other researchers who are Brenner, H. (2004). The empirical status of stance toward research” (Shedler, 2010, p.
like-minded to Shedler view overgeneral- empirically supported psychotherapies: As- 98).
ization in meta-analysis and the absence of sumptions, findings, and reporting in clinical
Shedler (2010) justified his blanket
mechanisms in empirically supported treat- trials. Psychological Bulletin, 130, 631– 663.
doi:10.1037/0033-2909.130.4.631 dismissal of criticisms of evidence support-
ments as problems to be reckoned with, ing psychodynamic psychotherapy on the
Westen, Novotny, and Thompson-Brenner Wood, J. M., Nezworski, M. T., & Stejskal,
W. J. (1996). The comprehensive system for basis of several published meta-analyses.
(2004) suggested that both approaches The validity of conclusions from meta-
the Rorschach: A critical examination. Psy-
have significant limitations, and they urged chological Science, 7, 3–10. doi:10.1111/ analyses depends on the quality of the ev-
practitioners to move to empirically in- j.1467-9280.1996.tb00658.x idence synthesized, the nature of the stud-
formed practice. Shedler appears to be ad- ies included, and the rigor of the statistical
vocating, in its current form, the most gen- analyses employed. Many meta-analyses,
eral form of psychotherapy, with emphasis Correspondence concerning this comment however, are not performed rigorously,
given to common approaches, such as should be addressed to Dean McKay, Depart-
which can result in treatment efficacy esti-
emotional expression and developmental ment of Psychology, Fordham University,
Bronx, NY 10458. E-mail: [email protected]
mates that obscure important intertrial dif-
events, over scientifically informed ap- ferences and that are unlikely to be repli-
proaches to practice. cated in clinical practice.
DOI: 10.1037/a0021248 Shedler’s (2010) only hint of possible
REFERENCES methodological issues in any of the meta-
Is There Room for Criticism of analyses he described was a footnote indi-
Abbass, A. A., Hancock, J. T., Henderson, J., & cating that effect sizes from a meta-analy-
Kisely, S. (2006). Short-term psychodynamic
Studies of Psychodynamic
sis by Leichsenring and Rabung (2008)
psychotherapies for common mental disor- Psychotherapy? were based on an “atypical method” that
ders. Cochrane Database of Systematic Re- “may provide an inflated estimate of effi-
views 2006, Issue 4, Article No. CD004687. Brett D. Thombs, Lisa R. Jewett, and cacy” (Shedler, 2010, p. 101). Rather than
doi:10.1002/14651858.CD004687 Marielle Bassel
Bornstein, R. F. (2001). The impending death of simply an “atypical” method, however, this
McGill University and Jewish General was a gross miscalculation that rendered
psychoanalysis. Psychoanalytic Psychology,
18, 3–20. doi:10.1037/0736-9735.18.1.2 Hospital reported effect sizes meaningless (Thombs,
Bornstein, R. F. (2002). The impending death of Bassel, & Jewett, 2009). Leichsenring and
psychoanalysis: From destructive obfuscation Shedler (February–March 2010) declared Rabung (2008) departed from standard
to constructive dialogue. Psychoanalytic Psy- unequivocally that “empirical evidence methods and erroneously calculated sepa-
chology, 19, 580 –590. doi:10.1037/0736-9735. supports the efficacy of psychodynamic rate within-group pre–post effect sizes for
19.3.580 therapy” (p. 98). He did not mention any long-term psychodynamic psychotherapy
Leichsenring, F., & Rabung, S. (2008). Effec- specific criticisms that have been made of (LTPP) and comparison groups followed
tiveness of long-term psychodynamic psycho- evidence on psychodynamic psychothera- by a point biserial correlation of group
therapy: A meta-analysis. Journal of the pies or address possible distinctions be- (LTPP vs. comparison) and within-group
American Medical Association, 300, 1551–
tween evidence for short-term versus long- effect sizes. As shown in the set of hypo-
1565. doi:10.1001/jama.300.13.1551
McKay, D., Abramowitz, J. S., & Taylor, S. term psychodynamic psychotherapies. thetical studies in Table 1, this method can
(2010). Cognitive-behavioral therapy for re- Instead, he attributed dissenting views to produce large, but meaningless, effect
fractory cases: Turning failure into success. biases in evidence dissemination and re- sizes. In this example, it produces an im-
Washington, DC: American Psychological view, which he suggested are rooted in a plausible standardized mean effect size of
Association. doi:10.1037/12070-000 “lingering distaste in the mental health pro- 34.6, even though differences between

148 February–March 2011 ● American Psychologist

 treatment and control groups are minimal therapy treatments, including psychody- sulting and Clinical Psychology, 69, 825–
(Bhar et al., 2010; Thombs et al., 2009). namic psychotherapy—would have met the 830. doi:10.1037/0022-006X.69.5.825
An additional criticism of the Leich- quality criteria proposed by Cuijpers et al. Cuijpers, P., van Straten, A., Bohlmeijer, E.,
senring and Rabung (2008) review that ap- (2010). Hollon, S. D., & Andersson, G. (2010). The
effects of psychotherapy for adult depres-
plies to the other meta-analyses cited by A recent high-quality trial (Knekt et
sion are overestimated: A meta-analysis of
Shedler (2010) relates to reliance on under- al., 2008), which was not included in any of study quality and effect size. Psychological
powered trials (Bhar et al., 2010; Thombs the meta-analyses reviewed by Shedler Medicine, 40, 211–223. doi:10.1017/
et al., 2009). Kraemer, Gardner, Brooks, (2010), provides an illustration of impor- S0033291709006114.
and Yesavage (1998) showed that the in- tant pragmatic issues not addressed by Shed- Knekt, P., Lindfors, O., Härkänen, T., Välikoski,
clusion of small, underpowered trials in ler. The trial by Knekt et al. included a M., Virtala, E., Laaksonen, M. A., . . . Hel-
meta-analyses results in substantially over- comparison of the effectiveness of a mean sinki Psychotherapy Study Group. (2008).
estimated pooled effect sizes, owing to of 232 LTPP sessions over 31 months with Randomized trial on the effectiveness of long-
a mean of 9.8 sessions of nurse-adminis- and short-term psychodynamic psychotherapy
confirmatory publication bias. Statistical
tered solution-focused therapy over 7.5 and solution-focused therapy on psychiatric
correction is impossible when all or most symptoms during a 3-year follow-up. Psycho-
studies in a meta-analysis are underpow- months. As measured by scores on the logical Medicine, 38, 689 –703. doi:10.1017/
ered. The eight studies pooled by Leichsen- Hamilton Depression Rating Scale, pa- S003329170700164X
ring and Rabung (2008), for instance, had tients who received LTPP had greater de-
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Kraemer, H. C., Gardner, C., Brooks, J. O., III,

pression symptom ratings at 7 months (d ⫽
This document is copyrighted by the American Psychological Association or one of its allied publishers.

15–30 patients in the treatment group and & Yesavage, J. A. (1998). Advantages of ex-
power to find a moderate effect size (e.g., 0.21) and 12 months (d ⫽ 0.16) but lower cluding underpowered studies in meta-analy-
␦ ⫽ 0.50) of 0.23– 0.48 (Bhar et al., 2010; ratings at 36 months (d ⫽ ⫺0.26). Results sis: Inclusionist versus exclusionist view-
on other outcome measures were similar. points. Psychological Methods, 3, 23–31. doi:
Thombs et al., 2009). Effect sizes of at least
All of the effect sizes for differences be- 10.1037/1082-989X.3.1.23
0.50 – 0.75 were required to achieve statis- Leichsenring, F., & Rabung, S. (2008). Effec-
tical significance in these studies. The tween groups were small and well below
tiveness of long-term psychodynamic psycho-
problem is even worse than that, however, what is typically considered to be of clini-
therapy: A meta-analysis. Journal of the
because small studies with true null effects cal significance (e.g., d ⫽ 0.50; National American Medical Association, 300, 1551–
that cross the p ⬍ .05 threshold do so by Institute for Clinical Excellence, 2004). On 1565. doi:10.1001/jama.300.13.1551
varying degrees. Kraemer et al. (1998) the basis of these minimal differences, National Institute for Clinical Excellence.
demonstrated that with 20 participants per Knekt et al. concluded that LTPP provided (2004). Depression: Management of depres-
group and a true null effect, the expected greater benefits, and they called for more sion in primary and secondary care. London,
research comparing different forms of England: Author.
standardized effect size in a meta-analysis Shedler, J. (2010). The efficacy of psychody-
of statistically significant trials will be short-term versus long-term therapies.
They did not note that the LTPP delivered namic psychotherapy. American Psychologist,
0.90 –1.00. Beyond sample size, critical as- 65, 98 –109. doi:10.1037/a0018378
in the study would cost approximately
sessments of the meta-analyses reviewed Thombs, B. D., Bassel, M., & Jewett, L. R.
$29,000 to $40,600, assuming an overall
by Shedler (2010) must consider the poor (2009). Analyzing effectiveness of long-term
cost of $125–$175 per hour, versus $735 to psychodynamic psychotherapy. Journal of the
quality of included studies, which gener-
$980 for the nurse-delivered solution-fo- American Medical Association, 301, 930 –
ally failed to protect against numerous
cused therapy, assuming a cost of $75– 931. doi:10.1001/jama.2009.178
sources of potential bias, as well as the
$100 per hour.
combining of overly heterogeneous trials in
Shedler (2010) provided an uncritical
terms of patients treated, interventions, review of meta-analyses of psychodynamic Correspondence concerning this comment
control groups, and outcomes (Bhar et al., psychotherapies and an unproven rationale should be addressed to Brett D. Thombs, McGill
2010). for why critics might question evidence University, Jewish General Hospital, 4333 Cote
Although it may be tempting to dis- supporting psychodynamic psychotherapy. Ste Catherine Road, Montréal, Québec H3T
miss critiques of study quality as academic, 1E4, Canada. E-mail: [email protected]
Rather than blanket approval or disap-
a recent study of psychotherapy for depres- proval of psychodynamic psychotherapies
sion clearly demonstrated the danger of or any other form of psychotherapy, there DOI: 10.1037/a0021190
such an attitude (Cuijpers, van Straten, is a need for careful evaluation of reason-
Bohlmeijer, Hollon, & Andersson, 2010). able criticisms that have been made of ex- When It Comes to Evaluating
On the basis of quality criteria involving isting research evidence, accompanied by a
sample size considerations and standards Psychodynamic Therapy,
consideration of pragmatic issues related to
intended to protect internal validity, Cuij- implementation and funding. the Devil Is in the Details
pers et al. found that the effect size for
high-quality studies was d ⫽ 0.22, com- Michael D. Anestis and Joye C. Anestis
REFERENCES
pared with d ⫽ 0.74 for all trials. Only one Florida State University and
psychodynamic psychotherapy trial, which Bhar, S. S., Thombs, B. D., Pignotti, M., Bassel, Psychotherapy Brown Bag
had a nonstatistically significant, small ef- M., Jewett, L. R., Coyne, J. C., & Beck, A. T.
fect size of d ⫽ 0.26, was included among (2010). Is longer-term psychodynamic psy- Scott O. Lilienfeld
studies classified as high quality. Out of all chotherapy more effective than shorter-term Emory University
the trials reviewed in the meta-analyses therapies? Review and critique of the evi-
cited by Shedler (2010), only one other dence. Psychotherapy & Psychosomatics, 79, As Shedler (February–March 2010) noted,
208 –216. doi:10.1159/000313689
trial (Crits-Christoph et al., 2001)—which some researchers have reflexively and stri-
Crits-Christoph, P., Siqueland, L., McCalmont,
found that standard drug counseling had a E., Weiss, R. D., Gastfriend, D. R., Frank, A., dently dismissed psychodynamic therapy
greater impact on drug use outcomes and a . . . Thase, M. E. (2001). Impact of psycho- (PT) as ineffective without granting out-
similar impact on associated psychological social treatments on associated problems of come studies on this modality a fair hear-
problems compared with several psycho- cocaine-dependent patients. Journal of Con- ing. We applaud Shedler’s efforts to bring

February–March 2011 ● American Psychologist 149