Psychological Medicine The effect of rehabilitation combined with

cambridge.org/psm
cognitive remediation on functioning in
persons with severe mental illness: systematic
review and meta-analysis
Review Article
Cite this article: van Duin D, de Winter L, Oud Daniëlle van Duin1,2,3, Lars de Winter1, Matthijs Oud2, Hans Kroon2,3,
M, Kroon H, Veling W, van Weeghel J (2019). Wim Veling4,5 and Jaap van Weeghel1,3
The effect of rehabilitation combined with
cognitive remediation on functioning in 1
persons with severe mental illness: systematic Phrenos Center of Expertise, Utrecht, the Netherlands; 2Trimbos Institute, Utrecht, the Netherlands; 3Tilburg
review and meta-analysis. Psychological School of Social and Behavioral Sciences, Tilburg, the Netherlands; 4University of Groningen, Groningen,
Medicine 49, 1414–1425. https://doi.org/ the Netherlands and 5University Medical Center Groningen, Groningen, the Netherlands
10.1017/S003329171800418X

Received: 5 April 2018

Abstract
Revised: 13 December 2018 Background. Psychiatric rehabilitation (PR) can improve functioning in people with severe
Accepted: 20 December 2018 mental illness (SMI), but outcomes are still suboptimal. Cognitive impairments have severe
First published online: 30 January 2019
implications for functioning and might reduce the effects of PR. It has been demonstrated
Key words: that performance in cognitive tests can be improved by cognitive remediation (CR).
Cognitive skills training; psychiatric However, there is no consistent evidence that CR as a stand-alone intervention leads to
rehabilitation; psychotic disorders; real-life improvements in real-life functioning. The present study investigated whether a combination
functioning; severe mental illness
of PR and CR enhances the effect of a stand-alone PR or CR intervention on separate domains
Author for correspondence: of functioning.
Daniëlle van Duin, E-mail: [email protected] Method. A meta-analysis of randomized controlled trials of PR combined with CR in people
and [email protected] with SMI was conducted, reporting on functioning outcomes. A multivariate meta-regression
analysis was carried out to evaluate moderator effects.
Results. The meta-analysis included 23 studies with 1819 patients. Enhancing PR with CR
had significant beneficial effects on vocational outcomes (e.g. employment rate: SMD =
0.41), and social skills (SMD = 0.24). No significant effects were found on relationships and
outcomes of community functioning. Effects on vocational outcomes were moderated by
years of education, intensity of the intervention, type of CR approach and integration of treat-
ment goals for PR and CR. Type of PR was no significant moderator.
Conclusions. Augmenting PR by adding cognitive training can improve vocational and social
functioning in patients with SMI more than a stand-alone PR intervention. First indications
exist that a synergetic mechanism also works the other way around, with beneficial effects of
the combined intervention compared with a stand-alone CR intervention.

Introduction
Psychotic disorders and other severe mental illnesses (SMIs) are associated with a high burden
of disease, which is reflected by a variety of impairments in social, economic and daily-life
functioning. Schizophrenia has the highest total social burden of disease compared with
other psychiatric disorders. A great majority of people with schizophrenia are unemployed
(72%), have left school at 16 years of age or earlier (58.1%), are living alone in single houses
and have shown impairments in self-care (29.8%) (Jablensky et al., 2000; Vargas et al., 2014).
One of the core features of psychotic disorders and other SMIs is a substantial impairment
in cognitive functioning (Goldberg and Green, 2002; Bowie and Harvey, 2006). Impairments
have been found in different neurocognitive domains, such as attention, working memory,
executive functioning and verbal learning (Goldberg and Green, 2002; Bowie and Harvey,
2006; Keefe and Harvey, 2012), social cognitive functioning (Green and Horan, 2010; Keefe
and Harvey, 2012) and an insight or metacognitive functioning (Brüne et al., 2011; Lysaker
et al., 2015). These cognitive limitations have severe implications for employment, social
and everyday living skills, and quality of life (Goldberg and Green, 2002; Green and Horan,
2010; Brüne et al., 2011; Keefe and Harvey, 2012; Lysaker et al., 2015). Cognitive impairments
are also related to a reduced response to psychiatric rehabilitation (PR) programs, with a nega-
tive impact on outcomes such as work, social skills and self-care (Wykes and Dunn, 1992;
© Cambridge University Press 2019 Smith et al., 1999; McGurk and Mueser, 2004).
For PR programs, evidence of the effect on functioning varies from indicative for general
programs, like the Boston PR Approach (Gigantesco et al., 2006; Swildens et al., 2011), to sub-
stantial for the vocational rehabilitation program Individual Placement and Support (IPS), a
model of supported employment (SE) (Kinoshita et al., 2013; Modini et al., 2016).
Psychological Medicine 1415

However, even in a highly evidence-based intervention like IPS a Search strategy

large proportion of people with SMI (around 40–50%) that are
Studies for the meta-analysis were identified by conducting
motivated to participate in regular employment and education
searches in PsycInfo, PubMed, Cochrane, CINAHL and
do not succeed. Cognitive impairments are one of the obstacles
EMBASE for English language articles published in peer-reviewed
for this (McGurk and Mueser, 2004), and training in cognitive
journals up until April 2017. Search terms were used for psychotic
skills and strategies might help to overcome the barriers to finding
disorders, SMI and CR training (see online supplementary data
and sustaining employment and education (McGurk et al., 2015).
Appendix 1 ). In addition, reference lists from included and excluded
Over the past few decades, several cognitive remediation (CR)
studies and from previous reviews were searched. When reported
techniques have been developed for people with SMI. The effect of
data were insufficient for quantitative data extraction, study authors
these CR interventions on cognitive test performance is shown in
were contacted to request additional study data.
multiple studies and meta-analyses (Kurtz et al., 2001; Pilling
et al., 2002; McGurk et al., 2007a, 2007b; Grynszpan et al.,
2011; Wykes et al., 2011). However, there is no consistent evi- Study inclusion
dence that such improvements generalize to better functioning
in real life (Pilling et al., 2002; Corrigan et al., 2008). This reduces Decisions on which studies to include were made independently
the clinical value of CR as a ‘stand-alone’ intervention for people by two authors (L.W., D.D.). Results were compared and disagree-
with SMI. Nevertheless, some studies have indicated the efficacy ments were resolved by consensus. All randomized controlled
of CR in relation to functioning, provided that CR is combined trials (RCTs) meeting the following criteria were included: (1) a
with PR (McGurk et al., 2007a, 2007b; Wykes et al., 2011). combined PR and CR intervention is compared with a
When further exploring the effectiveness of such a combined stand-alone PR or CR intervention. Studies with a ‘head to
intervention with CR and PR, it should be established on which head’ comparison, comparing the combination of PR and CR
domains of functioning beneficial results can be achieved. In add- with the combination of PR and another evidence-based interven-
ition, we need to know whether patient characteristics influence tion, were excluded; (2) the PR intervention is consistent with the
the results. For example, participants at a younger age seem to following definition of PR: to help persons with psychiatric dis-
be more responsive to cognitive interventions than those from abilities to increase their ability to function successfully and to
an older age group (McGurk and Mueser, 2008; Kontis et al., 2012; be satisfied in the environments of their choice with the least
Radhakrishnan et al., 2016). On the other hand, a meta-analysis amount of ongoing professional intervention (Anthony et al.,
on CR in early schizophrenia showed smaller effect sizes for CR 2002); (3) the majority of patients (⩾75%) have a diagnosis of a
in patients with a short duration of illness than in those with psychotic spectrum disorder or another SMI (according to the
chronic schizophrenia (Revell et al., 2015). Furthermore, it is cru- definition of the authors of the article concerned); (4) perform-
cial to determine which characteristics of the combined treatment ance is assessed using at least one ‘functioning’ outcome measure.
contribute to the optimal results. For example, we should know
whether the approach of the CR intervention moderates effects Quality assessment
on functioning. CR can be executed by repetitive exercise to
push intrinsic learning (drill and practice), by discussion and For each included study the risk of bias was assessed by the author
the use of methods and strategies to improve cognitive skills L.W. using the Cochrane Collaboration risk of the bias assessment
(strategy coaching), or by implementing a combination of ‘drill tool (Higgins and Green, 2008) (see online supplementary
and practice’ and strategy-based coaching (drill plus strategy) Appendix 2 for the Risk of bias table). Uncertainty concerning
(Hurford et al., 2011). Results regarding the most efficacious a study was discussed with a second author (M.O.) and resolved
method have been inconclusive so far. Previous research found by consensus. The risk of bias for each included study was rated
that ‘drill and practice’-based CR leads to better cognitive out- for six types of bias: selection bias; performance bias (no blinding
comes than a strategy-based approach (McGurk et al., 2007a, of participants and assessors); detection bias; attrition bias;
2007b), but to worse outcomes in real-life functioning (McGurk reporting bias; and ‘other sources’ of bias. Performance bias was
et al., 2007a, 2007b; Wykes et al., 2011). not rated for insufficient blinding of care providers, because in
The present meta-analysis aimed to test the hypothesis that a trials on the effectiveness of psychological interventions this is
combined intervention of PR and CR has a superior effect on real- hardly feasible. The risk of bias for each domain was rated as
life functioning and global cognition in people with SMI com- high risk (seriously weakens confidence in the results), low risk
pared with providing a stand-alone PR or CR intervention, and (unlikely to seriously alter the results) or unclear.
to determine in which domains of functioning this effect is pre- Confidence in the pooled results for each outcome measure was
sent. In order to establish whether treatment and patient charac- assessed independently by two authors (L.W., D.D.), using the
teristics moderate the effect of this combined PR plus CR Grades of Recommendation, Assessment, Development and
treatment, we conducted a meta-regression analysis. The type of Evaluation (GRADE) method (Guyatt et al., 2011). This is a struc-
CR approach, treatment goal integration, and the age and educa- tured assessment of the quality of evidence, taking into account the
tion level of the participants were expected to moderate the effect following factors: risk of bias, inconsistency, indirectness, impreci-
of this combined treatment. sion and publication bias. The results of the two assessors were
compared and disagreements were resolved by consensus. See
online supplementary Appendix 3. for the GRADE evidence profile.
Methods
The review is conducted and reported according the Preferred
Measures
Reporting Items for Systematic Reviews and Meta-analyses
(PRISMA) guidelines (Moher et al., 2009). Data were pre-specified Outcome measures for functioning were grouped into three cat-
in a review protocol, which was not registered in a public database. egories: (a) functioning in work or education (employment rate,
1416 Daniëlle van Duin et al.

hours worked, job duration, wages, work/education quality, work

interest/motivation); (b) social functioning (scales and subscales Box 1. Definitions for outcome measures and focus of psychiatric
covering specific social skills, number and quality of relation- rehabilitation programs
ships); and (c) community functioning (generic scales covering Outcome measures for domains of functioning:
independent/daily-life functioning, role adjustment and perform-
• Vocational functioning: employment rate, hours worked, job
ance, social and occupational functioning). Some overlap exists
duration, wages, work/education quality, work interest/
between the categories of social functioning and community func-
motivation
tioning. This could not be avoided because of the scales used in
• Social functioning: social skills, relationships
the included studies. See online supplementary Appendix 4 for
• Community functioning: independent/daily-life functioning, role
an overview of all instruments, subscales and parameters for func-
adjustment and performance, social and occupational
tioning outcomes in the included studies. To analyze the effect on
functioning
cognitive functioning, data on global cognition were extracted
when available.
Type of PR programs, with a focus on:
In addition to general study-characteristics (sample size, risk of
bias assessment), data on the following moderator variables were • Vocational skills: specific focus on skills in work and education
extracted: (1) treatment characteristics: (a) CR with or without • Social skills: specific focus on social skills
focus on enhancement of social cognition; (b) type of CR • Community reintegration skills (CRS): broad focus on skills in
approach with ‘drill & practice,’ ‘drill & strategy’ or ‘strategy- several domains such as work, education, social skills, leisure,
based’ execution; (c) focus of PR program on the domain ‘voca- daily life skills, etc.)
tional skills,’ ‘social skills,’ or ‘community reintegration skills’
(CRS) (e.g. broad focus on a combination of several domains
such as work, education, social skills, leisure, daily-living skills,
etc.); (d) treatment goal integration between CR and PR, with Calculating moderator effects
CR and PR executed separately with different treatment goals, For outcomes with a significant heterogeneity ( p < 0.10; I 2 >
CR and PR executed separately with integrated treatment goals 50%), with at least seven studies that provided data, or at least
or both interventions merged into one intervention; (e) treatment five studies analyzing the outcome and presenting significant
intensity of the experimental condition; (2) type of control group: treatment effects, a multivariate meta-regression analysis was car-
(a) single PR intervention; or (b) PR intervention combined with ried out. The aim of this analysis was to establish whether associa-
an ‘active’ control condition; and (3) patient characteristics: (a) tions were present between effect sizes and moderator variables
age; (b) years of education; and (c) severity of disorder at baseline. when adjusting for other variables.
For severity of disorder at baseline the mean was calculated from
all scores based on percentile scores of norm population ‘schizo-
phrenia’ or ‘psychopathology’, derived from multiple scales. For Results
an overview of definitions for outcome measures and focus of
Data from 23 studies (1819 subjects) on the effect of the com-
PR programs, see Box 1.
bined intervention of PR and CR were included in the
meta-analysis. A total of 2246 records were screened for eligibility.
Statistical analyses After selecting title and abstracts, 208 full-text articles remained
for assessment of eligibility. A total of 182 articles were excluded
Calculation of effect sizes
from the analysis because the article was not a (published) RCT
For continuous outcomes the standardized mean difference
(N = 68), the intervention was not a combination of PR and CR
(SMD) was calculated. For dichotomous outcomes the odds
(N = 66), the outcomes were not based on functioning measure-
ratio (OR) for events was converted to a SMD following the con-
ments (N = 33), the combined intervention was not compared
version method of Chinn (Chinn, 2000). All outcomes are
with a single PR or CR intervention (n = 14), or the article was
reported with 95% confidence intervals (CIs). Random effects
published in a non-English language (N = 1). Three of the
models were used to calculate the overall effect and were weighted
included articles were part of one of the included studies.
by the inverse variance (Higgins and Green, 2008). For each out-
Only one study was found that (also) compared the effect of a
come, missing data were noted. If information on missing cases
combination of PR and CR to a stand-alone CR intervention.
was not reported, we contacted the authors. When available,
Because the results of one single study cannot be pooled, this
data were used that controlled for missing data (for example,
comparison was excluded from the meta-analysis and the results
imputed using regression methods). In several of the included
from this comparison are described separately. Consequently, a
studies, time-to-event data were reported inconsistently, and
meta-analysis was conducted with one comparison, including
often incompletely. Therefore, it was impossible to analyze these
23 studies comparing the effect of the combined intervention
results.
of PR and CR to a stand-alone PR intervention. A more detailed
description of the study selection is presented in the flow chart
Meta-analytic procedure
in Fig. 1.
Meta-analyses were conducted using RevMan 5.2 (Nordic
Cochrane Centre, 2012). Statistical heterogeneity was assessed
by visual inspection of forest plots (crossing line of no effect),
Study characteristics
by χ2-tests (assessing the p-value) and by calculating the I 2 statis-
tic, which describes the percentage of observed heterogeneity that In the 23 included studies (N = 1819) the combination of CR ther-
would not be expected by chance. If p < 0.10 and I 2 exceeded 50%, apy and PR (N = 962) was compared with a stand-alone PR inter-
we considered heterogeneity to be substantial. vention (N = 857).
Psychological Medicine 1417

Fig. 1. Flow Chart selection of studies conform Prisma Guidelines.

Patient characteristics Characteristics of combined treatment

The mean age of all the participants in the studies was 38.15 years In six studies (26.1%) PR and CR were executed separately with
(S.D. = 4.00; range = 28.5–44.07), the mean duration of illness was different outcome goals, whereas in 12 studies (52.2%) PR and
14.36 years (S.D. = 5.61; range = 5.80–25.09), the mean number of CR were executed separately with integrated treatment goals,
years of education was 12.43 years (S.D. = 1.48; range = 9.75– and in five studies (21.7%) PR and CR were merged into one
15.00) and 36.6% of all participants were female (S.D. = 10.10%; intervention. The mean treatment intensity for PR programs
range = 10.59–54.05%). The mean percentile score (derived from could not be extracted. The mean treatment intensity in the CR
multiple scales) was 30.45 for baseline symptom severity (S.D. = intervention was 3.67 h per week. An overview of the study char-
19.84; range = 0.20–71.70), 56.17 for baseline severity in cognitive acteristics is presented in Table 1.
impairment (S.D. = 25.34; range = 8.80–99.90) and 57.24 for base-
line functional impairment (S.D. = 29.11; range = 14.70–99.90).
Synthesis of results
Characteristics of CR interventions
A meta-analysis was performed on studies comparing the effect of
In nine studies (39.1%) the CR was based on a ‘drill and practice’
PR plus CR to the effect of PR alone. The general outcomes of the
approach, while 12 studies (52.2%) had a ‘drill plus strategy’-
meta-analysis are presented per outcome category. A more
based CR approach and two studies (8.7%) had a ‘strategy-based’
detailed description of all outcomes is presented in Table 2.
CR approach. In six studies (26.1%) the CR intervention was
Forest plots of all outcomes are presented in online supplemen-
extended by social cognitive training.
tary Appendix 5, with the type of PR program categorized in sep-
arate subgroups.
Characteristics of PR programs
In 11 studies (47.8%) the PR intervention within the combined
intervention was focused on ‘vocational skills’ (work or educa- Global cognition
tion). In two studies (8.7%) the PR intervention was focused on The combination of CR and PR led to significant favorable out-
‘social skills,’ In ten studies (43.5%) a broad PR program was comes for global cognition [SMD = 0.31 (0.17–0.45); Z = 4.40;
used, with a focus on several domains of ‘CRS. p < 0.01].
1418 Daniëlle van Duin et al.

Vocational functioning [SMD = −0.19 (−0.41 to 0.03); Z = 1.68, p < 0.09] compared
The combination of CR and PR led to significant favorable out- with adding CR to all other types of vocational PR [SMD = 0.56
comes for employment rate [SMD = 0.41 (0.10–0.72); Z = 2.62; (0.26–0.86); Z = 3.67, p < 0.01]. There was no evidence that the
p < 0.01] compared with a single PR intervention. The combined type of control group (adding an active control condition or
intervention also had significant favorable outcomes for hours not), the type of PR program (focus on vocational, social or
worked [SMD = 0.31 (0.04–0.58); Z = 2.28; p < 0.05], job duration CRS), or the addition of social cognitive training to the CR inter-
[SMD = 0.48 (0.30–0.67); Z = 5.10; p < 0.01] and quality of vention was a moderator variable for any of the outcomes on
performance in work or education [SMD = 0.76 (0.15–1.36); functioning.
Z = 2.45; p < 0.05]. Only for wages were these effects not signifi-
cant [SMD = 0.25 (−0.07 to 0.58); Z = 1.53; p = 0.13].
Quality of evidence
Social functioning
Using the GRADE method (Guyatt et al., 2011), many outcomes
The combination of CR and PR had significant favorable out-
were downgraded by at least one level because of the risk of bias
comes for social skills [SMD = 0.24 (0.10–0.38); Z = 3.29; p <
(e.g. incomplete outcome data) and imprecision (the analyses
0.01) compared with a single PR intervention. However, these
included few participants or events). The GRADE quality level
effects were not significant for the number and quality of relation-
was ‘high’ for the evidence of job duration and global cognition,
ships [SMD = 0.07 (−0.18 to 0.33); Z = 0.55; p = 0.58].
with no serious risk of bias, inconsistency, indirectness or impre-
cision. A ‘moderate’ quality level was assigned to the evidence of
Community functioning
social and occupational functioning. The results for employment
The combination of CR and PR had no significant beneficial
rate, hours worked, the number and quality of relationships, and
effects for independent and daily-living skills [SMD = 0.22
for social skills were of ‘low’ quality. Finally, the outcomes with a
(−0.04 to 0.48); Z = 1.63; p = 0.10], role adjustment and perform-
‘very low’ GRADE quality level were: work and education quality,
ance [SMD = −0.14 (−0.64 to 0.36); Z = 0.54; p = 0.59], and
wages, role adjustment and performance, and independent and
social and occupational functioning [SMD = 0.06 (−0.09 to
daily-life functioning. These outcomes were assigned a ‘very
0.22); Z = 0.83; p = 0.41] when it was compared with a single
low’ level because of (very) serious risk of bias, imprecision, and
PR intervention.
inconsistency. The GRADE quality levels, and reasons for down-
grading, are presented for each outcome in Table 2 and with more
Moderating effect of patient and treatment variables detail in online supplementary Appendix 3.
Patient characteristics
The number of years of education was a significant negative
Discussion
predictor for employment rate (B = −0.94; p < 0.01), wages
(B = −0.74; p < 0.01) and hours worked (B = −0.60; p < 0.01), The present meta-analysis on the boosting effect of adding CR to
with a cutoff point (median split) of 12.16 years of education. different types of PR on real-life functioning in people with SMI is
There was no evidence that the age of the participants, or the the most comprehensive to date, containing 23 trials and 1819
baseline severity of the disorder, was a key moderator variable participants. A former meta-analysis on the topic of augmenting
for any of the functional outcomes. PR with CR (NICE, 2014), contained 533 participants in six stud-
ies published between the years 2005 and 2009. This review
Characteristics of CR intervention focused on the boosting effect of adding CR to vocational PR.
The type of CR approach was a significant predictor for employ- A more recent meta-analysis on the effect of CR on vocational
ment rate (B = 0.34; p < 0.01), indicating that the effects are stron- outcomes (Chan et al., 2015) contained nine studies. This review
ger when CR is more focused on a ‘drill plus strategy’ approach combined studies analyzing the effect of stand-alone CR with
than a ‘drill and practice’ approach or a ‘strategy-based’ approach. studies analyzing the boosting effect of adding CR to vocational
In addition, post-hoc analysis revealed that effects are significantly PR. Results of these analyses on vocational outcomes were incon-
smaller when CR is focused on a ‘strategy-based’ approach sistent, with Chan et al., (2015) finding beneficial results and
(B = 0.34; p < 0.05), indicating that mere ‘strategy-based’ CR is NICE (2014) concluding that the evidence was too limited to
the least effective type of CR. make a recommendation. The results of the present meta-analysis
on multiple functional outcomes confirm earlier findings of mod-
Characteristics of combined treatment erator analyses in two reviews on the effectiveness of CR, showing
Treatment intensity was a significant negative predictor for favorable effects for the combined intervention of PR and CR on
employment rate (B = −0.74; p < 0.01), with a cutoff point of real-life functioning (McGurk et al., 2007a, 2007b; Wykes et al.,
3.2 h of CR per week, and 33 CR sessions. Treatment goal integra- 2011). Our results indicate that beneficial effects of the combined
tion between PR and CR was a significant positive predictor for intervention over PR alone, are most prominent on the domains
employment rate (B = 0.50; p < 0.05). In addition, when the inter- of vocational and social functioning. In addition, a favorable effect
ventions were not merged, but executed separately, it also led to of the combined intervention on global cognition was found. No
significant beneficial effects on the employment rate (B = 0.77; beneficial effects were found on community functioning.
p < 0.01). Finally, post hoc sensitivity analyses on the beneficial Beneficial effects on vocational outcomes were best achieved in
effect of adjunctive CR on vocational outcomes, indicated no participants with fewer years of education (less than 12 years),
meaningful differences in effect for adding CR to SE programs lower intensity of the intervention (less than 33 CR sessions
including IPS (n = 8), as to non-SE programs (n = 8). However, and less than 3 h of CR each week), a ‘drill plus strategy’ CR
sensitivity analyses indicate a meaningful difference of effect on approach, and integration of treatment goals for PR and CR.
‘employment rate’ for adding CR to ‘integrated’ types of SE The type of PR program, focusing on vocational-, social- or a
 Psychological Medicine
Table 1. Summary of study characteristics and outcome measures

Study characteristics Characteristics of combined treatment Patients characteristics Outcome measures

Treatment Agef
CR with PR focus on Treatment Intensityc Severity DI Vocational Social Community Global
Study Treatment (N) CR social domain: goal (h/p/w) Disorder Years of (Treatment Functioning Functioning Functioning Cognition
(RCT) Control (N) approach cognition Approacha integrationb (sessions) Baselined educatione group) (mean SDM) (mean SDM) (mean SDM) ES

Spaulding PR + CR (49) Drill & Yes CRS: Broad Merged No report High 11.67 35.5 No 0.28 0.21 No
et al. (1999) PR + AC (42) strategy PR – C:12.50 (H) (Low) (young)
– F:99.9 (L) 11.79

Hadas-Lidor PR + CR (36) Strategy Yes CRS: Broad Merged Total: High High No report No report 0.40 No 0.75 No
et al. (2001) PR (36) based PR 2.5 C:8.80 (H) –
120

Bell et al. PR + CR (72) Drill & Yes Vocational Goal CR: High Low 13.20 42.0 (old) 0.08 No No No
(2005); PR (79) practice skills: Non-SE integrated 6 S:41.1 (L) (High) 19.5
Fiszdon and – C:61.80 (L)
Bell (2004)

Vauth et al. PR + CR (47) Strategy Yes Vocational Goal CR: Low Low 12.70 28.5 0.50 No No No
(2005) PR (46) based skills: Non-SE integrated 3 S:40.10 (L) (High) (young)
16 5.8

Silverstein PR + CR (20) Drill & No CRS: Broad Merged No report High 9.75 (Low) 38.94 (old) No No 0.01 No
et al. (2005) PR + AC (20) practice PR – S:12.00 (H) –
– F:90.1 (L)

McGurk et al. PR + CR (25) Drill & No Vocational Goal CR: Low Low 11.30 No report 0.98 No No 0.59
(2005, 2007a, PR (23) strategy skills: SE integrated 2 C:52.00 (H) (Low) –
2007b) 30 F:99.90 (L)

Linden- PR + CR (45) Drill & No CRS: Broad Goal CR: High High 10.69 43.58 (old) 0.42 No No 0.64
mayer et al. PR + AC (40) strategy PR integrated 3 C:46.00 (L) (Low) 25.09
(2008) 44 S:47.30 (H)

Bell et al. PR + CR (99) Drill & Yes Vocational Goal CR: High Low 12.22 41.94 (old) 0.25 No No No
(2008a, PR (75) practice skills: SE integrated 5.08 S:31.85 (H) (High) 17.01
2008b, 2014) – C:78.80 (L)

Cavallero PR + CR (58) Drill & No CRS: Broad Apart Total: High High 12.10 33.2 −0.05 0.27 0.34 No
et al. (2009) PR + AC (42) strategy PR 9.83 / - S:11.05 (H) (Low) (young)
CR: Low F:54.00 (L) 8.28
3/-

McGurk et al. PR + CR (18) Drill & No Vocational Goal CR: Low High 12.22 45.5 (old) 0.56 No No 0.26
(2009) PR (16) strategy skills: Non-SE integrated 1.75 C:50.00 (H) (High) 23.2
–

Silverstein PR + CR (47) Drill & No Social Skills: Merged No report Low 11.55 38.17 (old) No 0.50 No No
et al. (2009) PR (35) practice Conversation – F:52.00 (L) (Low) –
Skills –

Vita et al. PR + CR (30) Drill & No CRS: Broad Apart CR: Low Low 10.83 36.87 No No 0.11 0.29
(2011) PR + AC (28) practice PR 1.5 C:57.90 (L) (Low) (young)
(CACR)g 48 S:55.95 (L) 14.80
F:35.3 (H)

Vita et al. PR + CR (26) Drill & Yes CRS: Broad Goal CR: Low Low 10.00 37.15 No No −0.08 0.34
(2011) PR + AC (28) strategy PR integrated 1.5 C:77.30 (L) (Low) (young)
(IPT-Cog)g 48 S:71.70 (L) 14.94
F:39.45 (H)

1419
 1420
Bowie et al. PR + CR (38) Drill & No CRS: Broad Goal CR: Low Low 12.90 41.3 (old) 0.82 −0.08 −0.11 0.73 (PT)
(2012) PR + AC (38) strategy PR integrated 2 / 12 C:95.50 (L) (High) 20 0.71 (FU)
Total: Low S:38.20 (L)
2 / 24

Lee (2013) PR + CR (33) Drill & No CRS: Broad Apart CR: Low Low 12.87 43.53 (old) 1.59 0.09 −0.14 No
PR (33) practice PR 1.67 S:30.80 (H) (High) 17.75
– C:71.80 (L)

Tan and PR + CR (36) Drill & No CRS: Broad Goal CR: High Low 11.00 32.70 No 0.31 No No
King (2013) PR + AC (34) strategy PR integrated 5 F:84.10 (L) (Low) (young)
– S:52.00 (L) 9.28

Kidd et al. PR + CR (19) Drill & No Vocational Merged Total: High Low 11.90 33.7 0.68 No No No
(2014) PR (18) strategy skills: SE - / 55.5 S:14.77 (H) (Low) (young)
(education) CR: Low C:99.9 (L) 6.7
1.67 / -

Au et al. PR + CR (45) Drill & No Vocational Apart CR: High High 15.00 35.38 −0.22 No No No
(2015) PR (45) practice skills: SE 6 S:0.20 (H) (High) (young)
(integrated) – C:36.3 (H) 11.33
F:29.36 (H)

Kurtz et al. PR + CR (32) Drill & No Social Skills: Apart Total: Low Low 12.50 36.1 No 0.13 No No
(2015) PR + AC (32) practice Broad SS 4.42 / - S:36.63 (L) (High) (young)
CR: Low C:77.85 (L) 12.8
2.5 / - F:42.10 (H)

McGurk et al. PR + CR (57) Drill & No Vocational Goal No Report High No report 45.12 (old) 0.37 No No 0.50
(2015) PR + AC (50) strategy skills: SE integrated – C:52.00 (H) –
(enhanced) –

McGurk et al. PR + CR (28) Drill & No Vocational Goal CR: Low High No report 37.69 0.05 No No 0.33 (PT)
(2016) PR (26) Strategy skills: SE integrated 1.5 S:36.20 (L) (young) −0.06 (FU)
(enhanced) – C:55.00 (H) 10.84

Tsang et al. PR + CR (45) Drill & No Vocational Apart CR: High High 15.00 35.38 −0.17 No No No
(2016) PR + AC (45) practice skills: SE 6 C:36.30 (H) (High) (young)
(integrated) 36 S:0.20 (H) 11.33
F:14.70 (H)

Yamaguchi PR + CR (57) Drill & No Vocational Goal CR: Low High 14.56 34.66 0.85 No 0.18 0.61
et al. (2016) PR (54) strategy skills: SE integrated – S:20.63 (H) (High) (young)
24 F:42.10 (H) –

Total: Total: Total: Total: Total: Total: 6 CR high Total: Total: 9 older Total: Total: Total: Total:
23 RCT 1819 9 D&P; with 11 VS 6 apart; 11 CR low 12 low; 10 high 12 younger 16 7 9 9
(962/ 857) 12 D&S; social 2 SS 12 goal int; 3 total high 11 high 10 low
AC: 10 2 SB cogn:6 10 CRS 5 merged 2 total low

PR = Psychiatric Rehabilitation; CR = cognitive remediation; AC = Attention Control group; CRS = Community Reintegration skills (PR focused on different domains of community functioning); C = cognitive; S = symptoms; F = functioning; DI = duration of
illness; ES = effect size; PT = post treatment; FU = follow-up.
a
PR approach: SE = supported employment (first-place-then-train strategy, focus on job placement in competitive employment, including IPS); Non-SE = other programs for vocational rehabilitation; Integrated SE = SE augmented with work-related
social skills training; Enhanced SE = providing cognitive information to SE-coaches; VR = Vocational Rehabilitation; Conversation S = Conversation skill training (skills like recognizing (non-)verbal cues, starting and ending a conversation, keeping a
conversation going); Broad SS = Broad social skill training (different domains of social skills, like: conversation skills, assertiveness and friendship skills); Broad PR = Broad Psychiatric Rehabilitation (focus on multiple skills for community reintegration,
including: skills for work, education, social skills, self-care, daily living skills, medication management, leisure, transportation skills, etc).

Daniëlle van Duin et al.

b
Treatment goals: Apart = CRT and PR are separately executed without integration of and/or adaptation of one homogeneous treatment goal; Goal integrated = both CRT and PR are separately executed with treatment goals adapted to each other by
e.g. discussion/bridging groups; Merged = Both CRT and PR have been merged into one intervention;
c
Treatment Intensity: CRT: on average 3.23 h p/w of treatment and 33 sessions; cut off point: h p/w ⩽3.2; number of sessions ⩽33; Total: on average 4.69 h p/w and 43.88 sessions; cut off point: h p/w < 4.7; number of session <44; when both sessions and
h p/w has been indicated, overall judgement of treatment intensity has been based on the measurement which has relatively the highest deviation from the cutoff point;
d
Severity Disorder Baseline: All scores based on percentile scores of norm population schizophrenia or psychopathology, lower percentile score is higher baseline severity; Symptoms based on BPRS, PANSS, CDSS, SAPS & SANS; Functioning based on
GAF, QLS, QLS-B, MCAS, BCSM, AIPSS, HoNOS & MMLT; Cognition based on MMSE, WAIS III, WRAT-3, WAIS-R & Raven Progressive Matrices; median percentile score Functioning: 47.05; median percentile score Symptoms: 36.2; median percentile score
Cognition: 55;
e
Years of Education: median years of education is 12.16 years. Cut-off point is 12.16 years;
f
Mean age treatment group = 37.947; cut-off point age <37.95 years.
g
Study: Both interventions in this study were sufficient CR interventions, both combined with PR, and were both separately compared with PR alone. Based on this, these two interventions were considered as separate comparisons/studies.
Psychological Medicine 1421

Table 2. Summary of outcomes and GRADE level

Effect size Quality of

Number of Number of (SMD; 95% CI) and Heterogeneity Heterogeneity evidence
Outcome studies (K) participants (n) p* χ2 ( p) I2 (GRADE)

Vocational functioning
Employment Rate 11 967 0.41 (0.10–0.72) χ2: 64.87 85% Low2b
p < 0.01 ( p < 0.01)
Hours Worked 6 491 0.31 (0.04–0.58) χ2: 17.55 72% Lowb,e
p < 0.05 ( p < 0.01)
Job Duration (weeks) 6 399 0.48 (0.30–0.67) χ2: 6.91 28% High
p < 0.01 ( p = 0.23)
Wages 5 340 0.25 (−0.07 to χ2: 19.30 79% Very Low2b,d
0.58) ( p < 0.01)
p = 0.13
Work/Education 4 248 0.76 (0.15–1.36) χ2: 28.26 89% Very lowa,2b,2d
Quality p < 0.05 ( p < 0.01)
Social functioning
Social Skills 5 339 0.24 (0.10–0.38) χ2: 3.89 0% Lowa,d
p < 0.01 ( p = 0.42)
Relationships 4 251 0.07 (−0.18 to χ2: 4.34 31% Lowa,d
0.33) ( p = 0.23)
p = 0.58
Community functioning
Social and 7 459 0.06 (−0.09 to χ2: 2.22 0% Moderatea
Occupational 0.22) ( p = 0.90)
Functioning p = 0.43
Independent/Daily 4 284 0.22 (−0.04 to χ2: 6.65 55% Very
Life Functioning 0.48) ( p = 0.08) low2a,b,d,e
p = 0.10
Role Adjustment and 1 60 −0.14 (−0.64 to Not Applicable Not Applicable Very lowa,2d
Performance 0.36)
p = 0.59
Cognitive functioning
Global Cognition 9 565 0.31 (0.17–0.45) χ2: 5.25 0% High
p < 0.01 ( p = 0.73)
a. Risk of bias; b. Inconsistency; c. Indirectness; d. Imprecision; e. Publication/reporting bias.
The bold values are the ones that are significant.

broad range of community skills, was no significant moderator for relationships. This might indicate that the correlation between
any of the outcomes. cognitive limitations and the number of relationships is smaller
When focusing on vocational outcomes with a ‘low’ to ‘high’ than with social skills, which might be due to the fact that the
GRADE quality level of the pooled results (disregarding out- number of relationships can be influenced by many external fac-
comes with a ‘very low’ quality level), we found that augment- tors, such as social support (Couture et al., 2006), group context
ing PR with CR has a medium effect size on the employment (Gest et al., 2001), living circumstances, and financial strain
rate of participants, the amount of hours that people work (Mattsson et al., 2008).
and the duration of their job. These results are in line with The lack of favorable effects from providing CR as an adjunct
the fact that strong correlations are found between cognitive to PR on community functioning, means that increased cognitive
limitations in psychosis (and other SMI) and the level of func- skills show no enhanced results on a sample of different scales
tioning in work situations (McGurk and Meltzer, 2000; Bell measuring several domains of community functioning such as
et al., 2001; Gold et al., 2002; Dickinson et al., 2007). The bene- occupational functioning (leisure activities), independent daily
ficial effects of this combination, compared with PR alone, functioning and ‘role adjustment and performance’. These results
might be explained by: (a) a better uptake of PR lessons because are inconsistent with studies finding that cognition and commu-
of improved attention and recall; and (b) increased capacity to nity functioning are directly and indirectly correlated (Green,
apply PR lessons in work situations because of improved execu- 1996; Goldberg and Green, 2002; Aubin et al., 2009; Keefe and
tive functioning and planning. Harvey, 2012). This inconsistency may be due to differences in
The results on social functioning outcomes indicate that pro- the definition of community functioning that are used. Whereas
viding CR as an adjunct to PR has a small superior effect on social our definition excludes specific vocational outcomes into a separ-
skills. However, the enhancing effect is not seen in the number of ate domain, several of the before mentioned studies use a broad
1422 Daniëlle van Duin et al.

definition of community functioning, including outcomes on (2011), there was a narrow age range, with 65% of our included
employment and education. studies reporting a mean age between 30 and 40 years. Therefore,
Providing CR as an adjunct to PR shows a significant posi- as stated by Wykes et al., this conclusion is not as robust until
tive effect on global cognition, although the effect size in our other age groups have been tested in future research.
review (0.31) is smaller than seen in previous CR reviews of
Wykes et al., 2011 (0.45) and McGurk et al., 2007a, 2007b
(0.41). A possible explanation for this might be the differences Treatment moderator variables
in data management. Whereas we extracted data on global cog-
Several significant treatment moderators were detected for
nition as originally reported in the selected studies, and there-
employment outcomes. Our results confirm the finding of
fore could only include data from nine studies, the other two
Wykes et al. (2011) and McGurk et al. (2007a, 2007b) that the
reviews calculated global cognition for all included studies as
beneficial effects of PR programs plus CR are best achieved
an average across multiple reported cognitive outcomes.
with the ‘drill plus strategy’ approach of CR, rather than with a
However, the GRADE appraisal of evidence in our review
‘drill and practice’ approach. Our data add to this that a ‘drill
showed that the results on global cognition are ‘highly’ trust-
plus strategy’ approach is also superior to a mere ‘strategy-based’
worthy. Therefore, the small to moderate favorable effects on
approach. So, both massed practice and strategy training seem to
global cognition suggest that the cognitive training within the
be of critical importance for enhancing levels of daily functioning.
combined intervention works effectively in enhancing neuro-
The metacognitive component of strategy training is thought to
cognitive functioning, which might be an indication of a work-
help people to transfer skills from the training setting into their
ing mechanism for the combined intervention. Enhancing the
daily lives (Cella et al., 2015).
effect of a stand-alone PR intervention with CR might imply
When augmenting PR by adding CR, results indicate that the
that PR lessons can be learned and applied better because of
best vocational outcomes can be achieved when the interventions
improved cognitive skills. In addition, there might be other
are executed separately, but with integrated treatment goals. This
working mechanisms that we did not measure, like improved
means the goal of improving cognitive skills is targeted at the goal
metacognitive functioning, which might account for a better
of participating in employment or education and vice versa. To
transfer of cognitive gains to real-life functioning.
boost the effect of PR programs there should be a focus on how cog-
The type of PR program was no moderator variable for any of the
nitive limitations may impede functioning in PR activities and which
functioning outcomes. This indicates that superior effects of the
strategies can be used to deal with this. These results are in line with
combination of PR and CR are present both in specialized PR pro-
the review of Medalia and Saperstein (2013), which demonstrated
grams (e.g. IPS) as in broad PR programs (e.g. training of skills in
that CR is most likely to improve functional skills when the cognitive
various domains of functioning). In addition, the beneficial effects
training is linked to the specific demands in real-world settings.
were seen in PR programs with a variety of vocational and social
Finally, our results indicate that beneficial vocational effects are
goals, such as PR programs focusing on education, competitive
best achieved when the treatment intensity of the CR intervention
employment, sheltered employment and social skills. Only for one
is not too high (less than 33 CR sessions, with less than 3 h of
type of vocational PR (called ‘Integrated supported employment’
CR per week). This is in line with the finding that a relatively limited
in which SE is enriched with work-related social skills training),
amount of CR (e.g. 5–15 h) is sufficient to improve cognitive func-
no further augmenting effect of adjunctive CR was present. As a pos-
tioning (McGurk et al., 2007a, 2007b). Our result might be
sible explanation for this, the authors (Au et al., 2015; Tsang et al.,
explained by the fact that a higher CR intensity can be at the expense
2016) mention a plateau effect induced by the work-related social
of the time available for the PR program and that responsiveness in
skills training that had already pushed the effects to the upper limit.
obtaining work can be reduced when the combined treatment is so
The fact that beneficial effects of adjunctive CR are seen in PR
intensive that it forms an obstacle in searching for jobs, performing
programs with a variety of goals might be due to the fact that cog-
job interviews and being available on the job market.
nitive skills such as attention, working memory, executive func-
tioning, and verbal learning are basic requirements for The results of one single study (Bowie et al., 2012) give the first
vocational and social functioning. When cognitive limitations indication that providing PR as an adjunct to CR, compared with a
occur, this hampers real-life functioning in various activities stand-alone CR intervention, also has a favorable effect on voca-
within these domains. By training cognitive skills in relation to tional outcomes (0.49), social functioning (0.18) and global cogni-
the social context where they are required, one can improve real- tion (0.71). These results confirm earlier findings on moderator
life functioning in that particular context. variables in two meta-analyses on the effectiveness of CR
(McGurk et al., 2007a, 2007b; Wykes et al., 2011), in which
improvements in functioning were best achieved by combining
Patient moderator variables
PR with CR, to boost their effects. In their meta-analyses all
One significant patient moderator was detected for vocational out- included studies compared the combination of CR and PR with
comes; results indicate that beneficial effects can best be achieved in ‘any other intervention.’ The present study adds to these results
participants with fewer years of education. An explanation for this in that we compared the combined intervention specifically with
might be that patients with fewer years of education benefit most a stand-alone CR or PR intervention, which enables a tentative
from developing their cognitive skills during the rehabilitation pro- interpretation on the adjunctive effect of respectively CR and PR.
gram. This finding might imply that a possibility for better tailoring Enhancing the effect of a stand-alone CR intervention with PR
of the CR approach to patients with a higher education level is might suggest that improved cognitive skills can transfer better to
needed. No other patient moderators were found, suggesting that daily functioning when applied in the real-world directly, within
the combined intervention can be offered to a broad range of a PR program, and that enhanced cognitive skills become increas-
patients of various ages and baseline severities of the disorder. ingly relevant for clients when they can be employed within the
However, as seen in the Wykes review on the effects of CR context of a goal that they have prioritized for themselves.
Psychological Medicine 1423

Strengths and limitations program. At this point, no favorable effects on community func-
tioning were detected. In order to improve vocational functioning,
The strength of this study is the focus on the effect of the com-
it appears important to integrate and apply cognitive goals of CR
bined intervention of PR and CR on real-life functioning. This
within the vocational goals of the PR, and vice versa.
analysis is clinically relevant because it is highly unusual for CR
to be a stand-alone intervention in schizophrenia or other SMI Supplementary material. The supplementary material for this article can
these days. The study helps to understand the nature of the rela- be found at https://doi.org/10.1017/S003329171800418X.
tionship between PR and CR with greater specificity, by compar-
ing the combined intervention with a stand-alone PR and CR Author ORCIDs. Daniëlle van Duin, 0000-0001-8239-9999.
intervention, by establishing the effect on separate domains of Acknowledgments. We would like to acknowledge Angita Peters for her
functioning and by assessing moderator variables. help with the literature search and Filip Smit and Simone Onrust for their stat-
The main limitation of this meta-analysis is the amount and istical advice. This research received no specific grant from any funding
heterogeneity of included data. The number of included studies agency, commercial or not-for-profit sectors.
was scarcely high enough to enable analysis of moderator vari-
ables. Most included studies have a small sample size, with a Conflict of interest. Disclosures: The authors have declared that there are no
conflicts of interest in relation to the subject of this study.
mean sample of 78 participants (range 34–174). This problem
is partly resolved by pooling the results of multiple studies, but Ethical standards. All procedures performed in studies (of any of the
optimal information size still was not met for several outcomes. authors included in the literature review) involving human participants were
In addition, comparing the effects between different domains of in accordance with the ethical standards of the institutional and/or national
real-life functioning should be done with caution. As not all research committee and with the 1975 Helsinki declaration and its later
included studies reported outcomes on the separate domains of amendments or comparable ethical standards. For this type of study (literature
functioning (vocational-, social-, and community functioning), review within a guideline) additional formal consent is not required.
effects were calculated for different subsets of studies. As a conse-
quence, effect sizes in the separate domains of functioning might
be affected by characteristics of the subsets, including the number
of studies, the ceiling- and floor effects, and the variability within References
the subset. Also the effect on global cognition was analyzed for a Anthony WA, Cohen MR, Farkas M and Gagne C (2002) Psychiatric
subset of nine studies, originally reporting on the effects of the com- Rehabilitation, 2nd Edn. Boston, MA: Center for Psychiatric
bined intervention on global cognition. Another problem is that Rehabilitation, Boston University.
many studies did not measure outcomes both post-treatment and Au DWH, Tsang HWH, So WWY, Bell MD, Cheung V, Yiu MGC, Tam KL
at follow-up. Only five studies reported their outcomes on both and Lee GT (2015) Effects of integrated supported employment plus cog-
nitive remediation training for people with schizophrenia and schizoaffect-
measurement points (Bell et al., 2005; McGurk et al., 2005;
ive disorders. Schizophrenia Research 166, 297–303.
Silverstein et al., 2005; Bowie et al., 2012; Tan and King, 2013). Aubin G, Stip E, Gelinas I, Rainville C and Chapparo C (2009) Daily activ-
ities, cognition and community functioning in persons with schizophrenia.
Directions for future research Schizophrenia Research 107, 313–318.
Bell M, Bryson G, Greig T, Corcoran C and Wexler BE (2001)
Future research should focus more on the effects of CR on real-life Neurocognitive enhancement therapy with work therapy. effects on neuro-
functioning, and how these effects can by enhanced. Within stud- psychological test performance. Archives of General Psychiatry 58, 763–768.
ies on the combined intervention of CR and PR, there should be a Bell MD, Bryson GJ, Greig TC, Fiszdon JM and Wexler BE (2005)
greater focus on the effects of PR as an adjunct to CR compared Neurocognitive enhancement therapy with work therapy: productivity out-
with a single CR intervention. In this research, possible working comes at 6- and 12-month follow-ups. Journal of Rehabilitation Research &
mechanisms for the favorable effects of the combined interven- Development 42, 829–838.
tion should be further examined. For example by testing on boost- Bell MD, Zito W and Wexler BE (2008a) Neurocognitive enhancement therapy
ing effects of improved meta-cognition and interaction effects and competitive employment in schizophrenia: effects on clients with poor
community functioning. American Journal of Psychiatric Rehabilitation 11,
between improved cognitive skills and real-life functioning. In
109–122.
addition, a broad range of age groups should be tested in order Bell MD, Zito W, Greig T and Wexler BE (2008b) Neurocognitive enhance-
to obtain more knowledge on age as a possible moderator vari- ment therapy with vocational services: work outcomes at two-year
able. Furthermore, these studies should make a clear distinction follow-up. Schizophrenia Research 105, 18–29.
between post-treatment and follow-up results, in order to evaluate Bell MD, Choi KH, Dyer C and Wexler BE (2014) Benefits of cognitive
whether potential improved functioning is sustainable. Moreover, remediation and supported employment for schizophrenia patients with
larger sample sizes are needed in this specific field of expertise poor community functioning. Psychiatric Services 65, 469–475.
to achieve a better power on separate outcomes. Finally, Bowie CR and Harvey PD (2006) Cognitive deficits and functional outcome
meta-analyses on the effect of enhancing PR programs with CR in schizophrenia. Neuropsychiatric Disease and Treatment 2, 531–536.
should further analyze which specific domains of real-life func- Bowie CR, McGurk SR, Mausbach B, Patterson TL and Harvey PD (2012)
Combined cognitive remediation and functional skills training for schizo-
tioning the potential effects are most prominent.
phrenia: effects on cognition, functional competence, and real-world behav-
ior. American Journal of Psychiatry 169, 710–718.
Conclusions Brüne M, Dimaggio G and Lysaker PH (2011) Metacognition and social
functioning in schizophrenia: evidence, mechanisms of influence and treat-
The results of this study indicate that adding CR to a broad range ment implications. Current Psychiatry Reviews 7, 239–247.
of PR programs can boost their effects on vocational- and social Cavallero R, Anselmetti S, Poletti S, Bechi M, Ermoli E, Cocchi F, Stratta P,
functioning. First indications were found that a synergetic mech- Vita A, Rossi A and Smeraldi E (2009) Computer-aided neurocognitive
anism also works the other way around, in which effects of CR remediation as an enhancing strategy for schizophrenia rehabilitation.
can be generalized better to real-life functioning by adding a PR Psychiatry Research 169, 191–196.
1424 Daniëlle van Duin et al.

Cella M, Reeder C and Wykes T (2015) Lessons learnt? The importance of Database of Systematic Reviews 9: Art. No.: CD008297. doi: 10.1002/
metacognition and its implications for cognitive remediation in schizophre- 14651858.CD008297.pub2. (Accessed September 2013).
nia. Frontiers in Psychology 6, 1259. Accessed September 2015. Kontis D, Huddy V, Reeder C, Landau S and Wykes T (2012) Effects of age
Chan JYC, Hirai HW and Tsoi KKF (2015) Can computer-assisted cognitive and cognitive reserve on cognitive remediation therapy outcome in patients
remediation improve employment and productivity outcomes of patients with schizophrenia. American Journal of Geriatric Psychiatry 21, 218–230.
with severe mental illness? A meta-analysis of prospective controlled trials. Kurtz MM, Moberg PJ, Gur RC and Gur RE (2001) Approaches to cognitive
Journal of Psychiatric Research 68, 293–300. remediation of neuropsychological deficits in schizophrenia: a review and
Chinn S (2000) A simple method for converting an odds ratio to effect size for meta-analysis. Neuropsychology Review 11, 197.
use in meta-analysis. Statistics in Medicine 19, 3127–3131. Kurtz MM, Mueser KT, Thime WR, Corbera S and Wexler BE (2015) Social
Corrigan PW, Mueser KT, Bond GR, Drake RE and Solomon P (2008). skills training and computer-assisted cognitive remediation in schizophrenia.
Principles and Practice of Psychiatric Rehabilitation. New York/London: Schizophrenia Research 162, 35–41.
The Guilford Press. Lee WK (2013) Effectiveness of computerized cognitive rehabilitation training
Couture SM, Penn DL and Roberts DL (2006). The functional significance of on symptomatological, neuropsychological and work function in patients
social cognition in schizophrenia: a review. Schizophrenia Bulletin 32(suppl. with schizophrenia. Asia-pacific Psychiatry 5, 90–100.
1), S44–S63. Lindenmayer JP, McGurk SR, Mueser KT, Khan A, Wance D, Hoffman L,
Dickinson D, Bellack AS and Gold JM (2007) Social/communication skills, Wolfe R and Xie H (2008) A randomized controlled trial of cognitive
cognition, and vocational functioning in schizophrenia. Schizophrenia remediation among inpatients with persistent mental illness. Psychiatric
Bulletin 33, 1213–1220. Services 59, 241–247.
Fiszdon JM and Bell MD (2004) Remédiation cognitive et thérapie occupa- Lysaker PH, Vohs J, Minor KS, Irarrázaval L, Leonhardt B, Hamm JA,
tionnelle dans le traitement ambulatoire du patient souffrant de Kukla M, Popolo R, Luther L, Buck KD, Wasmuth S and Dimaggio G
schizophrénie. Santé mentale au Québec 29, 117–142.. (2015) Metacognitive deficits in schizophrenia: presence and associations
Gest SD, Graham-Bermann SA and Hartup WW (2001) Peer experience: with psychosocial outcomes. Journal of Nervous and Mental Disease 203,
common and unique features of number of friendships, social network cen- 530–536.
trality, and sociometric status. Social Development 10 , 23–40. Mattsson M, Topor A, Cullberg J and Forsell Y (2008) Association between
Gigantesco A, Vittorielli M, Pioli R, Falloon IRH, Rossi G and Morosini P financial strain, social network, and five-year recovery from first-episode
(2006) The VADO approach in psychiatric rehabilitation: a randomized psychosis. Social Psychiatry and Psychiatric Epidemiology 43, 947–952.
controlled trial. Psychiatric Services 57, 1778–1783. McGurk SR and Meltzer HY (2000) The role of cognition in vocational func-
Gold JM, Goldberg RW, McNary SW, Dixon LB and Lehman AF (2002) tioning in schizophrenia. Schizophrenia Research 45, 175–184.
Cognitive correlates of job tenure among patients with severe mental illness. McGurk SR and Mueser KT (2004) Cognitive functioning, symptoms, and
American Journal of Psychiatry 159, 1395–1402. work in supported employment: a review and heuristic model.
Goldberg TE and Green MF (2002) Neurocognitive functioning in patients Schizophrenia Research 70, 147–173.
with schizophrenia: an overview. In Davis KL, Charney D, Coyle JT and McGurk SR and Mueser KT (2008) Response to cognitive rehabilitation in
Nemeroff C (eds), Neuropsychopharmacology: The Fifth Generation of older versus younger persons with severe mental illness. American
Progress. Philadelphia: Lipincott, Williams & Wilkins, pp. 657–669. Journal of Psychiatric Rehabilitation 11, 90–105.
Green MF (1996) What are the functional consequences of neurocognitive McGurk SR, Mueser KT and Pascaris A (2005) Cognitive training and sup-
deficits in schizophrenia? American Journal of Psychiatry 153, 321–330. ported employment for persons with severe mental illness: one-year results
Green MF and Horan WP (2010) Social cognition in schizophrenia. Current from a randomized controlled trial. Schizophrenia Bulletin 31, 898–909.
Directions in Psychological Science 19, 243–248. McGurk SR, Mueser KT, Feldman K, Wolfe R and Pascaris A (2007a)
Grynszpan O, Perbal S, Pelissolo A, Fossati P, Jouvent R, Dubal S and Cognitive training for supported employment: 2–3 year outcomes of a ran-
Perez-Diaz F (2011) Efficacy and specificity of computer-assisted cognitive domized controlled trial. American Journal of Psychiatry 164, 437–441.
remediation in schizophrenia: a meta-analytical study. Psychological McGurk SR, Twamley EW, Sitzer DI, McHugo GJ and Mueser KT (2007b)
Medicine 41, 163–173. A meta-analysis of cognitive remediation in schizophrenia. American
Guyatt GH, Oxman AD, Schuneman HJ, Tugwell P and Knottnerus A Journal of Psychiatry 164, 1791–1802.
(2011) GRADE guidelines: a new series of articles in the Journal of McGurk SR, Mueser KT, DeRosa TJ and Wolfe R (2009) Work, recovery,
Clinical Epidemiology. Journal of Clinical Epidemiology 64, 380–382. and comorbidity in schizophrenia: a randomized controlled trial of cogni-
Hadas-Lidor N, Katz N, Tyano S and Weizman A (2001) Effectiveness of tive remediation. Schizophrenia Bulletin 35, 319–335.
dynamic cognitive intervention in rehabilitation of clients with schizophre- McGurk SR, Mueser KT, Xie H, Welsh J, Kaiser S, Drake RE, Becker DR,
nia. Clinical Rehabilitation 15, 349–359. Bailey E, Fraser G, Wolfe R and McHugo GJ (2015) Cognitive enhance-
Higgins JPT and Green S (2008) Cochrane Handbook for Systematic Reviews ment treatment for people with mental illness Who Do Not respond to sup-
of Interventions. Chichester, Hoboken, NJ, England: Wiley Blackwell. ported employment: a randomized controlled trial. American Journal of
Hurford IM, Kalkstein S and Hurford MO (2011) Cognitive Rehabilitation in Psychiatry 172, 852–861.
Schizophrenia. Psychiatric Times. Available at http://www.psychiatrictimes. McGurk SR, Mueser KT, Xie H, Feldman K, Shaya Y, Klein L and Wolfe R
com/schizophrenia/cognitive-rehabilitation-schizophrenia. (2016) Cognitive remediation for vocational rehabilitation nonresponders.
Jablensky A, McGrath J, Herrman H, Castle D, Gureje O, Evans M, Carr V, Schizophrenia Research 175, 48–56.
Morgan V, Korten A and Harvey C (2000) Psychotic disorders in urban Medalia A and Saperstein AM (2013) Does cognitive remediation for schizo-
areas: an overview of the study on Low prevalence disorders. Australian phrenia improve functional outcomes? Current Opinion in Psychiatry 26,
and New Zealand Journal of Psychiatry 34, 221–236. 151–157.
Keefe RSE and Harvey PD (2012) Cognitive impairment in schizophrenia. In Modini M, Tan L, Brinchmann B, Wang MJ, Killackey E, Glozier N,
Geyer MA and Gross G (eds), Novel Antischizophrenia Treatments. Handbook Mykletun A and Harvey SB (2016) Supported employment for people
of Experimental Pharmacology, vol. 213. Berlin-Heidelberg: Springer-Verlag, with severe mental illness: systematic review and meta-analysis of the inter-
pp. 11–37. doi: 10.1007/978-3-642-25758-2. national evidence. British Journal of Psychiatry 209, 14–22.
Kidd SA, Kaur J, Virdee G, George TP, McKenzie K and Herman Y (2014) Moher D, Liberati A, Tetzlaff J and Altman DG (2009) Reprint – preferred
Cognitive remediation for individuals with psychosis in a supported education reporting items for systematic reviews and meta-analyses: the PRISMA
setting: a randomized controlled trial. Schizophrenia Research 157, 90–98. statement. Physical Therapy 89, 873–880.
Kinoshita Y, Furukawa TA, Kinoshita K, Honyashiki M, Omori IM, National Institute for Health and Care Excellence (2014) Psychosis and
Marshall M, Bond GR, Huxley P, Amano N and Kingdon D (2013) Schizophrenia in adults: prevention and management. Clinical Guideline
Supported employment for adults with severe mental illness. Cochrane CG178. NICE.
Psychological Medicine 1425

Nordic Cochrane Centre (2012) Review Manager (RevMan) Version 5.2. psychiatric rehabilitation approach. Canadian Journal of Psychiatry 56,
Cochrane Collaboration. 751–760.
Pilling S, Bebbington P, Kuipers E, Garety P, Geddes J, Martindale B, Tan BL and King R (2013) The effects of cognitive remediation on functional
Orbach G and Morgan C (2002) Psychological treatments in schizophrenia: outcomes among people with schizophrenia: a randomised controlled study.
II. Meta-analyses of randomized controlled trials of social skills training and Australian and New Zealand Journal of Psychiatry 47, 1068–1080.
cognitive remediation. Psychological Medicine 32, 783–791. Tsang HWH, Bell MD, Cheung V, Tam KL and Yeung WS (2016) Integrated
Radhakrishnan R, Kiluk BD and Tsai J (2016) A meta-analytic review of supported employment plus cognitive remediation training for people with
non-specific effects in randomized controlled trials of cognitive remediation schizophrenia. Hong Kong Medical Journal 22, S15–S18.
for schizophrenia. The Psychiatric Quarterly 87, 57–62. Vargas G, Strassnig M, Sabbag S, Gould F, Durand D, Stone L,
Revell ER, Neill JC, Harte M, Khan Z and Drake RJ (2015) A systematic Patterson TL and Harvey PD (2014) The course of vocational functioning
review and meta-analysis of cognitive remediation in early schizophrenia. in patients with schizophrenia: re-examining social drift. Schizophrenia
Schizophrenia Research 168, 213–222. Research. Cognition 1, e41–e46. (Accessed March 2014).
Silverstein SM, Hatashita-Wong M, Solak BA, Uhlhaas P, Landa Y, Vauth R, Corrigan PW, Clauss M, Dietl M, Dreher-Rudolph M, Stieglitz RD
Wilkniss SM, Goicochea C, Carpiniello K, Schenkel LS, Savitz A and and Vater R (2005) Cognitive strategies versus self-management skills as
Smith TE (2005) Effectiveness of a two-phase cognitive rehabilitation inter- adjunct to vocational rehabilitation. Schizophrenia Bulletin 31, 55–66.
vention for severely impaired schizophrenia patients. Psychological Medicine Vita A, De Peri L, Barlati S, Cacciani P, Deste G, Poli R, Agrimi E,
35, 829–837. Cesana BM and Sacchetti E (2011) Effectiveness of different modalities
Silverstein SM, Spaulding WD, Menditto AA, Savitz A, Liberman RP, of cognitive remediation on symptomatological, neuropsychological, and
Berten S and Starobin H (2009) Attention shaping: a reward-based learn- functional outcome domains in schizophrenia: a prospective study in a real-
ing method to enhance skills training outcomes in schizophrenia. world setting. Schizophrenia Research 133, 223–231.
Schizophrenia Bulletin 35, 222–232. Wykes T and Dunn G (1992) Cognitive deficit and the prediction of rehabili-
Smith TE, Hull JW, Romanelli S, Fertuck E and Weiss KA (1999) Symptoms tation success in a chronic psychiatric group. Psychological Medicine 22,
and neurocognition as rate limiters in skills training for psychotic patients. 389–398.
American Journal of Psychiatry 156, 1817–1818. Wykes T, Huddy V, Cellard C, McGurk SR and Czobor P (2011) A
Spaulding WD, Reed D, Sullivan M, Richardson C and Weiler M (1999) meta-analysis of cognitive remediation for schizophrenia: methodology
Effects of cognitive treatment in psychiatric rehabilitation. Schizophrenia and effect sizes. American Journal of Psychiatry 168, 472–485.
Bulletin 25, 657–676. Yamaguchi S, Sato S, Horio N, Yoshida K, Shimodaira M, Taneda A,
Swildens W, van Busschbach JT, Michon H, Kroon H, Koeter MW, Ikebuchi E, Nishio M and Ito J (2016) Cost-effectiveness of cognitive
Wiersma D and van Os J (2011) Effectively working on rehabilitation remediation and supported employment for people with mental illness: a
goals: 24-month outcome of a randomized controlled trial of the Boston randomized controlled trial. Psychological Medicine 47, 53–65.
Copyright © Cambridge University Press 2019