Journal of Alzheimer’s Disease 86 (2022) 511–524 511

DOI 10.3233/JAD-215098
IOS Press

Review

A Neuropsychological Perspective on
Defining Cognitive Impairment in the
Clinical Study of Alzheimer’s Disease:
Towards a More Continuous Approach
Roos J. Juttena,1 , Louisa Thompsonb,c,1,∗ , Sietske A.M. Sikkesd , Paul Maruffe ,
José Luis Molinuevof,g , Henrik Zetterbergh,i,j,k , Jessica Alberl,m , David Faustn , Serge Gauthiero ,
Michael Goldp , John Harrisond,q,r , Athene K.W. Leeb,c and Peter J. Snyderl
a Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
b Department of Psychiatry and Human Behavior, Alpert Medical School of Brown University, Providence, RI,
USA
c Memory and Aging Program, Butler Hospital, Providence, RI, USA
d Alzheimer Center Amsterdam, Department of Neurology, Amsterdam UMC, Vrije Universiteit, Amsterdam,

the Netherlands
e The Florey Institute of Neuroscience and Mental Health, Melbourne, VIC, Australia
f Barcelonaβeta Brain Research Center (BBRC), Pasqual Maragall Foundation, Barcelona, Spain
g Alzheimer’s Disease and Other Cognitive Disorders Unit, Hospital Clinic, Barcelona, Spain
h Department of Psychiatry and Neurochemistry, Institute of Neuroscience & Physiology, the Sahlgrenska

Academy at the University of Gothenburg, Mölndal, Sweden

i Clinical Neurochemistry Laboratory, Sahlgrenska University Hospital, Mölndal, Sweden
j Department of Neurodegenerative Disease, UCL Institute of Neurology, Queen Square, London, UK
k UK Dementia Research Institute at UCL, London, UK
l Department of Biomedical and Pharmaceutical Sciences, College of Pharmacy, The University of Rhode Island,

Kingston, RI, USA

m George and Anne Ryan Institute for Neuroscience, University of Rhode Island, Kingston, RI, USA
n Department of Psychology, University of Rhode Island, Kingston, RI, USA
o McGill Centre for Studies in Aging, Montréal, Canada
p AbbVie Inc., North Chicago, IL, USA
q Metis Cognition Ltd, Kilmington Common, UK
r Institute of Psychiatry, Psychology & Neuroscience, King’s College London, UK

Handling Associate Editor: Katherine Gifford

Accepted 20 December 2021

Pre-press 31 January 2022

1 These authors contributed equally to this work.

∗ Correspondence to: Louisa Thompson, PhD, Memory &
Aging Program, Butler Hospital, 345 Blackstone Blvd., Provi-
dence, RI 02906, USA. E-mail: louisa [email protected].

ISSN 1387-2877/$35.00 © 2022 – IOS Press. All rights reserved.

512 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum

Abstract. The global fight against Alzheimer’s disease (AD) poses unique challenges for the field of neuropsychology.
Along with the increased focus on early detection of AD pathophysiology, characterizing the earliest clinical stage of the
disease has become a priority. We believe this is an important time for neuropsychology to consider how our approach to
the characterization of cognitive impairment can be improved to detect subtle cognitive changes during early-stage AD. The
present article aims to provide a critical examination of how we define and measure cognitive status in the context of aging
and AD. First, we discuss pitfalls of current methods for defining cognitive impairment within the context of research shifting
to earlier (pre)symptomatic disease stages. Next, we introduce a shift towards a more continuous approach for identifying
early markers of cognitive decline and characterizing progression and discuss how this may be facilitated by novel assessment
approaches. Finally, we summarize potential implications and challenges of characterizing cognitive status using a continuous
approach.

Keywords: Cognitive assessment, early diagnosis, neuropsychology, preclinical Alzheimer’s disease, progression

BACKGROUND This clinical neuropsychological approach to

assessment has influenced the role of cognitive test-
Alzheimer’s disease (AD) is currently understood ing in AD research and clinical trials, including
as a neurodegenerative disease starting with a pre- screening procedures targeting specific groups for
clinical phase lasting 15 years or more wherein recruitment and the stratification of patient cohorts.
neuropathological change occurs but clinical symp- In clinical research, characterizing individuals as
toms remain subtle or absent [1–5]. Despite this either cognitively healthy or impaired based on a
presumed long presymptomatic continuum charac- single, cross-sectional screening assessment allows
terized by subtle and gradual cognitive change, all for a straightforward design. However, this almost
clinical diagnostic systems for AD imply that cog- certainly leads to the loss of a large amount of infor-
nitive function should be classified as either normal mation, as the transitions between baseline cognitive
or impaired as if there were an abrupt change at one function, age-related amnestic changes, and disease-
point [6, 7]. While this classification is practical for related decline can be subtle and difficult to measure
clinical settings, it also leads to a lack of precision and with precision. Therefore, it is important to examine
can limit the design of research and clinical trials that the pitfalls and potential alternatives to this approach
focus on preventing AD-related cognitive decline. as we seek to better characterize the preclinical phase
Defined in its broadest terms, the field of neu- of AD and consider ways to update the role of clinical
ropsychology is concerned with understanding the neuropsychology in the future of AD research.
convergence of behavior, emotion, cognition, and Theories and definitions of normal cognitive aging,
brain function. In clinical neuropsychology practice, such as those established by Salthouse [12] and
decisions about cognitive impairment are typically Lidenberger & von Oertzen [13], are shifting as
guided by the comparison of performance on one or a result of improved understanding of amyloid-␤
more tests to cut-off scores for abnormality derived (A␤) deposition and other neuropathological changes
from relevant normative data [8, 9]. Measures of pre- that can identify individuals without dementia who
morbid intellectual functioning aid in estimating the have neurodegenerative disease, and therefore may
extent of cognitive change, especially when testing not have normal (i.e., non-pathological) cognitive
is only done at a single point in time. The extent to aging. For example, in a recent longitudinal anal-
which any detected cognitive impairment reflects AD ysis of cognitively normal older adults, Harrington
is then determined by the profile or pattern of cogni- and colleagues [14] found that estimates of age-
tive performance as well as the presence or absence related decline in cognition were accentuated when
of supportive neuroimaging or other biomarker data cognitively unimpaired individuals with elevated A␤
[10]. The practice of single time point assessment pathology were considered in the “normal aging”
has developed out of necessity and practicality for group. Although the clinical criteria for MCI and
diagnostic purposes, and numerous studies have con- dementia due to AD provide useful societal and
sistently and convincingly shown the diagnostic value medical benchmarks, clinicians and researchers are
of a cross-sectional neuropsychological evaluation in still grappling with how to address and define the
distinguishing normal cognition from mild cognitive preclinical stage of AD. The 2018 National Insti-
impairment (MCI) and dementia [11]. tute on Aging–Alzheimer’s Association (NIA-AA)
 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum 513

Alzheimer’s Diagnostic Framework provides a bio- incipient AD can be quite heterogeneous [21, 22].
marker-based system for AD staging, while plac- Psychometric limitations of commonly employed
ing the clinical syndrome on a separate continuum. tests (e.g., floor- and ceiling effects) as well as inter-
Advantages and limitations of the NIA-AA Frame- pretive factors may impede a formal computational
work have been discussed in depth elsewhere [15], pattern analysis of cognitive test data, as the com-
but some key challenges with this framework include plexity of potential patterns across these variables
the inherent discordance between results obtained increases. For example, marked variation in the level
from biomarker testing and a neuropsychological of test performance within and across individuals and
evaluation (e.g., cognitively unimpaired with pos- the mathematics of comparing an expanding num-
itive AD biomarkers), which can be confusing ber of test scores, which can exponentially diminish
for both providers and patients [16]. Additionally, reliability of test patterns as a whole [23–25].
within-subject discrepancies between the clinical and Altogether, these challenges underline that there
biomarker categorization may affect the outcomes is no universal consensus on the actual definition of
of research and clinical trials involving cognitively cognitive health, nor any single validated approach
unimpaired individuals as a control group, given evi- to assess this complex construct. We will now review
dence that approximately 30% of such individuals two main complexities inherent in conventional neu-
likely have elevated cerebral A␤ [17]. ropsychological approaches to determining cognitive
Thus, we believe the time is right for neuropsy- impairment, that is, the use of cross-sectional norma-
chologists to examine how our characterization of tive data to define cognitive status, followed by the
the cognitive features of AD can evolve to better inclusion of subjective complaints in defining cogni-
describe subtle cognitive decline in preclinical AD, tive health.
and to potentially better reflect the more continuous
nature of the clinical AD syndrome when possible. The use of normative data

Neuropsychological test results are often inter-

CONCEPTUAL ISSUES AND preted based on cross-sectional normative data with
CHALLENGES IN DEFINING adjustment for age, sex, and sometimes education.
‘COGNITIVE HEALTH’ IN AGING The reliance on historical normative data obtained
from cross-sectional samples carries limitations.
Beginning with a broad view of practices around First, it has been shown that cognitive test per-
assessing cognitive health, one of the first apparent formance is cohort dependent. That is, a healthy
challenges is the fact that definitions and interpreta- 50-year-old person performs differently on a test
tions of cognitive decline vary widely. For example, today relative to how a healthy 50-year-old person
cognitive decline during older age is interpreted performed on the same test twenty years ago, even
differently across cultures, with some being more after adjusting for the higher educational level of
likely to attribute memory loss and other AD-related more recent cohorts [26]. As a result, the ‘average
symptoms to natural aspects of aging as opposed to performance’ on a given test may drift over time and,
pathological processes [18–20]. Clinical disciplines consequently, normative data (norms) may become
may also vary in their approaches to defining cogni- rapidly outdated. In addition, even after correction
tive health. Neurologists may assess cognitive health for years of education, it remains difficult to capture
with a combination of neurological examination, cognitive decline among highly educated individu-
brief cognitive screening measures, and neuroimag- als who likely possess substantial cognitive reserve
ing, whereas neuropsychologists primarily rely on [27]. There also exists a major and well-established
multi-domain cognitive assessments, as well as infor- problem when applying normative data to both highly
mant and self-reports of cognitive and functional educated and less educated individuals, as both ends
changes. Consequently, the discipline of the clinician of the continuum are generally represented poorly
assessing the patient will influence how cognitive in normative samples. Indeed, norms are based typ-
health is defined. ically on relatively homogeneous samples, and may
In addition to these broader challenges, measuring not generalize to other populations or birth cohorts
AD-specific cognitive decline and distinguishing it that differ across a number of other key variables (e.g.,
from potential age-related cognitive changes remains socioeconomic status, ethnicity, cultural heritage,
particularly difficult because the clinical profile of language fluency, and varying quality and models
514 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum

of education). With the rapid growth of the aging self-reports and partner/caregiver-reports of cogni-
population worldwide, additional issues are ampli- tive change [39]. Subjective cognitive decline (SCD),
fied, such as known ethno-racial disparities in the generally defined as the perception of worsening
prevalence, diagnosis, and treatment of AD [28, 29]. cognitive performance in the absence of neuropsy-
Despite these significant gaps in care, we nonetheless chologically detectable cognitive deficits, may be
continue to have limited reference data for clinically both a normal part of cognitive aging and an impor-
underserved and underrepresented racial and ethnic tant feature of early pathological changes in cognitive
populations [30] as well as the oldest-old segment function [40–42]. In the clinical staging scheme pro-
of our population. Though recent efforts have made posed in the NIA-AA Framework [43], subjective
progress in addressing these gaps [31–33], broader complaints are described as a possible clinical fea-
issues in cultural competency among clinicians and ture of preclinical AD Stage 2, reflecting that, in the
researchers and monocultural assumptions and prac- context of research focused on the earliest stages of
tices also hinder the adoption of more equitable AD, the utility and role of subjective reporting is still
assessment approaches [34, 35]. unclear. This is in part because it has been difficult to
Another challenge is that shifts in test perfor- craft sensitive and broadly applicable clinical criteria
mance across editions of tests may be partly or in order to assess SCD [44, 45]. Currently, there are
largely artifactual and relate to changes in inclusion- a variety of measures and criteria designed to capture
ary and exclusionary criteria that alter the overall the phenomenon, making it challenging to compare
composition of normative samples. For example, data across studies and to investigate whether SCD
more exclusionary criteria were added across the suc- is an early, reliable marker of declining cognitive
cessive versions of the Wechsler Adult Intelligence health due to a neurodegenerative disease [45]. As
Scale, thus creating more select samples of higher is the case with cognitive testing, assessing SCD
functioning individuals on average, which in large at only a single time point is likely to be less reli-
part explains the need to recalibrate norms and the able than repeated assessments over time. SCD often
presumed tendency of test scores to ‘improve’ in the worsens with stress and commonly co-occurs with
general population over time (i.e., the Flynn effect anxiety and depression [41, 46]. While anxiety and
[36]). Another problem is that drawing normative depression have been associated with increased risk
samples from older populations may unintentionally for AD, they are by no means specific to AD and
result in the inclusion of individuals with preclinical are known to have independent deleterious effects on
stage AD or other neurodegenerative disease, lead- memory and other cognitive functions for individ-
ing to a misrepresentation of the ‘normal’ population. uals throughout the lifespan [47, 48]. Research has
This, in turn, results in poor sensitivity for identifying recently sought to parse these associations through
early AD-related cognitive changes when using the longitudinal monitoring of depression, cognitive per-
commonly employed cut-offs of 1.5 or 1.0 standard formance, and SCD in cognitively normal adults,
deviations below the mean [37, 38]. with early results suggesting that changes in depres-
Thus, it is a complex and expensive undertaking to sion symptoms may mediate associations between
collect a sufficient amount of data to provide reliable SCD and declines in objective memory performance
norms for common cognitive tests for each specific between subjects over time [49]. Another compli-
sub-population and across ages, and more impor- cating issue is that the setting in which participants
tantly, doing this would not necessarily guarantee are seen (memory clinic versus research program)
improved sensitivity to early-stage AD. Moreover, strongly influences whether SCD is a risk factor for
substantial proportions of the normative data derived developing dementia [50]. More research is therefore
from such a massive project would likely become out- needed to understand the utility of SCD in comple-
dated in less than a decade as newer generations of ment to subtle objective cognitive decline [44].
neuropsychological instruments are developed and as Comparing SCD reports with clinical outcomes,
individual differences that influence cognitive perfor- collateral reports (i.e., from a caregiver or study part-
mance become better understood. ner), and changes in cognitive functioning and AD
biomarkers, may be the most productive approach
The role of subjective cognitive complaints to deciphering the predictive utility of SCD across
the clinical AD spectrum, particularly when exam-
The process of differentiating between normal cog- ined in longitudinal studies [51–54]. Recent progress
nitive health, MCI, and AD dementia relies heavily on in this direction has shown that SCD distinguishes
 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum 515

between A␤(+) and A␤(−) older adults beyond the with single time point assessment, thereby providing
predictive utility of apolipoprotein E (APOE) geno- a more reliable method to evaluate cognitive health
type across several large community-based cohorts [59]. This point was well illustrated in a study by
of older adults [55]. In a longitudinal study of Darby and colleagues, in which only a minority of
cognitively normal older adults, SCD at baseline participants were consistently diagnosed as having
predicted more rapid cognitive decline on neuropsy- MCI on the basis of assessments at multiple time
chological assessments among A␤(+) individuals points within one day [60]. Additionally, since
[56]. Furthermore, collateral reports provide a useful within-person measurement will rely on an individ-
point of comparison for interpreting SCD. Partner- ual’s change scores (using either raw scores when
and self-reported subjective cognitive changes on they possess satisfactory psychometric properties,
the Cognitive Function Index (CFI) [57] have been such as a good approximation of interval measure-
shown to differentially predict longitudinal objective ment, or standardized scores), the availability of
cognitive decline at different stages of AD, with CFI normative data is no longer a strict prerequisite to
self-reports showing the greatest accuracy early in benchmark one’s performance. Hence, within-person
the disease course [57]. Overall, findings suggest that measurement is expected to be more cross-culturally
SCD is an important component of cognitive change applicable, though development and validation with
on the AD continuum, which should by factored into diverse samples would still be needed to avoid
AD research frameworks and monitored in longitu- potential cultural bias in test items and formats.
dinal clinical trials. Furthermore, a comparison of patterns of within-
The field of neuropsychology has made significant person decline across persons may be a promising
strides in addressing the above challenges, such as method to detect accelerated cognitive change due to
developing more inclusive normative datasets [58], pathological processes that are distinct from ongoing
and forming working groups that establish standards ‘age-related’ changes [61]. The greater sensitiv-
and unified approaches to constructs such as SCD ity of this approach has also been suggested by
in the context of the AD continuum [43, 44]. How- studies on preclinical AD showing that in older
ever, reliance on cross-sectional cut-off scores to adults who do not meet criteria for MCI, abnor-
determine cognitive impairment has remained essen- mally high A␤ is associated with within-subject
tially unchanged over the past 50 to 75 years. As the cognitive decline over time, but not with cognitive
larger field of AD research is growing rapidly and impairment at baseline [14]. Determining progres-
has become intensely interdisciplinary, neuropsy- sion markers may therefore be an especially useful
chology must also develop more flexible, progres- application to the measurement of subjective or sub-
sive approaches to defining cognitive dysfunction tle changes to evaluate individuals who report decline
using advances in methodology, technology, and compared to their previous level of functioning but
analytics. whose ‘objective’ performance still falls within the
normal range according to available reference data.
In these cases, multiple repeated assessments may
DEFINING COGNITIVE HEALTH USING allow for the detection of subtle changes associ-
PROGRESSION MARKERS ated with the earliest stages of neurodegenerative
disease.
To tackle the aforementioned challenges, we Utilizing a multiple time point assessment ap-
propose a shift toward a more continuous approach proach; however, also raises several challenges, such
to defining cognitive performance, by using multiple as the fact that it is time-consuming to do with current
repeated assessments across two or more narrow testing paradigms. Furthermore, it relies greatly on
and/or long-time windows. Using sensitive and participant adherence and assumes that the within-
reliable tools, change scores resulting from these person variance is equal across age, education, and
repeated assessments would enable the identification cross-cultural groups, which might not always be the
of ‘progression markers’, that is, cognitive decline case. In the next section, we aim to discuss several
determined by within-person change scores. The approaches to address these challenges, includ-
use of progression markers to define cognitive status ing the implementation of new neuropsychological
has several advantages when compared to single tools and strategies for repeated neuropsycholog-
time-point testing. Multiple repeated assessments ical assessment in the clinical study of aging
may reduce error from various sources associated and AD.
516 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum

NOVEL APPROACHES TO Americans (∼80%), indicating that app-based test-

CHARACTERIZE COGNITIVE ing could be useful for reaching a broad demographic
PROGRESSION MARKERS [82, 83].
Collecting cognitive data with smartphones comes
Digital cognitive testing
with several unique challenges (e.g., variable device
One promising direction for repeated neuropsy- specifications, privacy, popup interruptions), as does
chological assessment involves the ongoing shift remote cognitive testing in any format (e.g., envi-
toward digital assessment tools, which can be used ronmental variability, use of unallowed supports or
on their own or in combination with traditional neu- assistance from others). However, these approaches
ropsychological assessments [62]. Testing software also have the potential to reduce costs and patient
has the potential to reduce administration and scor- burden by avoiding the need for lengthy in clinic
ing errors, automatically tailor tasks to an individual’s testing, and thus allowing repeated assessments and
level of ability (i.e., computerized adaptive test- detection of within-person change within a short time
ing), capture nuanced performance information (e.g., frame. Adherence is major challenge with remote
response latencies and sequencing), rapidly compute cognitive assessment that must be considered at the
and compare scores, and generate meta-data. Digital onset of any clinical research study deploying these
assessment tools can also be used to easily capture methods. A recent survival analysis of adherence
and characterize speech and language using advanced across eight different remote digital assessment stud-
analytics, such as machine learning [63, 64]. All of ies (total combined N = 1,000) found that greater
these features are particularly attractive for apply- participant retention time was most strongly asso-
ing digital tools for the longitudinal assessment of ciated with: a) referral to the study by a clinician,
subtle, early cognitive changes associated with pre- b) compensation for participation, and c) having the
clinical AD. Numerous digital cognitive test batteries, clinical condition of interest in the study, and d) older
such as the NIH Toolbox and Cambridge Neuropsy- age [84]. An additional approach to improving adher-
chological Test Automated Battery (CANTAB) [65], ence may be the provision of feedback for participants
Cogstate Brief Battery [66], TabCAT [67], as well (either in real-time or post-assessment) to improve
as standalone tests, such as the DCTclockTM [68] engagement and commitment with testing [79]; how-
have been developed for face-to-face administration ever, this method carries challenges to assessment
in clinical and research settings [69]. Unsupervised validity and requires further research.
online neuropsychological testing has been used in New “burst” testing approaches (i.e., multiple brief
several AD clinical trials [66]. More recently, the assessments completed over a period of several days)
Online Repeated Cognitive Assessment [70] tool has provide multiple data points that can be averaged to
been found to be sensitive to detect cognitive changes generate more reliable indicators of cognitive perfor-
during the preclinical stage of AD, using metrics such mance and may be better tolerated and more robust
as learning curves across multiple days of assessment to variable adherence in data collection relative to
[71]. Mobile versions of existing cognitive tests, as more lengthy assessments ([79, 85]). Conducting
well as novel tasks designed specifically for mobile brief assessments via mobile app also has the poten-
use, have also been developed in recent years [72–75], tial to improve the ecological validity of cognitive
including from academic research programs such as tests by allowing patients to complete them in their
the Center For Healthy Aging at Penn State [76], typical environments across different time points both
the Harvard Aging Brain Study [77], the Dominantly within a day and across days [76]. While empirical
Inherited Alzheimer Network (DIAN) Observational support for mobile app-based assessment tools is still
Study [78], and Oxford University [79]. Remote limited, recent initiatives, such as the NIA Mobile
app-based cognitive testing has received increased Toolbox, aim to bring open source and easily acces-
attention recently in light of the COVID-19 pan- sible mobile assessment tools to wider scientific and
demic and is rapidly becoming more feasible [75, clinical audiences in the next few years, which may
80]. App-based testing in brief repeated sessions help accelerate validation studies.
has demonstrated similar or better reliability and
validity compared to standard in-clinic assessments Novel approaches to analyze conventional test data
[76, 81]. About two-thirds of baby boomers report
using smartphones, and rates of smartphone use are Another solution for improving the sensitivity of
similar among Black, Latino/Hispanic, and White assessment may involve new approaches to analyzing
 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum 517

longitudinal cognitive data obtained using conven- in an independent sample. This undertaking seems
tional measurements. Asken and colleagues [86] have worthwhile since head-to-head comparison studies
recently proposed a Discrepancy-based Evidence for have consistently shown that IRT scoring outper-
Loss of Thinking Abilities (DELTA) score as a new forms classic test scoring techniques, especially when
method for characterizing cognitive decline on a con- investigating an individual’s change or ‘growth’ over
tinuous spectrum. Using ADNI data, they derived time or the effectiveness of clinical interventions [94,
regression-based normative reference scores using 95]. An additional advantage is that, once calibrated,
age, sex, years of education, and word-reading ability IRT parameters can be used to ‘link’ scores across
from cognitively normal participants. DELTA scores different test versions and anchor measurements of
were calculated to reflect the degree of discrepancy change to determine the clinical meaningfulness, as
between predicted and observed scores, and hence well as minimal important change, of the latent trait
the strength of evidence of cognitive decline. This scores.
approach had a positive predictive value greater than The study of practice effects (PE), also referred
0.9 for AD biomarker classification cross-sectionally, to as learning or retest effects, provides another
and more recently, was shown to improve pre- opportunity to analyze repeated neuropsychological
diction of cognitive and functional decline above measurements. Practice effects are improvements in
and beyond biomarker variables longitudinally [87]. cognitive test performance due to repeated evaluation
These findings suggest that DELTA scoring could be a with the same or similar test materials [96]. It has
promising method for capturing cognitive function on been shown that subjects with late-life cognitive dis-
a continuum; however, this work has yet to be repli- orders show reduced practice effects as compared to
cated with more representative longitudinal samples their healthy peers [97]. Furthermore, diminished PE
of diverse older adults and in other clinical research may predict future decline, a future diagnosis of MCI,
studies with different cognitive tools and intervals and increased presence of cerebral pathology. Has-
between assessments. senstab and colleagues [98] showed that reduced PE
One other statistical strategy to increase the respon- on episodic memory tests were detectable in subjects
siveness of existing neuropsychological instruments with preclinical AD, and that the magnitude of these
involves the use of item response theory (IRT) anal- practice effects was inversely related to risk of pro-
ysis. IRT links responses for a specific set of items gression to AD. All together, these findings suggest
to an underlying construct resulting in a latent trait quantifying the magnitude of PE may be a valuable
score, assuming that items contribute differently to progression marker especially in early stages of AD
this latent trait score [88]. That is to say, the IRT [99]. However, at this point there is not much evidence
model considers that some items may be more dif- that supports the use of PE on an individual level, and
ficult to ‘endorse’ than others and thus carry more this warrants further investigation. Moreover, signifi-
weight in reflecting an individual’s level of cogni- cant challenges and limitations remain in determining
tive function. In addition, IRT-based items can be how we quantify PE (e.g., via simple within-subject
designed to provide high degree of separation within change scores, regression-based methods, or reliable
a narrow range of baseline capacity (sigmoid curves). change index), and whether there is an optimal time
Compared to classic scoring methods, such as creat- interval to study PE (e.g., over months [100], days
ing a simple sum score, an IRT score maximizes the [101], or even within a single day [60]). A further
sensitivity of responses and results in greater accu- question is how to handle variation in their magni-
racy in the assessment of individual change over time tude related to the number of test administrations,
[89–91]. For example, studies have shown that IRT as studies have consistently shown that PE seem to
might improve the precision of widely-applied cogni- be greatest over initial exposures followed by a pat-
tive tests such as the Alzheimer’s Disease Assessment tern of diminished gain over the course of repeated
Scale - Cognitive subscale [92, 93]. It should be noted assessments [97].
that calibrating robust and generalizable IRT mod-
els requires large sample sizes that cover the entire
spectrum of the latent trait (i.e., cognitive function) DISCUSSION
and provide an adequate representation of the tar-
get population (i.e., ranging from older adults with We described several promising methods and
normal cognition to individuals with dementia). Ide- approaches to complement conventional neuropsy-
ally, calibrated IRT parameters are then validated chological assessments and characterize cognitive
518 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum

progression markers based on repeated assessments. the preclinical stages, as not all individuals with
These progression markers could result in a scale AD neuropathology will go on to develop clinical
reflecting the continuous nature of cognitive aging symptoms [104]. More work is needed to under-
more reliably as compared to static, single time-point stand what leads and what lags in terms of both
testing that groups people as cognitively healthy or neurobiology and cognitive function. Therefore, as
impaired. The use of cognitive progression mark- a best practice we should avoid relying solely on
ers may be particularly relevant for individuals single biomarker-based outcomes for validation of
in the preclinical stage of AD, an early, presum- cognitive measures and characterization of early AD
ably transitional phase of cognitive decline without cognitive change, and instead focus on the predic-
objectifiable cognitive impairment, or “Stage 2” as tion of the emergence and progression of clinical
described in the clinical staging scheme of the NIA- symptoms.
AA Framework [43]. These Stage 2 individuals Ultimately, combining the predictive power of
are considered an important and clinically relevant cognitive and biomarker data as complementary
population, and they have become the main target approaches may be particularly useful for differ-
population of secondary AD prevention trials [4]. entiating healthy aging from preclinical AD [105]
However, identifying individuals in this ‘transitional and those at risk for progression to dementia [106].
stage’ remains challenging using current neuropsy- Yet many newer algorithms in development do not
chological paradigms that rely on cross-sectional, include cognitive data [107]. This again points to
single time-point testing [102]. Longitudinal assess- the need for innovative and more sensitive cogni-
ment to monitor for cognitive progression markers tive tools that are easy to administer and repeat as
could aid the identification of Stage 2 individu- part of a larger, multi-modal assessment approach. If
als, especially in combination with disease-specific the development of novel neuropsychological meth-
biomarker information. This approach has the poten- ods for detecting cognitive progression markers could
tial to both advance AD clinical trial screening go hand-in-hand with the analytical and clinical
procedures leading to more successful enrollment of validation of novel blood tests for AD pathology,
Stage 2 participants, and aid in the detection of indi- longitudinal studies of the interplay between cogni-
viduals with early cognitive decline in the memory tive function and AD-related brain changes would
clinic. become more feasible.
Harmonization of cognitive progression markers
with biomarker models, such as the NIA-AA Frame-
work, may improve our ability to identify high-risk Challenges and limitations
individuals for secondary prevention trials, espe-
cially if progression markers could be combined with While cross-sectional neuropsychological assess-
accurate AD blood biomarkers for screening [103]. ment will always be needed in certain situations,
This approach may aid in detecting subtle cognitive there is much to be gained by characterizing cog-
decline and assist with clinical trials by better pre- nitive decline as points along a continuum and
dicting cognitive trajectories in the disease course. moving away from the oft relied on dichotomous
For example, if we know that a given biomarker pat- “lumping” approach to cognitive aging assessment.
tern (e.g., tau deposition) evolves in a predictable way However, adopting an approach that involves char-
and predicts the onset of more rapid decline in a cog- acterization of change on a continuum creates
nitive domain (e.g., declarative memory), then we can both methodological and practical challenges, as
plan clinical trials where delays in the progression of well as limitations that are not easily addressed.
the pathology can potentially be tied to delayed wors- Included among these methodological challenges is
ening of the affected cognitive domain, and we can establishing the reliability and longitudinal valid-
avoid looking for signals where no treatment effect ity of measurement instruments that are used to
is likely to take place. identify progression markers. These measurement
When harmonizing cognitive progression markers properties are not self-evident, and thus novel
with a biomarker-based model of disease progression test paradigms require thorough validation efforts,
such as the NIA-AA Framework, it is tempting to con- including examination of their generalizability and
sider biology as the reference standard. However, it sensitivity to change over time in the target pop-
is important to realize that the clinical value of most ulation. Moreover, in the context of examining
AD biomarkers remains inconclusive particularly in progression markers, determining what constitutes
 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum 519

an ‘abnormal’ within-person change, and what the impairments after it is too late to do much. That,
appropriate length of time should be between tests, in turn, will require better and revised measurement
are additional methodological challenges that need strategies, but multiple factors create challenges in
to be addressed. A number of promising research this pursuit. For one, it is often difficult to pinpoint
studies investigating the feasibility, reliability, and when the disease process starts, or at least the period
validity of novel repeated assessment approaches before neurocognitive deficit is first manifested. Also,
exist and can set a precedent for future work pre-disease measurement benefits may not be realized
[76, 100, 101, 108]. if baseline assessment is not done effectively [110,
Another challenge relates to the use of progression 111]. Finally, variation or error in test performance
markers on a group level versus on a case-by-case or could obscure the appearance of disease specific
individual level. Further, it raises the question of what effects, especially if one’s goal is early detection,
the ideal time frame for longitudinal assessments given the 3 to 4 standard deviation range common
should be to in order to reliably stage individuals on across an individual’s highest and lowest test scores
the spectrum from cognitively normal to impaired. in a battery of tests [112–114].
From the patient perspective, not being able to receive It would be very desirable if cognitive screen-
immediate diagnostic feedback following an ini- ing tools were brief, widely available, affordable
tial evaluation of memory problems could lead to and reimbursed and if cognitive screening were part
heightened feelings of uncertainty and anxiety. One of routine healthcare, akin to measuring choles-
potential solution to this problem could involve the terol levels, with an emphasis on population-based
remote administration of repeated assessments via approaches. Brevity might be pursued through var-
online or smart phone-based testing prior to con- ious strategies, such as adaptive testing approaches,
ventional in-clinic cognitive evaluation, in order to which can greatly improve efficiency, and focusing
avoid delays in data interpretation. From a clinical in on functions that are most susceptible to change
trial perspective, repeated assessment could neces- across the condition(s) of interest. As for cost and
sitate a longer screening period, a concept that has accessibility, digital technology could go a long way
already been applied in the use of a “trial-ready- towards solving many of these issues.
cohort” or “run-in-study” in AD research, with the Some neuropsychologists may experience under-
expectation of faster clinical trial enrollment in the standable concern that brevity is incompatible with
long run. Even with brief assessments over short time- a complete neuropsychological assessment, could
intervals, repeated measures testing is more time-, deprive professionals of fair economic opportunity
labor- and cost-intensive than single-time testing, or ownership, or may lead non-neuropsychologists
and this is especially problematic when participants to take on tasks for which they are not necessarily
are lost to follow-up. Adherence is a major issue qualified. However, expansion of population-based,
in implementing mobile phone-based assessments, longitudinal cognitive screening should not preclude
and involvement of the target population in develop- a full neuropsychological assessment when indicated.
ing these digital tests is crucial. Relying on multiple There will always be benefits to thorough testing
assessments may therefore be less suitable for indi- when a clinical diagnosis is necessary, when there
viduals who are already on the more impaired end of are complex phenotypes (e.g., posterior variant AD)
the cognitive aging spectrum, as in those cases a sin- to characterize, and when recommendations for sup-
gle cognitive screening visit will likely be sufficient port and treatment are needed. Improved cognitive
to establish a diagnosis or screen for participation screening in different contexts (e.g., primary care
in a clinical trial. In fact, conventional neuropsycho- settings) could refine the referral pipeline for neu-
logical tests have proven to show high sensitivity ropsychologists and increase access for patients who
and specificity for establishing a dementia diagnosis are most need of a full evaluation. Additionally, a
[109]. move toward using novel methods (e.g., digital tech-
nology) for developing and adopting high quality
repeated assessment approaches more broadly in neu-
Implications ropsychological clinical practice, when appropriate
for detecting change, could provide incredible clini-
We believe that the future success of many efforts cal and research opportunities for the field and greatly
in neuropsychology will depend on the ability to enhance our contributions to human health and well-
detect subtle changes as opposed to identifying being.
520 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum

(ADDF), USA (#201809-2016862), and the UK

Box 1. Summary of recommendations for neu- Dementia Research Institute at UCL.
ropsychological assessment of early-stage AD AL and JA are partially supported by Institu-
tional Development Award Number U54GM115677
1) We recommend a shift to classifying a person’s from the National Institute of General Medical Sci-
level of cognitive functioning according to ences of the National Institutes of Health, which
some quantitative scale, not dichotomized. funds Advance Clinical and Translational Research
2) A person’s baseline cognitive status should (Advance-CTR). The content is solely the responsi-
ideally be determined only after a set of bility of the authors and does not necessarily represent
repeated testing to ensure accuracy/precision. the official views of the National Institutes of Health.
3) Cognitive testing should encompass both Authors’ disclosures available online (https://
traditional approaches as well as novel www.j-alz.com/manuscript-disclosures/21-5098r1).
technologies and analytical methods to
improve precision and predictive value.
REFERENCES

[1] Bateman RJ, Xiong C, Benzinger TL, Fagan AM, Goate A,

Fox NC, Marcus DS, Cairns NJ, Xie X, Blazey TM (2012)
CONCLUSION Clinical and biomarker changes in dominantly inherited
Alzheimer’s disease. N Engl J Med 367, 795-804.
This is a pivotal time for neuropsychologists to [2] Scheltens P, Blennow K, Breteler MMB, de Strooper
B, Frisoni GB, Salloway S, Van der Flier WM (2016)
reexamine and adjust how we assess and charac- Alzheimer’s disease. Lancet 388, 505-517.
terize the cognitive features of AD, in order to [3] Jack Jr CR, Knopman DS, Jagust WJ, Shaw LM, Aisen PS,
improve harmonization with a biomarker-based dis- Weiner MW, Petersen RC, Trojanowski JQ (2010) Hypo-
ease framework and detection of subtle cognitive thetical model of dynamic biomarkers of the Alzheimer’s
pathological cascade. Lancet Neurol 9, 119-128.
changes during the earliest stage of AD. We propose [4] Sperling RA, Aisen PS, Beckett LA, Bennett DA, Craft
a multipronged, flexible approach to allow for the S, Fagan AM, Iwatsubo T, Jack CR, Kaye J, Montine
assessment of cognitive change on the AD contin- TJ, Park DC, Reiman EM, Rowe CC, Siemers E, Stern
uum, particularly as a means to improve future AD Y, Yaffe K, Carrillo MC, Thies B, Morrison-Bogorad M,
Wagster MV, Phelps CH (2011) Toward defining the pre-
prevention clinical trial design, which incorporates clinical stages of Alzheimer’s disease: Recommendations
the development and use of ‘cognitive progres- from the National Institute on Aging-Alzheimer’s Associ-
sion markers’ based on repeated assessments. We ation workgroups on diagnostic guidelines for Alzheimer’s
addressed several promising approaches and meth- disease. Alzheimers Dement 7, 280-292.
[5] Sperling R, Mormino E, Johnson K (2014) The evolu-
ods to define these progression markers, any or all tion of preclinical Alzheimer’s disease: Implications for
of which may better capture the continuous nature of prevention trials. Neuron 84, 608-622.
cognitive aging and cognitive decline in the context [6] Albert MS, Dekosky ST, Dickson D, Dubois B, Feld-
man HH, Fox NC, Gamst A, Holtzman DM, Jagust
of AD (Box 1). Based on methodological and prac-
WJ, Petersen RC, Snyder PJ, Carrillo MC, Thies B,
tical issues, we argue that multiple assessments are Phelps CH (2011) The diagnosis of mild cognitive impair-
currently of particular relevance for defining ‘cogni- ment due to Alzheimer’s disease: Recommendations from
tive health’ and subtle cognitive impairments in the the National Institute on Aging-Alzheimer’s Association
workgroups on diagnostic guidelines for Alzheimer’s dis-
early clinical stages of AD, particularly in a research ease. Alzheimers Dement 7, 270-279.
setting. Neuropsychological paradigms that reliably [7] McKhann GM, Knopman DS, Chertkow H, Hyman BT,
assess clinically meaningful cognitive progression Jack CR, Kawas CH, Klunk WE, Koroshetz WJ, Manly
need to be further validated in order to apply them JJ, Mayeux R, Mohs RC, Morris JC, Rossor MN, Schel-
tens P, Carrillo MC, Thies B, Weintraub S, Phelps CH
in clinical practice. (2011) The diagnosis of dementia due to Alzheimer’s dis-
ease: Recommendations from the National Institute on
ACKNOWLEDGMENTS Aging-Alzheimer’s Association workgroups on diagnos-
tic guidelines for Alzheimer’s disease. Alzheimers Dement
7, 263-269.
HZ is a Wallenberg Scholar supported by grants [8] Bondi MW, Edmonds EC, Jak AJ, Clark LR, Delano-
from the Swedish Research Council (#2018-02532), Wood L, McDonald CR, Nation DA, Libon DJ, Au R,
Galasko D, Salmon DP (2014) Neuropsychological cri-
the European Research Council (#681712), Swedish
teria for mild cognitive impairment improves diagnostic
State Support for Clinical Research (#ALFGBG- precision, biomarker associations, and progression rates.
720931), the Alzheimer Drug Discovery Foundation J Alzheimers Dis 42, 275-289.
 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum 521

[9] Jak AJ, Bondi MW, Delano-Wood L, Wierenga C, Corey- [25] Ruscio J (2003) Holistic judgment in clinical practice. Sci
Bloom J, Salmon DP, Delis DC (2009) Quantification of Rev Mental Health Pract 2, 33-48.
five neuropsychological approaches to defining mild cog- [26] Dodge HH, Zhu J, Lee C-W, Chang C-CH, Ganguli M
nitive impairment. Am J Geriatr Psychiatry 17, 368-375. (2014) Cohort effects in age-associated cognitive trajec-
[10] Petersen RC (2004) Mild cognitive impairment as a diag- tories. J Gerontol A Biol Sci Med Sci 69, 687-694.
nostic entity. J Intern Med 256, 183-194. [27] Stern Y (2012) Cognitive reserve in ageing and
[11] Schmand B, Eikelenboom P, Van Gool WA (2011) Value of Alzheimer’s disease. Lancet Neurol 11, 1006-1012.
neuropsychological tests, neuroimaging, and biomarkers [28] Zhou Y, Elashoff D, Kremen S, Teng E, Karlawish J, Grill
for diagnosing Alzheimer’s disease in younger and older JD (2017) African Americans are less likely to enroll in
age cohorts. J Am Geriatr Soc 59, 1705-1710. preclinical Alzheimer’s disease clinical trials. Alzheimers
[12] Salthouse T (1991) Theoretical perspectives on cognitive Dement (N Y) 3, 57-64.
aging. Erlbaum, Hillsdale, NJ. [29] Powell WR, Buckingham WR, Larson JL, Vilen L, Yu
[13] Lindenberger U, von Oertzen T (2006) Variability in M, Salamat MS, Bendlin BB, Rissman RA, Kind AJ
cognitive aging: From taxonomy to theory. In Lifespan (2020) Association of neighborhood-level disadvantage
cognition: Mechanisms of change, Bialystok E, Craik with Alzheimer disease neuropathology. JAMA Netw Open
FIM, eds. Oxford University Press, pp. 297-314. 3, e207559-e207559.
[14] Harrington KD, Gould E, Lim YY, Ames D, Pietrzak RH, [30] Chin AL, Negash S, Hamilton R (2011) Diversity and
Rembach A, Rainey-Smith S, Martins RN, Salvado O, disparity in dementia: The impact of ethnoracial differ-
Villemagne VL, Rowe CC, Masters CL, Maruff P (2017) ences in Alzheimer’s disease. Alzheimer Dis Assoc Disord
Amyloid burden and incident depressive symptoms in cog- 25, 187.
nitively normal older adults. Int J Geriatr Psychiatry 32, [31] Wong CG, Thomas KR, Edmonds EC, Weigand AJ, Ban-
455-463. gen KJ, Eppig JS, Jak AJ, Devine SA, Delano-Wood L,
[15] McRae-Mckee K, Udeh-Momoh CT, Price G, Bajaj S, Libon DJ (2018) Neuropsychological criteria for mild cog-
De Jager CA, Scott D, Hadjichrysanthou C, McNaughton nitive impairment in the framingham heart Study’s old-old.
E, Bracoud L, Ahmadi-Abhari S, De Wolf F, Anderson Dement Geriatr Cogn Disord 46, 253-265.
RM, Middleton LT (2019) Perspective: Clinical relevance [32] Legdeur N, Binnekade T, Otten R, Badissi M, Schel-
of the dichotomous classification of Alzheimer’s disease tens P, Visser P, Maier A (2017) Cognitive functioning
biomarkers: Should there be a “gray zone”? Alzheimers of individuals aged 90 years and older without dementia:
Dement 15, 1348-1356. A systematic review. Ageing Res Rev 36, 42-49.
[16] Stites SD, Milne R, Karlawish J (2018) Advances in [33] Beker N, Sikkes SA, Hulsman M, Schmand B, Scheltens
Alzheimer’s imaging are changing the experience of P, Holstege H (2019) Neuropsychological test perfor-
Alzheimer’s disease. Alzheimers Dement (Amst) 10, mance of cognitively healthy centenarians: Normative data
285-300. from the Dutch 100-Plus Study. J Am Geriatr Soc 67,
[17] Jansen WJ, Ossenkoppele R, Knol DL, Tijms BM, Schel- 759-767.
tens P, Verhey FR, Visser PJ, Aalten P, Aarsland D, Alcolea [34] Byrd D (2021) Creating an antiracist psychology by
D (2015) Prevalence of cerebral amyloid pathology in addressing professional complicity in psychological
persons without dementia: A meta-analysis. JAMA 313, assessment. Psychol Assess 33, 279-285.
1924-1938. [35] Rivera Mindt M, Byrd D, Saez P, Manly J (2010) Increas-
[18] Cipriani G, Borin G (2015) Understanding dementia in the ing culturally competent neuropsychological services for
sociocultural context: A review. Int J Social Psychiatry 61, ethnic minority populations: A call to action. Clin Neu-
198-204. ropsychol 24, 429-453.
[19] Roberts JS, Connell CM, Cisewski D, Hipps YG, Demissie [36] Ahern DC (2010) Extreme group comparisons: Nature,
S, Green RC (2003) Differences between African Ameri- prevalence, and impact on psychological research (Dis-
cans and whites in their perceptions of Alzheimer disease. sertation), University of Rhode Island.
Alzheimer Dis Assoc Disord 17, 19-26. [37] Storandt M, Morris JC (2010) Ascertainment bias in the
[20] Gray HL, Jimenez DE, Cucciare MA, Tong H-Q, clinical diagnosis of Alzheimer disease. Arch Neurol 67,
Gallagher-Thompson D (2009) Ethnic differences in 1364-1369.
beliefs regarding Alzheimer disease among dementia fam- [38] Hassenstab J, Chasse R, Grabow P, Benzinger TL, Fagan
ily caregivers. Am J Geriatr Psychiatry 17, 925-933. AM, Xiong C, Jasielec M, Grant E, Morris JC (2016)
[21] Scheltens NM, Galindo-Garre F, Pijnenburg YA, van der Certified normal: Alzheimer’s disease biomarkers and
Vlies AE, Smits LL, Koene T, Teunissen CE, Barkhof normative estimates of cognitive functioning. Neurobiol
F, Wattjes MP, Scheltens P (2016) The identification of Aging 43, 23-33.
cognitive subtypes in Alzheimer’s disease dementia using [39] Marshall GA, Amariglio RE, Sperling RA, Rentz DM
latent class analysis. J Neurol Neurosurg Psychiatry 87, (2012) Activities of daily living: Where do they fit in
235-243. the diagnosis of Alzheimer’s disease? Neurodegener Dis
[22] Spaan PE (2016) Cognitive decline in normal aging and Manag 2, 483-491.
early Alzheimer’s disease: A continuous or discontinu- [40] Jessen F, Amariglio RE, Van Boxtel M, Breteler M, Cec-
ous transition? A historical review and future research caldi M, Chételat G, Dubois B, Dufouil C, Ellis KA, Van
proposal. Cogent Psychol 3, 1185226. Der Flier WM, Glodzik L, Van Harten AC, De Leon MJ,
[23] Faust D, Ahern DC (2012) Clinical judgment and pre- McHugh P, Mielke MM, Molinuevo JL, Mosconi L, Oso-
diction. In Coping with psychiatric and psychological rio RS, Perrotin A, Petersen RC, Rabin LA, Rami L,
testimony: Based on the original work by Jay Ziskin, Faust Reisberg B, Rentz DM, Sachdev PS, De La Sayette V,
D, ed. Oxford University Press, New York, pp. 147-208. Saykin AJ, Scheltens P, Shulman MB, Slavin MJ, Sper-
[24] Ruscio J (2003) Holistic judgment in clinical practice. Sci ling RA, Stewart R, Uspenskaya O, Vellas B, Visser PJ,
Rev Mental Health Pract 2, 33-48. Wagner M (2014) A conceptual framework for research
522 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum

on subjective cognitive decline in preclinical Alzheimer’s older adults with significant memory concern. Alzheimers
disease. Alzheimers Dement 10, 844-852. Dement 11, 1417-1429.
[41] Yates JA, Clare L, Woods RT, Matthews FE (2015) [54] Meiberth D, Scheef L, Wolfsgruber S, Boecker H, Block
Subjective memory complaints are involved in the rela- W, Traeber F, Erk S, Heneka MT, Jacobi H, Spottke A
tionship between mood and mild cognitive impairment. J (2015) Cortical thinning in individuals with subjective
Alzheimers Dis 48, S115-S123. memory impairment. J Alzheimers Dis 45, 139-146.
[42] Mitchell AJ, Beaumont H, Ferguson D, Yadegarfar M, [55] Buckley RF, Sikkes S, Villemagne VL, Mormino EC,
Stubbs B (2014) Risk of dementia and mild cogni- Rabin JS, Burnham S, Papp KV, Doré V, Masters CL,
tive impairment in older people with subjective memory Properzi MJ, Schultz AP, Johnson KA, Rentz DM, Sper-
complaints: Meta-analysis. Acta Psychiatr Scand 130, ling RA, Amariglio RE (2019) Using subjective cognitive
439-451. decline to identify high global amyloid in community-
[43] Jack CR Jr, Bennett DA, Blennow K, Carrillo MC, Dunn based samples: A cross-cohort study. Alzheimers Dement
B, Haeberlein SB, Holtzman DM, Jagust W, Jessen F, (Amst) 11, 670-678.
Karlawish J, Liu E, Molinuevo JL, Montine T, Phelps C, [56] Amariglio RE, Buckley RF, Mormino EC, Marshall GA,
Rankin KP, Rowe CC, Scheltens P, Siemers E, Snyder Johnson KA, Rentz DM, Sperling RA (2018) Amyloid-
HM, Sperling R; Contributors (2018) NIA-AA Research associated increases in longitudinal report of subjective
Framework: Toward a biological definition of Alzheimer’s cognitive complaints. Alzheimers Dement (N Y) 4,
disease. Alzheimers Dement 14, 535-562. 444-449.
[44] Molinuevo JL, Rabin LA, Amariglio R, Buckley R, Dubois [57] Amariglio RE, Donohue MC, Marshall GA, Rentz DM,
B, Ellis KA, Ewers M, Hampel H, Klöppel S, Rami L, Salmon DP, Ferris SH, Karantzoulis S, Aisen PS, Sperling
Reisberg B, Saykin AJ, Sikkes S, Smart CM, Snitz BE, RA (2015) Tracking early decline in cognitive function in
Sperling R, Van Der Flier WM, Wagner M, Jessen F (2017) older individuals at risk for Alzheimer disease dementia.
Implementation of subjective cognitive decline criteria in JAMA Neurol 72, 446.
research studies. Alzheimers Dement 13, 296-311. [58] de Vent NR, Agelink van Rentergem JA, Schmand BA,
[45] Rabin LA, Smart CM, Crane PK, Amariglio RE, Berman Murre JM, Huizenga HM, Consortium A (2016) Advanced
LM, Boada M, Buckley RF, Chételat G, Dubois B, Ellis Neuropsychological Diagnostics Infrastructure (ANDI):
KA (2015) Subjective cognitive decline in older adults: A normative database created from control datasets. Front
An overview of self-report measures used across 19 inter- Psychol 7, 1601.
national research studies. J Alzheimers Dis 48, S63-S86. [59] Spooner DM, Pachana NA (2006) Ecological validity in
[46] Thompson LI, Jones RN (2020) Depression screening neuropsychological assessment: A case for greater consid-
in cognitively normal older adults: Measurement bias eration in research with neurologically intact populations.
according to subjective memory decline, brain amyloid Arch Clin Neuropsychol 21, 327-337.
burden, cognitive function, and sex. Alzheimers Dement [60] Darby D, Maruff P, Collie A, McStephen M (2002) Mild
(Amst) 12, e12107. cognitive impairment can be detected by multiple assess-
[47] Modrego PJ, Ferrández J (2004) Depression in patients ments in a single day. Neurology 59, 1042-1046.
with mild cognitive impairment increases the risk of devel- [61] Rast P, Macdonald SW, Hofer SM (2012) Intensive mea-
oping dementia of Alzheimer type: A prospective cohort surement designs for research on aging. GeroPsych (Bern)
study. Arch Neurol 61, 1290-1293. 25, 45-55.
[48] Green RC, Cupples LA, Kurz A, Auerbach S, Go R, [62] Gold M, Amatniek J, Carrillo MC, Cedarbaum JM, Hen-
Sadovnick D, Duara R, Kukull WA, Chui H, Edeki T drix JA, Miller BB, Robillard JM, Rice JJ, Soares H, Tome
(2003) Depression as a risk factor for Alzheimer disease: MB, Tarnanas I, Vargas G, Bain LJ, Czaja SJ (2018) Digital
The MIRAGE Study. Arch Neurol 60, 753-759. technologies as biomarkers, clinical outcomes assessment,
[49] Hill NL, Bhargava S, Bratlee-Whitaker E, Turner JR, and recruitment tools in Alzheimer’s disease clinical trials.
Brown MJ, Mogle J (2021) Longitudinal relation- Alzheimers Dement (N Y) 4, 234-242.
ships between subjective cognitive decline and objective [63] Libon DJ, Baliga G, Swenson R, Au R (2021) Digital
memory: Depressive symptoms mediate between-person neuropsychological assessment: New technology for mea-
associations. J Alzheimers Dis 83, 1623-1636. suring subtle neuropsychological behavior. J Alzheimers
[50] Slot RE, Sikkes SA, Berkhof J, Brodaty H, Buckley R, Dis 82, 1-4.
Cavedo E, Dardiotis E, Guillo-Benarous F, Hampel H, [64] Thomas JA, Burkhardt HA, Chaudhry S, Ngo AD, Sharma
Kochan NA (2019) Subjective cognitive decline and rates S, Zhang L, Au R, Hosseini Ghomi R (2020) Assessing the
of incident Alzheimer’s disease and non–Alzheimer’s dis- utility of language and voice biomarkers to predict cogni-
ease dementia. Alzheimers Dement 15, 465-476. tive impairment in the Framingham Heart Study Cognitive
[51] Verfaillie SC, Slot RE, Tijms BM, Bouwman F, Benedic- Aging Cohort Data. J Alzheimers Dis 76, 905-922.
tus MR, Overbeek JM, Koene T, Vrenken H, Scheltens P, [65] Robbins TW, James M, Owen AM, Sahakian BJ, McInnes
Barkhof F (2018) Thinner cortex in patients with subjec- L, Rabbitt P (1994) Cambridge Neuropsychological Test
tive cognitive decline is associated with steeper decline of Automated Battery (CANTAB): A factor analytic study
memory. Neurobiol Aging 61, 238-244. of a large sample of normal elderly volunteers. Dement
[52] Koppara A, Wagner M, Lange C, Ernst A, Wiese B, König Geriatr Cogn Disord 5, 266-281.
H-H, Brettschneider C, Riedel-Heller S, Luppa M, Wey- [66] Lim YY, Baker JE, Bruns L, Mills A, Fowler C, Fripp J,
erer S (2015) Cognitive performance before and after the Rainey-Smith SR, Ames D, Masters CL, Maruff P (2020)
onset of subjective cognitive decline in old age. Alzheimers Association of deficits in short-term learning and A␤ and
Dement (Amst) 1, 194-205. hippocampal volume in cognitively normal adults. Neu-
[53] Risacher SL, Kim S, Nho K, Foroud T, Shen L, Petersen rology 95, e2577-e2585.
RC, Jack Jr CR, Beckett LA, Aisen PS, Koeppe RA [67] Possin KL, Moskowitz T, Erlhoff SJ, Rogers KM, John-
(2015) APOE effect on Alzheimer’s disease biomarkers in son ET, Steele NZR, Higgins JJ, Stiver J, Alioto AG,
 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum 523

Farias ST, Miller BL, Rankin KP (2018) The Brain Health [81] Hassenstab J, Aschenbrenner AJ, Balota D, McDade
Assessment for detecting and diagnosing neurocognitive E, Lim YY, Fagan AM, Benzinger TB (2020) Remote
disorders. J Am Geriatr Soc 66, 150-156. cognitive assessment approaches in the Dominantly Inher-
[68] Souillard-Mandar W, Davis R, Rudin C, Au R, Libon DJ, ited Alzheimer Network (DIAN). Alzheimers Dement
Swenson R, Price CC, Lamar M, Penney DL (2016) Learn- 16(Suppl 6), e038144.
ing classification models of cognitive conditions from [82] Vogels E (2019) Millennials stand out for their technology
subtle behaviors in the digital clock drawing test. Mach use, but older generations also embrace digital life. Fact
Learn 102, 393-441. Tank, Pew Research Center, https://www.pewresearch.
[69] Buckley RF, Sparks KP, Papp KV, Dekhtyar M, Martin C, org/fact-tank/2019/09/09/us-generations-technology-use/
Burnham S, Sperling RA, Rentz DM (2017) Computerized [83] Perrin A, Turner E (2019) Smartphones help blacks,
cognitive testing for use in clinical trials: A comparison Hispanics bridge some – but not all – digital gaps with
of the NIH Toolbox and Cogstate C3 Batteries. J Prev whites. Fact Tank, Pew Research Center, https://www.
Alzheimers Dis 4, 3-11. pewresearch.org/fact-tank/2019/08/20/smartphones-
[70] Baker JE, Bruns L Jr, Hassenstab J, Masters CL, Maruff help-blacks-hispanics-bridge-some-but-not-all-digital-
P, Lim YY (2020) Use of an experimental language gaps-with-whites/
acquisition paradigm for standardized neuropsychologi- [84] Pratap A, Neto EC, Snyder P, Stepnowsky C, Elhadad N,
cal assessment of learning: A pilot study in young and Grant D, Mohebbi MH, Mooney S, Suver C, Wilbanks J,
older adults. J Clin Exp Neuropsychol 42, 55-65. Mangravite L, Heagerty PJ, Areán P, Omberg L (2020)
[71] Baker JE, Pietrzak RH, Laws SM, Ames D, Villemagne Indicators of retention in remote digital health studies: A
VL, Rowe CC, Masters CL, Maruff P, Lim YY (2019) cross-study evaluation of 100,000 participants. NPJ Digit
Visual paired associate learning deficits associated with Med 3, 21.
elevated beta-amyloid in cognitively normal older adults. [85] Thompson LI, Harrington, KD, Roque N, Strenger J,
Neuropsychology 33, 964-974. Correia S, Jones RN, Salloway S, Sliwinski MJ (2022)
[72] Zorluoglu G, Kamasak ME, Tavacioglu L, Ozanar PO A highly feasible, reliable, and fully remote protocol
(2015) A mobile application for cognitive screening for mobile app-based cognitive assessment in in cogni-
of dementia. Comput Methods Programs Biomed 118, tively healthy older adults. Alzheimers Dement (Amst),
252-262. doi: 10.1002/dad2.12283.
[73] Van Mierlo LD, Wouters H, Sikkes SAM, Van Der [86] Asken BM, Thomas KR, Lee A, Davis JD, Malloy PF, Sal-
Flier WM, Prins ND, Bremer JAE, Koene T, Van Hout loway SP, Correia S (2020) Discrepancy-Based Evidence
HPJ (2017) Screening for mild cognitive impairment and for Loss of Thinking Abilities (DELTA): Development
dementia with automated, anonymous online and tele- and validation of a novel approach to identifying cognitive
phone cognitive self-tests. J Alzheimers Dis 56, 249-259. changes. J Int Neuropsychol Soc 26, 464-479.
[74] Schweitzer P, Husky M, Allard M, Amieva H, Pérès K, [87] O’Shea DM, Thomas KR, Asken B, Lee AK, Davis JD,
Foubert-Samier A, Dartigues JF, Swendsen J (2017) Fea- Malloy PF, Salloway SP, Correia S, Alzheimer’s Disease
sibility and validity of mobile cognitive testing in the Neuroimaging Initiative (2021) Adding cognition to AT
investigation of age-related cognitive decline. Int J Meth- (N) models improves prediction of cognitive and func-
ods Psychiatr Res 26, e1521. tional decline. Alzheimers Dement (Amst) 13, e12174.
[75] Koo BM, Vizer LM (2019) Mobile technology for cogni- [88] Reise SP, Waller NG (2009) Item response theory and
tive assessment of older adults: A scoping review. Innov clinical measurement. Annu Rev Clin Psychol 5, 27-48.
Aging 3, igy038. [89] Fok CCT, Henry D (2015) Increasing the sensitivity of
[76] Sliwinski MJ, Mogle JA, Hyun J, Munoz E, Smyth JM, measures to change. Prev Sci 16, 978-986.
Lipton RB (2018) Reliability and validity of ambulatory [90] Brouwer D, Meijer RR, Zevalkink J (2013) Measuring
cognitive assessments. Assessment 25, 14-30. individual significant change on the Beck Depression
[77] Rentz DM, Dekhtyar M, Sherman J, Burnham S, Blacker Inventory-II through IRT-based statistics. Psychother Res
D, Aghjayan SL, Papp KV, Amariglio RE, Schembri A, 23, 489-501.
Chenhall T (2016) The feasibility of at-home iPad cogni- [91] Jabrayilov R, Emons WH, Sijtsma K (2016) Compari-
tive testing for use in clinical trials. J Prev Alzheimers Dis son of classical test theory and item response theory in
3, 8. individual change assessment. Appl Psychol Measure 40,
[78] Hassenstab J, Aschenbrenner AJ, Balota DA, McDade E, 559-572.
Lim Y, Fagan AM, Benzinger TLS, Cruchaga C, Goate A, [92] Wouters H, van Gool WA, Schmand B, Zwinderman
Morris JC, Bateman RJ (2018) Comparing smartphone- AH, Lindeboom R (2010) Three sides of the same coin:
administered cognitive assessments with conventional Measuring global cognitive impairment with the MMSE,
tests and biomarkers in sporadic and dominantly inher- ADAS-cog and CAMCOG. Int J Geriatr Psychiatry 25,
ited Alzheimer disease. Alzheimers Dement 14, P224- 770-779.
P225. [93] Gross AL, Kueider-Paisley AM, Sullivan C, Schretlen
[79] Lancaster C, Koychev I, Blane J, Chinner A, Wolters L, D, International Neuropsychological Normative Database
Hinds C (2020) Evaluating the feasibility of frequent cog- Initiative (2019) Comparison of approaches for equating
nitive assessment using the Mezurio Smartphone App: different versions of the Mini-Mental State Examination
Observational and interview study in adults with elevated administered in 22 studies. Am J Epidemiol 188, 2202-
dementia risk. JMIR mHealth uHealth 8, e16142. 2212.
[80] Owens AP, Ballard C, Beigi M, Kalafatis C, Brooker H, [94] Reise SP, Haviland MG (2005) Item response theory and
Lavelle G, Brønnick KK, Sauer J, Boddington S, Velayud- the measurement of clinical change. J Person Assess 84,
han L, Aarsland D (2020) Implementing remote memory 228-238.
clinics to enhance clinical care during and after COVID- [95] Thomas ML (2011) The value of item response theory in
19. Front Psychiatry 11, 579934. clinical assessment: A review. Assessment 18, 291-307.
524 R.J. Jutten et al. / Neuropsychology for the AD Cognitive Continuum

[96] Duff K, Beglinger LJ, Moser DJ, Paulsen JS, Schultz SK, [106] Rhodius-Meester HF, Liedes H, Koikkalainen J, Wolfs-
Arndt S (2010) Predicting cognitive change in older adults: gruber S, Coll-Padros N, Kornhuber J, Peters O, Jessen
The relative contribution of practice effects. Arch Clin F, Kleineidam L, Molinuevo JL (2018) Computer-assisted
Neuropsychol 25, 81-88. prediction of clinical progression in the earliest stages of
[97] Calamia M, Markon K, Tranel D (2012) Scoring higher AD. Alzheimers Dement (Amst) 10, 726-736.
the second time around: Meta-analyses of practice effects [107] Khan TK (2018) An algorithm for preclinical diagnosis of
in neuropsychological assessment. Clin Neuropsychol 26, Alzheimer’s disease. Front Neurosci 12, 275.
543-570. [108] Papp KV, Samaroo A, Chou HC, Buckley R, Schneider
[98] Hassenstab J, Ruvolo D, Jasielec M, Xiong C, Grant E, OR, Hsieh S, Soberanes D, Quiroz Y, Properzi M, Schultz
Morris JC (2015) Absence of practice effects in preclinical A, Garcı́a-Magariño I, Marshall GA, Burke JG, Kumar R,
Alzheimer’s disease. Neuropsychology 29, 940. Snyder N, Johnson K, Rentz DM, Sperling RA, Amariglio
[99] Jutten RJ, Grandoit E, Foldi NS, Sikkes SAM, Jones RN, RE (2021) Unsupervised mobile cognitive testing for use
Choi SE, Lamar ML, Louden DKN, Rich J, Tommet D, in preclinical Alzheimer’s disease. Alzheimers Dement
Crane PK, Rabin LA (2020) Lower practice effects as a (Amst) 13, e12243.
marker of cognitive performance and dementia risk: A [109] Tsoi KKF, Chan JYC, Hirai HW, Wong SYS, Kwok TCY
literature review. Alzheimers Dement (Amst) 12, e12055. (2015) Cognitive tests to detect dementia: A systematic
[100] Samaroo A, Amariglio RE, Burnham S, Sparks P, Prop- review and meta-analysis. JAMA Intern Med 175, 1450-
erzi M, Schultz AP, Buckley R, Johnson KA, Sperling RA, 1458.
Rentz DM (2020) Diminished Learning Over Repeated [110] Echemendia RJ, Bruce JM, Bailey CM, Sanders JF,
Exposures (LORE) in preclinical Alzheimer’s disease. Arnett P, Vargas G (2012) The utility of post-concussion
Alzheimers Dement (Amst) 12, e12132. neuropsychological data in identifying cognitive change
[101] Lim YY, Baker JE, Mills A, Bruns L, Fowler C, Fripp J, following sports-related MTBI in the absence of baseline
Rainey-Smith SR, Ames D, Masters CL, Maruff P (2021) data. Clin Neuropsychol 26, 1077-1091.
Learning deficit in cognitively normal APOE ␧4 carri- [111] Haran FJ, Dretsch MN, Slaboda JC, Johnson DE,
ers with LOW ␤-amyloid. Alzheimers Dement (Amst) 13, Adam OR, Tsao JW (2016) Comparison of baseline-
e12136. referenced versus norm-referenced analytical approaches
[102] Loewenstein DA, Curiel RE, Duara R, Buschke H (2018) for in-theatre assessment of mild traumatic brain injury
Novel cognitive paradigms for the detection of memory neurocognitive impairment. Brain Injury 30, 280-286.
impairment in preclinical Alzheimer’s disease. Assess- [112] Binder LM, Iverson GL, Brooks BL (2009) To err
ment 25, 348-359. is human:“Abnormal” neuropsychological scores and
[103] Zetterberg H, Bendlin BB (2021) Biomarkers for variability are common in healthy adults. Arch Clin Neu-
Alzheimer’s disease—preparing for a new era of disease- ropsychol 24, 31-46.
modifying therapies. Mol Psychiatry 26, 296-308. [113] Schretlen DJ, Munro CA, Anthony JC, Pearlson GD
[104] Bennett DA, Wilson RS, Boyle PA, Buchman AS, Schnei- (2003) Examining the range of normal intraindividual
der JA (2012) Relation of neuropathology to cognition variability in neuropsychological test performance. J Int
in persons without cognitive impairment. Ann Neurol 72, Neuropsychol Soc 9, 864-870.
599-609. [114] Faust D, Ahern DC, Bridges AJ (2012) Neuropsychologi-
[105] Chen G, Shu H, Chen G, Ward BD, Antuono PG, Zhang Z, cal (brain damage) assessment. In Coping with psychiatric
Li S-J (2016) Staging Alzheimer’s disease risk by sequenc- and psychological testimony: Based on the original work
ing brain function and structure, cerebrospinal fluid, and by Jay Ziskin, Faust D, ed. Oxford University Press, pp.
cognition biomarkers. J Alzheimers Dis 54, 983-993. 363-469.