Quantitative Psychology The 85th Annual Meeting of the

Psychometric Society, Virtual

Visit the link below to download the full version of this book:

https://medipdf.com/product/quantitative-psychology-the-85th-annual-meeting-of-t
he-psychometric-society-virtual/

Click Download Now

Preface

This volume represents presentations given at the 85th annual meeting of the
Psychometric Society, that due to the pandemic of covid-19 was held virtually.
This is the first IMPS meeting held only over internet and it was given during
July 14-17, 2020. There were 230 abstracts submitted (154 oral presentations, 89
posters, and 3 symposia). The virtual meeting attracted 378 participants, 54 of
whom also participated in the virtual short course pre-conference workshop. There
were three keynote presentations, three invited presentations, eight spotlight speaker
presentations, and one dissertation award presentation.
Since the 77th meeting in Lincoln, Nebraska, Springer publishes the proceedings
volume from the annual meeting of the Psychometric Society to allow presenters at
the annual meeting to spread their ideas quickly to the wider research community,
while still undergoing a thorough review process. This is especially important
now as meeting in person was difficult in 2020. The previous eight volumes of
the meetings were received successfully, and we expect these proceedings to be
successful as well.
The authors were asked to use their presentation at the meeting as the basis
of their chapters, possibly extended with new ideas or additional information.
The result is a selection of 42 stateof- the-art chapters addressing a diverse set
of psychometric topics, including but not limited to item response theory, factor
analysis, test equating, cognitive diagnostic models, response time, IRT as well as
psychometric applications within different fields.

Umeå, Västerbottens Län, Sweden Marie Wiberg

Amsterdam, The Netherlands Dylan Molenaar

Santiago, Chile Jorge González

Evanston, IL, USA Ulf Böckenholt

Madison, WI, USA Jee-Seon Kim

v
Contents

A Rotation Criterion That Encourages a Hierarchical Factor Structure . . 1

Chen Tian and Yang Liu
Comparison Between Different Estimation Methods of Factor
Models for Longitudinal Ordinal Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Silvia Bianconcini and Silvia Cagnone
An Efficient Scheduling Algorithm for Parallel Planar Rotations
of Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Yiu-Fai Yung and W. Clay Thompson
Explanatory Response Time Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Daniella Rebouças-Ju and Ying Cheng
Response Time Relationships Within Examinees: Implications
for Item Response Time Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Susan Embretson
Nonlinear Latent Effects in Diagnostic Classification Modeling
Incorporating Response Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Xin Qiao, Manqian Liao, and Hong Jiao
Sequential Monitoring of Aberrant Test-Taking Behaviors Based
on Response Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Suhwa Han and Hyeon-Ah Kang
Estimating Approximate Number Sense (ANS) Acuity . . . . . . . . . . . . . . . . . . . . . . 81
Anne Thissen-Roe and Lewis Baker
Differences in Symbolic and Non-symbolic Measures of
Approximate Number Sense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Lewis Baker and Anne Thissen-Roe

vii
viii Contents

Formulas of Multilevel Reliabilities for Tests with Ordered

Categorical Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Zhenqiu (Laura) Lu, Minju Hong, and Seohyun Kim
Polytomous IRT Models Versus IRTree Models for Scoring
Non-cognitive Latent Traits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Francisca Calderón and Jorge González
On the Coefficient Alpha in High-Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Kentaro Hayashi, Ke-Hai Yuan, and Regan Sato
IRT Analysis of Dimensional Structure and Item Wording Effects . . . . . . . . 141
Ki Cole, Ronna Turner, and Sohee Kim
Item Level Measurement of Extreme Response Style . . . . . . . . . . . . . . . . . . . . . . . . 153
Tongtong Zou and Daniel M. Bolt
On the Marginal Effect Under Partitioned Populations:
Definition and Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Eduardo Alarcón-Bustamante, Ernesto San Martín, and Jorge González
Range-Preserving Confidence Intervals and Significance Tests
for Scalability Coefficients in Mokken Scale Analysis . . . . . . . . . . . . . . . . . . . . . . . 175
Letty Koopman, Bonne J. H. Zijlstra, and L. Andries van der Ark
Equating Nonequivalent Groups Using Propensity Scores: Model
Misspecification and Sensitivity Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
Gabriel Wallin and Marie Wiberg
Possible Factors Which May Impact Kernel Equating
of Mixed-Format Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
Marie Wiberg and Jorge González
Population Invariance of Equating for Subgroups Differing
in Achievement Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
Dongmei Li and Shalini Kapoor
Comparison of Outlier Detection Methods in NEAT Design . . . . . . . . . . . . . . . . 219
Chunyan Liu and Daniel Jurich
An Illustration on the Quantile-Based Calculation of the
Standard Error of Equating in Kernel Equating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Jorge González and Gabriel Wallin
Improving the Measurement Efficiency in Test Construction
Related to Cognitive Diagnosis Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
Ya-Hui Su and Ken-Hsien Chu
Exploring Temporal Functional Dependencies Between Latent
Skills in Cognitive Diagnostic Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
Athul Sudheesh and Richard M. Golden
Contents ix

Sample Size for Latent Dirichlet Allocation of

Constructed-Response Items. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
Jordan M. Wheeler, Allan S. Cohen, Jiawei Xiong, Juyeon Lee, and
Hye-Jeong Choi
The Asymptotic Power of the Lagrange Multiplier Tests for
Misspecified IRT Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
Lucia Guastadisegni, Silvia Cagnone, Irini Moustaki,
and Vassilis Vasdekis
Residual Analysis in Rasch Counts Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
Naiara Caroline Aparecido dos Santos and Jorge Luis Bazán
A Bayesian Solution to Non-convergence of Crossed Random
Effects Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
Mingya Huang and Carolyn Anderson
Priors in Bayesian Estimation Under the Two-Parameter Logistic Model 309
Seock-Ho Kim, Elaine Duong, Constanza Mardones, Madeline Schellman,
Jordan Wheeler, Jiawei Xiong, Guoguo Zheng, Selay Zor,
and Allan S. Cohen
Increasing Measurement Precision of PISA Through Multistage
Adaptive Testing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
Hyo Jeong Shin, Kentaro Yamamoto, Lale Khorramdel,
and Frederic Robin
Simulation Studies of Item Bias Estimation Accuracy . . . . . . . . . . . . . . . . . . . . . . . 335
Ritesh K. Malaiya and Richard M. Golden
Multiple Answer Multiple Choice Items: A Problematic Item Type? . . . . . . 347
Magdalen Beiting-Parrish, Jay Verkuilen, Sydne McCluskey,
Howard Everson, and Claire Wladis
Modified Method of Drawing Classical ICCs Comparable to
IRT-Based ICCs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
Sayaka Arai and Gen Hori
Ontological and Methodological Barriers to the Incorporation of
Event Data in Psychometric Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
Tiago Caliço
Psychometrics for Forensic Fingerprint Comparisons . . . . . . . . . . . . . . . . . . . . . . . 385
Amanda Luby, Anjali Mazumder, and Brian Junker
After Thematic Analysis: Introducing the Fuzzy Thematic
Network Analysis in Psychological Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
Hojjatollah Farahani, Parviz Azadfallah, and Kazhal Rashidi
x Contents

Psychometric Models for a New State Science Assessment

Aligned to the Next Generation Science Standards . . . . . . . . . . . . . . . . . . . . . . . . . . 407
Jing Chen, Jonghwan Lee, Paul Nichols, and M. Christina Schneider
Diagnostic Classification Using a Polytomous Measure of Korean
Organizational Commitment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
Jungwon Rachael R. Ahn and Leah Feuerstahler
An Empirical Study of Developing Automated Scoring Engine
Using Supervised Latent Dirichlet Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
Jiawei Xiong, Jordan M. Wheeler, Hye-Jeong Choi, Juyeon Lee, and
Allan S. Cohen
Where the Choice of Model Leads Us: An Empirical Comparison
of Dyadic Data Analysis Frameworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
Hanna Kim and Jee-Seon Kim
Generalized Additive Modeling for Learning Trajectories
in E-Learning Environments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
Jung Yeon Park, JinHo Kim, Dries Debeer, and Wim Van den Noortgate
Students Ratings Their Open Classroom Discussion. . . . . . . . . . . . . . . . . . . . . . . . . 463
Diego Carrasco, Ernesto Treviño, Natalia López Hornickel,
and Carolina Castillo
A Generalizability Study of Teach, a Classroom Observation Tool. . . . . . . . . 477
Diego Luna-Bazaldua, Ezequiel Molina, and Adelle Pushparatnam

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487
A Rotation Criterion That Encourages
a Hierarchical Factor Structure

Chen Tian and Yang Liu

1 Introduction

In Yung, Thissen, and McLeod’s terminology (1999), a hierarchical factor model

may have several layers of factors: Each manifest variable loads on exactly one of
the factors in each layer. Hierarchical factor structures are common in educational
and psychological testing. For example, the big five personality traits can be divided
into many aspects, and each aspect can further be divided into facets (e.g., Allen &
DeYoung, 2017). These relationships between personality traits, aspects, facets, and
manifest variables can be represented and analyzed using the higher-order factor
model, a special case of hierarchical factor model with proportionality constraints.
Another example is the testlet effect: Both the construct and the testlet factors
contribute to the observed responses in a compensatory fashion. In addition to
explaining the correlated errors, testlets may also explicitly represent higher-order
facets within the hierarchy of interested constructs (Cooke et al., 2007).
Despite the wide-spread usage in theorizing constructs, it remains challenging to
directly obtain hierarchical structures in Exploratory Factor Analysis (EFA) since
there lacks a suitable rotation criterion. Rotation to a partially specified target
may be used, but it requires fully specifying the positions of zero loadings in the
target matrix (Browne, 1972, 2001). For circumstances where we have limited
prior knowledge on the exact pattern of factor-item dependencies, Jennrich and
Bentler (2011) discussed rotation criterions that encourage a bifactor structure
which is the simplest hierarchical model with one general factor and one layer
of specific factors. A rotation criterion function measures the discrepancy from an
exact bifactor structure, which requires each item to load on at most one specific

C. Tian () · Y. Liu

University of Maryland, College Park, MD, USA
e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 1

M. Wiberg et al. (eds.), Quantitative Psychology, Springer Proceedings
in Mathematics & Statistics 353, https://doi.org/10.1007/978-3-030-74772-5_1
2 C. Tian and Y. Liu

Fig. 1 Generalizing the rotation cretirion from a bi-factor structure to a hierarchical structure

factor. One example from Jennrich and Bentlerp (2011)  is to apply the quartimin
rotation criterion to the specific factors: i=1 kr=2 ks=r+1 λ2ir λ2is , where λir is
the loading of item i on factor r, p is the number of items, and k is the number of
factors.
Inspired by Jennrich and Bentler’s exploratory bifactor analysis using the
quartimin rotation criterion, the goal of this study is to propose a generalized rotation
criterion for a two-layer hierarchical structures in EFA. Fig. 1 displays an example
of such a structure: The corresponding factor loading matrix is expressed as eq. (1),
in which asterisks denote non-zero loading entries, and the columns from left to
right represent Fg and F1-F6. In the sequel, we say that a higher level factor is the
parent of an adjacent lower level factor if all the items loading on the lower level
factor also load on the higher level factor, and that two lower level factors having
the same parent are sibling factors. In Fig. 1, F1 is the parent of F3 and F4, and F2
is the parent of F5 and F6.

⎛ ⎞
∗ ∗ 0 ∗ 0 0 0
⎜∗ ∗ ∗ 0⎟
⎜ 0 0 0 ⎟
⎜ ⎟
⎜∗ ∗ 0 ∗ 0 0 0⎟
⎜ ⎟
⎜∗ ∗ 0 0 ∗ 0 0⎟
⎜ ⎟
⎜∗ ∗ 0 0 ∗ 0 0⎟
⎜ ⎟
⎜∗ ∗ 0 0 ∗ 0 0⎟
Λ=⎜
⎜∗
⎟ (1)
⎜ 0 ∗ 0 0 ∗ 0⎟⎟
⎜∗ ∗ ∗ 0⎟
⎜ 0 0 0 ⎟
⎜ ⎟
⎜∗ 0 ∗ 0 0 ∗ 0⎟
⎜ ⎟
⎜∗ 0 ∗ 0 0 0 ∗⎟
⎜ ⎟
⎝∗ 0 ∗ 0 0 0 ∗⎠
∗ 0 ∗ 0 0 0 ∗
A Rotation Criterion That Encourages a Hierarchical Factor Structure 3

2 Methods

2.1 Proposed Rotation Criterion Function

The proposed rotation criterion function should first be able to encourage a simple
structure within each layer, and this can be achieved by summing up the quartimin
criterion applied to different layers. In our case with F1 and F2 in the first level, there
p
is only one pair to constrain, so the corresponding term is i=1 λ2i1 λ2i2 . Minimizing
this non-negative term encourages either λ2i1 or λ2i2 in the F1-F2 pair to be close to
4
0. With four factors in the second level, F3-F6, there are = 6 pairs of factors,
2
p
and the corresponding quartimin term is i=1 λ2i3 λ2i4 + λ2i3 λ2i5 + λ2i3 λ2i6 + λ2i4 λ2i5 +
λ2i4 λ2i6 + λ2i5 λ2i6 . Summing up those terms from two layers allows us to simplify the
within-layer structure simultaneously for the two layers.
The rotation criterion should also be able to constrain the between-layer rela-
tionship to avoid items being loaded on the same second-level factor but different
first-level factors. Ideally, this constraint on the parent-child relationship can be
achieved using indicator functions. In the hierarchical structure shown in eq. (1),
we consider the product of two sums for each child factor. If F3 is a child of F1, as
shown in Fig. 1 and p Eq. (1), for all the items loading on F1, the sum of their squared
loadings on F3, i=1 I λ2i1 = 0 · λ2i3 , should be non-zero; for all the items loading
p
on F2, the sum of their squared loadings on F3, i=1 I λ2i2 = 0 · λ2i3 , should be
zero. Taking the product of these two sums encourages F3 to be the child of either
F1 or F2: In other words, it penalizes the case when F3 is the child of both F1 and
F2. Similarly, we can encourage that F1 is the only parent of F4, and F2 is the only
parent of F5 and F6.
The proposed rotation criterion function designed for a two-layer binary-split
hierarchical structure can be written in the following equation:
p  
P (Λ) = λ2i1 λ2i2 + λ2i3 λ2i4 + λ2i3 λ2i5 + λ2i3 λ2i6 + λ2i4 λ2i5 + λ2i4 λ2i6 + λ2i5 λ2i6
 p
i=1   p 
   
+ I λ2i1 = 0 · λ2i3 × I λ2i2 = 0 · λ2i3
 i=1   i=1 
p   p  
+ I λi1 = 0 · λi4 ×
2 2
I λi2 = 0 · λi4
2 2

 p
i=1   p i=1 
   
+ I λi1 = 0 · λi5 ×
2 2
I λi2 = 0 · λi5
2 2

 p
i=1   p i=1 
   
+ I λi1 = 0 · λi6 ×
2 2
I λi2 = 0 · λi6
2 2
(2)
i=1 i=1
4 C. Tian and Y. Liu

The first seven terms constrain the within-layer relationship between factors
in the same level, and the others consider the parent-child relationship between
two layers of factors. As indicated by the notation, the proposed rotation criterion
assumes the position of parent and children factors in the loading matrix: The
second and third columns should be the parent factors F1 and F2, and the fourth
to seventh columns should be the child factors F3 to F6. This rotation criterion can
be generalized to cases where one parent has more than two children by adding more
terms that control the relationship between a low-level factor and all other high-level
factors. For example, if we have
p three 2first-level 2factors,
the between-layer term
p
for F3 can be generalized to I λ = 0 · λ × I λ2i2 = 0 · λ2i3 +
p pi=1 i1 i3 pi=1
I λi1 = 0 · λi3 ×
2 2
i=1 I λi3 = 0 · λi3 +
2 2
i=1 I λi2 = 0 · λi3 ×
2 2
i=1
p
i=1 I λi3 = 0 · λi3 , which ensures that F3 has only one parent. It can also
2 2

be generalized beyond two layers of factors by adding another collection of terms

constraining the children-grandchildren relationship.

2.2 Computational Techniques

The non-continuous indicator function was approximated by the smooth exponential

function such that the criterion function is differentiable as required by the optimiza-
tion algorithm. The exponential function used in this simulation is y = e−αx , with
α = 1,000,000.
The Riemannian trust-region algorithm (Liu, 2020) was used to perform the
orthogonal rotation, a second-order optimization algorithm for numerical search on
the orthogonal group, i.e., the space of rotation matrices. It converges much faster
than the gradient projection algorithm (Jennrich, 2001) with fewer iterations.
As the criterion function is sensitive to starting values and may converge to local
minima, we used multiple random starts and chose the best as the final solution. It
is a common practice for rotation criterions that are sensitive to starting values (e.g.,
Kiers, 1994; Rozeboom, 1992). In this study, we used 30 random starts and pick the
solution with the minimum resulted function value from all 30 solutions.

2.3 Simulation Design

To understand the tolerance of the criterion function to the non-perfect and diverse
EFA practices, three design factors were manipulated in the simulation study:
loading matrices having (a) different numbers of rows/items; (b) magnitude of
small errors replacing zeros; and (c) equal/unequal numbers of manifest variables
among sibling factors. This study considers hierarchical structures with binary split:
The numbers of items are 16, 32, and 64. To generate a scenario that is more
realistic than the condition with exact hierarchical patterns, slight departures from
A Rotation Criterion That Encourages a Hierarchical Factor Structure 5

the exact structure were generated by substituting exact zero loadings with random
variates independently sampled from Uniform [−0.05, 0.05] or Uniform [−0.1,
0.1]. The non-zero loadings were simulated from Uniform [0.3, 1]. For the third
factor involving the balancing condition of items loading on sibling factors, we
considered three levels: balanced loading matrices with 50:50 items loading on
sibling factors (balanced), unbalanced loading matrices with 35:65 items loading on
sibling factors (unbalanced 1), and unbalanced loading matrices with 20:80 items
loading on sibling factors (unbalanced 2). Note that F3 and F4 are siblings because
their parent is F1, and F3 and F5 are not siblings because they do not have the
same parent. For the unbalanced conditions, we may have non-integer numbers of
items per factor (e.g. 16×0.2=3.4), and those numbers were rounded to the closest
integers.
An initial loading matrix with an unrecognizable structure, which mimics the
result of EFA, was the matrix to be rotated using the proposed rotation criterion.
We need to make sure the initially unrecognizable matrix is finally recognizable.
Therefore, the initial matrix was created by randomly rotating the true loading
matrix. Thirty replications were done for each of the 3×3×3 conditions. The
convergence tolerance was set to 10−5 . To evaluate the results, we calculate the
scaled Frobenius-norm error between the true matrix and the rotated loading matrix
averaged across 30 replications:
  2
 p 
   k ˆij − λij
1    i=1 j =1 λ (3)
√ Λ̂ − Λ = ,
pk F pk

where k is the total number of factors or columns in the complete loading matrix 
and p is the number of items.

3 Results

For the 30 replications of each of the 3×3×3 conditions, all the final solutions
converged, and the mean of minimized criterion function values over 30 replications
are summarized in Table 1. The results show that when the true loading matrix has
some errors, the function value will be greater than 0 because the true value itself is
greater than 0. Holding the error range and the extent of balance constant, the more
items we have, the larger the minimized function value; holding the number of items
and the error range constant, the more balanced the true loading matrix, the larger
the minimized function value.
The scaled Frobenius-norm error was summarized in Table 2 and depicted in
Fig. 2. The values in Table 2 reflect the estimated error per entry and can be seen as
the “averaged distance” between true and estimated loading matrices, considering
the size of the matrices. Table 2 shows that holding the error range and the number
of items constant, the more unbalanced the true matrix is, the larger the distances
6 C. Tian and Y. Liu

Table 1 The mean of minimized criterion function values over 30 replications for all conditions
Balanced (50%) Unbalanced 1 (35%) Unbalanced 2 (20%)
16 items 0 errors 0 0 0
Unif (−0.05, 0.05) 0.007 0.006 0.005
Unif (−0.1, 0.1) 0.023 0.018 0.008
32 items 0 errors 0 0 0.001
Unif (−0.05, 0.05) 0.019 0.019 0.016
Unif (−0.1, 0.1) 0.071 0.069 0.033
64 items 0 errors 0 0 0
Unif (−0.05, 0.05) 0.043 0.043 0.043
Unif (−0.1, 0.1) 0.174 0.163 0.11

Table 2 The scaled Frobenius-norm error averaged across 30 replications for all conditions
Balanced (50%) Unbalanced 1 (35%) Unbalanced 2 (20%)
16 items 0 errors 0 0 0.042
Unif (−0.05, 0.05) 0.057 0.081 0.258
Unif (−0.1, 0.1) 0.180 0.241 0.327
32 items 0 errors 0 0 0.017
Unif (−0.05, 0.05) 0.035 0.044 0.172
Unif (−0.1, 0.1) 0.114 0.223 0.335
64 items 0 errors 0 0 0
Unif (−0.05, 0.05) 0.034 0.035 0.047
Unif (−0.1, 0.1) 0.079 0.123 0.328

Fig. 2 The scaled Frobenius-norm error averaged across 30 replications for all conditions

between true and estimated loading matrix. Holding the balance extent and the
number of items constant, the larger the errors, the larger the distances. Holding
the balance extent and the error range constant, the more items we have, which
means more pieces of information, the smaller the distances are.
A Rotation Criterion That Encourages a Hierarchical Factor Structure 7

4 Discussions

This proposed function is a generalization of Jennrich and Bentler’s exploratory

bifactor analysis to a two-layer hierarchical structure, which facilitates recovering
and testing of a more complicated hierarchical structure in EFA with limited prior
knowledge about item-factor dependency. Starting from an initial matrix with an
unrecognizable structure, we can find a rotation matrix such that the rotated matrix
is as close to a matrix with the hierarchical structure as possible. Our simulation
results suggest that the proposed criterion function is generally robust to realistic
scenarios when slight to moderate departures from a perfect hierarchical structure
are present and when the matrix is moderately unbalanced. We also observe that
parameter recovery is worsened by the magnitude of imperfect factor structure and
the unbalancedness while improved as the number of items increases (Fig. 2). When
the error ranges from −0.1 to 0.1, given that the meaningful loading value ranges
from 0.3 to 1, we may have a too-small signal-to-noise ratio to recover the true
hierarchical loading matrix. The negative effect of errors was intensified when we
also have a severely unbalanced true matrix.
There are some limitations of the current study. First, the proposed rotation
criterion requires us to know exactly how many factors we have at each layer
and locks the positions of factors of a specific layer. In other words, the criterion
function implicitly assumes that the general factor lies in the first column, the two
first-level factors lie in the second and third columns, and the four second-level
factors lie in the fourth to seventh columns. The problem is that, if we reorder the
columns of a loading matrix with a perfect hierarchical structure in some ways, say,
let the general factor lie in the fourth column and a second-level factor lie in the
first column, then the function value is not zero, even though the re-ordered matrix
has a perfect hierarchical structure. An ideal criterion function should always give
a zero for matrices with the perfect hierarchical structure regardless of the ordering
of columns, which is not satisfied by the current criterion function. In our numerical
experiments, we observed local minima for the simpler bifactor structure as well, but
not as severe as the two-layer structure which has more constrains to the column-
wise relationships. Second, the proposed criterion function has many local minima.
Some initial matrices may be easily rotated to a point which does not give a zero
function value, then be further distorted when forcing the function value to be zero.
Using random starts is only a heuristic solution, and more investigation is needed.

References

Allen, T. A., & DeYoung, C. G. (2017). Personality neuroscience and the five factor model. In
Oxford handbook of the five factor model (pp. 319–352). Oxford University Press.
Browne, M. (1972). Orthogonal rotation to a partially specified target. British Journal of
Mathematical and Statistical Psychology, 25(1), 115–120.
8 C. Tian and Y. Liu

Browne, M. W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivari-

ate Behavioral Research, 36(1), 111–150.
Cooke, D. J., Michie, C., & Skeem, J. (2007). Understanding the structure of the psychopathy
checklist–revised: An exploration of methodological confusion. The British Journal of Psychi-
atry, 190(S49), s39–s50.
Jennrich, R. I. (2001). A simple general procedure for orthogonal rotation. Psychometrika, 66(2),
289–306.
Jennrich, R. I., & Bentler, P. M. (2011). Exploratory bi-factor analysis. Psychometrika, 76(4), 537–
549.
Kiers, H. A. L. (1994). SIMPLIMAX: Oblique rotation to an optimal target with simple structure.
Psychometrika, 59, 567–579.
Liu, Y. (2020). Riemannian Newton and trust-region algorithms for analytic rotation in exploratory
factor analysis. British Journal of Mathematical and Statistical Psychology.
Rozeboom, W. W. (1992). The glory of suboptimal factor rotation: Why local minima in
analytic optimization of simple structure are more blessing than curse. Multivariate Behavioral
Research, 27, 585–599.
Yung, Y. F., Thissen, D., & McLeod, L. D. (1999). On the relationship between the higher-order
factor model and the hierarchical factor model. Psychometrika, 64(2), 113–128.
Comparison Between Different
Estimation Methods of Factor Models
for Longitudinal Ordinal Data

Silvia Bianconcini and Silvia Cagnone

1 Introduction

In recent years, common statistical applications have dealt with multivariate longi-
tudinal data with the purpose of measuring changes in constructs over time, such as
attitudes, opinions, performances and abilities. In this context, both the multifaceted
nature of the data and the longitudinal evolution of the underlying constructs have to
be studied jointly, and Generalised Linear and Latent Variable Models (GLLVMs)
(Dunson, 2003; Cagnone et al., 2009) represent a useful framework. GLLVMs
assume that the entire set of the responses given by an individual to a certain number
of items at different occasions, called the response pattern, can be expressed as a
function of one or more latent variables and random effects through a monotone
differentiable link function.
A potential barrier to the application of these latent variable models is the
computational challenge presented by typically large datasets. Panel studies usually
have several thousands of respondents which, when combined with multiple waves
of measurement and a large choice set, renders unfeasible existing estimation
approaches (likelihood-based and Bayesian one). Even when cross-sectional models
are used, if the observed variables are of different nature, continuous and discrete,
the estimation of these models is cumbersome. It can be carried out using a full
information maximum likelihood method via either the EM algorithm or direct
maximisation, but, in both cases, the integrals involved in the likelihood compu-
tation have no analytical solutions and need to be approximated. This problem is
more evident in presence of longitudinal data since the number of latent variables
and random effects increases proportionally to the number of observed items. In

S. Bianconcini () · S. Cagnone

Department of Statistical Sciences, University of Bologna, Bologna, Italy
e-mail: [email protected]; [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 9

M. Wiberg et al. (eds.), Quantitative Psychology, Springer Proceedings
in Mathematics & Statistics 353, https://doi.org/10.1007/978-3-030-74772-5_2