SIAM J.

APPLIED DYNAMICAL SYSTEMS c 2019 Society for Industrial and Applied Mathematics
Vol. 18, No. 3, pp. 1586–1609

Online Dynamic Mode Decomposition for Time-Varying Systems∗

Hao Zhang† , Clarence W. Rowley† , Eric A. Deem‡ , and Louis N. Cattafesta‡
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Abstract. Dynamic mode decomposition (DMD) is a popular technique for modal decomposition, flow analysis,
and reduced-order modeling. In situations where a system is time varying, one would like to update
the system’s description online as time evolves. This work provides an efficient method for computing
DMD in real time, updating the approximation of a system’s dynamics as new data becomes available.
The algorithm does not require storage of past data and computes the exact DMD matrix using
rank-1 updates. A weighting factor that places less weight on older data can be incorporated in
a straightforward manner, making the method particularly well suited to time-varying systems. A
variant of the method may also be applied to online computation of “windowed DMD,” in which
only the most recent data are used. The efficiency of the method is compared against several existing
DMD algorithms: for problems in which the state dimension is less than about 200, the proposed
algorithm is the most efficient for real-time computation, and it can be orders of magnitude more
efficient than the standard DMD algorithm. The method is demonstrated on several examples,
including a time-varying linear system and a more complex example using data from a wind tunnel
experiment. In particular, we show that the method is effective at capturing the dynamics of surface
pressure measurements in the flow over a flat plate with an unsteady separation bubble.

Key words. data-driven modeling, dynamic mode decomposition, online methods, rank-1 updating algorithm,
system identification, time-varying systems

AMS subject classifications. 37M10, 93B30, 37N10, 37N35, 65P99, 37E99

DOI. 10.1137/18M1192329

1. Introduction. Modal decomposition methods are widely used in studying complex

dynamical systems such as fluid flows. In particular, dynamic mode decomposition (DMD)
[25, 24] has become increasingly popular in the fluids community. DMD decomposes spatio-
temporal data into spatial modes (DMD modes) each of which has simple temporal behavior
characterized by single frequency and growth/decay rate (DMD eigenvalues). DMD has been
successfully applied to a wide range of problems, for instance, as discussed in [23, 18]. The
idea of DMD is to fit a linear system to observed dynamics. However, DMD is also a promising
technique for nonlinear systems, as it has been shown to be a finite-dimensional approximation
to the Koopman operator, an infinite-dimensional linear operator that captures the full be-
havior of a nonlinear dynamical system [24, 27]. A few methods [33, 8] have been proposed to
assess the accuracy (quality) of the DMD approximated Koopman eigenfunctions/eigenvalues.

∗
Received by the editors June 5, 2018; accepted for publication (in revised form) by T. Sauer July 4, 2019;
published electronically September 12, 2019.
https://doi.org/10.1137/18M1192329
Funding: This work was supported by AFOSR through grant FA9550-14-1-0289 and DARPA through award
HR0011-16-C-0116.
†
Mechanical and Aerospace Engineering, Princeton University, Princeton, NJ 08544 ([email protected],
[email protected]).
‡
Mechanical Engineering, Florida State University, Tallahassee, FL 32310 ([email protected], [email protected]).
1586

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 ONLINE DMD FOR TIME-VARYING SYSTEMS 1587

Recently, several algorithms have been proposed to compute DMD modes efficiently for
very large datasets, for instance, using randomized methods [10, 9]. In situations in which
the incoming data is “streaming” in nature, and one does not wish to store all of the data,
a “streaming DMD” algorithm performs online updating of the DMD modes and eigenvalues
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

[16]. Streaming DMD keeps track of a small number of orthogonal basis vectors and updates
the DMD matrix projected onto the corresponding subspace. Another related method uses an
incremental SVD algorithm to compute DMD modes on the fly [20]. The work proposed here
may be viewed as an alternative to streaming DMD, in that we provide a method for updating
the DMD matrix in real time, without the need to store all the raw data. Our method differs
from streaming DMD in that we compute the exact DMD matrix, rather than a projection
onto basis functions; in addition, we propose various methods for better approximation of
time-varying dynamics, in particular by “forgetting” older snapshots, or giving them less
weight than more recent snapshots.
It is worth emphasizing that our proposed algorithm relies on an important assumption:
the number of snapshots is much larger than the state dimension. In practice, DMD is often
applied to fluid problems for which the opposite is true: the state dimension is high and
is much larger than the number of snapshots. In this paper, we explore an algorithm for
real-time updating of the linear model (DMD matrix), with the ultimate goal of real-time
modeling and control. For offline data analysis, one can have access to datasets with large
numbers of states (e.g., measurements of the full flow field). However, in real-time modeling,
it is often the case that we have only a small number of measurements (for instance, pressure
measurements from an array of sensors). Given this limited amount of information, we are
trying to find adaptive real-time models.
The paper is organized as follows. In section 2, we give an overview of DMD and describe
the online DMD algorithm. In section 3, we discuss a variant called windowed DMD, in which
only the most recent data are used. In section 4, we briefly describe how these methods may
be used in online system identification, and in section 5 we compare the different algorithms
on various examples.
2. Online dynamic mode decomposition.
2.1. The problem. We first give a brief summary of the standard DMD algorithm, as
described in [27]. Suppose we have a discrete-time dynamical system given by

xj+1 = F (xj ),

where xj ∈ Rn is the state vector, and F : Rn → Rn defines the dynamics. For a given state
xj , let yj = F (xj ); we call (xj , yj ) a snapshot pair. For DMD, we assume we have access to
a collection of snapshot pairs (xj , yj ) for j = 1, . . . , k. (It is often the case that xj+1 = yj ,
corresponding to a sequence of points along a single trajectory, but this is not required.)
DMD seeks to find a matrix A such that yj = Axj , in an approximate sense. DMD
modes are then eigenvectors of the matrix A, and DMD eigenvalues are the corresponding
eigenvalues. In the present work, we are interested in obtaining a matrix A that varies in
time, giving us a local linear model for the dynamics, but in the standard DMD approach,
one seeks a single matrix A.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1588 ZHANG, ROWLEY, DEEM, AND CATTAFESTA

Given snapshot pairs (xj , yj ) for j = 1, . . . , k, we form matrices

   
(1) Xk = x1 x2 · · · xk , Yk = y1 y2 · · · yk ,
which both have dimension n × k. We wish to find an n × n matrix Ak such that Ak Xk = Yk
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

approximately holds; in particular, we are interested in the overconstrained problem, in which

k > n. When the problem is underconstrained , the model will tend to overfit the data, and
any noise present in the data will lead to poor performance of the model [4]. Note that
DMD has typically been used on underconstrained problems, in which the number of states
is greater than the number of snapshots (e.g., in [27]), while in this paper we consider the
overconstrained case. In either case, the DMD matrix Ak is found by minimizing the cost
function [25, 24]
k
X
(2) Jk = kyi − Ak xi k2 = kYk − Ak Xk k2F ,
i=1

where k · k denotes the Euclidean norm on vectors and k · kF denotes the Frobenius norm on
matrices. The unique minimum-norm solution to this least-squares problem is given by
(3) Ak = Yk Xk+ ,
where Xk+ denotes the Moore–Penrose pseudoinverse of Xk .
Here, we shall assume that Xk has full row rank, in which case Xk XkT is invertible, and

−1
(4) Xk+ = XkT Xk XkT .
This assumption is essential for the development of the online algorithm, as we shall see
shortly. Under this assumption, the Ak given above is the unique solution that minimizes Jk .
This corresponds to the case in which the number of snapshots k is large, compared with the
state dimension n.
Our primary focus here is systems that may be slowly varying in time, so that the matrix
Ak should evolve as k increases. In the following section, we will present an efficient algorithm
for updating Ak as more data becomes available. Furthermore, if the system is time varying,
it may make sense to weight more recent snapshots more heavily than less recent snapshots.
In this spirit, we will consider minimizing a modified cost function
k
X
(5) J˜k = ρk−i kyi − Ak xi k2
i=1

for some constant ρ with 0 < ρ ≤ 1. When ρ = 1, this cost function is the same as (2), and
when ρ < 1, errors in past snapshots are discounted. Our algorithm will apply to this min-
imization problem as well, with only minor modifications and no increase in computational
effort.
A sketch of the online DMD setup is shown in Figure 1. Suppose we have already computed
Ak for a given dataset. As time progresses and a new pair of snapshots (xk+1 , yk+1 ) becomes
available, the matrix Ak+1 may be updated according to the formula given in (3). If Ak+1 is
computed directly in this manner, we call this the “standard approach.”

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 ONLINE DMD FOR TIME-VARYING SYSTEMS 1589

Time
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Figure 1. A cartoon of the online DMD setup. Ak is the optimal (least-squares) fit that maps
Xk = [x1 , x2 , . . . , xk ] to Yk = [y1 , y2 , . . . , yk ]. Arrow indicates the information flow, and box denotes
block of information. At time k + 1, Ak is updated to find Ak+1 , using the information from time k, and
new available snapshot pair xk+1 , yk+1 at time k + 1. Ak+1 is the optimal (least-squares) fit that maps
Xk+1 = [x1 , x2 , . . . , xk , xk+1 ] to Yk+1 = [y1 , y2 , . . . , yk , yk+1 ].

There are two drawbacks to the “standard approach.” First, it requires computing the
pseudoinverse of Xk whenever new snapshots are acquired, and for this reason it is computa-
tionally expensive. In addition, the method requires storing all the snapshots (i.e., storing the
matrix Xk ), which may be challenging or impossible as the number of snapshots k increases.
2.2. Algorithm for online DMD. To overcome the above two shortcomings, we propose
a different approach to find the solution to (3) in the “online setting,” in which we want to
compute Ak+1 given a matrix Ak and a new pair of snapshots (xk+1 , yk+1 ). The algorithm
we present is based on the idea that Ak+1 should be close to Ak in some sense. For the online
updating, our approach is similar to the classical recursive least-squares estimation algorithm
as formulated in [17]. Here we focus on updating the DMD matrix in real time, but the
recursive least-squares algorithm updates a vector in real time.
First, observe that, using (4), we may write (3) as

(6) Ak = Yk XkT (Xk XkT )−1 = Qk Pk ,

where Qk and Pk are n × n matrices given by

(7a) Qk = Yk XkT ,
(7b) Pk = (Xk XkT )−1 .

The condition that Xk has rank n ensures that Xk XkT is invertible, and hence Pk is well
defined. Note also that Pk is symmetric and strictly positive definite.
At time k + 1, we wish to compute Ak+1 = Qk+1 Pk+1 . Clearly, Qk+1 , Pk+1 are related to
Qk , Pk :
T
T
= Yk yk+1 Xk xk+1 = Yk XkT + yk+1 xTk+1 ,
 
Qk+1 = Yk+1 Xk+1
−1 T
T
= Xk xk+1 Xk xk+1 = Xk XkT + xk+1 xTk+1 .
 
Pk+1 = Xk+1 Xk+1

Because Xk already has rank n, and adding an additional column cannot reduce the rank of
a matrix, it follows that Xk+1 also has rank n, so Pk+1 is well defined. The above equation
shows that, given Qk and Pk−1 , we may find Qk+1 and Pk+1−1
with simple rank-1 updates:

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1590 ZHANG, ROWLEY, DEEM, AND CATTAFESTA

Qk+1 = Qk + yk+1 xTk+1 ,

−1
Pk+1 = Pk−1 + xk+1 xTk+1 .

The updated DMD matrix is then given by

Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

−1
Ak+1 = Qk+1 Pk+1 = Qk + yk+1 xTk+1 Pk−1 + xk+1 xTk+1


(8) .

Then the problem is reduced to how to find Pk+1 from Pk in an efficient manner. Comput-
ing the inverse directly would require O(n3 ) operations and would not be efficient. However,
because Pk+1 is the inverse of a rank-1 update of Pk−1 , we may take advantage of a matrix
inversion formula known as the Sherman–Morrison formula [26, 13].
Suppose A is an invertible square matrix, and u, v are column vectors. Then A + uv T
is invertible if and only if 1 + v T A−1 u 6= 0, and in this case, the inverse is given by the
Sherman–Morrison formula

−1 A−1 uv T A−1
(9) A + uv T = A−1 − .
1 + v T A−1 u
This formula is a special case of the more general matrix inversion lemma (or Woodbury
formula) [31, 13].
Applying the formula to the expression for Pk+1 , we obtain

(10a) Pk+1 = (Pk−1 + xk+1 xTk+1 )−1 = Pk − γk+1 Pk xk+1 xTk+1 Pk ,

where
1
(10b) γk+1 = .
1+ xTk+1 Pk xk+1

Note that, because Pk is positive definite, the scalar quantity 1 + xTk+1 Pk xk+1 is always
nonzero, so the formula applies. Therefore, the updated DMD matrix may be written

Ak+1 = Qk + yk+1 xTk+1 Pk − γk+1 Pk xk+1 xTk+1 Pk

(11) = Qk Pk − γk+1 Qk Pk xk+1 xTk+1 Pk

+ yk+1 xTk+1 Pk − γk+1 yk+1 xTk+1 Pk xk+1 xTk+1 Pk .

We can simplify the last two terms, since

−1
yk+1 xTk+1 Pk − γk+1 yk+1 xTk+1 Pk xk+1 xTk+1 Pk = γk+1 yk+1 γk+1 − xTk+1 Pk xk+1 xTk+1 Pk

= γk+1 yk+1 xTk+1 Pk ,

where we have used (10b). Substituting into (11), we obtain

Ak+1 = Qk Pk − γk+1 Qk Pk xk+1 xTk+1 Pk + γk+1 yk+1 xTk+1 Pk

= Ak − γk+1 Ak xk+1 xTk+1 Pk + γk+1 yk+1 xTk+1 Pk ,

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 ONLINE DMD FOR TIME-VARYING SYSTEMS 1591

and hence

(12) Ak+1 = Ak + γk+1 (yk+1 − Ak xk+1 )xTk+1 Pk .

Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

The above formula gives a rule for computing Ak+1 given Ak , Pk and the new snapshot pair
(xk+1 , yk+1 ). In order to use this formula recursively, we also need to compute Pk+1 using (10),
given Pk and xk+1 .
There is an intuitive interpretation for the update formula (12). The quantity (yk+1 −
Ak xk+1 ) can be considered as the prediction error from the current model Ak , and the DMD
matrix is updated by adding a term proportional to this error.
The updates in (10) and (12) together require only two matrix vector multiplications
(Ak xk+1 and Pk xk+1 , since Pk is symmetric), and two vector outer products, for a total of
4n2 floating-point multiplies. This is much more efficient than applying the standard DMD
algorithm, which involves an SVD or pseudoinverse, and requires O(kn2 ) multiplies, where
k > n. In our approach, two n × n matrices need to be stored (Ak and Pk ), but the large
n × k snapshot matrices (Xk , Yk ) do not need to be stored.
It is worth emphasizing that the update formulas (10) and (12) compute the DMD matrix
+
Ak+1 = Yk+1 Xk+1 exactly (up to machine precision). That is, with exact arithmetic, our
formulas give the same results as the standard DMD algorithm. The matrix Pk does involve
“squaring up” the matrix Xk , which could in principle lead to difficulties with numerical
stability for ill-conditioned problems [32, 1]. However, we have not encountered problems
with numerical stability in the examples we have tried (see section 5).
Initialization. The algorithm described above needs a starting point. In particular, to apply
the updates (10) and (12), one needs the matrices Pk and Ak at time step k. The initialization
technique is similar to the initialization of recursive least-squares estimation described in [17].
Two practical approaches are discussed below. The most straightforward way to initialize the
algorithm is to first collect at least n snapshots (more precisely, enough snapshots so that
Xk as defined in (1) has rank n), and then compute Pk and Ak using the standard DMD
algorithm, from (6) and (7):

−1
(13) Ak = Yk Xk+ , Pk = Xk XkT .

If for some reason this is not desirable, then an alternative approach is to initialize A0 to a
random matrix (e.g., the zero matrix), and set P0 = αI, where α is a large positive scalar.
Then in the limit as α → ∞, the matrices Pk , Ak computed by the updates (10) and (12)
converge to the true values given by (13).
Multiple snapshots. In our method, the DMD matrix Ak gets updated at every time step
when a new snapshot pair becomes available. In principle, one could update the DMD matrix
less frequently (for instance, every 10 time steps). The above derivation can be appropriately
modified to handle this case, using the more general Woodbury formula (see (23)) [31, 13].
However, if s is the number of new snapshots to be incorporated, the computational cost of
a single rank-s update is roughly the same as applying the rank-1 formula s times, so there
does not appear to be a benefit to incorporating multiple snapshots at once.
Extensions. As is the case for most DMD algorithms (including streaming DMD), the
online DMD algorithm described above applies more generally to extended DMD (EDMD)

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1592 ZHANG, ROWLEY, DEEM, AND CATTAFESTA

[29], simply replacing the state observations xk , yk by the corresponding vectors of observables.
In addition, the algorithm can be used for real-time online system identification, including
both linear and nonlinear system identification, as we shall discuss in section 4.
Summary. To summarize, the algorithm proceeds as follows:
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

1. Collect k snapshot pairs (xj , yj ), j = 1, . . . , k, where k > n is large enough so that

Rank Xk = n (where Xk is given by (1)).
2. Compute Ak and Pk from (13).
3. When a new snapshot pair (xk+1 , yk+1 ) becomes available, update Ak and Pk accord-
ing to (12) and (10).
Implementations of this algorithm in both MATLAB and Python are publicly available at [34].
2.3. Weighted online DMD. As mentioned previously, the online DMD algorithm de-
scribed above is ideally suited to cases for which the system is varying in time, so that we
want to revise our estimate of the DMD matrix Ak in real time. In such a situation, we might
wish to place more weight on recent snapshots, and gradually “forget” the older snapshots,
by minimizing a cost function of the form (5) instead of the original cost function (2). This
weighting scheme is analogous to that used in real-time least-squares approximation [17]. This
idea may also be used with streaming DMD and in fact has been considered before (the confer-
ence presentation [15] implemented such a “forgetting factor” with streaming DMD, although
it did not appear in the associated paper). It turns out that the online DMD algorithm can
be adapted to minimize the cost function (5) with only minor modifications to the algorithm.
We now consider the cost function
k
X
J˜k = ρk−i kyi − Ak xi k2 , 0 < ρ ≤ 1,
i=1

where ρ is the weighting factor. For instance, if we wish our snapshots to have a “half-life”
of m samples, then we could choose ρ = 2−1/m . In practice, ρ should be chosen according to
how fast the dynamics are changing. There is a trade-off between faster tracking and noise
filtering: a smaller ρ will result in faster tracking, while inevitably making the identified model
more sensitive to noise in the data (since we are forgetting old samples). For convenience, let
us take ρ = σ 2 , where 0 < σ ≤ 1, and write the cost function as
k
X 2
J˜k = σ k−i yi − Ak σ k−i xi .
i=1

If we define matrices based on scaled versions of the snapshots as

X̃k = σ k−1 x1 σ k−2 x2 · · · xk ,

 

Ỹk = σ k−1 y1 σ k−2 y2 · · · yk ,

 

then the cost function can be written as

2
J˜k = Ỹk − Ak X̃k .
F

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 ONLINE DMD FOR TIME-VARYING SYSTEMS 1593

The unique least-squares solution that minimizes this cost function (assuming X̃k has full row
rank) is given by
 −1
Ak = Ỹk X̃k+ = Ỹk X̃kT X̃k X̃kT = Q̃k P̃k ,
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

where we define

Q̃k = Ỹk X̃kT ,

 −1
P̃k = X̃k X̃kT .

At step k+1, we wish to compute Ak+1 = Q̃k+1 P̃k+1 . We write down X̃k+1 , Ỹk+1 explicitly as

X̃k+1 = σ k x1 σ k−1 x2 · · · σxk xk+1 = σ X̃k xk+1 ,

   

Ỹk+1 = σ k y1 σ k−1 y2 · · · σyk yk+1 = σ Ỹk yk+1 .

   

Therefore, Q̃k+1 can be written

T
  T
Q̃k+1 = Ỹk+1 X̃k+1 = σ Ỹk yk+1 σ X̃k xk+1
= σ 2 Ỹk X̃kT + yk+1 xTk+1
= ρQ̃k + yk+1 xTk+1 ,

and similarly
−1
(14) P̃k+1 = ρP̃k−1 + xk+1 xTk+1 .

The updated DMD matrix is then given by

  −1
Ak+1 = Q̃k+1 P̃k+1 = ρQ̃k + yk+1 xTk+1 ρP̃k−1 + xk+1 xTk+1 .

As before, we can apply the Sherman–Morrison formula (9) to (14) and obtain

P̃k P̃k P̃k

P̃k+1 = − γk+1 xk+1 xTk+1 ,
ρ ρ ρ
where
1
γk+1 = .
1+ xTk+1 (P̃k /ρ)xk+1

Let us rescale P̃k , and define

P̃k 1 −1
P̂k = = X̃k X̃kT .
ρ ρ
Then after some manipulation, the formula for Ak+1 becomes

(15) Ak+1 = Ak + γk+1 (yk+1 − Ak xk+1 )xTk+1 P̂k ,

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1594 ZHANG, ROWLEY, DEEM, AND CATTAFESTA

where
1 
(16a) P̂k+1 = P̂k − γk+1 P̂k xk+1 xTk+1 P̂k ,
ρ
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

1
(16b) γk+1 = T
.
1 + xk+1 P̂k xk+1

Observe that the update (15) for Ak+1 is identical to the update (12) from the previous
section, with Pk replaced by P̂k , and the update rule (16) for P̂k+1 differs from (10) only
by a factor of ρ. When ρ = 1, of course, the above formulas are identical to those given in
subsection 2.2.
3. Windowed dynamic mode decomposition. In subsection 2.3, we presented a method
for gradually “forgetting” older snapshots, by giving them less weight in a cost function. In this
section, we discuss an alternative method, which uses a hard cutoff: in particular, we consider
a “window” containing only the most recent snapshots, for instance, as used in [12, 19].
3.1. The problem. If the dynamics are slowly varying with time, we may wish to use
only the most recent snapshots to identify the dynamics. Here, we consider the case where
we use only a fixed “window” containing the most recent snapshots. Here, we present an
“online” algorithm to compute windowed DMD efficiently, again using low-rank updates, as
in the previous section. We refer to the resulting method as “windowed DMD.”
At time tk , suppose we have access to past snapshot pairs {(xj , yj )}kj=k−w+1 in a finite
time window of size w. We would like to fit a linear model Ak , such that yj = Ak xj (at least
approximately) for all j in this window. Let
   
(17) Xk = xk−w+1 xk−w+2 · · · xk , Yk = yk−w+1 yk−w+2 · · · yk ,

both n × w matrices. Then we seek an n × n matrix Ak such that Ak Xk = Yk approximately

holds. More precisely (as explained in subsection 2.1), the DMD matrix Ak is found by
minimizing

(18) Jk = kYk − Ak Xk k2F .

As before, we assume that the rank of Xk is n ≤ w, so that there is a unique solution to this
least-squares problem, given by

(19) Ak = Yk Xk+ ,

where Xk+ = XkT (Xk XkT )−1 is the Moore–Penrose pseudoinverse of Xk . (Note, in particular,
that we require that the window size w be at least as large as the state dimension n, so that
Xk XkT is invertible.)
A sketch of the windowed DMD setup is shown in Figure 2. As time progresses, we
can update Ak according to the formula given in (19). However, evaluating (19) involves
computing a new pseudoinverse and a matrix multiplication, which are costly operations. We
may compute this update more efficiently using an approach similar to that in the previous
section, as we describe below.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 ONLINE DMD FOR TIME-VARYING SYSTEMS 1595

Time
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Figure 2. A cartoon of the windowed DMD setup. At time k, Ak depends only on the w most recent
snapshots. At time k + 1, one new snapshot is added, and the oldest snapshot is dropped.

3.2. Algorithm for windowed DMD. As in the approach presented in subsection 2.2,
observe that (19) can be written as

−1
(20) Ak = Yk Xk+ = Yk XkT Xk XkT = Qk Pk ,
where
k
X
Qk = Yk XkT = yi xTi ,
i=k−w+1
k
!−1
X
(21) Pk = (Xk XkT )−1 = xi xTi ,
i=k−w+1

where Qk and Pk are n × n matrices. The condition that Xk has rank n ensures that Xk XkT
is invertible, so Pk is well defined.
At step k + 1, we need to compute Ak+1 = Qk+1 Pk+1 . Clearly, Qk+1 , Pk+1 are related to
Qk , Pk . To show this, we write them down explicitly as
k+1
X
T
Qk+1 = Yk+1 Xk+1 = yi xTi = Qk − yk−w+1 xTk−w+1 + yk+1 xTk+1 ,
i=k−w+2
k+1
X
−1
Pk+1 T
= Xk+1 Xk+1 = xi xTi = Pk−1 − xk−w+1 xTk−w+1 + xk+1 xTk+1 .
i=k−w+2

There is an intuitive interpretation to this relationship: Qk , Pk forgets the oldest snapshot

and incorporates the newest snapshot and gets updated into Qk+1 , Pk+1 . As in subsection 2.2,
where we used the Sherman–Morrison formula (9) to update Pk , we may use a similar approach
to update Pk in this case.
Letting
 
    −1 0
U = xk−w+1 xk+1 , V = yk−w+1 yk+1 , C= ,
0 1
we may write Qk+1 , Pk+1 as
Qk+1 = Qk + V CU T ,
−1
Pk+1 = Pk−1 + U CU T ,

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1596 ZHANG, ROWLEY, DEEM, AND CATTAFESTA

therefore

−1
Ak+1 = Qk+1 Pk+1 = Qk + V CU T Pk−1 + U CU T


(22) .
Now, the matrix inversion lemma (or Woodbury formula) [31, 13] states that
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

−1
(23) (A + U CV )−1 = A−1 − A−1 U C −1 + V A−1 U VA
whenever A, C, and A + U CV are invertible. Applying this formula to our expression for
Pk+1 , we have
(24a) Pk+1 = Pk − Pk U Γk+1 U T Pk ,
where

−1
(24b) Γk+1 = C −1 + U T Pk U .
Substituting back into (22), we obtain
Ak+1 = Qk + V CU T Pk − Pk U Γk+1 U T Pk

(25) = Qk Pk − Qk Pk U Γk+1 U T Pk
+ V CU T Pk − V CU T Pk U Γk+1 U T Pk .
The last two terms simplify, since
V CU T Pk − V CU T Pk U Γk+1 U T Pk = V C Γ−1 T T


k+1 − U Pk U Γk+1 U Pk
= V CC −1 Γk+1 U T Pk = V Γk+1 U T Pk ,
where we have used (24b). Substituting into (25), we obtain
Ak+1 = Qk Pk − Qk Pk U Γk+1 U T Pk + V Γk+1 U T Pk
= Ak − Ak U Γk+1 U T Pk + V Γk+1 U T Pk ,
and hence
(26) Ak+1 = Ak + (V − Ak U )Γk+1 U T Pk .
Notice the similarity between this expression with the updating formula (12) for online DMD.
Γk+1 is the matrix version of γk+1 in (10b). The matrix (V − Ak U ) can also be considered
as the prediction error based on current model Ak , and the correction to DMD matrix is
proportional to this error term.
The updates in (26), (24) require two products of n × n and n × 2 matrices (to compute
Ak U and Pk U , since Pk is symmetric), and two products of n × 2 and 2 × n matrices, for
a total of 8n2 multiplies. This windowed DMD approach is much more efficient than the
standard DMD approach, solving (20) directly (O(wn2 ) multiplies, with w ≥ n). Windowed
DMD can be initialized in the same manner as online DMD, discussed in subsection 2.2.
In order to implement windowed DMD, we need to store two n × n matrices (Ak , Pk ),
as well as the w most recent snapshots. Thus, the storage required is more than in online
DMD, or the weighted online DMD approach discussed in subsection 2.3, which also provides a
mechanism for “forgetting” older snapshots. It is worth pointing out that the update formulas
+
(24), (26) give the exact solution Ak+1 = Yk+1 Xk+1 from (22), without approximation.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 ONLINE DMD FOR TIME-VARYING SYSTEMS 1597

Larger window stride size. We can in principle move more than one step for windowed
DMD, i.e., forgetting multiple snapshots and remembering multiple snapshots. If we would
like to move the sliding window for s steps (s < n/2), then after similar derivations we can
show that the computational cost is 8sn2 multiplies, which is the same as applying the rank-2
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

formulas s times. Therefore, there is no obvious advantage to incorporating multiple snapshots

at one time.
Extensions. Similar to online DMD, we can also incorporate an exponential weighting
factor into windowed DMD. In particular, consider the cost function as
k
X
J˜k = ρk−i kyi − Ak xi k2 , 0 < ρ ≤ 1,
i=k−w+1

where ρ is the weighting factor. Then, proceeding as in subsection 2.3, we obtain the update
formulas

(27) Ak+1 = Ak + (V − Ak U )Γ̃k+1 U T P̂k ,

1
(28a) P̂k+1 = (P̂k − P̂k U Γ̃k+1 U T P̂k ),
ρ
where
−ρw 0
 
(28b) Γ̃k+1 = (C̃ −1 + U T P̂k U )−1 , C̃ = .
0 1

As with the online DMD algorithm, the above windowed DMD algorithm applies generally
to EDMD [29] as well, if xk , yk are simply replaced by the observable vector of the states. In
addition, the algorithm can be used for real-time online system identification, including both
linear and nonlinear system identification, as discussed in section 4.
Summary. To summarize, the algorithm proceeds as follows:
1. Collect w snapshot pairs (xj , yj ), j = 1, . . . , w, where w ≥ n is large enough so that
Rank Xk = n (where Xk is given by (17)).
2. Compute Ak and Pk from (13), where Xk , Yk is given by (17).
3. When a new snapshot pair (xk+1 , yk+1 ) becomes available, update Ak and Pk accord-
ing to (26) and (24).
Implementations of this algorithm in both MATLAB and Python are publicly available at [34].
4. Online system identification. As previously mentioned, the online and windowed
DMD algorithms discussed above can be generalized to online system identification with con-
trol in a straightforward manner. For a review of system identification methods, see [2].
4.1. Online linear system identification. DMD can be used for system identification, as
shown in [22]. Suppose we are interested in identifying a (discrete-time) linear system given by

(29) xk+1 = Axk + Buk ,

where xk ∈ Rn , uk ∈ Rp are the states and control input, respectively, A ∈ Rn×n , B ∈ Rn×p .

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1598 ZHANG, ROWLEY, DEEM, AND CATTAFESTA

At time k, assume that we have access to x1 , x2 , . . . , xk+1 and u1 , u2 , . . . , uk . Letting

 
  x1 x2 · · · xk  
Ỹk = x2 x3 · · · xk+1 , X̃k = , Ã = A B ,
u1 u2 · · · uk
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

we may write (29) in the form

(30) Ỹk = ÃX̃k .

The matrices A, B may then be found by minimizing the cost function

(31) Jk = kỸk − Ãk X̃k k2F .

As before, the solution is given by

(32) Ãk = Ỹk X̃k+ .

At time k+1, we add a new column to X̃k and Ỹk , and we would like to update Ãk+1 using
our previous knowledge of Ãk . Using the same approach as in section 2, it is straightforward
to extend the online DMD and windowed DMD algorithms to this case. In particular, the
square matrix Ak from section 2 is replaced by the rectangular matrix Ãk defined above, and
the vector xk in the formulas in section 2 is replaced by the column vector
 
xk
.
uk
4.2. Online nonlinear system identification. The efficient online/windowed DMD algo-
rithms apply to nonlinear system identification as well. In general, nonlinear system iden-
tification is a challenging problem; see [21] for an overview. Some interesting methods are
to use linear-parameter-varying models [28, 14], or to consider a large dictionary of potential
nonlinear functions, and exploit sparsity to select a small subset [5].
Suppose we are interested in identifying a nonlinear system

xk+1 = f (xk , uk )

directly from data, where xk ∈ Rn , uk ∈ Rp are the state vector and control input, respectively.
The specific form of nonlinearity is unknown, but in order to proceed, we have to make
some assumptions about the nonlinear form. Assume that we have q (nonlinear) observables
zi (x, u), i = 1, 2, . . . , q, such that the underlying dynamics can be approximately described by

(33) xk+1 = Azk ,

where A ∈ Rn×q , and

 T
zk = z1 (xk , uk ) z2 (xk , uk ) · · · zq (xk , uk ) .

To illustrate how this representation works, we take x ∈ R, u ∈ R, for example, and assume
the nonlinear dynamics is given by

xk+1 = a1 xk + a2 x2k + a3 uk + a4 u2k + a5 xk uk .

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 ONLINE DMD FOR TIME-VARYING SYSTEMS 1599

Then by setting z1 (x, u) = x, z2 (x, u) = x2 , z3 (x, u) = u, z4 (x, u) = u2 , z5 (x, u) = xu, we can

write the dynamics in the form (33), with
 
A = a1 a2 a3 a4 a5 .
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Note that in the above, the state xk still evolves nonlinearly (i.e., xk+1 depends in a nonlinear
way on xk and uk ), but we are able to identify the coefficients ak using linear regression (i.e.,
finding the matrix A in (33)).
This approach is related to Carleman linearization [3, 11], although in Carleman lineariza-
tion, the goal is to find a true linear representation of the dynamics in a higher-dimensional
state space, and for most nonlinear systems of practical interest, it is not possible to ob-
tain a finite-dimensional linear representation. This is also related to EDMD [29] and kernel
DMD [30], where Koopman eigenfunctions are used to determine coordinates in which the
nonlinear system becomes linear. Our approach is to define a collection of observables in
order to write the original nonlinear system in a linear fashion. It is generally difficult to
determine what observables to use, and sparsity-promoting system identification [5] is one
possible method. If one knows something about the form of the nonlinearity (e.g., the non-
linear terms are quadratic), then this can inform the choice of observables (e.g., including all
quadratic couplings of the states).
In summary, by assuming a particular form of the nonlinearity, we can find the coefficients
of a nonlinear system using the same techniques as used in linear system identification, writing
the nonlinear system in the form (33).
5. Application and results. In this section, we illustrate the methods on a number of
examples, first showing results for simple benchmark problems, and then using data from a
wind tunnel experiment.
5.1. Benchmarks. We now present a study of the computational time of various DMD
algorithms. Two benchmark tasks are considered here. In the first task, we wish to know the
DMD matrix only at the final time step, at which point we have access to all of the data. In
the second task, we wish to compute the DMD matrix at each time, whenever a new snapshot
is acquired. The first task thus represents the standard approach to computing the DMD
matrix, while the second task applies to situations where the system is time varying, and we
wish to update the DMD matrix in real time.
Asymptotic cost. First, we examine how the various algorithms scale with the state di-
mension n and the number of snapshots m, for the two tasks described above. In particular,
we are concerned with the overconstrained case in which n < m. For the standard algorithm,
in which the DMD matrix is computed directly using (3), one must compute an n × m pseu-
doinverse and an n × m, m × n matrix multiplication. For the first task, the computational
cost (measured by the number of multiplies) is thus

Tstandard = O(nm min(m, n) + mn2 ) = O(mn2 ).

For the second task, in which we compute the DMD matrix at each time, we refer to the
standard algorithm as “batch DMD,” since the snapshots are processed all in one batch. The

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1600 ZHANG, ROWLEY, DEEM, AND CATTAFESTA

method is initialized and applied after m0 snapshots are gathered (and in the examples below,
we will take m0 = n), so the computational cost is
 
m
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

X
nk min(k, n) + kn2  = O m2 n2 .



Tbatch = O 
k=m0

Next, we consider windowed DMD, for a window containing w snapshots (with n < w < m).
In this case, we refer to the standard algorithm, in which the DMD matrix is computed directly
using (19), as “mini-batch DMD.” The computational cost is given by
m
!
X
nw min(n, w) + wn2 = O mwn2 .



Tmini-batch = O
k=w

For streaming DMD [16] for a fixed rank r, the cost of one iteration is O(r2 n), and for full-rank
streaming DMD, the cost of one iteration is O(n2 ). Thus, for either task, the overall cost after
m snapshots is
m
!
X
r=n
n2 = O mn2


Tstreaming =O
k=1

and
m
!
X
r<n
r2 n = O mr2 n .


Tstreaming =O
k=1

(If, in streaming DMD, the compression step (step 3 in the algorithm in [16]) is performed only
every r steps, then the cost is reduced to O(mrn).) Finally, for both online and windowed
DMD algorithms, discussed in subsections 2.2 and 3.2, the cost per time step is O(n2 ). The
algorithms are applied after w snapshots are gathered, so the overall cost of either algorithm
is
m
!
X
n2 = O mn2 .


Tonline = Twindow = O
k=w+1

Results. We now compare the performance of the different algorithms on actual examples
for the two tasks described above. In particular, we consider a system with state x ∈ Rn , where
n varies between 2 and 1024. The entries in the n×n matrix A are chosen randomly, according
to a normal distribution (zero-mean with unit variance). The snapshots x1 , . . . , xm are also
chosen to be random vectors, whose components are also chosen according to the standard
normal distribution. In the tests below, we use a fixed number of snapshots m = 104 . For
minibatch DMD and windowed DMD, the window size is fixed at w = 2048, and online DMD
and windowed DMD are both initialized after the first w snapshot pairs. For streaming DMD
with a fixed rank r, we take r = 16. The simulations are performed in MATLAB (R2016b)
on a personal computer equipped with a 2.6 GHz Intel Core i5 processor.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 ONLINE DMD FOR TIME-VARYING SYSTEMS 1601

3 3
10 10

10 2 10 2 Batch

Mini-batch
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Time (sec)

Time (sec)
10 1 Streaming 10 1
(rank = r)
Streaming
Streaming
10 0 10 0 (rank = r)
(rank = n) Windowed
Online Standard Online
10 -1 10 -1

10 -2 10 -2
10 0 10 1 10 2 10 3 10 4 10 0 10 1 10 2 10 3 10 4
State dimension n State dimension n
(a) Task: compute DMD matrix at final step. (b) Task: compute DMD matrix at each step

Figure 3. Performance of the different DMD algorithms on the benchmark cases described in subsection 5.1.
For low-rank streaming, the dimension is limited to r = 16.

The results are shown in Figures 3(a) and 3(b). For the first task (computing the DMD
matrix only at the final step), the standard DMD algorithm is the most efficient, for the
problem sizes considered here. However, note that streaming DMD with a fixed rank r scales
much better with the state dimension n and would be the fastest approach for problems with
larger state dimension.
Our primary interest here is in the second task, shown in Figure 3(b), in which the DMD
matrix is updated at each step. For problems with n < 256, online DMD is the fastest approach
and can be orders of magnitude faster than the standard batch and minibatch algorithms. For
problems with larger state dimension, streaming DMD is the fastest algorithm, as it scales
linearly in the state dimension (while the other algorithms scale quadratically). However,
note that streaming DMD does not compute the exact DMD matrix: rather, it computes a
projection onto a subspace of dimension r (here 16). By contrast, online DMD and windowed
DMD both compute the full DMD matrix, without approximation.
These results focus on the time required for these algorithms, but it is worth pointing out
the memory requirements as well. Streaming DMD and online DMD do not require storage
of any past snapshots, while windowed DMD and mini-batch DMD require storing the w
snapshots in the window, and batch DMD requires storage of all past snapshots.
5.2. Linear time-varying system. We now test the online DMD and windowed DMD
algorithms on a simple linear system that is slowly varying in time. In particular, consider
the system

(34a) ẋ(t) = A(t)x(t),

where x(t) ∈ R2 , and the time-varying matrix A(t) is given by

 
0 ω(t)
(34b) A(t) = ,
−ω(t) 0

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1602 ZHANG, ROWLEY, DEEM, AND CATTAFESTA

1 2

1.8
0.5
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

x1 (t), x2 (t)

1.6

Im(λ)
0
1.4

-0.5
1.2

-1 1
0 2 4 6 8 10 0 2 4 6 8 10
Time t Time t
(a) State evolution (b) Frequency prediction

Figure 4. Solution of the linear time-varying system (34), and frequencies predicted by various DMD
algorithms. For minibatch and windowed DMD, the window size is w = 10. For online DMD, smaller values
of the parameter ρ result in faster tracking of the time-varying frequency.

where

ω(t) = 1 + t.

We take  = 0.1, so that the system is slowly varying in time. The eigenvalues of A(t) are
±iω(t), and it is straightforward to show that kx(t)k is constant in t. We simulate the system
for 0 < t < 10 from initial condition x(0) = (1, 0)T , and the snapshots are taken with time
step ∆t = 0.1 as shown in Figure 4(a). It is evident from the figure that the frequency is
increasing with time.
Given the snapshots, we apply both brute-force batch DMD and minibatch DMD as
benchmarks, then we compare streaming DMD, online DMD, and windowed DMD with these
two benchmark brute-force algorithms. The finite time window size of minibatch DMD and
windowed DMD is w = 10. Batch DMD takes into account all the past snapshots, while
minibatch DMD only takes the recent snapshots from a finite time window. Streaming DMD,
online DMD, and windowed DMD are initialized using the first w = 10 snapshot pairs, and
they start iteration from time w + 1. Batch DMD and minibatch DMD also start from time
w + 1. The results for streaming DMD, online DMD (ρ = 1, ρ = 0.95, ρ = 0.8), and windowed
DMD are shown in Figure 4(b). DMD finds the discrete-time eigenvalues µDMD from data, and
the figure shows the continuous-time DMD eigenvalues λDMD , which are related to these by

(35) µDMD = eλDMD ∆t ,

where ∆t is the time spacing between snapshot pairs. We show the DMD results starting
from time w + 1, and the true eigenvalues are also shown for comparison.
Observe from Figure 4(b) that the eigenvalues computed by the standard algorithm (batch
DMD) agree with those identified by streaming DMD and online DMD with ρ = 1, as expected.
Similarly, windowed DMD perfectly overlaps with minibatch DMD. When the weighting ρ in

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 ONLINE DMD FOR TIME-VARYING SYSTEMS 1603

online DMD is smaller than 1, the identified frequencies shift slightly toward those identi-
fied by windowed DMD. If we further decrease the weighting factor (ρ = 0.8), online DMD
aggressively forgets old data, and the identified frequency adapts more quickly. This exam-
ple demonstrates that windowed DMD and weighted online DMD are capable of capturing
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

time variations in dynamics, with an appropriate choice of the weight ρ. A MATLAB script
implementing this example can be found in [34]

5.3. Pressure fluctuations in a separation bubble. We now demonstrate the algorithm

on a more complicated example, using data obtained from a wind tunnel experiment. In
particular, we study the flow over a flat plate with an adverse pressure gradient and investigate
the dynamics of pressure fluctuations in the vicinity of a separation bubble.
The setup of the wind tunnel experiment is shown in Figure 5(a). A flat plate with an
elliptical (4:1) leading edge is placed in the flow, and zero-net mass flux suction and blowing
are applied at the ceiling of the wind tunnel in order to impose an adverse and subsequent
favorable pressure gradient along the surface of the plate, causing the boundary layer to
separate and then reattach. More information about the separation system and the plate
model can be found in [6].
These experiments were conducted in the Florida State Flow Control open-return wind
tunnel. The cross-sectional dimensions of the test section are 30.5 cm×30.5 cm, and the length
is 61.0 cm. The contraction ratio of the inlet is 9:1. An aluminum honeycomb mesh and two
fine, antiturbulence screens condition the flow at the inlet and provide a freestream turbulence
intensity of 0.5%. The suction/blowing on the ceiling of the wind tunnel test section is provided
by a variable-speed fan mounted within a duct fixed to the ceiling of the wind tunnel, which
pulls flow from the ceiling and reintroduces it immediately downstream. The chord of the
flat plate model is c = 40.2 cm, and the height is h = 0.095c. For these experiments, the
freestream velocity is U∞ = 3.9 m/s and the Reynolds number is Rec = U∞ c/ν = 105 .

35
Rotation rate (Hz)

30

25

20
0 5 10 15 20 25 30
Time (sec)
(a) Experiment setup (b) Separation fan roation speed

Figure 5. Schematic of the flat plate model and flow separation system. Separation fan rotation speed
(rotation per second).

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1604 ZHANG, ROWLEY, DEEM, AND CATTAFESTA

Unsteady surface pressure fluctuations within the separated flow are monitored by an array
of 13 surface-mounted Panasonic WM-61A electret microphones located within the separation
region. The microphones are placed at the centerline of the plate, between x/c = 0.70 and
0.94, with a spacing of 4x/c = 0.02. More details regarding this microphone array can be
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

found in [6]. These 13 microphone signals provide the data we use for online DMD.
We change the rotation speed of the separation fan slowly, in order to impose a time-
varying boundary condition and induce time-varying dynamics in the separation bubble. The
size of the separation bubble increases with the rotation speed of the separation fan. The
rotation speed starts at about 25 Hz (rotation per second) and increases slowly to reach a
high of about 30 Hz, then back to initial speed of 25 Hz. The blade rotation rate is shown
in Figure 5(b). The pressure data sampling frequency is fs = 104 , and we collect data for
T = 30 sec. The total number of samples is m = 3 × 105 , and the state dimension is n = 13
(there are 13 pressure sensors).
To study the frequencies from the pressure data, we first present a spectral analysis of
the pressure data using the short-time discrete Fourier transform. Figure 6 shows the results
for the seventh (medium) pressure sensor; other pressure sensors have similar results. A win-
dow size of fs = 104 is used (along with a Hamming window of the same length), with 90%
overlap of samples between adjointing sections. It’s observed that there is no persistent peak
frequency, and the dominant frequencies are time-varying. In order to gain a comprehensive
understanding of the frequency variations in all the pressure sensors, and how these might be
related to one another, we apply online DMD and windowed DMD to all the pressure data.
The number of snapshots m = 3 × 105 is much larger than the state dimension n = 13,
so the overconstrained assumption is satisfied. Recall that our proposed algorithm relies on
the assumption that the number of snapshots is larger than the state dimension. Therefore,
we can apply both online DMD and windowed DMD for this problem. The dynamics of
the pressure fluctuations can be characterized by the DMD frequencies, which may be slowly
varying in time. The DMD frequency is defined as

200
Power spectral density (dB/Hz)

-50

150 -60
Frequency (Hz)

-70

-80
100
-90

-100
50
-110

-120
0
5 10 15 20 25
Time (sec)
Figure 6. Power spectral density of the pressure measurement at the seventh (medium) pressure sensor.
There is no persistent peak frequency, and the dominant frequencies are time-varying.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 ONLINE DMD FOR TIME-VARYING SYSTEMS 1605

80 80 80

60 60 60
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

40 40 40

20 20 20

0 0 0
0 10 20 30 0 10 20 30 0 10 20 30

Figure 7. DMD frequencies identified by different DMD algorithms from 13 pressure signals, as described
in subsection 5.3. The two vertical lines indicate the time when the fan rotation rate starts to increase (t = 13
sec) and finally settles down to the original speed (t = 27 sec).

Im(λDMD )
fDMD = ,
2π
where Im(λDMD ) is the imaginary part of the continuous-time DMD eigenvalues computed
from (35). (The discrete-time eigenvalues µDMD are eigenvalues of the 13 × 13 matrix ma-
trix Ak .) The frequencies computed by the various DMD algorithms are shown in Figure 7.
There are 13 DMD eigenvalues in total, and one of them is f0 = 0 Hz corresponding to the
mean. The remaining DMD eigenvalues consist of six complex-conjugate pairs, corresponding
to six nonzero DMD frequencies. The DMD computation reveals three zero frequencies for
all cases we run, indicating that there are only five active (nonzero) frequencies present in
the system. Online DMD with ρ = 1 is run to obtain the standard DMD result. We use a
weighting factor of ρfs = 0.5 (corresponding half-decay life is fs samples, i.e., one second.)
for weighted online DMD and window size of w = fs = 104 for windowed DMD. The DMD
algorithms are initialized with the first w snapshots, and the resulting DMD frequencies are
computed starting from time step w + 1.
Recall that with ρ = 1, online DMD coincides with the standard DMD algorithm. From
Figure 7, we see that for this case, the three leading frequencies remain more or less constant
in time. However, we expect the dynamics in the separation bubble to be time-varying. In
response to the change of fan speed, the frequencies should start to change at t = 13 sec and
return close to their original values at t = 27 sec (with some lag). The weighted online DMD
and windowed DMD both act as desired, while the standard DMD is clearly erroneous. With
weighting factor ρfs = 1/2, online DMD behaves more like windowed DMD: in particular,
the method is better at tracking variations in the frequency. The weighting factor in online
DMD acts like a soft cutoff for the old snapshots, compared with the hard cutoff imposed
by windowed DMD. Online DMD gradually forgets the old snapshots, so it is expected to
be smoother than windowed DMD. From the results, we confirm that the online DMD is
smoother compared to the windowed DMD. While the frequency variations shown in Figure 7
appear to be rapid or noisy, note that the time interval shown in the figure is quite long (about
300 periods for the lowest frequency of around 10 Hz), so it is reasonable to consider these
frequencies as slowly varying in time.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1606 ZHANG, ROWLEY, DEEM, AND CATTAFESTA

Due to the slowly time-varying boundary condition imposed by the separation fan, the
dynamics of the pressure fluctuations in the separation bubble is expected to be slowly time-
varying too. When we simply use standard DMD, then the frequencies (and linear models)
we identify will not be adaptive to the flow conditions in real time. In contrast, weighted
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

online DMD and windowed DMD successfully track the time variations from the pressure sig-
nals. Therefore, the proposed online algorithms are demonstrated to be beneficial in real-time
modeling of time-varying systems. In fact, the online DMD algorithm has been applied to
this particular separation control problem [7] for system identification and real-time control
purposes. The idea is to use online DMD to find an adaptive linear model from pressure
measurements and control input history (see section 4.1) and apply linear control based on
this adaptive model.

6. Conclusion and outlook. In this work, we have developed efficient methods for com-
puting online DMD and windowed DMD. The proposed algorithms are especially useful in
applications for which the number of snapshots is very large compared to the state dimension
and when the dynamics are slowly varying in time. The number of snapshots is assumed
to be larger than state dimension, because we are primarily interested in real-time model-
ing (updating the DMD matrix). In real time, it is often the case that we have a relatively
small number of measurements, and as time progresses, eventually we will always have more
snapshots than measurements. A weighting factor can be included easily in the online DMD
algorithm, which is used to weight recent snapshots more heavily than older snapshots. This
approach corresponds to using a soft cutoff for older snapshots, while windowed DMD uses a
hard cutoff, from a finite time window. The proposed algorithms can be readily extended to
online system identification, even for time-varying systems.
The efficiency is compared against the standard DMD algorithm, both for situations in
which one computes the DMD matrix only at the final time and for situations in which one
computes the DMD matrix in an “online” manner, updating it as new snapshots become
available. The latter case is applicable, for instance, when one expects the dynamics to be
time varying. For the former case, the standard DMD algorithm is the most efficient, while for
the latter case, the new online and windowed DMD algorithms are the most efficient and can
be orders of magnitude more efficient than the standard DMD algorithm. Table 1 provides
a brief comparison of the main characteristics and features of standard DMD, batch DMD,
minibatch DMD, (low rank) streaming DMD, online DMD, and windowed DMD.
The algorithms are further demonstrated on a number of examples, including a linear time-
varying system, and data obtained from a wind tunnel experiment. As expected, weighted
online DMD and windowed DMD are effective at capturing time-varying dynamics.
In this paper we assign different weights to different samples, and in particular a weighting
factor is used to forget old snapshots. This choice of weight is especially useful for time-varying
systems, as older information becomes outdated. As one reviewer pointed out, some real-world
applications might involve measurements with different degrees of reliability (e.g., different
levels of uncertainty, different measurement methods). One could in principle place different
weights on different samples, according to how much one “trusts” them, which would lead
to a weighted linear regression problem. This could be an interesting direction for future
work.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 ONLINE DMD FOR TIME-VARYING SYSTEMS 1607

Table 1
Characteristics of the various DMD algorithms considered. Relevant parameters are state dimension n,
total number of snapshot pairs m  n, window size w such that n < w  m, low rank r < n, and discrete
time k > n. Computational time denotes the required floating-point multiplies for one iteration (computing the
DMD matrix).
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Aspect Standard Batch Minibatch Streaming Online Windowed

Computational
O(mn2 ) O(kn2 ) O(wn2 ) O(r2 n) 4n2 8n2
time
Memory mn kn wn O(rn) 2n2 wn + 2n2
Store
Yes Yes Yes No No Yes
past snapshots
Track
No No Yes Yes Yes Yes
time variations
Real-time
No Yes Yes Yes Yes Yes
DMD matrix
Exact DMD matrix Yes Yes Yes No Yes Yes

Another relevant direction for future work is a more detailed study of the application
of proposed online/windowed DMD algorithms to system identification. In situations where
there are variations in dynamics, or where we desire real-time control, it is crucial to build
accurate and adaptive reduced order models for effective control, and the methods proposed
here could be useful in these cases.
Acknowledgment. The authors thank Professor Maziar S. Hemati for helpful suggestions
about references.

REFERENCES

[1] E. L. Allower and K. Georg, Update methods and their numerical stability, in Numerical Continuation
Methods, Springer Ser. Comput. Math. 13, Springer, New York, 1990, pp. 252–265, https://doi.org/
10.1007/978-3-642-61257-2 16.
[2] K. J. Åström and P. Eykhoff, System identification—a survey, Automatica, 7 (1971), pp. 123–162,
https://doi.org/10.1016/0005-1098(71)90059-8.
[3] R. Bellman and J. M. Richardson, On some questions arising in the approximate solution of nonlinear
differential equations, Quart. Appl. Math., 20 (1963), pp. 333–339, https://doi.org/10.1090/qam/
144472.
[4] C. M. Bishop, Pattern Recognition and Machine Learning, Springer, New York, 2006.
[5] S. L. Brunton, J. L. Proctor, and J. N. Kutz, Discovering governing equations from data by sparse
identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci. USA, 113 (2016), pp. 3932–3937,
https://doi.org/10.1073/pnas.1517384113.
[6] E. Deem, L. Cattafesta, H. Zhang, C. Rowley, M. Hemati, F. Cadieux, and R. Mittal,
Identifying dynamic modes of separated flow subject to ZNMF-based control from surface pres-
sure measurements, in Proceedings of the 47th AIAA Fluid Dynamics Conference, 2017, https:
//doi.org/10.2514/6.2017-3309.
[7] E. A. Deem, L. N. Cattafesta, H. Yao, M. Hemati, H. Zhang, and C. W. Rowley, Experimental
implementation of modal approaches for autonomous reattachment of separated flows, in Proceedings
of the AIAA Aerospace Sciences Meeting, 2018, p. 1052.
[8] Z. Drmac, I. Mezic, and R. Mohr, Data driven modal decompositions: analysis and enhancements,
SIAM J. Sci. Comput., 40 (2018), pp. A2253–A2285, https://doi.org/10.1137/17M1144155.
[9] N. B. Erichson, S. L. Brunton, and J. N. Kutz, Randomized Dynamic Mode Decomposition, preprint,
arXiv:1702.02912, 2017.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1608 ZHANG, ROWLEY, DEEM, AND CATTAFESTA

[10] N. B. Erichson and C. Donovan, Randomized low-rank dynamic mode decomposition for motion de-
tection, Comput. Vision Image Understanding, 146 (2016), pp. 40–50, https://doi.org/10.1016/j.cviu.
2016.02.005.
[11] A. Germani, C. Manes, and P. Palumbo, Polynomial extended kalman filter, IEEE Trans. Automat.
Control, 50 (2005), pp. 2059–2064, https://doi.org/10.1109/TAC.2005.860256.
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

[12] J. Grosek and J. N. Kutz, Dynamic Mode Decomposition for Real-Time Background/Foreground Sep-
aration in Video, preprint, arXiv:1404.7592, (2014).
[13] W. W. Hager, Updating the inverse of a matrix, SIAM Rev., 31 (1989), pp. 221–239, https://doi.org/
10.1137/1031049.
[14] M. S. Hemati, S. T. Dawson, and C. W. Rowley, Parameter-varying aerodynamics models for aggres-
sive pitching-response prediction, AIAA J., 55 (2017), pp. 1–9, https://doi.org/10.2514/1.J055193.
[15] M. S. Hemati, E. A. Deem, M. O. Williams, C. W. Rowley, and L. N. Cattafesta, Improving
Separation Control with Noise-Robust Variants of Dynamic Mode Decomposition, in Proceedings of
the 54th AIAA Aerospace Sciences Meeting, 2016, https://doi.org/10.2514/6.2016-1103.
[16] M. S. Hemati, M. O. Williams, and C. W. Rowley, Dynamic mode decomposition for large and
streaming datasets, Phy. Fluids, 26 (2014), 111701, https://doi.org/10.1063/1.4901016.
[17] T. C. Hsia, System Identification, Lexington Books, Lanham, MD, 1977.
[18] J. N. Kutz, S. L. Brunton, B. W. Brunton, and J. L. Proctor, Dynamic Mode Decomposition:
Data-Driven Modeling of Complex Systems, SIAM, Philadelphia, 2016, https://doi.org/10.1137/1.
9781611974508.
[19] S. Maćešić, N. Črnjarić-Žic, and I. Mezić, Koopman Operator Family Spectrum for Nonautonomous
Systems–Part 1, preprint, arXiv:1703.07324, 2017.
[20] D. Matsumoto and T. Indinger, On-the-Fly Algorithm for Dynamic Mode Decomposition Using In-
cremental Singular Value Decomposition and Total Least Squares, preprint, arXiv:1703.11004, 2017.
[21] O. Nelles, Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy
Models, Springer, New York, 2013, https://doi.org/10.1007/978-3-662-04323-3.
[22] J. L. Proctor, S. L. Brunton, and J. N. Kutz, Dynamic mode decomposition with control, SIAM J.
on Appl. Dyn. Syst., 15 (2016), pp. 142–161, https://doi.org/10.1137/15M1013857.
[23] C. W. Rowley and S. T. Dawson, Model reduction for flow analysis and control, Annu. Rev. Fluid
Mech., 49 (2017), pp. 387–417, https://doi.org/10.1146/annurev-fluid-010816-060042.
[24] C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter, and D. S. Henningson, Spectral analysis of non-
linear flows, J. Fluid Mech., 641 (2009), pp. 115–127, https://doi.org/10.1017/S0022112009992059.
[25] P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656
(2010), pp. 5–28, https://doi.org/10.1017/S0022112010001217.
[26] J. Sherman and W. J. Morrison, Adjustment of an inverse matrix corresponding to a change in one
element of a given matrix, Ann. Math. Statist., 21 (1950), pp. 124–127, https://doi.org/10.1214/
aoms/1177729893.
[27] J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton, and J. N. Kutz, On dynamic mode
decomposition: Theory and applications, J. Comput. Dyn., 1 (2014), pp. 391–421, https://doi.org/10.
3934/jcd.2014.1.391.
[28] V. Verdult and M. Verhaegen, Kernel methods for subspace identification of multivariable LPV and
bilinear systems, Automatica, 41 (2005), pp. 1557–1565, https://doi.org/10.1016/j.automatica.2005.
03.027.
[29] M. O. Williams, I. G. Kevrekidis, and C. W. Rowley, A data–driven approximation of the Koopman
operator: Extending dynamic mode decomposition, J. Nonlinear Sci., 25 (2015), pp. 1307–1346, https:
//doi.org/10.1007/s00332-015-9258-5.
[30] M. O. Williams, C. W. Rowley, and I. G. Kevrekidis, A kernel-based method for data-driven
koopman spectral analysis, J. Comput. Dyn., 2 (2015), pp. 247–265.
[31] M. A. Woodbury, Inverting Modified Matrices, Memorandum report, 42, Statistical Research Group,
Princeton University, 1950.
[32] E. L. Yip, A note on the stability of solving a rank-p modification of a linear system by the Sherman–
Morrison–Woodbury formula, SIAM J. Sci. Stat. Comput., 7 (1986), pp. 507–513, https://doi.org/
10.1137/0907034.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 ONLINE DMD FOR TIME-VARYING SYSTEMS 1609

[33] H. Zhang, S. T. Dawson, C. W. Rowley, E. A. Deem, and L. N. Cattafesta, Evaluating the

Accuracy of the Dynamic Mode Decomposition, preprint, arXiv:1710.00745, 2017.
[34] H. Zhang and C. W. Rowley, Online DMD and Window DMD Implementation in Matlab and Python,
https://github.com/haozhg/odmd (2017).
Downloaded 09/20/22 to 222.233.136.147 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.