U NIVERSITY OF C AMBRIDGE

C ENTRE FOR M ATHEMATICAL S CIENCES

D EPARTMENT OF A PPLIED M ATHEMATICS
& T HEORETICAL P HYSICS

Lie-group methods
in geometric numerical integration

Arieh Iserles

1
OUTLINE

1. Why geometric numerical integration?

2. Elements of differential geometry

3. The exponential and trivializations

4. Runge–Kutta–Munthe-Kaas methods

5. Magnus and Magnus-type expansions

6. Multivariate quadrature

7. Graded Lie algebras

8. The matrix exponential

2
Why geometric numerical integration?
A classical paradigm of applied mathematics:
Do rigorous, pure mathematics as much as possible. Determine qualitative
features of your problem. And then, when exact analysis has reached its
limits, resort to computation.
The problem: having spent great effort and ingenuity on finding precise qual-
itative information on the behaviour of our problem, which often has deep
physical significance, we produce numerical solution that does not respect
this qualitative information.

Invariants represent important qualitative information about the differential

system: it is often advantageous to respect them under discretization. This
means designing numerical methods that share qualitative features of
the differential equation(s).
This is precisely the goal of geometric numerical integration (GNI).
3
Major themes in GNI:

• Symplectic methods for DEs with Hamiltonian structure

(Newton, Störmer, Verlet, de Vogelaerre, Feng Kang, Sanz-Serna, Scovel, Hairer & Lu-
bich, Bennetin & Gorgili, . . . )

• Volume and energy conservation in DEs

(Feng Kang, McLachlan & Quispel, . . . )

• Methods respecting Lie-Poisson structure

(Marsden, Lewis & Simo, Ratiu, . . . )

• Methods replacing symplectic structure by a ‘nearby’ symplectic structure

(Moser & Veselov, Marsden, Bridges & Reich, Hong, . . . )

• Methods for problems evolving on a differentiable manifold, in particular

on a homogeneous manifold
(Crouch, Munthe-Kaas, AI & Nørsett, Zanna, Owren, . . . )
4
Major techniques in GNI:

• Identification of ‘classical’ methods that retain geometric invariants

(Sanz-Serna, Scovel, Skeel, . . . )

• Splitting and composition methods

(Yošida, McLachlan & Quispel, Sanz-Serna & Murua, Blanes & Casas, . . . )

• Backward error analysis

(Naishtadt, Bennetin & Gorgili, Reich, Hairer & Lubich, . . . )

• Clever asymptotic expansions

(Cohen, Hairer & Lubich)

• Structure-preserving projection techniques

(Hairer & Lubich, Calvo, AI & Zanna, . . . )

• Trivializations, group actions and Lie-algebraic

techniques
(Lewis & Simo, Crouch, Munthe-Kaas, AI, Nørsett, Zanna, P. Olver, Owren, . . . )

• Tricks from linear and abstract algebra

(Munthe-Kaas & Owren, Munthe-Kaas, Quispel & Zanna, AI & Zanna, . . . )
5
Elements of differential geometry
A smooth manifold is a smooth domain which locally ‘looks like’ an Euclidean
space: more formally, it can be covered by a smooth atlas made out of local
coordinate charts. Conceptually, think of

A tangent vector at p ∈ M is dρ(t)/dt|t=0, where ρ(t) ∈ M is a smooth

curve s.t. ρ(0) = p.

The linear space of all tangents at p is the tangent space TM|p, while

TM = ∪p∈MTM|p
is the tangent bundle.
6
The cotangent space T⋆M|p consists of all linear functionals acting on ele-
ments of TM|p.

Differential equations and tangents

A vector field on a manifold M is a smooth function F (p) ∈ TM|p, p ∈ M.

The set of all vector fields over M is denoted by X(M) and, clearly, it is a
linear space.

The differential equation

y ′ = F (y), t ≥ 0, y(0) = y0 ∈ M,
where F ∈ X(M), evolves on the manifold M.

The flow of this DE is

y(t) = Ψt,F (y0), t ≥ 0.

7
Therefore
d
F (y) = Ψt,F (y0)|t=0.
dt
In other words, F is the infinitisimal generator of the flow.

Noting that Ψα,F = Ψ1,αF , we define

Ψ1,F = exp(F ), hence exp(tF ) = Ψt,F .

This is the exponential map.
[This should not be confused with the concept of a semigroup in PDE theory, although at a
deeper abstract level the two are actually quite similar!]

In general, flows fail to commute. Thus, let

Φs,t = exp(sF ) ◦ exp(tG) ◦ exp(−sF ) ◦ (−tG).

8
If flows commute then Φs,t(y) = y, but this isn’t in general true.
Φs,ts (y0 )
· ·
* i
PP
PP
Ψs,F · · ·· PPs
y0
· · 
·
·· 

· ·· 
·· 
s
6
·· 
 Ψt,G


Ψt,G 

Ψs,F
· · · · · · · · · · · · · · · · · · ·- s
s

The local measure of lack of commutativity is the commutator H = [F, G]

where (translating to the standard Euclidean coordinates in Rn)
n (
X
)
∂Fi(y) ∂Gi(y)
Hi(y) = Gj (y) − Fj (y) .
j=1 ∂yj ∂yj

9
Note that for every F, G, H ∈ X(M) and scalar α we have the following fea-
tures of the bracket operation:
Skew symmetry: [F, G] = −[G, F ];
Bilinearity:

[αF, G] = α[F, G],

[F + G, H] = [F, H] + [G, H];
The Jacobi identity

[F, [G, H]] + [G, [H, F ]] + [H, [F, G]] = 0.

Therefore, X(M) is a Lie algebra.

In the important case of linear DE y ′ = ay (where a is a matrix) we have

Ψt,F (y) = exp(ta)y0, with the familiar matrix exponential
∞
X 1 m m
eta = t a .
m=0 m!
In that case [a, b] = ab − ba.
10
Examples of manifolds:

• The sphere Sn ⊂ Rn: x ∈ Rn s.t. kxk = 1;

• The regular torus Tn ⊂ Rn: x ∈ Rn which are 1-periodic in each coordi-

nate;

• The orthogonal group SO(n) of n × n orthogonal matrices;

• The Grassmannian Gn,m of real n × m matrices, m ≤ n, consisting of

unit-length columns which are orthogonal to each other and equivalenced
by orthogonal transformations;

• The isospectral orbit In(y0) of all n × n symmetric matrices which are

similar to the symmetric matrix y0;

• The projective space Pn of all lines in Rn that pass through the origin.

11
Lie groups

Lie group G is smooth manifold, endowed with group structure which is con-
tinuous with respect to the topology of the manifold.

O(n, R): real n × n orthogonal matrices (orthogonal group);

SL(n, R): real n × n matrices with unit determinant (the special linear group);

SU(n, C): complex n × n unitary matrices with unit determinant (the special
unitary group);

SO(n, m, R): real n × n matrices a s.t. apa⊤ = p, p = diag [1m, −1n−m],

with unit determinant. (n = 4, m = 1: the Lorenz group);

E(n, R): The Euclidean group of all translations and length-preserving linear
transformations in Rn;

A(n, R): The affine group of all translations and area-preserving linear trans-
formations in Rn.
12
Why are Lie groups useful?

Lie groups are important since they provide an appropriate formalism to in-
vestigate symmetries, invariants and qualitative behaviour of differential equa-
tions.

There are many other good reasons why Lie groups are important. We’ll see one (group
actions on manifolds) but there are many others, not least in number theory.

Many physical laws are conveniently formulated with built-in Lie-group sym-
metries.
For example, laws of motion have SO(3) symmetry, equations of special rela-
tivity evolve in SO4,1 and theory of superstrings can be formulated in SU(32)
and in E(8) × E(8).

A finite-dimensional Lie group can be usually (but not always!) represented

as a subgroup of the set of n × n nonsingular square matrices, GLn[F], for
some n ≥ 1 and a field F. Such groups are called matrix Lie groups.
13
The tangent space of a Lie group

Let G be a Lie group and g = TG|I the tangent space at identity. Since G is a
group, it follows at once that
TG|X = gX for all X ∈ G.
Since (as we have already seen) there is a natural isomorphism between
(finite-dimensional) linear vector fields and square matrices, we deduce that

• g is a Lie algebra (cf. next slide)

• The exponential map is the classical matrix exponential.

• Let ρ and σ be smooth curves on G s.t. ρ(t) = I + ta + · · · and

σ(t) = I + tb + · · ·. Then a, b ∈ g and
∂2
[a, b] = ρ(s)σ(t)ρ(−s)|t=s=0 = ab − ba.
∂t∂s

14
Lie algebras

We say that the abstract set g is a Lie algebra if it is a linear space, which in
addition is closed under the binary operation [ · , · ] : g × g → g which obeys
the following axioms:
1. Linearity: For every a, b, c ∈ g and scalars α, β it is true that
[αa + βb, c] = α[a, c] + β[b, c].

2. Skew-symmetry: For every a, b ∈ g

[a, b] = −[b, a].

3. The Jacobi identity: For every a, b, c ∈ g

[[a, b], c] + [[b, c], a] + [[c, a], b] = 0.

Lie algebras can be fairly strange objects and not all of them originate as tangent spaces
of Lie groups. A nice example is the modular group SL(n, Z) of integer matrices with unit
determinant, which is important in number theory.
15
Examples of Lie algebras

1. Lie group: O(n, R), SO(n, R)

Lie algebra: The set so(n, R) of real n × n skew symmetric matrices;

2. Lie group: SL(n, R):

Lie algebra: The set sl(n, R) of real n × n matrices with zero trace;

3. Lie group: SO(n, m, R):

Lie algebra: The set so(n, m, R) of real n × n matrices a such that
ap + pa⊤ = 0, where p = diag[1m, −1n−m]. With greater generality,
for any nonsingular symmetric p and the quadratic Lie group

{x ∈ GL(n, R) : xpx⊤ = p}
the corresponding Lie algebra is

{a ∈ gl(n, R) : ap + pa⊤ = 0.

16
Group actions

An action of a Lie group G on a manifold M is a smooth map Λ : G × M → M

s.t.
Λ(I, y) = y, y ∈ M,
Λ(p, Λ(q, y)) = Λ(pq, y), p, q ∈ G, y ∈ M.
• Each group acts on itself;

• O(n, R) acts on on the sphere Sn, Λ(p, y) = py;

• SO(n, R) acts on the Grassmannian

G(n, m) = S(n)/(SO(m) × SO(n − m))
• . . . and on the isospectral orbit In(y0) via
Λ(p, y) = pyp−1.

As a matter of fact, all previous examples of manifolds are subject to transitive group action:
every point in M is reachable from any other point via the group action. In that case M is a
homogeneous manifold.
17
Differential equations and group actions

Given a homogeneous manifold M and a group action Λ we define

λ∗ : g → X(M) as
d
λ∗(a)(y) = Λ(ρ(s), y)|s=0,
ds
where ρ(s) = I + at + · · · is a smooth curve in G.

Suppose that G is a matrix group. Then, for a ∈ g the flow of λ∗, i.e.

y ′ = λ∗(a)(y), t ≥ 0, y(0) = y0 ∈ M,
can be expressed in the form

y(t) = Λ(s(t), y0), t ≥ 0,

where
s′ = as, t ≥ 0, s(0) = I.

18
This can be generalized from fixed a ∈ g to a sufficiently smooth function
a : R+ × M → g: The solution of the differential equation

y ′λ∗(a(t, y))(y), t ≥ 0, y(0) = y0 ∈ M,

can be represented as y(t) = Λ(s(t), y0), where

s′ = a(t, Λ(s, y0))s, t ≥ 0, s(0) = I,

evolves in G.

Therefore,
instead of solving the equation in M, i.e. finding yN +1 given yN , say, find a
group action that takes yN to yN +1.

We conclude that
if we can devise a numerical method that respects a Lie-group structure then
it can be extended to respect every homogeneous manifold structure acted
upon by that group.
19
The main paradigm of Lie-group methods

Instead of a group action, we can consider an algebra action. Specifically, if

G ∋ x = ev , where v ∈ g, then µ(v, y) = Λ(x, y). Note that this is less gen-
eral than a group action: it is entirely possible that there is no single v ∈ g s.t.
x = ev , although in a finite-dimensional group there always exist v1, v2, . . . , vr
s.t. x = ev1 · · · evr .
For example, each element of SL(2) can be obtained as a product of two
exponentials of elements from sl(2).

We follow the pattern

yN DE on a manifold yN +1
6

?
pN DE on a Lie group pN +1
6

? DE on a Lie algebra
ωN - ωN +1
20
Some Lie-group methods do not follow this pattern:

• The method of Crouch & Grossman represents the equation in the form
P
y ′ = dl=1 αl (t)ql , using a rigid frame ql ∈ TM| y , l = 1, . . . , d, and
composes the solution from one-dimensional ‘steps’.

• The method of McLachlan, Quispel & Robidoux represents the equation in

the skew-gradient form y ′ = S(t, y)∇g(y), where S is skew-symmetric,
and suitably discretizes the gradient.

• The approach of cannonical coordinates of the second kind (Marthinsen

& Owren) writes
G ∋ y(t) = eθ1(t)φ1 eθ2(t)φ2 · · · eθd(t)φd y0,
where dim g = d, {φ1, . . . , φd} is a basis of g and θ1, . . . , θd are scalar
functions. It is then possible to derive differential equations for the un-
knowns θk . The CCSK approach is particularly useful when the basis
corresponds to the root space decomposition of g, since this simplifies
the equations a great deal.

21
The exponential and trivializations
An equation evolving on a homogeneous manifold M can be written in the
form

y ′ = µ∗(a(t, y))(y), t ≥ 0, y(0) = y0 ∈ M,

where the function µ∗ : g → X(M) is a Lie-algebra homomorphism, X(M)

is the set of vector fields on M and a : [0, ∞) × M → g.
Many such ODEs occur in applications:
O(n, R) (Orthogonal group): mechanical systems, robotics, computer vision,
computation of Lyapunov exponents, isospectral flows, numerical linear alge-
bra;
SL(n, R) (Special linear group): conservation of volume, Riccati systems,
Sturm–Liouville problems, image processing;
Sp(n, R) (Symplectic group): Hamiltonian and Lie–Poisson systems;
SO(1, 3) (Lorenz group): relativity theory (also SO(2, 5));
SU(n, C) (Unitary group): quantum mechanics.
22
Example 1: ‘Classical’ Lie-group equations:
y ′ = a(t, y)y, t ≥ 0, y(0) = y0 ∈ G,
where a : [0, ∞) × G → g.

Example 2: Isospectral flows:

y ′ = [b(t, y), y], t ≥ 0,
where y(0) = y0 ∈ Symn[R] and b : [0, ∞) × I[y0] → son[R].

Example 3: Equations on a sphere:

y ′ = a(t, y ) × y , t ≥ 0, y (0) = y 0 ∈ S2,
where a : [0, ∞) × S2 → R3 and × is a vector product. E.g., the Lagrange
top equations of a rigid body in body coordinates are
Π′b = Πb × Ωb + M g l Γb × χ, Γb = Γb × Ωb,
where Πb is angular momentum, Ωb the angular velocity (Ωb = Ib−1Πb, Is
the inertia tensor) and Γs the gravity, while M is the mass, g the gravitational
constant, l the distance between the centre of mass and the centre of frame
of reference and χ the unit vector on the body axis.
23
Trivialization

We solve homogeneous-manifold equations by five conceptual steps, to be

implemented in every time step,

1. Transform the equation from M to G;

2. Transform the equation from G to g;

3. Discretize the equation in g;

4. Transform the outcome from g to G;

5. Transform the outcome from G to M.

The advantage of this approach is that

while in general M is a nonlinear surface, the Lie algebra g is a linear space.
As long as we discretize there with just linear operations and commutators,
we are bound to respect its structure!

Steps 1 and 5 are already clear from our discussion, at present we focus on steps 2–4.
24
A trivialization is a smooth mapping Φ : g → G such that Φ(0) = I. The main
idea is write the solution p of a Lie-group equation in the form

p(t) = Φ(ω(t)), t≥0

and replace the equation for p by the Lie-algebraic equation for ω.

Formally,

p′ = a(t, p)p = dΦ(ω, ω ′)

⇒ ω ′ = dΦ−1
ω a(t, p)
⇒ ω ′ = dΦ−1
ω a(t, Φ(ω)),
with the initial condition ω(0) = 0.

Note that a trivialization might be valid only in (hopefully, large) neighbourhood of the origin:
this represents a possible restriction on the step size of the discretization method.

25
An obvious (and most useful) trivialization is exponentiation,
p(t) = eω(t)pN .
The outcome is the dexpinv equation

∞
X Bm
ω = dexp−1
ω a= adm
ω a, ω(tN ) = 0
m=0 m!

(Hausdorff), where
m times
z }| {
m
adω a = [ω, [ω, . . . , [ω, a] · · · ]]
and {Bm}m≥0 are Bernoulli numbers,
∞
X Bm m z 1 z2 − 1 z4 + 1 z 6.
z = z ≈1−1
2 z + 12 720 30240
m=0 m! e − 1

Note that the dexpinv equation is always nonlinear and consists of an infinite
sum. However, infinite sums can be truncated and nonlinearity is a worthwhile
price to pay.
26
Specifically, the dexpinv equation is
1 [ω, a] + 1 [ω, [ω, a]] − 1 [ω, [ω, [ω, [ω, a]]]]
ω′ = a − 2 12 720
1 [ω, [ω, [ω, [ω, [ω, [ω, a]]]]]] + · · · .
+ 30240

For a quadratic Lie group, i.e. G = {p : pqp⊤ = q}, where q ∈ Symn[R] is

nonsingular, an interesting alternative is the Cayley trivialization
1z
1+2
Φ(z) = .
1−1
2z
It results in the dcayinv equation

ω′ = a − 1
2 [ω, a] + 1 ωaω.
4
The Cayley trivialization has two advantages vis-á-vis the standard exponen-
tial: the equation is much simpler and the evaluation of Φ(ω) much cheaper, in
particular for large n. On the other hand, of course, it applies only to quadratic
Lie groups.
27
Runge–Kutta–Munthe-Kaas methods
We apply an RK method to the dexpinv equation, in place of the original Lie-
group equation. E.g., instead of the familiar 3rd-order scheme
k1 = a(tN , yN )yN
k2 = a(tN +1/2, yN + 12 hk 1 )(y N + 1 hk )
2 1
k3 = a(tN+1, yN − hk1 + 2hk2)(yN − hk1 + 2hk2)
1k + 2k + 1k )
∆ = h( 6 1 3 2 6 3
yN +1 = yN + ∆,
we use
k1 = a(tN , yN )
1 hk
k2 = a(tN +1/2 , e 2 1 yN )
k3 = a(tN +1, e−hk1+2hk2 yN )
1k + 2k + 1k )
∆ = h( 6 1 3 2 6 3
1 h[∆,k ]
∆+ 6
yN +1 = e 1 yN .
This is again a 3rd-order method, except that it is guaranteed to evolve on G.
28
• The beauty of this approach is that we can take any RK method and con-
vert it into a Lie-group method without any major effort.

• Each step of an ν-stage RK–MK method requires ν function evaluations

and ν computations of the matrix exponential. The latter can be fairly
expensive for high ν.

• The number of commutators increases rapidly with order (in particular if

we want really high order methods!). This can be alleviated by techniques
from graded Lie algebras, which will be considered later.

• Implicit RK–MK methods can be used, but they require the solution of un-
derlying nonlinear algebraic equations. Owren & Welfert have presented
an extension of Newton’s method that respects Lie group structure but,
again, it ain’t cheap.

• Similar approach is valid for other trivializations, in particular for the dcay-
inv equations.

29
Magnus and Magnus-type expansions
Consider (for simplicity) the linear equation

y ′ = a(t)y, t ≥ 0, y(0) ∈ G, a(t) ∈ g.

Recall that y(t) = eω(t)y0, where ω was given by the dexpinv equation.

Wilhelm Magnus showed that

Z t Z tZ ξ
1
ω(t) = a(ξ)dξ − 1
2 [a(ξ2), a(ξ1)]dξ2dξ1
0 0 0
Z tZ ξ Z ξ
1 2
+1
4 [[a(ξ3), a(ξ2)], a(ξ1)]dξ3dξ2dξ1
0 0 0
Z tZ ξ Z ξ
1 1 1
+ 12 [a(ξ3), [a(ξ2), a(ξ1)]]dξ3dξ2dξ1
0 0 0
+ ··· .
Extensive use of Magnus expansions in theoretical physics, quantum chem-
istry, control theory, stochastic DEs, . . . .
30
Systematic theoretical analysis and numerical implementation of the Magnus
expansion by AI & Nørsett, using graph theory.

• Graph: The pair G = hV , E i, where V = {v1, . . . , vr } are vertices and

E ⊆ V × V the edges;
q
• Path from vi to vj : Ordered set {(vsl , vsl+1 )}l=1 of distinct vertices s.t.
s1 = i and sq+1 = j;

• A graph is connected if all vertices connected by a path and it is a tree if

such path is unique;

• Rooted tree: The pair T = (G, w), where G is a tree and w (the root) is
one of its vertices.

• A rooted tree admits a natural partial ordering of ancestor/successor and

parent/child. Note that root has no parent. Vertices with no children are
called leaves.
u u
@
u @u
@
u
a leaf
@u u u u
@
@u
Q
u
Q 
u
the root
Q
u

31
It is easy to see that all terms in a Magnus expansion of ω ′ are of the form
Z 
simpler term, another simpler term .
This can be expressed by recursion (the proof follows by Picard iteration):

1. We commence from a(t);

2. If H1(t) and H2(t) already feature in the expansion, so does

Z t 
H1(ξ)dξ, H2(t) .
0

We model this with rooted binary trees,

1. The tree u
corresponds to a(t);

2. If Hτ1 ; τ1, Hτ2 ; τ2 have been already derived, then

τ1
Z t  τ2
u
Hτ1 (ξ)dξ, Hτ2 (t) ; @@u .
0
32
Magnus in terms of trees

Let Tm be the set of trees with m vertical lines (i.e., corresponding to terms
with m integrals). Then
∞
X X Z t
ω(t) = α(τ ) Hτ (ξ)dξ,
m=0 τ ∈Tm 0

where the constants α(τ ) are obtained recursively: Any τ ∈ Tm, m ≥ 1, can
be written uniquely in the form
τs
u u
@ @u
τ2 q
τ1 u u
q
q

u @@u
@
@u ,
where
α( u) = 1,
s
Bs Y
α(τ ) = α(τl ).
s! l=1

33
AI, Nørsett & Rasmussen: Appropriately truncated “by power”, a Magnus ex-
pansion yn+1 = Υhyn is time symmetric, i.e. Υh ◦ Υ−h = Id. Therefore
Υh = eΨh , where Ψh is an odd function and it follows that such an expan-
sion is always of an even order – if we truncate it to produce odd order, we
gain for free an extra unit of order.

Sixth-order Magnus expansion:

u
u u u u
u u u u @@u u u
@ @
u u u u @u u u @u
u @@u u u u @@u @@u u u u
@ @
u u u u u @u u u u u u @u
@ @ @ @ @ @
@u @u @u @u @u @u
ω(t) ; − 1 1 1 1 −1 1
u

2
u u +4 u + 12 u − 24 u
8 u − 24 u .

Note that we truncate the expansion by throwing away high-order terms (trun-
cation “by power”), rather than trees of sufficiently high m. This results in much
smaller number of surviving trees.
34
Convergence of the Magnus expansion

Theorem The Magnus expansion converges if

Z t Z 2π
dξ
ka(ξ)kdξ ≤ ≈ 1.086869.
0 0 4 + ξ(1 − cot 2ξ )

Original proof by Blanes, Casas, Oteo & Ros. Shorter proof by Moan: Integra-
tion & triangle inequality imply that
Z t Z t X
∞
−1 |Bk |
kω(t)k ≤ kdexpω(ξ)a(ξ)kdξ ≤ (2kω(ξ)k)k a(ξ)dξ
0 0 k=0 k!
Z t
= g(2kω(ξ)k)ka(ξ)kdξ,
0
where g(x) = 2 + 2 x (1 − cot x ). Bihari-type inequality: Let h, g, v ∈ C[0, t∗ )
2 Rt
positive, g nondecreasing. Then h(t) ≤ 0 g(h(ξ))v(ξ)dξ implies
Z t  Z x
dξ
h(t) ≤ g̃ −1 v(ξ)dξ , g̃(x) = .
0 0 g(ξ)
Letting h(t) = 2kω(t)k, v(t) = ka(t)k completes the proof. 2
35
The Fer expansion

We approximate the solution of y ′ = a(t)y in the form

Z t 
y(t) = exp a(ξ)dξ v(t).
0
Then
v ′ = ferR t a(ξ)dξ a(t) v, t ≥ 0, v(0) = y(0),
0
where
(I + adb)e−adb − I
ferba = a
adb
∞
X k
= (−1)k adkb a.
k=1
(k + 1)!
This procedure can be iterated: the outcome is the Fer expansion
Rt Rt
y(t) = e 0 ω0 (ξ)dξ e 0 ω1 (ξ)dξ · · · y(0), t ≥ 0.
 
AI: ωm(t) = O t 2m+2 −2 , hence the order increases very fast.

36
Also the Fer expansion can be expanded in rooted trees. Thus, for order 6,
just two terms are required and

ω0 : u
u
u u u u
@
u u u u u u @u
u u u u @
u u @u u u @@u
@ @ @
u @u u @u u @u
ω1 : 1 u
@
@u
u
1
+3 @
@u
1
+8 @
@u
1
+ 30 @
@u
2

Detailed complexity analysis of Lie-group methods (Celledoni, AI, Nørsett &

Orel) demonstrates that in general the Magnus expansion is always cheaper
than the Fer expansion. A major reason is that Fer requires more evaluations
of the exponential. However, Fer might be useful for ‘niche’ computations.

37
The Cayley expansion

For a quadratic Lie group with the Cayley trivialization we let

I+2 1 ω(t)
y(t) = cayω(t)y0 = 1
y0 , t ≥ 0,
I − 2 ω(t)
consequently

ω′ = a − 1
2 [ω, a] + 1 ωaω,
4 t ≥ 0, ω(0) = 0.
In line with the Magnus expansion, we can show that
Z t Z tZ ξ
1
ω(t) = a(ξ)dξ − 1
2 [a(ξ2), a(ξ1)]dξ2dξ1
0 0 0
Z tZ ξ Z ξ
1 1 2
+4 [[a(ξ3), a(ξ2)], a(ξ1)]dξ3dξ2dξ1
0 0 0
Z tZ ξ Z ξ
1 1
−1
4 a(ξ2)dξ2a(ξ1) a(ξ2)dξ2dξ1 + · · · .
0 0 0
This Cayley expansion can be also expanded in rooted trees, except that we
require slightly more complicated structures.
38
AI: Specifically, we have the following composition rules:

R
1. 0t a(ξ)dξ ;
u
u is a term.

2. If H ;τ is a term, then so is
Z t τ u
@
@u
[H(ξ), a(ξ)]dξ ; u .
0

3. If H1 ;τ1 and H2 ;τ2 are terms, then so is

Z t τ1 τ2
@e
H1(ξ)a(ξ)H2(ξ)dξ ; u .
0

We obtain u
u u
@
u @u u u
u u u u u u
@ @ @e
@u @u
ω(t) ;
u
1
−2 1
+4 −1 + ··· .
u u u
4 u

39
Multivariate quadrature
Each term in a Magnus, Fer or Cayley expansion corresponds to a multivariate
integral over a different polytope. In principle, multivariate integration is very
expensive: we need many function evaluations for every polytope. Fortunately,
everything simplifies!
All integrals in question are of the form
Z
I(h) = L(a(ξ1), a(ξ2), . . . , a(ξr ))dξ ,
hS
where L is a multilinear form and S is a polytope,

S = {x ∈ Rr : 0 ≤ xk ≤ xik , k = 1, 2, . . . , r},
with x0 = 1 and ik ≤ k − 1, x0 = 1, k = 1, 2, . . . , r. Specifically, for the first
few integrals we have the following polytopes S and multilinear forms L:

40
 Tree r L(z1, . . . , zr ) S
u
u u 1 z1
u u @
@@u @
@
u u 2 [z2, z1] @
u u
@
@u
X
u u @X X
X
@ @ C
@u @ C
u
u 3 [[z3, z2], z1] @C
@
@
u u u
@
u @u @ @
@@u @ @
@
u 3 [z3, [z2, z1]] @

Let c1, . . . , cν ∈ [0, 1] be distinct quadrature points and ak = a(tN + ck h).

We approximate I(h) with
X
Q(h) = hr βiL(ai1 , ai2 , . . . , air ),
i∈Cνr
where Cνr = [1, 2, . . . , ν]r . Note that we repeatedly recycle just ν function
evaluations of a!

41
Theorem (AI & Nørsett) The order of quadrature is ν + s, where
Z 1
ζ i−1q(ζ)dζ = 0, i = 1, 2, . . . , s,
0
Qν
and q(x) = k=1(x − ck ).

The proof is long and complicated. We’ll prove instead a simpler statement.
For this (and for much future use) we require the

Alekseev–Gröbner Lemma Given the ODE y ′ = f (t, y), suppose that u is

any C1 function s.t. u(tN ) = y(tN ). Then
Z t
u(t) − y(t) = Φ(t − x)[f (x, u(x)) − u′(x)]dx,
tN

where Φ is the solution of the variational equation Φ′ = ∂f

∂y Φ, Φ(0) = I.

Theorem (Zanna) The order of the approximation to truncated Magnus expan-

sion with the above quadrature is ν + s.
42
To prove this, we replace the matrix a by its interpolating polynomial â s.t.
â(tN + ck h) = a(tN + ck h), k = 1, . . . , ν. Let u′ = âu, u(tN ) = y(tN ).
Therefore, by the A–G Lemma
Z t
N +1
u(tN +1) = y(tN +1) + Φ(tN +1 − x)[â(x) − a(x)]u(x)dx.
tN
The integrand vanishes at c1, . . . , cν . Therefore, once we discretize the in-
tegral
 by the
 quadrature formula, all the is left is the quadrature error, i.e.
O hν+s+1 . 2

Corollary If c1, . . . , cν are Gauss–Legendre points in [0, 1] then the order is

2ν.

Example: A 4th-order approximantion

Z hZ ξ √
1
[a(ξ2), a(ξ1)]dξ ≈ 63 h2[a1, a2],
0 0
√ √
1 − 3, c = 1 + 3.
where c1 = 2 6 2 2 6
43
To construct a quadrature formula in greater detail, let ℓk ∈ Pν−1 be the kth
cardinal polynomial of Lagrangian interpolation,

ℓk (ck ) = 1, ℓk (cj ) = 0, j 6= k.
Then
Z r
Y
βi = ℓik (ξk )dξ .
S k=1

GOOD NEWS
It takes just ν evaluations of the matrix a to compute all the integrals in the
expansion to order 2ν.

BAD NEWS
The numbers of commutators increases very fast, in line with the exponential
growth in the number of combinations in Cνr . Therefore, the volume of linear
algebra required in every time step becomes prohibitive for large orders.

44
Graded Lie algebras
The number of commutators can be vastly reduced by an idea due to Munthe-
Kaas & Owren.

Replace ha1, . . . , haν by b0, b1, . . . , bν−1, where

ν−1
X
1 (c − 1 )l b = ha , k = 1, 2, . . . , ν.
l! k 2 l k
l=0
Then bl ≈ hl+1a(l)( 2 1 h) = O(hl+1 ): the term b is of grade l + 1. The
l
grades propagate naturally under commutation: if the grade of xi is wi for
i = 1, 2, then the grade of [x1, x2] is w1 + w2. Therefore, the term
L(bi1 , bi2 , . . . , bir )
P
is of grade |i| = il . We obtain the order-(2ν) quadrature
X
f (h) =
Q β̃iL(bi1 , bi2 , . . . , bir ),
|i|≤2ν
where
Z r
Y
β̃i = (ξl − 1
2 ) il dξ · · · dξ .
r 1
S l=1

45
Free Lie algebras

For future use we require the concept of a Free Lie algebra. Given generators
Φ = φ1, φ2, . . . , φν , we say that

F = FLA(φ1, φ2, . . . , φν )
is the free Lie algebra generated by Φ if it is the closure of the generators
under commutation and linear combinations. Trivially, it is a Lie algebra.

We attach to each φl the grade g(φl ) = κl ∈ N and let grades propagate by

commutation:

g(t1) = µ1, g(t2) = µ2 ⇒ g([t1, t2]) = g(t1) + g(t2).

ν
We denote by Km the linear space of all terms in the FLA of grade m. Of
ν
course F = Sp{Km : m ≥ 1}, although we will not make much use of it.

ν
The surprisingly small dimension of Km is the key to our goal, to reduce the
number of commutators.
46
The great commutator throw-away

Three mechanisms allow us to get rid of most terms:

1. All terms of grade ≥ 2ν + 1 are not required and can be thrown out.

2. It is possible to show that the discrete expansion is time symmetric. There-

fore sums of terms of even grade vanish and we don’t need to compute
them.

3. The classical Witt–Birkhoff formula gives the (surprisingly small) dimen-

ν
sion of Km for unit grades κl ≡ 1 (corresponding to xl = hal ). Munthe-
Kaas & Owren have extended the Witt–Birkhoff formula to arbitrary grades.
Specifically, let λk , k = 1, 2, . . . , s = max κl , be the zeros of the polyno-
mial
ν
X
1− z κi .
i=1

47
 1. Then
s
ν 1 X X m/d
dim Km = µ(d) λi ,
m d|m i=1

where µ is the Möbius function,


 1,
 d = 1,
µ(d) = (−1)q , ni = 1, i = 1, 2, . . . , q,

 0, otherwise,
n n n
where d = p11 p22 · · · pq q is the prime-number decomposition of d.
ν
They also gave a recursive procedure for the formation of a basis of Km
(a generalisation of Hall and Lyndon bases).

The number of required terms:

order: 2 4 6 8 10
Naive 1 5 80 3304 1256567
Clever 1 2 7 22 73
48
Is this the optimal Magnus expansion?

Further economy is possible by clever algorithmic tricks and aggregating terms.

This has been worked out for even orders 4–8 by Blanes, Casas & Ros.

For example, for order 6, we compute

√ √
c1 = 1
2 − 15 , c = 1 , c = 1 + 15 ,
10 2 2 3 2 10
ak = ha(tN + ck ), k = 1, 2, 3,
√
b1 = ha2, b2 = 315 (a3 − a1),
b3 = 10
3 (a3 − 2a1 + a1),
v1 = [b1, b2],
v2 = [b1, 2b3 + v1],
1 v ],
v3 = [−20b1 − b3 + v1, b2 − 60 2
1b + 1 v ,
ωN = b1 + 12 3 240 3
yN +1 = eωN yN .
Thus,
just three function evaluations, three commutators and one exponential!
49
The matrix exponential
It is not enough to “approximate” the exponential: we must do so while map-
ping an element from g to G! Otherwise all the hard work in designing Lie-
group methods would have been in vain.

Standard methods for the calculation of the matrix exponential:

• Eigenvalue/eigenvector decomposition:
Exceedingly expensive (in particular for large systems) and ill conditioned.

• Polynomial and rational approximations:

We cannot expect the result to live in G, with one exception: when the Lie group is
quadratic and we use diagonal Padé approximations (Feng Kang). On the other hand,
no rational approximation can map sl(n) to SL(n) for n ≥ 3.

• Krylov subspace approximation:

Very effective means for large matrices (Hochbruck & Lubich). However, there is no
reason for them to map a Lie algebra to ‘its’ Lie group.
50
Splitting methods (Celledoni & AI)

Given a ∈ g, we split its exponential,

eta ≈ etb1 etb2 · · · etbs

such that

P
1. k bk = a, each bk lives in g;

2. It is easy to evaluate each etbk exactly; and

3. The overall error is consistent with the order of the Lie-group method

Note that necessarily etA ∈ G.

If bk = bs+1−k ∀k then the approximation is time symmetric. In that case the

order can be further enhanced with the Yos̆ida device.
51
Practical splittings decompose a into low-rank matrices, using the following
explicit formula for the matrix exponential. Suppose that u and v are both
n × r matrices, r ≪ n, and c = uv ⊤. Then

etc = I + tud−1(etd − I)v ⊤,

where d = v ⊤u is r × r, hence ‘small’.

For example, for g = son we let r = 2, s = n − 1, set

[0] [0] [0]
b[0] = a = [b1 , b2 , . . . , bn ]
and choose
[0] ⊤ [0] ⊤
b1 = b1 e1 − e1b1 ∈ so(n)
and b[1] = b[0] − b1. Observe that the first row and column of b[1] vanish. We
continue in this manner, similar to LU factorization, letting b[i] = b[i−1] − bi
and
[i−1] ⊤ [i−1] ⊤
bi = bi ei − eibi ∈ son, i = 1, . . . , n − 1.
Note that rank bi = 2.
52
The Yošida device

Let Φh be a time-reversible numerical method of order p for the solution of

y ′ = f (t, y).
Time reversibility implies that p is even.
We consider a new method,
Ψh = Φαh ◦ Φβh ◦ Φαh.

1. The method Ψh is time reversible.

2. Suppose that
1 21/(p+1)
α= , β=− .
2 − 21/(p+1) 2 − 21/(p+1)
Then Ψh is of order p + 2.

This approach (originally developed by Haruo Yošida for symplectic methods)

can be used to improve the order of our approximation of the exponential.
53
Coordinates of the second kind (Celledoni & AI)

Let d = dim g and let {φ1, φ2, . . . , φd} be its basis. We seek scalar functions
θ1, θ2, . . . , θm s.t.
eta = eθ1(t)φ1 eθ2(t)φ2 · · · eθd(t)φd .
Such functions always exist but their practical evaluation in closed form is vir-
tually impossible for d ≥ 3 (Wei & Norman). Alternatively, we approximate the
functions θk to requisite order.

The structure constants of g are numbers

d
X
j j
ck,l , k, l, j = 1, . . . , d, such that [φk , φl ] = ck,l φj .
j=1
The Taylor expansion of each θk can be derived explicitly in terms of the struc-
ture constants (which, of course, depend on the choice of the basis). Let
d
X
a= αk φk .
k=1

54
We have

θk (0) = 0,
θk′ (0) = αk ,
d l−1
X X
θk′′(0) = αl ckl,j αj ,
l=1 j=1
Xd l−1
X
θk′′′(0) = 2 ckl,j [θl′′(0)αj + θj′′(0)αl ]
l=1 j=1
X X j−1
d l−1 d
X X d X
X d l−1
X j
+2 cm c k αα α +
l,j i,m l j i cl,icki,j αl α2
i
l=1 j=1 i=1 m=1 l=1 j=1 i=1
and so on. More and more summation!
Assume, though, that the structure constants are sparse: they are almost all
zero. In that case the cost reduces a very great deal! This can be accom-
plished by choosing as our basis a root space decomposition of g: the number
of summations typically drops by a factor of two and the cost of a second-order
approximation in so(n) is just O(n).
55
Generalized polar decomposition (Munthe-Kaas, Quispel & Zanna)

An involutory automorphism of a Lie group G is a one-to-one map σ : G → G

s.t.

σ(x · y) = σ(x) · σ(y), x, y ∈ G,

σ(σ(x)) = x, x ∈ G.
Each G-automorphism can be lifted to the underlying Lie algebra, resulting in
a g-automorphism,
d
dσ(a) = σ(eta)|t=0.
dt
We use dσ to define the sets

k = {a ∈ g : dσ(a) = a},
p = {a ∈ g : dσ(a) = −a}.
While k is a Lie subalgebra, p is a Lie triple system: a linear space s.t.

a, b, c ∈ g ⇒ [a, [b, c]] ∈ g.

56
We have
g = k ⊕ p.
Specifically, g ∋ a = p + k, where
p=1
2 [a − dσ(a)],
1 [a + dσ(a)].
k=2
Note that
a, b ∈ k : [a, b] ∈ k,
a, b ∈ p : [a, b] ∈ k,
a ∈ k, b ∈ p : [a, b] ∈ p,
a ∈ p, b ∈ k : [a, b] ∈ p.

It is possible to prove that the Lie group can be factorized into generalized
polar decomposition
G ∋ z = xy, σ(y) = y, σ(x) = x−1.
At the algebra level, this corresponds to
G ∋ ec = eaeb, a ∈ p, b ∈ k.
The main idea is to obtain the leading terms of a, b from p, k.
57
We have
1 [p, k]t2 − 1 [k, [p, k]]t3
a = pt − 2 6  
+ ( 24 [p, [p, [p, k]]] − 24 [k, [k, [p, k]]])t + O t5 ,
1 1 4
 
b = kt − 12 [p, [p, k]]t + O t5 .
1 3

Suppose now that dim p is very small. In that case ea is very cheap to com-
pute. This is not the case with eb, but recall that k is a subalgebra! Hence, we
can go on splitting it!

We thus obtain a sequence of algebra authomorphisms and low-dimensional

triple Lie systems pk for k = 1, 2, . . . , m, such that
ea = ep1 ep2 · · · epm , pk ∈ p k .

A convenient way of generating such dσk is through involutory inner autho-

morphisms
σ(x) = sxs⊤, dσ(a) = sas⊤,
where s ∈ G ∩ O(n).
58
Let sj = diag(1 − 2ej ). Then elements in each pj are nonzero just along the
j row and column, hence rank pj ≡ 2. We are ‘peeling’ the matrix a from the
top down.

AI & Zanna: Suppose that we have brought a to an upper Hessenberg form,

a = qãq ⊤, where q ∈ O(n). The above algorithm does not respect this form.
But suppose that we ‘peel’ the matrix from the bottom, not from the top! In
that case, moving to each subsequent subalgebra kj ‘contaminates’ just few
elements under the first subdiagonal, at the bottom of the matrix, which can
be ‘cleaned’ with fre Givens rotations.

This yields an algorithm which is competitive with classical methods to com-

pute the matrix exponential, even when the conservation of Lie-group structure
is not at issue!

59