Optimization with partial differential equations

Manuel Schaller
October 23, 2024

These lecture notes were partly written during the lecture Optimierung mit elliptischen partiellen
Differentialgleichungen held at the summer semester 2023 at the TU Ilmenau. They can not
replace your own notes, as they do not contain any graphical sketches and might miss some details
in particular topics. Please consider the lecture notes for personal use only, i.e., do not distribute
them.
The notes are mainly based on the two textbooks [4, 8] and the (unpublished) lecture notes of
Prof. Anton Schiela (University of Bayreuth). We will make excessive use of central theorems of
functional analysis. For the proofs of these results I refer you, e.g., to the (german) textbook [9].
Particular thanks go to the students of the first edition of this course which significantly im-
proved these lecture notes by valuable comments and to Cinja Arndt, Till Preuster and Sandra
Ritter for correcting many typos.

1 Introduction
In this lecture we consider optimization problems (or more precisely, optimal control problems)
over infinite dimensional function spaces. The constraints of these optimization problems will
partly be given by partial differential equations. Abstractly, these problems may be written as

min f (x) s.t. e(x) ≤ 0.

x∈X

where X is an (infinite-dimensional) Banach space, f : X → R is a cost functional and e : X → P

models the constraints (where we assume that P may be endowed with some ordering).
An important subclass of these problems are given as optimal control problem, where the
optimization variable splits splits into x = (y, u), where u ∈ U is the control variable, and y ∈ Y
is the state variable:

min J(y, u) s.t. c(y, u) = 0, g(y) ≤ 0, h(u) ≤ 0,

(y,u)∈Y ×U

The nonlinear operator c : Y × U → P models the PDE and g : Y → Z and h : U → W encode

state and control constraints, respectively.
There are many applications of PDE-constrained optimization, such as:
• Optimal retinal laser treatment [6]:
– Model: Heat equation with Beer-Lambert law for absorption of laser.
– Control: Laser power.
– Objective: Heat up tissue (effectivity of treatment), but do not overshoot (safety of
treatment).
• Hyperthermia for cancer treatment [2]
– Model: Heat equation.
– Control: Microwave antennas.
– Objective: Heat up tissue to prepare for radiotherapy.

1
 • Design of semiconductors [4, Kapitel 4.1]

– Model:Drift-Diffusion model.
– Control: Doping (dt. Dotierung; Einbringen von Fremdatomen in die Schaltung zur
Optimierung der Materialeigenschaften).
– Objective: Optimize material behavior.
• Shape optimization [7]

– Model: Depends on application.

– Control: Domain of the PDE.
– Objective: Minimize drag of trains, maximize lift of airplanes, maximize robustness of
bridges...

• Implant shape design for facial reconstruction [5]

– Model: static (highly!) nonlinear elasticity models.
– Control: Material properties of implant, shape of implant, position of implant...
– Objective: Maximize aesthetics and durability.

• Fluid dynamics:
– Model: Navier-Stokes-Equations, Oseen-Equations, Stokes-Equations...
– Control: e.g. velocity field at the boundary.
– Objective: e.g. optimal flow of nutrients in biochemical reactors [1].

• Wave propagation in networks [3]

– Coupled Telegrapher-equations, Euler-Equations on Graphs...
– Control: Pressure at compressor stations, voltage at generators.
– Objective: Stable network, meet the need of the customers.

Our main focus in this lecture is on

• Existence of optimal solutions
• First and second order optimality conditions
Main difficulty: (y, u) belongs to an infinite-dimensional function space.

1.1 Elliptic model problem: Poisson equation

In this lecture, we will mainly consider the following optimal control problem, where Ω ⊂ Rd ,
d ∈ {1, 2, 3} is an open domain with boundary Γ:
Z
min
1 2
|y(ω) − yref (ω)|2 + λ|u(ω)|2 dω
(y,u)∈H (Ω)×L (Ω) Ω
−∆y(ω) = u(ω) ∀ω ∈ Ω
y(γ) = 0 ∀γ ∈ Γ
u ≤ u(ω) ≤ u ∀ω ∈ Ω

where λ > 0 is a Tikhonov penalization parameter, u, u ∈ R are control bounds and ∆ is the
Laplace operator, that is,
d
X ∂2
∆y = y(ω).
i=1
∂ωi2

2
1.2 Parabolic model problem: Heat equation
Our model problem will be the heat equation on the time horizon T > 0 and spatial domain
Ω ⊂ Rd , d ∈ N with temperature field y : [0, T ] × Ω → R, i.e., y(t, ω) provides the temperature at
time point t ∈ [0, T ] and space point ω ∈ Ω.

∂
y(t, ω) − ∆y(t, ω) = f (t, ω) on [0, T ] × Ω
∂t
y(t, γ) = 0 on [0, T ] × ∂Ω
0
y(0, ω) = y (ω) on Ω

where y0 : Ω → R is an initial state and f : [0, T ] × Ω → R is a source term. Then, an optimal

control problem could look like
Z T Z
min |y(t, ω) − yref (ω)|2 + λ|u(ω)|2 dω
(y,u) 0 Ω
∂
y(t, ω) − ∆y(t, ω) = u(t, ω) ∀(t, ω) ∈ [0, T ] × Ω
∂t
y(t, γ) = 0 ∀(t, γ) ∈ [0, T ] × Γ
0
y(0, ω) = y (ω) ∀ω ∈ Ω
u ≤ u(t, ω) ≤ u ∀ω ∈ Ω

2 Existence results in unconstrained minimization

In this part, for f : X → R, we consider the problem

min f (x). (1)

x∈X

If not stated otherwise, (X, ∥ · ∥) is a Banach space.

Definition 2.1 (Radial unboundedness). A function f : X → R is called radially unbounded if

∥x∥ → ∞ =⇒ f (x) → ∞.

Example: f (x) = ∥x − 5∥2 .

Definition 2.2 (Sublevel set). Consider a function f : X → R. For α ∈ R, the α-sublevel set is
defined as

Nα (f ) := {x ∈ X | f (x) ≤ α}.

Lemma 2.3. If f : X → R is radially unbounded and continuous, all α-sublevel sets are closed
and bounded.
Proof. Let α ∈ R and w.l.o.g. assume that Nα (f ) is non-empty. Consider a sequence (xn )n∈N ⊂
Nα (f ) and assume that xn is unbounded, i.e., for all K ∈ R there is n ∈ N such that ∥xn ∥ ≥ K.
Due to the radial unboundedness, this implies that f (xn ) is also unbounded which contradics that
f (xn ) ≤ α.
To show closedness, consider now that xn → x. Then, by continuity f (xn ) → f (x) and thus
f (x) ≤ α. This implies that x ∈ Nα (f ) and thus closedness of Nα (f ).

Note that the closedness in the previous lemma can also be deduced by Nα (f ) = f −1 ((−∞, α])
as by continuity, pre-images of closed sets are closed.

3
2.1 What about the finite dimensional case?
Before going to infinite dimensional problems, we revisit the finite dimensional case, that is, we
set X = Rn .
Theorem 2.4 (Weierstrass theorem). Let f : Rn → R be continuous and radially unbounded.
Then there exists a minimizer x∗ ∈ Rn of (1).

Proof. Consider an infimizing sequence (xn )n∈N such that f (xn ) → inf f . Note that inf f always
exists, but it may be −∞. W.l.o.g., there is α ∈ R such that (xn )n∈N ⊂ Nα (f ). By radial
unboundedness and Lemma 2.3, Nα (f ) is closed and bounded, i.e., compact and thus, (xn )n∈N has a
convergent subsequence in Nα (f ), which we again will denote (xn )n∈N such that xn → x∗ ∈ Nα (f ).
Now, we conclude by continuity of f that

inf f ≤ f (x∗ ) = f ( lim xn ) = lim f (xn ) = inf f.

n→∞ n→∞

This means that inf f = f (x∗ ) and thus, x∗ is indeed a minimizer of (1).
Problem in infinite dimensions: closed and bounded sets are not necessarily compact. Exam-
ple: fn (x) = xn in C([0, 1]). Here, fn (x) converges pointwise to the discontinuous function
(
1 x=1
f (x) = .
0 else

k→∞
Assume now that there is a subsequence (fnk )k∈N ⊂ C([0, 1]) such that ∥fnk − g∥C([0,1]) → 0.
Then, as limits of uniformly convergent sequences of continuous functions are necessarily contin-
uous, g ∈ C([0, 1]). This however contradicts the pointwise convergence to a discontinuous function.

But: bounded sequences in reflexive spaces admit weakly convergent subsequences.

2.2 Functional analytic toolbox

Definition 2.5 (Linear mappings, dual space and reflexivity).
1. A linear mapping l : X → R is called a linear functional on X.
|l(x)|
2. The associated norm for such linear mappings will be denoted by ∥l∥ := supx̸=0 ∥x∥ .

3. The space X ∗ = L(X, R) = {l : X → R | ∥l∥ < ∞} is called the dual space of X (all
continuous linear functionals on X). This space is also a Banach space.
4. If X ∗∗ ∼
= X, we call X reflexive.
Example 2.6. Integral operator on L1 (Ω): y 7→ Ω y(ω) dω, or differentiation operator on C 1 (Ω):
R

y 7→ y ′ . Note: L1 (Ω) and C 1 (Ω) are not reflexive.

Reflexive spaces: For 1 < p < ∞: Lp (Ω) and W m,p (Ω), all Hilbert spaces.
Not reflexive: C k (Ω), W m,∞ (Ω), L1 (Ω) and L∞ (Ω).
Having defined the dual space, we introduce the important notion of weak convergence.
Definition 2.7 (Weak convergence). A sequence (xn )n∈N ⊂ X is called weakly convergent to
x ∈ X, written xn ⇀ x if
k→∞
l(xk ) → l(x) ∀l ∈ X ∗

We call a set M ⊂ X weak sequentially closed if xn ⇀ x implies x ∈ M . Further, we call M ⊂ X

weakly sequentially compact, if any sequence (xn )n∈N ⊂ M has a weakly convergent subsequence
with weak limit in M .

4
 We have the following important results from functional analysis.
Theorem 2.8. The following hold:
1. Any convex and closed set in a Banach space is weakly sequentially closed.
2. Any bounded and weakly sequentially closed set in a reflexive Banach space is weakly sequen-
tially compact.
Proof. See [8, Satz 2.10 und Satz 2.11].
Definition 2.9 (Notions of weak sequential continuity). A mapping F : X → Y , X, Y Banach
spaces is called weakly sequential continuous (in the following denoted by weakly continuous), if
xn ⇀ x ⇒ F (xn ) ⇀ F (x).
A functional f : X → R is called weakly lower semi-continuous, if
xn ⇀ x ⇒ lim inf f (xn ) ≥ f (x).
n→∞

Definition 2.10 (Convex function). A functional f : X → R is called convex, if

f (λx + (1 − λ)v) ≤ λf (x) + (1 − λ)f (v)
for all λ ∈ (0, 1) and x, v ∈ X. We call f strictly convex, if the above inequality holds strictly (<
instead of ≤) for all x ̸= v.
Lemma 2.11. The following hold:
1. Any linear continuous operator is weakly continuous.
2. Any convex and continuous functional is weakly lower semi-continuous.
Proof. For the first claim see Exercises, for the second claim see [8, Satz 2.12].
We will revisit the result of Theorem 2.4 using the machinery introduced in the previous
subsections.

2.3 Existence of solutions: Infinite-dimensional case

Lemma 2.12. Sublevel sets of convex functions are convex.
Proof. Exercises.
Note that the converse statement of Lemma 2.12 is not true.
Lemma 2.13. Let X be a reflexive Banach space and let f : X → R be continuous, convex and
radially unbounded. Then the sublevel sets Nα (f ) are weakly sequentially compact.
Proof. As f is convex and by Lemma 2.12, Nα (f ) is convex, as well. Together with the closedness
shown in Lemma 2.3, in view of Theorem 2.8 1., we obtain the weak sequential closedness. Thus,
together with reflexivity of X and boundedness of the sublevel sets as shown in Lemma 2.3 by
means of Theorem 2.8 2. we conclude weak sequential compactness.
Theorem 2.14 (Existence of solutions). Let X be a reflexive Banach space and let f : X → R be
continuous, convex and radially unbounded. Then there exists a minimizer x∗ ∈ X of (1).
Proof. Consider an infimizing sequence (xn )n∈N such that f (xn ) → inf f . Note that inf f always
exists, but it may be −∞. Thus, there is α ∈ R such that (xn )n∈N ⊂ Nα (f ). From Lemma 2.13, we
conclude that Nα (f ) is weakly sequentially compact and thus, (xn )n∈N has a weakly convergent
subsequence in Nα (f ), which we will denote by (xn )n∈N such that xn ⇀ x∗ ∈ Nα (f ). As f is
convex and continuous, it is weakly lower semi-continuous due to Lemma 2.11 2., and hence
inf f ≤ f (x∗ ) ≤ lim inf f (xn ) = inf f
n→∞

which concludes the proof.

5
 Last, we briefly inspect the case when f (x) is strictly convex, yielding uniqueness of the mini-
mizer.
Theorem 2.15 (Existence of unique solutions). Let X be a reflexive Banach space and let f :
X → R be continuous, strictly convex and radially unbounded. Then there exists a unique minimizer
x∗ ∈ X of (1).
Proof. In view of Theorem 2.14, we only have to proof the uniqueness using strict convexity.
Assume that there is another minimizer y ∗ ∈ X with y ∗ ̸= x∗ . As x∗ and y ∗ are minimizers,
f (x∗ ) = f (y ∗ ) ≤ f (x) for all x ∈ X. However, then

f ((x∗ + y ∗ )/2) < f (x∗ )/2 + f (y ∗ )/2 = f (x∗ ), (2)

which yields a contradiction.

2.4 Optimality conditions for unconstrained problems

Here, we derive optimality conditions that have to hold at minimizers. As in the finite dimensional
case, these conditions will be stated by means of derivatives.
In the following definitions X and Y will be Banach spaces.
Definition 2.16 (Directional derivative). Let F : X → Y be a mapping and x0 , δx ∈ X. If the
one-sided limit
F (x0 + tδx) − F (x0 )
F ′ (x0 ; δx) := lim
t↓0 t
exists, then F ′ (x0 ; δx) is called the directional derivative of F at x0 in direction δx.
It is clear that the above directional derivative is unique (by uniqueness of limits) and it is
positively homogeneous, i.e., for all λ ≥ 0, F ′ (x0 ; λδx) = λF ′ (x0 , δx).
Note that, already in finite dimensions. the directional derivative is not necessarily linear in δx.
Definition 2.17 (Gâteaux derivative). Let F : X → Y be a mapping and assume the directional
derivatives at x0 in all directions δx ∈ X exists and satisfy

F ′ (x0 ; δx) = Aδx ∀δx ∈ X

for A ∈ L(X, Y ) (linear and bounded operator). Then, F is called Gâteaux differentiable at x0 and
F ′ (x0 ) = A is called the Gâteaux derivative at x0 .
Note that if Y = R, then A ∈ L(X, R) = X ∗ , that is, the Gâteaux derivative is an element of
the dual space.
We briefly consider two examples, cf. [8, p.45].
Example 2.18 (Nonlinear point evaluation). Let X = C(0, 1) := C([0, 1]; R), Y = R and f :
C(0, 1) → R defined by

f (x) = sin(x(1)).

Let us compute the directional derivative at x ∈ C(0, 1) in direction δx ∈ C(0, 1):

f (x + tδx) − f (x) 1
f ′ (x; δx) = lim = lim (sin(x(1) + tδx(1)) − sin(x(1)))
t↓0 t t↓0 t
∗ d
= sin(x(1) + tδx(1))|t=0
dt
= cos(x(1) + tδx(1))δx(1)|t=0 = cos(x(1))δx(1).

where in ∗ we used that sin : R → R is continuously differentiable. Clearly, the expression on the
right-hand side is linear, such that we have that the Gâteaux derivative is given by

f ′ (x)δx = f ′ (x; δx) = cos(x(1))δx(1).

6
 Here, we observe a crucial difference to the finite-dimensional case: One usually has to include
a direction into the definition of the derivative, i.e., we can not define above Gâteaux derivative
without including δx due to the point evaluation on the right-hand side. In finite dimensions, one
simply passes to the matrix representation of the linear mapping by inserting all basis vectors as
directions. However, in the infinite dimensional case and if Y = R, F ′ (x) ∈ X ∗ and we have to
specify the action of this linear mapping on elements of X explicitly.
Example 2.19 (Derivative of squared norm in Hilbert space). Let (H, ⟨·, ·⟩) be a real Hilbert space
with norm ∥x∥2 := ⟨x, x⟩. We now set X = H, Y = R and defined f : H → R via

f (x) = ∥x∥2 .

For x ∈ X and a direction δx ∈ X, we compute

f (x + tδx) − f (x) 1
f ′ (x; δx) = lim ∥x + tδx∥2 − ∥x∥2 .


= lim
t↓0 t t↓0 t

A simple computation yields

∥x + tδx∥2 = ⟨x + tδx, x + tδx⟩ = ⟨x, x⟩ + 2t⟨x, δx⟩ + t2 ⟨δx, δx⟩

such that
1
f ′ (x; δx) = lim 2t⟨x, δx⟩ + t2 ⟨δx, δx⟩ = 2⟨x, δx⟩.


t↓0 t
Clearly, this functional is again linear such that

f ′ (x)δx = f ′ (x; δx) = 2⟨x, δx⟩.

Note that in view of Riesz representation theorem, there is a isometric isomorphism M : H → H ∗ ,

with M x = ⟨x, ·⟩ called the Riesz map such that we can set M −1 F ′ (x) = 2x.
Besides directional differentiability and Gâteaux differentiablity, there is the stronger notion of
the Fréchet derivative.
Definition 2.20 (Fréchet derivative). A mapping F : X → Y is called Fréchet differentiable at
x0 ∈ X if there is an operator A ∈ L(X, Y ) and r(x0 , · ) : X → Y such that for all δx ∈ X

F (x0 + δx) = F (x0 ) + Aδx + r(x0 , δx)

with
∥r(x0 , δx)∥Y
→0 for ∥δx∥X → 0.
∥δx∥X
In this case, A is called the Fréchet derivative of F at x0 and we will write A = F ′ (x0 ).
Equivalently, we may show Fréchet differentiablity at x0 ∈ X via
∥F (x0 + δx) − F (x0 ) − Aδx∥Y
→0 for ∥δx∥X → 0.
∥δx∥X
Note that we used the same symbol for the Fréchet derivative as for the Gâteaux derivative. Which
derivative we will use will be clear from the context in the following.
We briefly gather some facts of the Fréchet derivative. First, we have that Fréchet differ-
entiablity implies continuity, similar to the finite dimensional case. Second, if F (x) = Ax for
A ∈ L(X, Y ), then clearly

F (x0 + δx) = Ax0 + Aδx + 0 (3)

that is, r(x0 , ·) ≡ 0. Hence, we may state that the Fréchet derivative of a linear operator is the
linear operator itself. Last, it is important to note that there are functions which are Gâteaux
differentiable but not Fréchet differentiable (e.g., functions which are not continuous at particular
points). However, both functions provided in Examples 2.18 and 2.19 are Fréchet differentiable.

7
References
[1] Lino J Alvarez-Vázquez and Francisco J Fernández. Optimal control of a bioreactor. Applied
mathematics and computation, 216(9):2559–2575, 2010.

[2] Peter Deuflhard, Anton Schiela, and Martin Weiser. Mathematical cancer therapy planning in
deep regional hyperthermia. Acta Numerica, 21:307–378, 2012.
[3] Mirko Hahn, Sven Leyffer, and Victor M Zavala. Mixed-integer PDE-constrained optimal
control of gas networks. Mathematics and Computer Science, page 113, 2017.

[4] Michael Hinze, René Pinnau, Michael Ulbrich, and Stefan Ulbrich. Optimization with PDE
constraints, volume 23. Springer Science & Business Media, 2008.
[5] Lars Lubkoll. An Optimal Control Approach to Implant Shape Design : Modeling, Analysis
and Numerics. PhD thesis, University of Bayreuth, 2015.
[6] Manuel Schaller, Mitsuru Wilson, Viktoria Kleyman, Mario Mordmüller, Ralf Brinkmann,
Matthias A Müller, and Karl Worthmann. Parameter estimation and model reduction for
model predictive control in retinal laser treatment. Control Engineering Practice, 128:105320,
2022.
[7] Jan Sokolowski and Jean-Paul Zolésio. Introduction to shape optimization. Springer, 1992.

[8] Fredi Tröltzsch. Optimale Steuerung partieller Differentialgleichungen: Theorie, Verfahren und
Anwendungen. Vieweg+Teubner Verlag Wiesbaden, 2009.
[9] Dirk Werner. Funktionalanalysis. Springer-Verlag, 2006.