Lectures in Functional Analysis

Roman Vershynin
Department of Mathematics, University of Michigan, 530 Church
St., Ann Arbor, MI 48109, U.S.A.
E-mail address: [email protected]

Preface
These notes are for a one-semester graduate course in Functional Analysis,
which is based on measure theory. The notes correspond to the course Real Analysis
II, which the author taught at University of Michigan in the Fall 2010. The course
consists of about 40 lectures 50 minutes each.
The student is assumed to be familiar with measure theory (both Lebesgue and
abstract), have a good command of basic real analysis (epsilon-delta) and abstract
linear algebra (linear spaces and transformations).
The course develops the theory of Banach and Hilbert spaces and bounded
linear operators. Main principles of are covered in depth, which include HahnBanach theorem, open mapping theorem, closed graph theorem, principle of uniform boundedness, and Banach-Alaoglu theorem. Fourier series are developed for
general orthogonal systems in Hilbert spaces. Compact operators and basics of
Fredholm theory are covered.
Spectral theory for bounded operators is studied in the second half of the course.
This includes the spectral theory for compact self-adjoint operators, functional
calculus and basic spectral theory of general (non-compact) operators, although
the latter needs to be expanded a bit.
Topics not covered include: Krein-Milman theorem (although this can be done
with one extra lecture), unbounded linear operators, and Fourier transform. Most
applications to ODE and PDE are not covered, however the integral operators serve
as a main example of operators in this course.
The material has been compiled from several textbooks, including Eidelman,
Milman and Tsolomitis Functional Analysis, Kirillov and Gvishiani Theorems
and problemsin functional analysis, Reed and Simon Methods of modern mathematical physics. I. Functional analysis, V. Kadets A course in functional analysis (Russian), and P. Knyazev, Functional analysis. Minor borrowings are made
from Yoshida Functional analysis, Rudin Functional analysis, and Conway A
course in functional analysis. For some topics not covered, one may try R. Zimmer
Essential results of functional analysis.
Acknowledgement. The author is grateful to his students in the Math 602
course Real Analysis II, Winter 2010, who suggested numerous corrections for these
notes. Special thanks are to Matthew Masarik for his numerous thoughtful remarks,
corrections, and suggestions, which improved the presentation of this material.

Contents
Preface

iv

Chapter 1. Banach and Hilbert spaces

1.1. Linear spaces and linear operators
1.2. Normed spaces
1.3. Banach spaces
1.4. Inner product spaces
1.5. Hilbert spaces
1.6. Fourier series

1
1
7
15
19
25
28

Chapter 2. Bounded linear operators

2.1. Bounded linear functionals
2.2. Representation theorems for linear functionals
2.3. Hahn-Banach theorem
2.4. Bounded linear operators

39
39
42
47
56

Chapter 3. Main principles of functional analysis

3.1. Open mapping theorem
3.2. Closed graph theorem
3.3. Principle of uniform boundedness
3.4. Compact sets in Banach spaces
3.5. Weak topology
3.6. Weak topology. Banach-Alauglus theorem

69
69
74
76
83
87
91

Chapter 4. Compact operators. Elements of spectral theory

4.1. Compact operators
4.2. Fredholm theory
4.3. Spectrum of a bounded linear operator
4.4. Properties of spectrum. Spectrum of compact operators.

94
94
98
101
103

Chapter 5. Self-adjoint operators on Hilbert space

5.1. Spectrum of self-adjoint operators
5.2. Spectral theorem for compact self-adjoint operators
5.3. Positive operators. Continuous functional calculus
5.4. Borel functional calculus. Spectral theorem for self-adjoint operators

110
110
113
116
122

Bibliography

126

Index

127

CHAPTER 1

Banach and Hilbert spaces

1.1. Linear spaces and linear operators
Lec.1: 09/08/10

1.1.1. From individual functions to function spaces. In this section, our

discussion is general and not very precise. For now, our goal is to see a big picture.
? In antiquity, properties of individual numbers were of great importance. Whether
2 is rational, for example, was a source of a great discussion. Later, with the
development of calculus in the XVII century, the focus shifted from numbers to
functions. A function wraps up individual numbers into some assignment rule,
and it is the assignment that now matters rather than individual numbers. Later,
in the second half of XIX century, the interest of mathematicians shifted further
from studying individual functions (their differentiability, integrability) to function
spaces. One may say that a function space wraps up functions of interest into one
geometric object. The geometry of the function space reflects important properties
of functions. This lead to the development of the field of functional analysis.
In functional analysis, we view functions as points or vectors in a function
space. Since we can add functions on a common domain (say ra, bs) by defining
pf gqpxq : f pxq gpxq and multiply them by defining paf qpxq  af pxq, we see
that a function space is a linear vector space.
Furthermore, we can envision a kind of distance on a function space, which
would quantify similarity (or dissimilarity) of functions. The choice of a distance
depends on application. One choice of distance between f and g is the supdistance
}f
g}8 : sup |f pxq
gpxq|.

Pr s

x a,b

This is clearly a metric, so a function space becomes not only a linear vector space
but also a metric space. Such spaces will be called normed spaces later. Another
natural choice of a distance would be the
b

}f
g}1 : |f pxq
gpxq| dx

a
Heuristically, a control of }f
g }8 forces the values of f and g stay close everywhere
on ra, bs, while a control of }f
g }1 forces the values of f and g stay close on
average.

1.1.2. Examples of linear vector spaces. Classical example of linear vector

spaces given in linear algebra courses are Rn and Cn . Here are some examples of
function spaces that are linear vector spaces (check!):
1. F  tall functions R Ru. This space is too large, and is never studied.
2. {all solutions of a linear homogeneous PDE}
3. L1 ra, bs  tall Lebesgue integrable functions on ra, bsu
1

1.1. LINEAR SPACES AND LINEAR OPERATORS

4.
5.
6.
7.
8.
9.

L8 ra, bs  tall bounded almost everywhere functions on ra, bsu

C ra, bs  tall continuous functions on ra, bsu
C 1 ra, bs  tall continuously differentiable functions on ra, bsu
C 8 ra, bs  tall infinitely differentiable functions on ra, bsu
P pxq  tall univariate polynomialsu
Pn pxq  tall univariate polynomials of degree at most nu

Also there are many natural examples of sequence spaces that are linear vector
spaces (check!):
1. s  tall sequences of real numbers pan q8
n1 u. This space is too large, and is
never studied.

2. `1  tall absolutely summable sequences, i.e. satisfying n |an | 8u

3. `8  tall bounded sequences of real numbersu
4. c  tall convergent sequences of real numbersu
5. c0  tall sequences of real numbers converging to zerou
6. c00  tall sequences of real numbers with finite supportu
1.1.3. Subspaces. A (linear) subspace of a linear vector space is a subset that
is closed under the operations of addition of vectors and multiplication by scalars:
Definition 1.1.1. A subset A1 of a linear vector space E is called a (linear)
subspace if x, y P E1 , a, b P R implies ax by P E1 .
Example 1.1.2. One can check that the following set-theoretic inclusions hold:1
Pn pxq P pxq C 8 ra, bs C 1 ra, bs C ra, bs L8 ra, bs L1 ra, bs F,
c00

`1 c0 c `8 s.

Some of these inclusions encode non-trivial theorems of undergraduate analysis.

All these inclusions are subspace inclusions, i.e. Pn pxq is a subspace of P pxq,
etc. (check this!)
Exercise 1.1.3. Let E be a linear vector space. Show that t0u and E
are subspaces of E. Show that the intersection of an arbitrary collection
of subspaces of E is again a subspace of E.
Lec. 2: 09/10/10

1.1.4. Hamel basis. As we know, every finite dimensional linear vector space
E has a basis tx1 , . . . , xn u. A basis is a maximal linearly independent subset of
vectors in E. The number n of basis elements is called the dimension of E; this
number is independent of the choice of the basis. Every vector x P E can be
uniquely expressed as a linear combination of the basis elements:
(1.1)

x

ak xi ,

for some ak

P R.

k 1

The notion of basis can be generalized to arbitrary, possibly infinite dimensional

linear vector spaces E.

pq pq

1For consistency of these inclusions, we restrict the functions in P x , P x and F onto

n

ra, bs.

1.1. LINEAR SPACES AND LINEAR OPERATORS

Definition 1.1.4 (Hamel basis). A subset X of a linear vector space E is called

a Hamel basis of E if every vector x P E can be uniquely expressed as a finite linear
combination of some elements of X :
x

(1.2)

k 1

for some nonzero scalars ak and vectors xk

ak xk

P X.

Exercise 1.1.5. Show that each of the following two statements gives
an equivalent definition of Hamel basis:
(1) A Hamel basis is a maximal linearly independent2 subset X E.
(2) A Hamel basis is a linearly independent subset of E which spans
E. The latter means that the linear span of X , defined as
!

SpanpX q : x 

ak xk : ak

P R, xk P X , n P N

k 1

coincides with E.
Since we have no topology on E, we have to consider finite sums in (1.2). This
requirement is too strong to be put in practice, which makes Hamel bases essentially
impractical (except in theory). We will come across the more practical notion of
Schauder basis later.
Proposition 1.1.6. Every linear vector space E has a Hamel basis.
For finite dimensional spaces E, this result is usually proved in undergraduate
linear algebra using induction. One keeps adding linearly independent elements
to a set until it spans the whole E. This argument can be pushed into infinite
dimensions as well, where the usual induction is replaced by transfinite induction.
The transfinite induction is best done with Zorns lemma (review a Wikipedia article
on Zorns lemma if you are uncomfortable with all notions it uses):
Lemma 1.1.7 (Zorns lemma). A partially ordered set in which every chain has
an upper bound contains a maximal element.
Proof of Proposition 1.1.6. Consider a family F of all linearly independent subsets of E, which is partially ordered by inclusion. We claim that F has a
maximal element; this would obviously complete the proof by Exercise 1.1.5. We
will get a maximal element from Zorns lemma. Let us check its assumption. Consider a chain pX q of elements in F. The elements X are linearly independent
subsets of E totally ordered by inclusion. Therefore, their union Y X is again a
linearly independent subset of E (check!) Hence this union is an element of F, and
it is clearly an upper bound for the chain pX q. The assumption of Zorns lemma
is therefore satisfied, and the proof is complete

As in the finite dimensional case, the cardinality of Hamel basis of E is called
the dimension of E; one can show that the dimension is independent of the choice
of a Hamel basis.
Example 1.1.8. Here we consider some of the examples of linear vector spaces
given in Section 1.1.2.
2Linear independence means that every finite subset of X is linearly independent in the

ordinary sense. In other words, if n

0 for some ai R, xk X , n N, then all ak 0.
k1 ak xk

1.1. LINEAR SPACES AND LINEAR OPERATORS

1.
2.
3.
4.

dimpRn q  n, dimpCn q  n.
dimpPn pxqq  n 1, the monomials t1, x, x2 , . . . , xn u form a basis.
dimpP pxqq  8, the monomials t1, x, x2 , . . .u form a Hamel basis.
dimpc00 q  8, the coordinate vectors p0, . . . , 0, 1, 0, . . .q form a Hamel basis.

Remark 1.1.9. Unfortunately, the notion of Hamel basis is too strong. Except
in spaces P pxq and c00 (which are isomorphic - why?) no explicit constructions are
known in any other infinite dimensional vector space. It would be great to have
a construction of a Hamel basis in C r0, 1s, for example. However, Hamel bases
usually have to be uncountable; see a later exercise.
1.1.5. Quotient spaces. The notion of quotient space allows one easily to
collapse some directions in linear vector spaces. One reason for doing this is when
one has unimportant directions and would likes to neglect them; see the construction
of L1 below.
Definition 1.1.10 (Quotient space). Let E1 be a subspace of a linear vector
space E. Consider an equivalence relation on E defined as
xy

if

x
y

P E1 .

The quotient space E {E1 is then defined as the set of equivalence classes (cosets)
rxs for all x P E.
The quotient space is a linear space, with operations defined as

rxs rys : rx

y s,

arxs : raxs

for x, y

P E, a P R.

The dimension of the quotient space is called the codimension of E1 , thus

codimpE1 q : dimpE {E1 q.

Exercise 1.1.11. Prove that the operations above are well defined,
and that quotient space is indeed a linear space.
Remark 1.1.12. 1. Observe that rxs is an affine subspace:

rxs  x

E1 : tx

h : h P E1 u.

2. The definition of the equivalence relation x  y is meant to ignore the

directions in E1 , and thus to identify points x, y if they only differ by a vector from
E1 .
3. From undergraduate linear algebra we know that if E is finite dimensional
then all of its subspaces E1 satisfy
dimpE1 q

codimpE1 q  dimpE q.

Example 1.1.13 (Space L1 ). The notion of quotient space comes handy when
we define the space of integrable functions L1  L1 p, , q where p, , q is an
arbitrary measure space. We first consider
E : tall integrable functions f on p, , qu.

To identify functions that are equal -almost everywhere, we consider the subspace
we would like to neglect:
E1 : tall functions f

Then we define
L1

 0 -almost everywhereu

 L1 p, , q : E {E1 .

1.1. LINEAR SPACES AND LINEAR OPERATORS

This way, the elements of L1 are, strictly speaking, not functions but classes of
equivalences.3 But in practice, one thinks of an f P L1 as a function, keeping in
mind that functions that coincide -almost everywhere are the same.
Example 1.1.14 (Space L8 ). A similar procedure is used to define the space
of essentially bounded functions L8  L8 p, , q. A real valued function f on
is called essentially bounded if there exists a bounded function g on such that
f  g -almost everywhere. Similar to the previous example, we consider the linear
vector space
E : tall essentially bounded functions f on p, , qu
and the subspace we would like to neglect:
E1 : tall functions f
Then we define
L8

 0 -almost everywhereu

 L8 p, , q : E {E1 .

Example 1.1.15. As we know, the space c0 of sequences converging to zero is

a subspace of the space c of all convergent sequences. Let us observe that
codimpc0 q  1.

Indeed, every sequence x P c can be uniquely represented as

x  a1

for some a P R, z

P c0

where 1  p1, 1, . . .q. (How do we choose the value of a?). Hence

rxs  ar1s rzs  ar1s.

It follows that every element rxs P c{c0 is a constant multiple of the element r1s.
Therefore, dimpc{c0 q  1 as claimed.

This example shows that the space c0 makes up almost the whole space c,
except for one dimension given by the constant sequences. This explains why the
space c is rarely used in practice; one prefers to work with c0 which is almost the
same as c but has the advantage that we know the limits of all sequences there
(zero).
1.1.6. Linear operators. This is a quick review of the classical linear algebra
concept.
Definition 1.1.16 (Linear operator). A map T : E F between two linear
vector spaces E and F is called a linear operator if it preserves the operations of
addition of vectors and multiplication by scalars, i.e.
T pax

by q  aT pxq

bT py q

for all x, y

The kernel and image of T is defined respectively as

kerpT q  tx P E : T x  0u;

P E, a, b P R.

ImpT q  tT x : x P E u.

3Even more strictly speaking, the representative functions f in L may take infinite values,
1
too. However, every integrable function is finite a.e. So every such function is equivalent to a
function that is finite everywhere.
4One usually writes T x instead of T x

pq

1.1. LINEAR SPACES AND LINEAR OPERATORS

Example 1.1.17 (Differential operator). The simplest example of a differential

operator is given by taking the derivative of a function:
T pf q  f 1 .

Such operator is well be defined e.g. on the space of polynomials T : P pxq P pxq.
But usually one prefers to have a differential operator on a larger space; for example
T : C 1 r0, 1s C r0, 1s is also well defined.
Example 1.1.18 (Embedding and quotient map). Given a subspace E1 of a
linear vector space E, there are two canonical linear operators associated with it:
1. Embedding : E1 E, which acts as an identity pxq  x;
2. Quotient map q : E E {E1 , which acts as q pxq  rxs.
Example 1.1.19 (Shifts on sequence spaces). On any sequence space such as
c00 , c0 , c, `8 , `1 , one can define the right and left shift operators respectively as
Rpxq  p0, x1 , x2 , . . .q;

Lpxq  px2 , x3 , . . .q

for x  px1 , x2 , . . .q.

Exercise 1.1.20. Compute the images and kernels of the embedding,

quotient map, and the shift operators in the examples above.

1.1.7. Additional Exercises.

Exercise 1.1.21. Show that the intersection of an arbitrary collection
of subspaces of a linear vector space E is again a subspace of E.
Exercise 1.1.22. Show that every linearly independent subset of a
linear vector space E can be extended to a Hamel basis of E.
Exercise 1.1.23. [Complementary subspaces] Let E1 be a subspace of
a linear vector space E. Prove that there exists a subspace E2 of E such
that
E1 X E2  t0u, SpanpE1 Y E2 q  E.
(Hint: extend a Hamel basis from E1 onto E; use the extension to construct E2 ). Such subspaces E1 , E2 are called complementary to each
other. Show that E1 , E2 are complentary if and only if every vector
x P E can be uniquely represented as the sum
x  x1

x2

for some x1

P E1 , x2 P E2 .

Exercise 1.1.24. [Injectivization] This is a linear version of the fundamental theorem on homomorphisms for groups. Consider a linear
operator T : E F acting between linear spaces E and F . The operator
T may not be injective; we would like to make it into an injective operator. To this end, we consider the map T : X { ker T Y which sends
every coset rxs into a vector T x, i.e. Trxs  T x.
(i) Prove that T is well defined, i.e. rx1 s  rx2 s implies T x1  T x2 .
(ii) Check that T is a linear and injective operator.
(iii) Check that T is surjective then T is also surjective, and thus T is a
linear isomorphism between X { ker T and Y .

1.2. NORMED SPACES

(iv) Show that T  T  q, where q : X X { ker T is the quotient map. In

other words, every linear operator can be represented as a composition of a surjective and injective operator.

1.2. Normed spaces

Lec. 3: 09/13/10

1.2.1. Definition and examples. A norm is a general concept of length of

vectors. Once we have a norm we can geometrize analysis in some sense, because
we would have a metric on our linear vector spaces. For example, this would allow
us to study functions through geometry of function spaces.
A norm is an assignment of a non-negative number }x} to every vector x in a
linear vector space E. In order to have a meaning of length, this assignment must
satisfy some natural axioms:
Definition 1.2.1 (Normed space). Let E be a linear vector space. A norm on
E is a function }  } : E R which satisfies the following axioms:
(i) }x} 0 for all x P E; }x}  0 if and only if x  0;
(ii) }ax}  |a|}x} for all x P E, a P R (or C);
(iii) }x y } }x} }y } for all x, y P E.

The linear vector space E equipped with the norm

and denoted X  pE, }  }q.

}  } is called a normed space,

Axiom (iii) is called triangle inequality for the following reason. Given an
arbitraty triangle in E with vertices x, y, z P E, its lengths satisfy the inequality
(1.3)

}x
z} }x
y} }y
z},

which follows from norm axiom (iii). For the usual Euclidean length on the plane,
this is the ordinary triangle inequality.
The normed space is naturally a metric space, with the metric defined by
dpx, y q : }x
y }.
The norm axioms, and in particular triangle inequality (1.3), show that this is
indeed a metric (check!)
Exercise 1.2.2. [Normed spaces `8 , c, c0 , `1 , C pK q, L1 , L8 ] Many of
linear vector spaces introduced in Section 1.1.2 and Example 1.1.12 are
in fact normed spaces. Check the norm axioms for them:
1. The space of bounded sequences `8 is a normed space, with the norm
defined as
(1.4)

}x}8 : sup |xi |.

i

2. The spaces c and c0 are normed spaces, with the same sup-norm as in
(1.4).
3. The space of summable sequences `1 is a normed space, with the norm
defined as

}x}1 :

i 1

|xi |.

1.2. NORMED SPACES

4. The space C pK q of continuous functions on a compact topological

space K is a normed spaces with the norm5

5.

}f }8 : max
|f pxq|.
K
The space L1  L1 p, , q is a normed space, with the norm defined
as6

}f }1 :

|f pxq| d.

Note that `1 is a partial case of the space L1 p, , q where  N and

is the counting measure on N.
6. The space L8  L8 p, , q is a normed space, with the norm defined
as the essential supremum:

}f }8 : esssup |f ptq| : ginf

sup |g ptq|.
f a.e. tP
tP
Here the infimum is taken over all g P L8 that are equal to f -almost
everywhere. Note that `8 is a partial case of the space L8 p, , q
where  N and is the counting measure on N.
Exercise 1.2.3. [Essential supremum] Show that the norm in L8 p, , q

can be equivalently computed as

}f }8  pinf
Aq0

sup |f ptq|

P z

t A

where the infimum is over all subsets A of measure zero.

Exercise 1.2.4. [Continuity of norms] Prove that the norm assignment
x }x} is a continuous function on the normed space. Specifically, show
that if }xn
x} 0 then }xn } }x}.
1.2.2. Convexity of norms and balls. The geometry of a normed space
can be very different from that of the usual Euclidean geometry. The balls do not
need to be round anymore. For example, the ball of `8 looks like a cube (why?)
Nevertheless, one important property still holds: the balls are always convex sets,
and the norm is a convex function. The convexity considerations are very helpful
when one works in normed spaces.
Let us first recall some notions coming from geometry of metric spaces.
Definition 1.2.5 (Balls, spheres of normed spaces). Let X be a normed space.
A (closed) ball centered at a point x0 P X and with radius r 0 is defined as
BX px0 , rq : tx P X : }x
x0 } ru.

The (closed) unit ball of X is defined as

BX : BX p0, 1q  tx P X : }x} 1u.

The unit sphere of X is the boundary of the unit ball, that is

SX : tx P X : }x}  1u.

5The maximum is attained because K is compact.

6Formally, as we know the elements of L are cosets f
1

rs

rather than functions. We should

define the norm f 1 :

f x d where f is an arbitrary element in the coset f . Check
that this way, the norm is well defined. The same concerns the definition of L8 below.

}r s} 

| p q|

rs

1.2. NORMED SPACES

Definition 1.2.6 (Convex functions and sets). Let E be a linear vector space.
A function f : E R is convex if for all x, y P E, P r0, 1s one has
f px

p1
qyq f pxq p1
qf pyq.

A set K E is convex if for all x, y P K, P r0, 1s one has
x p1
qy P K.

A geometric meaning of convexity is the following. A function f is convex on

E if its graph restricted to any interval rx, y s E lies below the interval joining
the points px, f pxqq and py, f py qq; see the picture. A set K is convex if, together
with any two points x, y P K, it contains the interval rx, y s.

Figure 1.1. Convex function f on a linear vector space E

Proposition 1.2.7 (Norm axioms imply convexity). Let X be a normed space.
Then:
1. The function x }x} is convex on X.
2. The unit ball BX is a closed, origin-symmetric7, and convex set in X.
Proof. 1. Convexity of the norm follows from the norm axioms. Indeed, for
every x, y P E, P r0, 1s we have

}x p1
qyq} }x} }p1
qyq}  }x} p1
q}y}.

2. Closedness of BX follows from continuity of the norm (see Exercise 1.2.4).

Origin-symmetry follows from norm axiom (ii) with 
1. Finally, to prove
convexity of BX we choose arbitrary x, y P BX , P r0, 1s and use inequality above
to obtain

}x p1
qy} }x} p1
q}y} p1
q  1.

It follows that x p1
qy P BX as required.

The converse to Proposition 1.2.7 also holds:

Proposition 1.2.8 (Convexity implies triangle inequality). Let x }x} be a
real-valued function defined on a linear vector space E. Assume that this function
satisfies norm axioms (i) and (ii). Then:
7Origin-symmetric means that x

P BX implies
x P BX

1.2. NORMED SPACES

10

1. If the function x }x} is convex, then the triangle inequality is satisfied, and
}  } is a norm on E.
2. If the sublevel set tx P X : }x} 1u is convex, then }  } is a norm on E.

P E, P r0, 1s we have
}x p1
qyq} }x} p1
q}y}.
Triangle inequality follows from this for  1{2.
Proof. 1. Convexity ensures that for every x, y

2. This statement is less trivial, and can not be obtained from the first one
directly. Indeed, while it is true that the sublevel sets of a convex functions are
convex sets, the converse statement may fail (construct an example!)
The assumption states that, for u, v P E we have:

}u p1
qv} 1.
Let x, y P E; we want to show that }x y } }x} }y }. This is equivalent to


y
x



}x} }y} }x} }y}  1.
(1.5)

if }u} 1, }v } 1, P r0, 1s,

then

We obtain this inequality from (1.5) with

u

x
}x} , v

 }yy} ,  }x}}x}}y} .

This completes the proof.


Lec. 4: 09/15/2010

1.2.3. Spaces Lp . Minkowski inequality. We have already come across

the spaces L1 and `1 . They are partial cases of a big family of spaces Lp and `p
which we will study now.
Consider a measure space p, , q and an exponent p P r1, 8q. We define
the space of p-integrable functions Lp  Lp p, , q as the set of all measurable
functions f : R such that

|f pxq|p d 8.

Proposition 1.2.9. Lp p, , q is a linear vector space for p P r1, 8q.

Proof. The only non-trivial point is that Lp is closed under addition, that is
f, g P Lp implies f g P Lp . We will obtain this by the convexity of the function
z |z |p on R for p 1. The convexity implies the pointwise inequality

| pq


 f t


g ptq| p


|f ptq|

|gptq|p ,

t P .

Integrating both sides of this inequality, we obtain the required claim.

We turn Lp into a normed space by defining

}f }p :

1{p

|f pxq|p d

for f

P Lp p, , q.

Proposition 1.2.10. Lp p, , q is a normed spaces with the norm }f }p for

p P r1, 8q.

1.2. NORMED SPACES

11

Proof. Norm axioms (i) and (ii) are straightforward. Axiom (iii), triangle
inequality, will follow from Proposition 1.2.8. To this end, it suffices to check that
the sublevel set
Bp : tf P Lp : }f }p 1u
is a convex set. To prove this, let us fix f, g P Bp and P r0, 1s. Since the function
z |z |p is convex on R for p 1, we have a pointwise inequality

f t

p q p1
qgptqp |f ptq|p p1
q|gptq|p .

Integrating both sides of this inequality implies


f t

p q p1
qgptqp d |f ptq|p d p1
q |gptq|p d p1
q  1.

We have showed that }f p1
qg }p 1, hence f p1
qg P Bp . Therefore,

the sublevel set Bp is convex. The proof is finished by Proposition 1.2.8.

Writing out the triangle inequality }f

obtain the classical Minkowski inequality:

g }p

}f }p }g}p in analytic form, we

Theorem 1.2.11 (Minkowski inequality in Lp ). Let p P r1, 8q. Then, for every
two functions f, g P Lp p, , q one has

1{p

|f ptq

g ptq|p d

|f ptq|p d

1{p

|gptq|p d

1{p

.


1.2.4. Spaces `p and `np . An important partial case of the space Lp p, , q

is the space `p obtained by choosing  N and to be the counting measure on
N. Equivalently, for p P r1, 8q, the space of p-summable sequences `p is defined to
consist of sequences x  pxi q8
i1 for which

|xi |p 8.

i 1

We turn `p into a normed space with the norm

}x}p :

8




|xi |p

1{p

i 1

Writing down Minkowski inequality for this specific measure space, we obtain:
Theorem 1.2.12 (Minkowski inequality in `p ). Let p P r1, 8q. Then, for every
two sequences x, y P `p one has

8




|xi

yi |p

1{p

8

i 1

|xi |p

8

1{p

i 1

|yi |p

1{p

i 1

A remarkable family of finite-dimensional spaces Lp p, , q is formed by considering to be a finite set, say  t1, . . . , nu and to be the counting measure on
. The resulting space is called `np . The functions in `np can be obviously identified
with vectors in Rn . Thus `np  pRn , }  }p q with the norm

}x}p :

n

i 1

|xi |p

1{p

1.2. NORMED SPACES

12

When p  2, this space is the usual Euclidean space Rn . However, for p  2,

the geometry of `np is quite different from Euclidean. Indeed, in two-dimensional
spaces, the unit ball of `21 is a diamond with vertices p1, 0q, p0, 1q, p
1, 0q, p0,
1q.
The unit ball of `28 is the square with vertices p1, 1q, p1,
1q, p
1, 1q, p
1,
1q.
Exercise 1.2.13. [`8 as the limit of `p ] This exercise explains the index

8 in the name of the spaces `8 , L8 .

1. Show that if x P `p for some p0 1 then
}x}p }x}8 as p0 p 8.
2. Consider the space L8  L8 p, , q with finite
Show that if f P L8 then
}f }p }f }8 as p 8.
0

total measure pq.

1.2.5. Subspaces of normed spaces.

Definition 1.2.14 (Subspace). Let X be a normed space. A subspace Y of X
is a linear subspace equipped with the norm induced from that of X.
This concept should be familiar from topology, where a subspace is a subset of
a topological space with the induced topology.
Example 1.2.15. 1. The space of polynomials P pxq is a dense subspace of
C r0, 1s. This is the statement of Weierstrass approximation theorem.
2. The set of all continuous functions C r0, 1s forms a dense subspace of L1 r0, 1s. (of
course, both spaces are considered in the L1 norm!) This follows from a theorem
in measure theory that states that an integrable function can be approximated
by a continuous function (why?)
Exercise 1.2.16. 1. Show that the set of convergent sequences c and
the set of sequences converging to zero c0 are closed subspaces of `8 .
2. For all p P r1, 8q, show that the set of p-summable sequences `p is a
closed subspace of `8 but is a dense subspace of c0 .
1.2.6. Quotient spaces of normed spaces. In Section 1.1.5, we defined
quotient spaces of linear vector spaces. If the ambient space is a normed space,
then we can also induce the norm onto the quotient space as follows.
Definition 1.2.17 (Quotient space of a normed space). Let X be a normed
space and Y be a closed subspace of X. We define a norm on X {Y as follows. For
every coset rxs  x Y , we put

}rxs} : yinf
}x
PY

y }.

It is easy to understand the norm in the quotient space geometrically as the

distance from the origin to the coset rxs. Indeed, let distpb, Aq denote the distance
in X from a point b to a set A:
distpb, Aq  inf }b
a}.

a A

Then clearly
(1.6)

}rxs}  distp0, rxsq.

1.2. NORMED SPACES

Proposition 1.2.18. The definition of

X {Y .

13

}rxs} above indeed produces a norm on

Proof. First we observe that the number }rxs} is well defined, i.e. it does not
depend on a choice of a representative x in the coset rxs. This clearly follows from
the geometric definition (1.6).
Next, we have to check the three norm axioms.
(i) Assume that }rxs}  0. Then, from the geometric definition (1.6) we see
that 0 is a limit point of rxs. Since Y is closed, so is rxs  x Y . Therefore 0 P rxs.
Hence rxs  r0s, which verifies norm axiom (i).
(ii) Let x P X and P R. Then

}rxs}  yinf
}x
PY

y }  inf }x

y }   inf }x

y Y

y Y

y }  }rxs}.

This verifies norm axiom (ii).

(iii) Let us fix x1 , x2 P X; we want to show that }rx1 x2 s} }rx1 s} }rx2 s}.
To this end, fix an arbitrary 0. By the definition of the quotient norm there
exist y1 , y2 P Y so that

}x1

y1 }
}rx1 s} }x1

y1 },

}x2

y2 }
}rx2 s} }x2

y2 }.

Using triangle inequality for the norm in X, we obtain

}x1

x2

y2 } }x1

y1

y1 }

} x2

y2 } }rx1 s}

}rx2 s}

2.

We conclude that

}rx1

x2 s}  inf }x1

y } }x1

x2

y Y

x2

y1

y2 } }rx1 s}

}rx2 s}

Since 0 is arbitrary, this completes the proof of norm axiom (iii).

2.


Exercise 1.2.19. Consider the subspace Y of C pK q that consists of

constant functions. Derive the following formula for the norm in the
quotient space C pK q{Y :

}rf s}  21

max f ptq
min f ptq

t K

t K

for f

P C pK q.

Exercise 1.2.20. Derive the following formula for the norm in the
quotient space `8 {c0 :

}ras}  lim sup |ai |

for a  pai q8
i1

P `8 .

1.2.7. Additional Exercises.

Exercise 1.2.21. [Direct sum of two normed spaces] Let X and Y be
two normed spaces. Consider their direct (Cartesian) product
X
Show that X

`1 Y

`1 Y  tpx, yq : x P X, y P Y u.

is a normed space, with the norm defined as

}px, yq} : }x} }y}.

1.3. BANACH SPACES

14

Exercise 1.2.22. [Minkowski functional] Consider a closed, convex,

origin-symmetric set K in Rn with nonempty interior. Minkowski functional of K is the function defined on Rn by

}x}K : inf t 0
Show that
is K.

}  }K

: x{t P K .

is a norm on Rn , and the unit ball of this normed space

Exercise 1.2.23. [Seminorms] A seminorm on a linear vector space

E is a function }  } : E R which satisfies all norm axioms except the
second part of axiom (i). That is, there may exist nonzero vectors x for
which }x}  0.
Show that one can convert a seminorm into a norm by factoring out
the zero directions. Mathematically, show that kerppq : tx P E : }x}  0u
is a linear subspace of E. Show that the quotient space E { kerppq is a
normed space, with the norm defined as

}rxs} : }x}

for x P E.

Illustrate this procedure by constructing the normed space L8 from

the semi-normed space of all essentially bounded functions, with the
essential sup-norm.
Exercise 1.2.24. [Convex hull] The convex hull of a subset A of a linear
vector space X is defined to be the smallest convex set that contains A.
The convex hull of A is denoted convpAq. (See the picture for an example).

Figure 1.2. The shaded region is the convex hull convpx1 , . . . , x5 q

A convex combination of vectors x1 , . . . , xn in a linear vector space X
is any vector of the form
x

k xk

k 1

where k 0 are some numbers such that k1 k  1. Prove that convpAq
coincides with the set of all convex combinations of a finite number of
vectors from A.

1.3. BANACH SPACES

15

1.3. Banach spaces

1.3.1. Definition. Completeness of C pK q. It turns out that the concept of
normed space is deficient; many results in analysis can not be obtained just based
on the norm axioms. An additional axiom is needed, which is completeness.
Recall that a metric space X is called complete if every Cauchy sequence in X
converges to a point in X. For example, R is a complete metric space while Q is
not.
Specializing to normed spaces X, recall that a sequence pxi q8
i1 in X is Cauchy
if
}xn
xm } 0 as n, m 8,
i.e. for every 0 there exists N  N pq such that

}xn
xm }

for all n, m N.

Definition 1.3.1 (Banach space). A complete normed space X is called a

Banach space.
Exercise 1.3.2. [Subspaces, quotients of Banach spaces] Let X be a
Banach space and Y be a (linear) subspace of X. Show that:
1. Y is a Banach space if and only if Y is closed.
2. If Y is closed8, then X {Y is a Banach space.
Many classical spaces are Banach spaces.
Theorem 1.3.3. For a compact topological space K, C pK q is a Banach space.
Proof. Most proofs of completeness work out by reducing the problem to the
completeness of R. To this end, consider a Cauchy sequence pfn q in C pK q, that is

}fn
fm }8 0, n, m 8.
Therefore, for every t P K, we have |fn ptq
fm ptq| 0.
(1.7)

In other words, pfn ptqq

is a Cauchy sequence in R for every t. By completeness of R, this sequence has a
limit which we call f ptq. We have constructed a function f ptq such that fn f
pointwise.
We now claim that fn f uniformly, i.e. }fn
f }8 0. This would
complete the proof, since the limit of a uniformly convergent sequence of continuous
functions on the compact space K is a continuous function (as we know from the
undergraduate analysis). Let us prove our claim then. By (1.7), for every there
exists N  N pq such that

|fn ptq
fm ptq|

for all n, m N, t P K.

Letting m 8 in this inequality (while keeping everything else fixed), we conclude

that
|fn ptq
f ptq| for all n, m N, t P K.
This means that }fn
f }8 8, which is what we wanted.

Exercise 1.3.4. [Banach spaces `8 , c0 , L8 ]
1. Show that `8 and L8 are Banach spaces. (Hint: modify the proof of
Theorem 1.3.3.)
2. Show that c0 is a Banach space (Hint: use Exercise 1.2.16.)

8Recall that Y has to be closed in order for X Y to be well defined.

Lec.5: 09/17/10

1.3. BANACH SPACES

16

3. Show that the set of functions

tf P C r0, 1s : f p0q  f p1q  0u

is a Banach space with respect to the sup-norm. (Hint: check that
this is a closed subspace of C r0, 1s.)
4. Show that the set of polynomials P pxq restricted to r0, 1s is not a
Banach space with respect to the sup-norm. (Hint: recall that P pxq
is dense in C r0, 1s by Weierstrass approximation theorem.)
5. Show that c00 is not a Banach space with respect to any }  }p norm,
1 p 8. (Hint: show that c00 is dense in `p for p P r1, 8q, and is not
closed in `8 .)
Lec.6: 09/20/10

1.3.2. Series in Banach spaces. Completeness of Lp . We are going to

give a useful criterion of completeness of normed spaces in terms of convergence of
series rather than sequences. We shall use this criterion to prove the completeness
of the Lp spaces.
Definition 1.3.5. Series Let pxk q be a sequence of vectors in a normed space
X. If the partial sums
sn :
converge to some vector x P X as n
converges in X, and we write

A series

xk

k 1

8,
xk

then we say that the series

xk

 x.

k 1

xk is called absolutely convergent if

}xk } 8.

k 1

Recall that in the scalar case, where X  R or C, absolute convergence of

series implies convergence (but not vice versa). As the following theorem shows,
this happens precisely because of the completeness of R and C.
Theorem 1.3.6 (Completeness criterion). A normed space X is a Banach space
if and only if every absolutely convergent series in X converges in X.
Proof. 1. Necessity. Let X be a Banach space, and consider an absolutely
convergent series, i.e.

(1.8)

}xk } 8.

k 1

We want to prove that the series k xk converges. By completeness of X, it suffices

to show that the partial sums of this series are Cauchy, i.e. that }sn
sm } 0 as
n m 8. To this end, we use triangle inequality and our assumption (1.8) to
obtain
m


}sn
sm }  

k n 1

This completes the proof of necessity.




xk 

k n 1

}xk } 0.

1.3. BANACH SPACES

17

2. Sufficiency. Assume that X in incomplete; we want to construct a divergent

series which is absolutely convergent. By incompleteness, there exists a Cauchy
sequence pvn q in X which diverges.
Every subsequence of pvn q diverges (check!). Therefore, there exists a subsequence pwn q of pvn q which diverges but which is rapidly Cauchy, i.e.

}w2
w1 } 21 , }w3
w3 } 212 , }w4
w4 } 213 , . . .

(Construct it!) It follows that the sequence pxk q defined as
x1 : w2
w1 , x2 : w3
w2 , x3 : w4
w3 , . . .
(1.9)

forms the terms of an absolutely convergent series:

k 1

}xk } 12

1
22

1
23

    1.

Nevertheless, the partial sums

n

xk

 wn 1
w1

k 1

diverge. So we have constructed an absolutely convergent series in X which diverges. This completes the proof.

Exercise 1.3.7. Validate the two missing steps in the proof of Theorem 1.3.6. Let Xn be a normed space.
1. Let pvn q be a Cauchy sequence X which diverges. Prove that every
subsequence of pvn q diverges.
2. Let pvn q be a Cauchy sequence in X. Construct a rapidly Cauchy
subsequence pwn q of pvn q, i.e. one that satisfies (1.9).
Theorem 1.3.8. For every p P r1, 8q, the space Lp
space.

 Lp p, , q is a Banach

Proof. Let functions pfn q in Lp form the terms of an absolutely convergence

series, i.e.

}fk }p : M 8.

k 1

By
the completeness criterion, Theorem 1.3.6, it suffices to show that the series
k fk converges in Lp .
n
Case 1: all fk 0 pointwise. The partial sums k1 fk form a pointwise non8
decreasing sequence of functions. Denote the pointwise limit by k1 fk ; it may
be infinite at some points.
The triangle inequality (which is Minkowskis inequality) implies that the partial sums are bounded:
n




fk 


k 1

which in other words is

n

k 1

}fk }p M,

k 1

fk

d M p .

1.3. BANACH SPACES

18

We
monotone convergence theorem for the sequence of functions
napplyLebesgue
p
f
and
get
k
k 1


n

fk

8

k 1

fk

d.

k 1

The right hand

side exists and is finite since the left hand side is bounded by M p
8
for all n. Thus k1 fk P Lp .
8
It remains to check that the series k1 fk P Lp converges not only pointwise
but also in Lp . By the pointwise convergence and positivity, the tails of this series
rn :

fk

k n 1

form a pointwise non-increasing sequence of positive functions in Lp that pointwise

converge to 0. Applying Lebesgue monotone convergence theorem
8 for the sequence
rnp , we conclude that }rn }p 0. We conclude that the series k1 fk converges in
Lp as required.
Case 1: arbitraty fk . As is usual in the theory of Lebesgue integration, the claim
for arbitrary (not necessarily non-negative) functions fk will follow by decomposing
them into positive and negative parts:
fk : maxpfk , 0q,

Then fk  fk
fk
. Since 0 fk

8

fk
: maxp
fk , 0q

pointwise.

|fk |, we have

}fk }p

k 1

k 1

}fk }p 8.
8

So, by the first part of the argument, the series k1 fk converges in Lp . Similarly
8
8
8
8
we show that k1 fk
converges in Lp . Therefore, k1 fk  k1 fk
k1 fk

converges in Lp . This completes the proof.

Lec.7: 09/22/10

1.3.3. Completion. From a course in point-set topology we know that every

metric space can be turned into a complete metric space. This procedure is called
completion. We will illustrate it in the context of normed spaces.
Theorem 1.3.9 (Completion). Let X be a normed space. There exists a Bap called the completion of X, with the following properties. One can
nach space X,
p such that:
find a linear map : X X
p
(i) }x}  }x} for all x P X (i.e. is an isometric embedding of X into X;
p
(ii) Impq is dense in X.
The completion of X is unique up to an isometry.9

Exercise 1.3.10. Prove Theorem 1.3.9. Mimic the construction of R

from Q, or the more general construction of completion of metric spaces.
Namely, consider the space Y of all Cauchy sequences pxk q in X, equipped
with the semi-norm
}pxk q} : lim }xk }.
k

9Precisely, this means that for any other completion X

p 1 , there exists an injective and bijective
p Xp 1 such that T x
linear map T : X
x for all x X.

} }} }

1.4. INNER PRODUCT SPACES

19

p by taking quotient over the

Then turn this space into a normed space X
kernel of the semi-norm (see Exercise 1.2.23).

The concept of completion suggests an alternative construction of the space

Lp ra, bs, p P r1, 8q. From measure theory we know that the set of continuous
functions is a dense subset of Lp ra, bs (why?). Furthermore, Lp ra, bs is a complete normed space. Therefore (by the uniqueness of completion), Lp ra, bs is the
completion of the space C ra, bs in the }  }p norm.
This is an alternative definition of Lp ra, bs. This gives also an alternative construction of Lebesgue integral. Indeed, on for continuous functions, Riemann and
Lebesgue integrals coincide. Therefore, the space C ra, bs with }  }1 can be constructed using only Riemann integral. But its completion gives rise to Lebesgue
integral.
1.3.4. Additional Exercises.
Exercise 1.3.11. [Space of continuous periodic functions] Show that
the set of functions
tf P C r0, 1s : f p0q  f p1qu
is a Banach space with respect to the sup-norm. (Hint: identify this
space with C pTq where T is a one-dimensional torus.)
Exercise 1.3.12. [Space of continuously differentiable functions] Show
that the space C k r0, 1s of k-times differentiable functions is not a Banach
space with respect to the sup-norm.
Show that C k r0, 1s is a Banach space with respect to the norm

}f }C  }f }8 }f 1 }8    }f pkq }8 .
k

Exercise 1.3.13. [Completeness of a direct sum] Let X and Y be Banach spaces. Show that the direct sum X `1 Y defined in Exercise 1.2.21
is a Banach space.
1.4. Inner product spaces
1.4.1. Definition. Cauchy-Schwarz inequality. Hilbert spaces form an
important and simplest class of Banach spaces. Speaking imprecisely, Hilbert spaces
are those Banach spaces where the concept of orthogonality of vectors is defined.
Hilbert spaces will arise as complete inner product spaces.
Definition 1.4.1 (Inner product space). Let E be a linear space over C. An
inner product on E is a function x, y : E  E C which satisfies the following
three axioms:
(i) xx, xy 0 for all x P E; xx, xy  0 if and only if x  0;
(ii) xax by, z y  axx, z y bxy, z y for all x, y, z P E and a, b P C;
(iii) xx, y y  xy, xy for all x, y P E.
The space E with an inner product is called an inner product space.
Inner products over R are defined similarly, except there is no conjugation in
axiom (iii).

1.4. INNER PRODUCT SPACES

20

Remark 1.4.2. The inner product is (congjugate) linear in the second argument:
xx, ay bzy  axx, yy bxx, zy.
This follows from axioms (ii) and (iii) of the inner product.
Definition 1.4.3 (Orthogonality). If xx, y y
are orthogonal and write x K y.

 0 we say that vectors x and y

Example 1.4.4. The canonical example of a finite-dimensional Hilbert space

is the Euclidean space Cn equipped with the inner product

xx, yy 

xk yk .

k 1

Theorem 1.4.5 (Cauchy-Schwarz inequality). Let X be an inner product space.

Then every two vectors x, y P X satisfy

|xx, yy| xx, xy1{2 xy, yy1{2 .

xx, yy P R. For all t P R, by the axioms of the inner product

Proof. Case 1:
we have:
0 xx ty, x ty y  t2 xy, y y 2txx, y y xx, xy.
A quadratic polynomial that is everywhere non-negatve must have a non-positive
discriminant, i.e.
xx, yy2
xx, xyxy, yy 0.
This is precisely Cauchy-Schwarz inequality.
Case 2: xx, y y P C arbitrary. We will multiply y by a unit scalar so that xx, y y
becomes a real number, and use the first part. Indeed, polar decomposition

xx, yy  |xx, yy|ei Argxx,yy

implies that

|xx, yy|  xx, y1 y

where y 1  ei Argxx,yy y.
Now using the first part of the proof we conclude that

|xx, yy|  xx, y1 y xx, xyxy1 , y1 y xx, xyxy, yy

as required.

Corollary 1.4.6 (Norm in an inner product space). Let X be an inner product

space. Then X is a normed space, with the norm defined as

}x} : xx, xy1{2 .

Proof. Of the three norm axioms, only the triangle inequality is non-trivial.
Let us check it. For x, y P X we have by Cauchy-Schwarz inequality that

}x

y }2

 xx y, x yy  xx, xy xx, yy xy, xy xy, yy

 }x}2 2 Rexx, yy }y}2 }x}2 2|xx, yy| }y}2
 p}x} }y}q2 .

This completes the proof.

Remark 1.4.7 (Pythagorean theorem). The calculation above clearly implies

Pythagorean theorem: if x K y then

}x

y }2

 }x}2 }y}2 .

1.4. INNER PRODUCT SPACES

21

Remark 1.4.8 (Angle between vectors). The concept of inner product makes
it possible to define the angle between two vectors x, y in an inner product space
X. Recall that in Euclidean space Rn , the inner product can be computed by the
formula
xx, yy  }x}}y} cos px, yq
where px, y q is the angle between x and y. Therefore, in a general inner product
space X, it makes sense to define the angle between x, y by
xx, yy .
cos px, y q 
}x}}y}

Cauchy-Schwarz inequality guarantees that the right hand side lies in r
1, 1s, so
the angle exists. Nevertheless, the concept of angle is rarely used; one prefers to
work with inner product directly.

Lec.8: 09/24/10

1.4.2. The space L2 . H

olders inequality. The basic example of an inner
product space is L2  L2 p, , q:
Proposition 1.4.9 (Canonical inner product on L2 ). For f, g
tity

xf, gy :

P L2 , the quan-

f g d

is finite, and it defines an inner product on L2 . This inner product obviously agrees
with the L2 norm, i.e. }f }2  xf, f y1{2 .

Proof. The only non-trivial fact to prove is that xf, g y is finite, i.e. that
f g is integrable. Since f , g and f
g belong to L2 , we have that f 2 , g2 and
2
2
2
pf gq  f f g g are integrable. Hence f g is integrable, as required. 
We can recast Cauchy-Schwarz inequality in this specific space L2 as follows.
Corollary 1.4.10 (Cauchy-Schwarz inequality in L2 ). For every f, g
has





 f g d

|f |2 d

1{2

1{2

|g|2 d

P L2 one

The left
hand side of Cauchy-Schwarz inequality can be replaced by the larger
quantity |f g | d. (This can be seen by applying Cauchy-Schwarz inequality for
|f |, |g|.) Thus Cauchy-Schwarz inequality can be written as

}f g}1 }f }2 }g}2 .

Cauchy-Schwarz inequality is a partial case of the more general Holders inequality:

Corollary 1.4.11 (H
olders inequality for functions). Let p, q
adjoint, i.e. p1 1q  1. Then for every f P Lp and g P Lq one has




 f g d

|f |

1{p

1{q

|g|q d

P p1, 8q

be

Before proving this result, note that similarly to Cauchy-Schwarz inequality,

the
left hand side in H
olders inequality can be replaced by the larger quantity

|f g| d. Thus Holders inequality can be written as

(1.10)

}f g}1 }f }p }g}q .

1.4. INNER PRODUCT SPACES

22

Proof of Corollary 1.4.11. We are going to prove inequality (1.10). By

rescaling, we can assume that }f }p  }g }q  1 (In order to see this, divide both
sides by }f }p and }g }q ). We shall use Youngs inequality
ap
bq
for a, b 0,
p
q
which follows by taking logarithm of both sides and using the concavity of the
function logpxq. Therefore, we have the pointwise inequality
ab

|f ptqgptq| |f pptq|

|gptq|q

Integrating yields

|f g| p1

1
q

for all t P .

1

as required.

Using H
olders inequality we can clarify the scale of spaces Lp  Lp p, , q
for various p. Assume that is a finite measure (this is important!) Then L8 is
the smallest space, L1 is the largest, and all other Lp , p P r1, 8q lie in between:

Corollary 1.4.12 (Scale of Lp spaces). Let p, , q is a probability space,

and 1 r s 8. Then

}f }r }f }s

for all f

P Ls  Ls p, , q.

In particular, we have the inclusion

Ls

Lr .

Proof. For s  8 this inequality is clearly true, so let s 8. We apply

H
olders inequality for |f |r and the constant function 1, and with p  s{r:

}f }rr  |f |r d  |f |r  1 d


|f |s

r{s

|f |rs{r

r{s

1{q

1q d

pq  }f }rs .

This completes the proof.

1.4.3. The space `2 . Recall that the space of square-summable sequences `2

is a particular case of L2  L2 p, , q for the counting measure on  N.
Therefore, `2 is also an inner product space. The inner product formula reads as

xx, yy 

xi yi

for x  pxi q, y

 pyi q P `2 .

i 1

Let us write out Cauchy-Schwarz inequality in this case:

Corollary 1.4.13 (Cauchy-Schwarz inequality in `2 ). For every two sequences
x  pxi q and y  pyi q in `2 one has

8




xi yi 



i 1

8




i 1

|xi |2

8
1{2



|yi |2

1{2

i 1

As before, the left hand side can be replaced by the larger quantity
Similarly, H
olders inequality in this case takes the following form:

 |xi yi |.

i 1

1.4. INNER PRODUCT SPACES

23

Corollary 1.4.14 (H
olders inequality for sequences). Let p, q P p1, 8q be
adjoint, i.e. p1 1q  1. Then for every x  pxi q P `p and y  pyi q P `q one has

|xi yi |

8




i 1

|xpi

8
1{p

|yiq

i 1

1{q

i 1

However, Corollary 1.4.12 on the scale of Lp (on finite measure spaces) does
not hold for `p , because the counting measure on N is not finite. In fact, the scale
is completely reversed in this case: `1 is the smallest space, `8 is the largest, and
the other `p , p P r1, 8q lie in between:
Corollary 1.4.15 (Scale of `p spaces). Let 1 r

}x}s }x}r

for all x P `r .

s 8. Then

In particular, we have the inclusion

`r

`s .

Exercise 1.4.16. Prove Corollary 1.4.15. You dont have to use H

olders
inequality.
1.4.4. The space of matrices. For arbitrary fixed dimensions m, n P N, we
consider the space of matrices
Mm,n : tall m  n matrices with complex entriesu.
One can turn Mm,n into an inner product space by defining the trace inner product
as
(1.11)

xA, B y : trpAB  q 

m
n

 

aij bij .

i 1j 1

This is clearly an inner product. One way to see this is to identify Mm,n with Cmn
by concatenating the rows of a matrix A P Mm,n into a long vector in Cmn . Then
the canonical inner product in Cmn is the same as the right hand side of (1.11).
The norm defined by the inner product on Mm,n is called Hilbert-Schmidt or
Frobenius norm of matrices:
(1.12)

}A}HS  xA, Ay1{2 

m
n

 

|aij |2

1{2

i 1j 1

Note some similarity between the forms of the inner product in L2 , which is
f g d and in Mm,n , which is xA, B y : trpAB  q the integral is replaced
by the trace, functions by matrices, complex conjugation by transposition, and
product of functions by product of matrices.

xf, gy 

1.4.5. A space of random variables. The space L2 arises in probability

theory in a natural way as a space of random variables with finite variance. Indeed,
consider a probability space p, , Pq. Recall that a random variable X is a measurable real-valued function defined on . The expectation of X is, by definition,
the integral of X:

EX

X p q dPp q.

1.4. INNER PRODUCT SPACES

24

Therefore, the space L2 p, , Pq consists of all random variables X with finite

second moment:
}X }2  pEX 2 q1{2 8.

Exercise 1.4.17. Show that X P L2 p, , Pq if and only if X has finite

variance:
VarpX q  EpX
EX q2  EX 2
pEX q2 8.
The concepts of covariance and correlation coefficient have some geometric
meaning, too. Consider two random variables X and Y , and for simplicity assume
that they have mean zero, i.e. EX  EY  0. Then the covariance of X and Y is
nothing else than the inner product in L2 :
covpX, Y q : EpX

EX qpY
EY q  EXY  xX, Y y.

Similarly, the correlation coefficient between X and Y is

corrpX, Y q :

covpX, Y q
VarpX q1{2 VarpY q1{2

Yy
 pEX 2 qEXY
 }XxX,
1{2 pEY 2 q1{2
} }Y }
2

Hence the correlation coefficient is nothing else as the cosine of the angle between
random variables X and Y considered as vectors in L2 (see Remark 1.4.8). This
demonstrates the geometric meaning of correlation the more random variables X
and Y are correlated, the less the angle between them, and vice versa.
1.4.6. Parallelogram law. The parallelogram law in planar geometry states
that for every parallelogram, the sums of squares of the diagonals equals the sum of
squares of the sides. This statement remains to be true in all inner product spaces:
Proposition 1.4.18 (Parallelogram law). Let X be an inner product space.
Then for every x, y P X one has
(1.13)

}x

y }2

}x
y}2  2}x}2

2}y }2 .

Proof. The result follows once we recall that }x y }2  xx y, x

}x} 2 Rexx, yy }y}2 and similarly }x
y}2  }x}2
2 Rexx, yy }y}2 .
2

yy

The parallelogram law characterizes inner product spaces. First recall that in
inner product spaces, the inner product determines the norm (}x}  xx, xy1{2 ).
Vice versa, the inner product is uniquely determined by the norm, and it can be
reconstructed through the polarization identity:
Proposition 1.4.19 (Polarization identity). Let X be an inner product space.
Then for every x, y P X one has

(1.14)

xx, yy  14 }x

y }2
}x
y }2

i}x

iy }2

i}x
y }2 .

Exercise 1.4.20. Prove the polarization identity.

The parallelogram law characterizes inner product spaces:
Theorem 1.4.21 (Characterization of inner product spaces). Let X be a normed
space over C such that every two vectors x, y P X satisfy the parallelogram law
(1.13). Then X is an inner product space. Precisely, polarization formula (1.14)
defines the inner product on X that agrees with the norm, i.e. }x}  xx, xy1{2 .

Lec.9: 09/27

1.5. HILBERT SPACES

25

The proof of this result is deferred to the exercises for this section.
It follows from Theorem 1.4.21 that being an inner product space is a local
property, since checking the parallelogram law involves just two (arbitrary) vectors.
In particular, if all two-dimensional linear subspaces of a normed space X are innerproduct spaces (with respect to some inner product, possibly different for each
subspace), then X is an inner product space (and there the inner product on all
subspaces is actually the same, induced from X!)
1.4.7. Additional Exercises.
Exercise 1.4.22. [Direct sum of inner product spaces] Let X, Y be
inner product spaces. Show that their direct sum
X

`2 Y : tpx, yq : x P X, y P Y u

is also an inner product space, with the inner product defined as

xpx1 , y1 q, px2 , y2 qy  xx1 , x2 y xy1 , y2 y.

Derive a formula for the norm in X `2 Y . Show that if X, Y
spaces then so is X `2 Y .

are Hilbert

Exercise 1.4.23. Show that the inner product is a continuous function

on the direct sum X `2 X of a Hilbert space X with itself.
Exercise 1.4.24. [Characterization of inner product spaces] Prove Theorem 1.4.21. For simplicity, do this for spaces over R. (Formulate the
polarization identity in this case). Axioms (i) and (iii) of inner product
are simple to check. To prove axiom (iii) (linearity), you may follow this
sequence of steps:
(i) Show that xx1 x2 , y y xx1
x2 , y y  2xx1 , y y for all x1 , x2 , y P X.
(ii) Deduce the additivity property: xu, y y xv, y y  xu v, y y for all
u, v, y P X.
(iii) From additivity property, deduce the multiplicativity property xtx, y y 
txx, y y for all x, y P X, t P R. Indeed, for t P N this follows by induction; deduce the same for all t P Q and extend to all t P R by
continuity.
Exercise 1.4.25. Show that C pK q, c0 , Lp r0, 1s, `p for p P r1, 8s, p  2, are
not inner product spaces. (More accurately, it is not possible to define
an inner product on those spaces which would agree with their norms).
Use Theorem 1.4.21.
1.5. Hilbert spaces
1.5.1. Definition. Orthogonal complements. As in the case of normed
spaces, we will really need an extra axiom completeness in order to study
geometry of inner product spaces.
Definition 1.5.1. A complete inner product space is called a Hilbert space.

1.5. HILBERT SPACES

26

Example 1.5.2. As we know, L2 p, , q is a Hilbert space. In particular,

`2 is a Hilbert space. Also, Cn and Rn are Hilbert spaces; therefore the space of
matrices Mm,n is also a Hilbert space.
The geometry of Hilbert spaces is dominated by the concept of orthogonality.
We will exploit it in detail now.
Definition 1.5.3 (Orthogonal complement). Let A be a subset of an inner
product space X. The orthogonal complement of A is defines as
AK : tx P X : xx, ay  0 for all a P Au.

Proposition 1.5.4. Let A be a subset of an inner product space X. Then AK

is a closed linear subspace of X. Moreover,
AK X A t0u.

Proof. It is easy to check that AK is a linear subspace of X. To show that

AK is a closed set, express it as

AK

tauK .

aPA
K
Therefore it suffices to check that tau is a closed set for every a P A. So we fix
a P A and consider a sequence xn P tauK such that xn x for some x P X. We
would like to show that x P tauK . To this end, notice that continuity of the inner
10

product (Exercise 1.4.23) implies that

0  xxn , ay xx, ay.

Hence xx, ay  0 as required.

Finally, to show that AK X A t0u, consider x
xx, xy  0 which implies x  0. The proof is complete.

AK

X A;

it follows that


1.5.2. Orthogonality principle. Orthogonal decompositions.

Theorem 1.5.5 (Orthogonality principle). Let Y be a closed linear subspace of
a Hilbert space X, and let x P X. Then the following holds.
(i) There exists a unique closest point y P Y to x, i.e. such that

1
}x
y}  ymin
1 PY }x
y }.

(ii) The point y is the unique vector in Y such that

x
y

P Y K.

The point y is called the orthogonal projection of x onto the subspace Y .

Proof. (i). Existence. Denote the distance by
d : inf
}x
y1 }.
y 1 PY

Let us choose a sequence pyn q in Y which satisfies

}x
yn } d.

Since Y is closed, it is a Banach space. Therefore it suffices to show that pyn q is a

Cauchy sequence. (By continuity of the norm it would follow for y  limn yn that
}x
y}  d.)
10Recall that the intersection of closed sets is closed.

1.5. HILBERT SPACES

27

To bound }yn
ym }, we use parallelogram law. We apply it for the parallelogram
with vertices x, yn , ym (and whose fourth vertex is determined by these three, see
the picture.)

Figure 1.3. Proof of Theorem 1.5.5

Parallelogram law then yields

}yn
ym }2

4}x

1
pyn
2

ym q}2

 2}x
yn }2

2}x
ym }2 .

By definition of d, we have }x
12 pyn ym q} d, and by construction we have

}x
yn } d, }x
ym } d. With this, we conclude that
0 lim inf }yn
ym }2

2d2

2d2
4d2

 0.

Therefore pyn q is a Cauchy sequence as required.

Uniqueness. Suppose there existed two different closest points y1 , y21 for x.
Then the alternating sequence py1 , y2 , y1 , y2 , . . .q would not be Cauchy, contradicting the argument above. Part (i) of Theorem is proved.
(ii). Orthogonality. Assume that x
y R Y K , so

xx
y, y1 y  0

for some y 1

P Y.

By multiplying y 1 by an appropriate complex scalar, we can assume that xx
y, y 1 y

is a real number. We will show that by moving y in the direction of y 1 , one can
improve the distance from x, which will contradict the definition of y. Namely, for
every t P R the definition of y implies that

}x
y}2 }x
y

ty 1 }2

 }x
y}2

2txx
y, y 1 y

t2 }y 1 }2 .

This implies that the quadratic polynomial in t satisfies

}y1 }2 t2 2xx
y, y1 yt 0 for all t P R.

This can only happens if xx
y, y 1 y  0, which contradicts our assumption.
Uniqueness. Suppose there are two vectors y 1 , y 2 P Y which satisfy x
y 1 P Y K ,
x
y 2 P Y K . Since Y K is a linear subspace, subtracting yields that y 1
y 2 P Y K .
But Y is also a linear subspace, so y
y 1 P Y . Since Y X Y K  t0u, it follows that
y
y 1  0, hence y 1  y 2 . Theorem is proved.


Lec.10: 09/29

1.6. FOURIER SERIES

28

In the proof of part (i) of Theorem 1.5.5, we used convexity rather than linearity
of Y . (Indeed, we needed that together with two points yn , ym P Y their midpoint
1
ym q is contained in Y ). Therefore, our argument implies the following more
2 pyn
general result:
Theorem 1.5.6 (Hilberts projection theorem). Given a closed convex set Y
in a Hilbert space X and a point x P X, there exists a unique closest point y P Y .
The map that takes x into the closest point y is called a projection onto convex
set Y and is abbreviated POCS. This map appears in several applied fields.
The orthogonality principle immediately implies that a Hilbert space X can be
decomposed into the orthogonal sum of a subspace Y and its complement Y K :
Corollary 1.5.7 (Orthogonal decomposition). Let X be a Hilbert space and
Y be a closed subspace. Then every vector x P X can be uniquely represented as
xy

z,

P Y,

P Y K.

This orthogonal decomposition is usually abbreviated as

X

 Y ` Y K.

Definition 1.5.8 (Orthogonal projection). In the setting of Corollary 1.5.7,

the map
PY : X X, PY x  y
is called the orthogonal projection in X onto Y .
Exercise 1.5.9. Show that the orthogonal projection PY is a linear
map. Check that ImpPY q  Y and kerpPY q  Y K . Also check that the
identity map IX on X can be decomposed as
IX

 PY

PY K .

1.5.3. Additional Exercises.

Exercise 1.5.10. Let A be a subset of a Hilbert space. Show that
AK

 AK

where A denotes the closure of A.

Exercise 1.5.11. [Projection onto constants] Let Y denote the subspace of constant functions in L2  L2 p, , q. Compute PY f for an
arbitrary function f P L2 .
1.6. Fourier series
Hilbert space gives a geometric framework for studying Fourier series. The
classical Fourier analysis studies situations where a function f : r
, s C can
be expanded as Fourier series

f ptq 
k


8

fppk qeikt

1.6. FOURIER SERIES

with Fourier coefficients

fppk q 

(1.15)

1
2

29

f ptqe
ikt dt

are called the Fourier coefficients of f . In order to make Fourier analysis rigorous,
one has to understand what functions f can be written as Fourier series, and in
what sense the Fourier series converges.
In order to do so, it is of great advantage to depart from this specific situation
and carry out Fourier analysis in an abstract Hilbert space. We will regard the
function f ptq as a vector in the function space L2 r
, s. The exponential functions
e
ikt will form a set of orthogonal vectors in this space. Fourier series will then
become an orthogonal decomposition of a vector f with respect to an orthogonal
system of coordinates.
1.6.1. Orthogonal systems.
Definition 1.6.1 (Orthogonal system). A sequence pxk q in a Hilbert space X
is called an orthogonal system if

xxk , xl y  0

for all k

 l.

If additionally }xk }  1 for all k, the sequence pxk q is called an orthonormal system.
Equivalently, pxk q is an orthonormal system if

xxk , xl y  kl

where kl equals 1 if k

 l and 0 otherwise (it is called Kroneckers delta function).

Example 1.6.2 (Canonical basis of `2 ). In the space `2 , consider the vectors

xk

 p0, . . . , 0, 1, 0, . . .q

whose all coordinates are zero except the k-th equals 1. The sequence pxk q8
k1 is
clearly an orthonormal system in `2 .
Example 1.6.3 (Fourier basis in L2 ). In the space11 L2 r
, s, consider the
exponentials
1
(1.16)
ek ptq  ? eikt , t P r
, s.
2
8
Then pek qk
8 is an orthonormal system in L2 r
, s (check!).
Example 1.6.4 (Trigonometric system in L2 ). Closely related to the Fourier
basis is the trigonometric system. Note that we can write the exponentials from
the previous example as

1
fk ptq  ?
cospktq i sinpktq .
2
Considering the real and imaginary parts separately, we see that the system
! 1
)
? , ?1 cosptq, ?1 sinptq, ?1 cosp2tq, ?1 sinp2tq, . . .
2
is an orthonormal system in L2 r
, s (check!)

r

p q

11The space L
, can be identified with L2 T where T is the unit torus in C. We can
2
think of elements of this space as 2-periodic functions.

1.6. FOURIER SERIES

30

Exercise 1.6.5. Prove that an orthogonal system is a linearly independent set.

1.6.2. Orthogonal series. The main interest in orthogonal systems is that
they allow us to form orthogonal expansions of every vector x P X. Such expansions
are infinite series. So our first task will be to clarify when orthogonal series converge.
Theorem 1.6.6 (Convergence of orthogonal series). Let pxk q be an orthogonal
system in a Hilbert space X. Then the following are equivalent:

(i) k xk converges in X;
(ii) k }xk }2 8;
(iii) k xk converges unconditionally in X, i.e. for every reordering of terms.
In case of convergence, we have
 2


xk 


(1.17)

}xk }2 .

The proof of this result is based on its finite version, which may be called the
Pythagorean theorem in higher dimensions:
Lemma 1.6.7 (Pythagorean theorem). Let pxk q be an orthogonal system in a
Hilbert space X. Then for every n P N one has
n

2


xk 


(1.18)

k 1

}xk }2 .

k 1

Proof. Using orthogonality, we see that the left hand side of (1.18)
n
A

k 1

xk ,

xk

k 1

xxk , xl y 

k,l 1

xxk , xk y

k 1

which equals the right hand side of (1.18).

(ii). By the Cauchy criterion, the series

x
converges
if
and
only
if
its
partial
sums form a Cauchy sequence in X, i.e.
k k

Proof of Theorem 1.6.6. (i)

(1.19)

m

2


xk 


as n, m 8.

k n

Note
m that 2by Pythagorean theorem (Lemma 1.6.7), the quantity in (1.19) equals
criterion again we see that (1.19) is equivalent to the
kn }xk } . So using Cauchy

convergence of the series k }xk }2 , as

required.

(ii) (iii). The scalar series k }xk }2 converges absolutely, therefore also
unconditionally (as we know from an analysis
course). Hence, by the equivalence

of (i) and (ii) proved above, the series k xk converges unconditionally.

(iii) (i) is trivial.
The last part of the theorem, identity (1.17), follows by taking limit in Pythagorean
identity (1.18). The theorem is proved.

Exercise 1.6.8. For what coefficients ak does the series
converge in L2 r
, s?

8
k


8 ak e

ikt

1.6. FOURIER SERIES

31

Lec.11: 10/01

1.6.3. Fourier series.

Definition 1.6.9 (Fourier series). Consider an orthonormal system pxk q8
k1 in
a Hilbert space X and a vector x P X. The Fourier series of x with respect to pxk q
is the formal series

xx, xk yxk .
k

The coefficients xx, xk y are called the Fourier coefficients of x.

In order to understand the convergence of Fourier series, we will first focus on
the finite case, and study the partial sums of Fourier series
Sn pxq :

xx, xk yxk .

k 1

Lemma 1.6.10. Sn pxq is the orthogonal projection of x onto12 Spanpx1 , . . . , xn q.

Proof. By the definition of the orthogonal projection (see the orthogonality
principle, Theorem 1.5.5), the lemma states that
x
Sn pxq P pSpanpx1 , . . . , xn qqK .

It suffices to check that x
Sn pxq K xk for all k

 1, . . . , n. We have
xx
Sn pxq, xk y  xx, xk y
xSn pxq, xk y.
By definition of Sn pxq and orthonormality of pxk q we see that xSn pxq, xk y  xx, xk y.
Therefore we conclude that xx
Sn pxq, xk y  0 as required.

Let us estimate the size of Sn pxq. Since x
Sn pxq K Sn , by Pythagorean
theorem we have }Sn pxq}2 }x
Sn pxq}2  }x}2 . Hence
}Sn pxq}2 }x}2 .
On the other hand, by Pythagorean theorem and orthonormality,

}Sn pxq}2 

}xx, xk yxk }2 

k 1

Combining these two inequalities and letting n

result.

|xx, xk y|2 .

k 1

8, we arrive at the following

Theorem 1.6.11 (Bessels inequality). Let pxk q be an orthonormal system in

a Hilbert space X. Then for every x P X one has

|xx, xk y|2 }x}2 .

This result along with the convergence criterion for orthogonal series, Theorem 1.6.6, shows that Fourier series always converge.
Corollary 1.6.12.
Let pxk q be an orthonormal system in a Hilbert space X.
Then the Fourier series k xx, xk yxk of every vector x P X converges in X.
12

(
n Recall that the
k1 ak xk : ak P C .

linear span of vectors x1 , . . . , xn is defined as Span x1 , . . . , xn

q

1.6. FOURIER SERIES

32

In general, Fourier series of x needs not converge to x (why?) Still we can

compute the point where it converges. Indeed, letting n 8 in Lemma 1.6.10
and interpreting the conclusion via the orthogonality principle (Theorem 1.5.5), we
conclude:
Theorem 1.6.13 (Optimality of Fourier series). Consider an orthonormal system pxk q8
k1 in a Hilbert space X and a vector x P X.

(i) The Fourier series k xx, xk yxk is the orthogonal projection of x onto Spanpxk q
(the closure of the linear span).

(ii) Among all convergent series of the form S  k ak xk , the approximation

error }x
S } is minimized by the Fourier series of x.
Exercise 1.6.14. Prove Theorem 1.6.13.

1.6.4. Orthonormal bases. Fourier expansions. It is now easy to identify

an extra condition so that the Fourier series of every vector x converges to x.
Definition 1.6.15 (Complete systems). A system of vectors pxk q in a Banach
space X is called complete if
Spanpxk q  X.
A complete orthonormal system in a Hilbert space X is called an orthonormal basis
of X.
Theorem 1.6.16 (Fourier expansions). Let pxk q be an orthonormal basis of a
Hilbert space X. Then every vector x P X can be expanded in its Fourier series:
(1.20)

x

xx, xk yxk .

Consequently, Parsevals identity holds:

}x}2 

|xx, xk y|2 .

Proof. The first part follows from the Optimality Theorem 1.6.13, since by
completeness the orthogonal projection onto Spanpxk q  X is the identity map in
X. Parsevals identity follows from Fourier expansion (1.20), Pythagorean identity
(1.17) for orthogonal series, and the normalization condition }xk }  1.

Exercise 1.6.17. Prove that Parsevals identity holds for an orthonormal system pxk q if and only if pxk q is complete. Therefore the equality
cases of Bessels inequality hold exactly when the system is complete.
Now we describe some classical examples of complete sets and orthonormal
bases.
Example 1.6.18 (Monomials). Weierstrass approximation theorem states that
the system of monomials ptk q8
k0 is a complete system in C r0, 1s. We claim that
this is also a complete system in L2 r0, 1s.
Indeed, C r0, 1s is dense in L2 r0, 1s. This means that for every f P L2 r0, 1s
and 0, there exists g P C r0, 1s such that }f
g }2 {2. By Weierstrass
approximation theorem, there exists h P Spanptk q8
k0 such that }g
f }8 {2.
Hence }g
f }2 }g
f }8 {2, so by triangle inequality we conclude that
}f
h}2 {2 {2  . We have proved that Spanptk q8k0 is dense in L2 r0, 1s as
required.

1.6. FOURIER SERIES

33

Example 1.6.19 (Exponentials). By a general version of Weierstrass approximation theorem (called Stone-Weierstrass theorem), the exponential monomials
peitk qkPZ is a complete system in C r
, s. Repeating the argument in Example 1.6.18, we can check that this is also a complete system in L2 r
, s.
Therefore, the system of exponentials
1
(1.21)
xk ptq  ? eikt , k P Z,
2
forms an orthonormal basis of L2 r
, s. Reformulating Theorem 1.6.16 in this
case, we obtain a basic result in classical Fourier analysis:
Theorem 1.6.20 (Classical Fourier series). Every function f
be represented by its Fourier series

f ptq 
k


8

fppk qeikt ,

fppk q 

where

1
2

P L2 r
, s can

f ptqe
ikt dt.

The coefficients fppk q are all finite; the Fourier series converges in L2 r
, s.
Example 1.6.21 (Trigonometric system). In a similar way we can show that
the trigonometric system considered in Example 1.6.4 is an orthonormal basis in
L2 r
, s (do this!) Therefore a version of Theorem 1.6.20 holds for the trigonometric system, and it reads as follows:
f ptq 
where

8
a0 
ak cospktq
2 k1

bk sinpktq

1
1
ak 
f ptq cospktq dt, bk 
f ptq sinpktq dt.




This again holds for every function f P L2 r
, s; the coefficients ak , bk are all
finite, and the Fourier series converges in L2 r
, s.
1.6.5. Gram-Schmidt orthogonalization. There is a general way of creating an orthonormal basis phk q in a Hilbert space X out of some other, possibly
non-orthogonal system pxk q. One orthogonalize the system pxk q one element at a
time. This procedure is called Gram-Schmidt orthogonalization.
So let us consider a linearly independent system of vectors pxk q8
k1 in X. We
define the system phk q8
inductively
as
follows:
k 1
h1

 }xx1 } ,
1

hn

 }PPn xxn 1 } ,
n n 1

n  1, 2, . . .

where Pn denotes the orthogonal projection in X onto Spanph1 , . . . , hn qK . Geometrically, one rotates the new vector xn 1 so it becomes orthogonal to the vectors
hk constructed earlier, normalizes it, and calls hn 1 ; see the picture.
One can effectively compute the vectors Pn xn 1 used in this process. Indeed,
by Lemma 1.6.10, the orthogonal projection of a vector x onto Spanph1 , . . . , hn q is
the partial sum of Fourier series:
Sn pxq 

xx, hk yhk .

k 1

Lec.12: 10/04

1.6. FOURIER SERIES

34

Figure 1.4. Gram-Schmidt orthogonalization

Now, Pn pxq is the othogonal projection onto the orthogonal complement, so
Pn x  x
Sn pxq  x

xx, hk yhk .

k 1

So
Pn x n

1  xn 1

xxn

1 , hk

yhk .

k 1

Proposition 1.6.22. Let pxk q be a linearly independent system in a Hilbert

space X. Then the system phk q obtained by Gram-Schmidt orthogonalization of
pxk q is an orthonormal system in X, and
Spanphk qnk1

 Spanpxk qnk1

for all n P N.

Proof. The system phk q is orthonormal by construction. Also, we obviously

have the inclusion of the subspaces Spanphk qnk1 Spanpxk qnk1 , and the dimensions of these subspaces both equal n by construction. Therefore, these subspaces
are equal.

Exercise 1.6.23. [Legendre polynomials] Prove that Gram-Schmidt
orthogonalization of the monomials ptk q8
k0 in the space L2 r
1, 1s gives
the system of Legendre orthogonal polynomials
1 dk 2
pt
1qk ,
2k k! dtk
up to normalization constants. More precisely, Pk ptq form an orthogonal
basis in L2 r
1, 1s and }Pk }22  2{p2k 1q.
Pk ptq 

In a similar way, by orthogonalization of monomials in different spaces, one

produces other classical systems of orthogonal polynomials, such as Hermite (see
Exercise 1.6.38 and Chebychev polynomials.
1.6.6. Existence of orthogonal bases. As a consequence of Proposition 1.6.22,
we will obtain that every Hilbert space that is not too large has an orthonormal
bases. Such are all separable Hilbert spaces.
Recall that a metric space is separable if it contains a countable dense subset.
Exercise 1.6.24. Prove that the following Banach spaces X are separable by describing a countable dense subset in X: C r0, 1s, L2 r0, 1s, c0 , `p
for p P r1, 8q.

1.6. FOURIER SERIES

35

Exercise 1.6.25. Prove that the spaces `8 and L8 are not separable
by constructing an uncountable separated subset A, i.e. such that
inf t}x
y } : x, y

P A, x  yu 0.

For Banach spaces, separability is formally equivalent to a slightly stronger

property:
Lemma 1.6.26 (Separable spaces). A Banach space X is separable if and only
if it contains a system of vectors pxk q8
k1 whose linear span is dense in X, i.e.
Spanpxk q  X.

Proof. Necessity. If X is separable, it contains a system of vectors pzk qnk1

whose linear span is dense in X. We construct pxk q inductively as a subset of pzk q.
Namely, we include z1 , and if zn 1 R pz1 , . . . , zn q we include zn for n  1, 2, . . .. By
construction, pxk q is linearly independent and Spanpx1 , . . . , xn q  Spanpz1 , . . . , zn q
for all n  1, 2, . . . Letting n 8 we conclude that Spanpx1 , . . . , xn q  Spanpz1 , . . . , zn q 
X as required.
Sufficiency.
If Spanpxk q is dense in X, so is the set of all finite linear combinan
tions k1 ak xk with ak P Q, which is a countable set. The lemma is proved. 
Theorem 1.6.27. Every separable Hilbert space has an orthonormal basis.
Proof. Let X be a separable Hilbert space. By Lemma 1.6.26, there is a
system of vectors pxk q in X such that Spanpxk q  X. Applying the Gram-Schmidt
orthogonalization, we obtain an orthonormal system phk q in X. Using Proposition 1.6.22 and letting n 8, we conclude that
Spanphk q  Spanpxk q  X.

This proves the completeness of phk q. Hence phk q is an orthonormal basis in X. 

Exercise 1.6.28. Prove the converse for Theorem 1.6.27. Namely, if
a Hilbert space X has an orthonormal basis then X is separable.
Remark 1.6.29. We developed the theory for countable orthogonal systems
and bases. One can generalize it for systems of arbitrary cardinality, but we will
not do this here.
1.6.7. Isometry of all separate Hilbert spaces. We are ready to show
that all Hilbert spaces of the same cardinality have the same geometry:
Theorem 1.6.30. All infinite-dimensional separable Hilbert spaces are isometric to each other. Precisely, for every such spaces X and Y , one can find a linear
bijective map T : X Y which preserves the inner product, i.e.
(1.22)

xT x, T yy  xx, yy for all x, y P X.

Let pxk q and pyk q be orthonormal bases of spaces X and Y respectively.

Proof.
Let T be the map that takes xk to yk . More precisely, define T by
T

k

ak xk

a k yk .

1.6. FOURIER SERIES

36

Note that every x P X has the form x  k ak xk for some (Fourier) coefficients ak ,
so the definition makes sense. Also, by Parsevals identity,
(1.23)


2


a k xk 


|ak |2  

2


ak yk  .

Therefore, T is well defined on X, its inverse is also well defined as

T
1

ak yk

ak xk ,

so T is bijective and clearly linear. Additionally, (1.23) shows that

}T x}  }x}

for all x P X.

So T preserves the norm. Since by polarization formula, the inner product is

uniquely determined by the norm, T must also preserve the inner product, i.e.
(1.22) holds. This completes the proof.

Remark 1.6.31. It follows from Theorem 1.6.30 that }T x}  }x} for all x P X.
Thus by linearity,

}T x
T y}  }x
y}

for all x, y

P X.

Thus T preserves all pairwise distances; hence the name isometry.

Remark 1.6.32. Since `2 and L2 r0, 1s are separable Hilbert spaces, it follows
that every Hilbert space is isometric to `2 and L2 r0, 1s.

1.6.8. Additional Exercises.

Exercise 1.6.33. [Frames] Consider the space L2 ra, bs where
a
b . Show that the system of exponentials (1.21) satisfies Parsevals
identity for every function x P L2 ra, bs, although it is not an orthogonal system. General systems that satisfy Parsevals identity are called
frames. They are important in signal processing.
Exercise 1.6.34. [Haar system] Consider the function
hptq 

1,

1,

t P r0, 1{2q
t P r1{2, 1q

and define the functions

hkl ptq  hp2k t
lq,

 0, 1, 2, . . . ,

l  0, 1, 2, . . . , 2k
1.

Together with the constant function 1, this system of functions is called

the Haar system; the individual functions is called the Haar wavelets
and the function hptq is called the Haar mother wavelet. Show that,, the
Haar system is an orthonormal basis in L2 r0, 1s. (Hint: first show that
Haar system is complete in C r0, 1s.)

1.6. FOURIER SERIES

37

Exercise 1.6.35. [Rademacher system] Consider the functions

rk ptq  p
1qr2

s,

P N Y t0u,

where rs denotes the integer part of a number. This system of functions
is called the Rademacher system. Show that the Rademacher system is
an orthonormal system in L2 r0, 1s, but is not complete (therefore not an
orthonormal basis).
Exercise 1.6.36. [Walsh system] Consider the functions wA ptq, A
N Y t0u (indexed by subsets A rather than numbers!) defined by
wA ptq 

rk ptq

k A

where rk ptq are the Rademacher functions. This system of functions is

called the Walsh system. Show that the Walsh system is an orthonormal
basis in L2 r0, 1s.
Exercise 1.6.37. [Characterization of complete systems] Consider a
system of vectors pxk q (not necessarily orthogonal) in a Hilbert space X.
Prove that pxk q is complete if and only if the only vector orthogonal to
all of xk is zero.
Exercise 1.6.38. [Hermite polynomials] Suppose wptq is a continuous
weight function R R . Consider the Hilbert space L2 pR, wptq dtq, i.e. the
measure on R is given by wptq dt. The Gram-Schmidt orthogonalization of
monomials ptk q8
k0 produces a system orthogonal polynomials Pk ptq with
repsect to the weight wptq, i.e.

Pk ptqPl ptqwptq dt  kl .

Now consider the weight wptq  ?12 e
t {2 , i.e. the standard normal
density. Prove that the orthogonal polynomials with respect to this
weight is the system of Hermite polynomials
2

Pk ptq  p
1qk et

{2 d e
t2 {2 ,
k
k

dt
up to normalization constants. More precisely, Pk ptq form an orthogonal
basis in L2 pR, wptq dtq and }Pk }22  k!.
Exercise 1.6.39. [Space of almost periodic functions] A function f :
R C is called almost periodic if it is a superposition of a finite number
of frequencies, i.e. f has the form
f ptq 

ak eiwk t ,

where ak

P C,

wk

P R,

n P N.

k 1

Note that the frequencies wk are allowed to take arbitrary real values.
Denote the space of almost periodic functions by X0 , and equip it with
the inner product

xf, gy  Tlim
8

1 T

2T

1{2

|f ptq|2 dt

1.6. FOURIER SERIES

38

Show that the inner product is well defined on X0 , and

xf, gy 

k 1

ak bk

where f ptq 

k 1

ak eiwk t , g ptq 

bk eiwk t

k 1

(and where the frequencies wk are all different). The completion X of the
inner product space X0 is called the space of almost periodic functions.
Prove that X is a non-separable Hilbert space by showing that the
system of functions
teitw , w P Ru
is an orthonormal system in X.

CHAPTER 2

Bounded linear operators

Lec.13: 10/06

In this chapter we study certain transformations of Banach spaces. Because

these spaces are linear, the appropriate transformations to study will be linear
operators. Furthermore, since Banach spaces carry topology, it is most appropriate
to study continuous transformations, i.e. continuous linear operators. They are
also called bounded linear operators for the reasons that will become clear shortly.
2.1. Bounded linear functionals
The most basic but rich class of linear operators are linear functionals, those
that map a space into R or C.
2.1.1. Definition and examples. At this moment, the topology does not
matter, so we define linear functionals on general linear vector spaces.
Definition 2.1.1 (Linear functionals). Let E be a linear space over C (the
real case is similar). A linear functional on E is a linear operator f : E C.
Equivalently, a function f : E C is a linear functional if
f pax

by q  af pxq

bf py q

for all x, y

P E, a, b P C.

Example 2.1.2 (Integration). The integral of an integrable function is a basic

example of a linear functional. Specifically, the map

F pg q 

g ptq d

is clearly a linear functional on L1 p, , q.

Similarly, for a fixed weight function w P L1 r0, 1s, the map
F pg q 

1
0

g ptqwptq dt

defines a linear functional on L8 r0, 1s. (Check!)

Example 2.1.3 (Point evaluation functional). For a fixed t0
(2.1)

F pg q  g pt0 q

P r0, 1s, the map

is clearly a linear functional on C r0, 1s. It is called the point evaluation functional
at t0 .
Physicists view the point evaluation functional as a special case of integration
with weight:
(2.2)

g pt0 q 

1
0

g ptq pt
t0 q dt

The weight here is given by Dirac delta function ptq, which is zero for all values t
1
except pt0 q  8, and such that 0 ptq dt  1. Dirac delta function does not exist
39

2.1. BOUNDED LINEAR FUNCTIONALS

40

as a function r0, 1s R, and should be understood as a linear functional. The

integral (2.2) of a function against the Dirac delta function should be understood
as the point evaluation functional (2.1).
Exercise 2.1.4. [Functionals on Cn ] Show that every linear functional
f on Cn has the form
f p xq 

xk yk

 xx, yy,

x  px1 , . . . , xn q,

k 1

for some y

 py1 , . . . , yn q P Cn .

Example 2.1.5. More generally, we will soon show that every liner functional
on a Hilbert space X has the form

for some y
functional.

f pxq  xx, y y
X. For now, we note that f defined this way is indeed a linear

2.1.2. Continuity and boundedness. Dual space.

Definition 2.1.6 (Continuity, boundedness). Let f be a linear functional on
a normed space X.
(i) Recall that f is continuous if
xn

x in X

implies f pxn q f pxq.

(ii) f is called bounded if there exists a number C such that

|f pxq| C }x}

for all x P X.

Proposition 2.1.7. Continuity and boundedness of linear functionals are equivalent.

x. Then
|f pxn q
f pxq|  |f pxn
xq| C }xn
x} 0.

Proof. Assume that f is bounded, and let xn

Thus f is continuous.
Vice versa, assume that f is not bounded. Then we can find a sequence pxn q
of nonzero vectors in X such that

|f pxn q| n}xn },
Dividing both sides by n}xn } we obtain


xn 

f
 1,
n}x }
n

On the other hand, n}xxnn }

that f is not continuous.

n  1, 2, . . .
n  1, 2, . . .

0 as the norm of these vectors equals 1{n. This implies

Exercise 2.1.8. Prove that if f is continuous at a single point x0

then f is continuous (everywhere on X).

PX

2.1. BOUNDED LINEAR FUNCTIONALS

41

Definition/Proposition 2.1.9 (Dual space). Let X be a normed space. The

space of all linear functionals f on X is a linear vector space. It is called the dual
space and is denoted X  . The dual space is a normed space, with the norm defined
as
}f }  sup |f}pxx}q|  sup |f pxq|, f P X  .
x0
}x}1
Exercise 2.1.10. Prove the facts stated in this definition/proposition
that X  is a normed space, the identity between the two norm definitions,
and that }f } indeed defines the norm on X  .
Remark 2.1.11. The definition implies the following useful inequality:

|f pxq| }f } }x}

for all x P X, f

P X .

Also, }f } is the smallest number in this inequality that makes it valid for all x P X.
Exercise 2.1.12. Compute the norms of the integration and the point
evaluation functionals considered in Examples 2.1.2 and 2.1.3.
2.1.3. Hyperplanes as level sets of linear functionals. General functions,
and in particular linear functionals f , on a linear vector space E may be visualized
by describing their level sets

tx P X : f pxq  cu

for various values c P C. The level set corresponding to c  0 is the kernel of f .

It turns out that ker f is a hyperplane, i.e. a subspace of E of codimension
1. All other level sets of f are obviously the translates of ker f , see the picture.
Moreover, there is a canonical correspondence between the linear functionals and
the hyperplanes in X. This is clarified in the following proposition.

Figure 2.1. Level sets of a linear functional f on a linear vector

space E
Proposition 2.1.13 (Linear functionals and hyperplanes). Let E be a linear
vectors space.
(i) For every linear functional f on E, ker f is a hyperplane in E, i.e. codimpker f q 
1.
(ii) If f, g  0 are linear functionals on E such that ker f  ker g, then f  ag
for some scalar a  0.

2.2. REPRESENTATION THEOREMS FOR LINEAR FUNCTIONALS

(iii) For every hyperplane H

ker f  H.

42

E there exists a linear functional f  0 such that

Proof. (i) Follows from a linear version of the fundamental theorem on homomorphisms, Exercise 1.1.24. Indeed, the injectivization f : E { ker f C of f
establishes a linear bijection (isomorphism) between E { ker f and the range C of f .
Thus dimpE { ker f q  dimpCq  1, so ker f is a hyperplane in E.
(ii) Since ker f  ker g : H, the injectivizations f, g : E {H C are linear
functionals on the one-dimensional space E {H. A moments thought yields that
such linear functionals must be equal up to some constant factor a, i.e. f 
a
g . Then frxs  a
g rxs for all x P E. On the other hand, by construction of
injectivization, f pxq  frxs and g pxq  grxs. Therefore f pxq  ag pxq as required.
(iii) Since dimpE {H q  1, we have
E {H

 tarx0 s : a P Cu
for some x0 P E. Let x P E be arbitrary; then rxs  arx0 s for some a  apxq P C,
which implies x  ax0 h for some h P H. Let us define f on E by f pxq  a. Then
f is a linear functional (check!), and clearly ker f  H.

Proposition 2.1.14. Let f be a bounded linear functional, i.e. f P X  then
ker f is closed.

Proof. ker f is the pre-image of the closed set t0u under the continuous map
f , so it must be closed.

Remark 2.1.15. Using injectivization of f , one can show that the converse also
holds. So, a linear functional f is bounded if and only if ker f is closed. It follows
that the kernel of a linear functional is either closed or dense in X. (Why?)
Lec.14: 10/08

2.2. Representation theorems for linear functionals

In concrete Banach spaces, the bounded linear functionals usually have a specific and useful form. Generally speaking, all linear functionals on function spaces
(such as Lp and C pK q) act by integration of the function (with respect to some
weight or measure). Similarly, all linear functionals on sequence spaces (such as `p
and c0 ) act by summation with weights.
2.2.1. Dual of a Hilbert space: Riesz representation theorem. We
start by characterizing bounded linear functionals on a Hilbert space X. The
following theorem says that every functional f acts as an inner product with some
vector in X.
Theorem 2.2.1 (Riesz representation theorem). Let X be a Hilbert space.
(i) For every y
(2.3)

P X, the function
f pxq  xx, y y,

xPX

is a bounded linear functional on X, and its norm is }f }  }y }.

(ii) Conversely, for every bounded linear functional f P X  there exists a unique
vector y P X such that (2.3) holds. Moreover, }f }  }y }.

2.2. REPRESENTATION THEOREMS FOR LINEAR FUNCTIONALS

43

Proof. (i) By Cauchy-Schwarz inequality,

|f pxq|  |xx, yy| }x} }y}, x P X.

Hence f is bounded, and }f } }y }. Conversely, for x  y we have
f pxq  xx, xy  }x}2 .
Hence }f } }y }.
(ii) Let f P X  . By Proposition 2.1.13, ker f is a hyperplane in X.

Since f is
bounded, ker f is closed (see Proposition 2.1.14). Therefore, X can be represented
by the orthogonal decomposition
X

 ker f ` Spanpy0 q

for some y0

P pker f qK .

(Use Theorem 1.5.7). Consider the map

g pxq : xx, y0 y,

We have g P X  by part (i). Moreover,

ker g

x P X.

 ty0 uK  ker f.

Therefore, by Proposition 2.1.13, the functionals f and g are equal up to some

constant factor a, that is f  ag. It follows that
f pxq  axx, y0 y  xx, ay0 y,

and the conclusion follows with y : ay0 .

The uniqueness of y is simple. (Why?)

Remark 2.2.2. In a concise form, the statement of Riesz representation theorem can be expressed as
X   X.

Although X  and X are formally different spaces, they can be canonically identified
as in Riesz representation theorem.
Let us rewrite Riesz representation theorem for the Hilbert space L2 .
Corollary 2.2.3 (L2  L2 ). Consider the space L2  L2 p, , q.
(i) For every weight function g P L2 , integration with weight
Gpf q :

f g d,

P L2

is a bounded linear functional on L2 , and its norm is }G}  }g }2  p |g |2 dq1{2 .

(ii) Conversely, every bounded linear functional G P L2 can be represented as
integration with weight for some unique weight function g P L2 . Moreover,
}G}  }g}2 .
2.2.2. Application: proof of Radon-Nikodym theorem. Riesz representation theorem can be used to give a soft proof of Radon-Nikodym theorem in
measure theory. This argument is due to von Neumann (1940).
Consider two measures , on the same -algebra. Recall that is called
absolutely continuous with respect to , abbreviated ! , if
pAq  0

for measurable sets A.

implies

pAq  0

2.2. REPRESENTATION THEOREMS FOR LINEAR FUNCTIONALS

44

Theorem 2.2.4 (Radon-Nikodym theorem). Consider two finite measures1 ,

such that ! . Then can be expressed as
pAq 

g d

for all measurable sets A,

where g 0 is a measurable function. Moreover, g is uniquely determined -a.e.;

it is called the Radon-Nikodym derivative and is denoted g : d {d.
Proof. We shall prove the existence part only.
The linear functional
F pf q :

f d

is a bounded linear functional on the space L2 pq, and therefore also on the space
L2 p q. By Riesz representation theorem, there exists h P L2 p q such that

(2.4)

f d 

f h d p

q 

f h d

f h d

for all f

P L2 p

q.

Rearranging the terms, we obtain

(2.5)

f h d

f p1
hq d for all f

P L2 p

q.

We claim that
(2.6)

0h1

-a.e.

Indeed, consider the set A 

th 0u and the indicator function f  1A . In this
case (2.4) becomes pAq  A h dp q 0, hence pAq  0. Similarly, consider
the set B  th 1u and the indicator function f  1B . If pB q 0 then (2.4)
becomes pB q  B h dp q p qpB q pB q, a contradiction. This proves
(2.6).
Since ! , we moreover have
(2.7)

0h1

q-a.e.

By Monotone Convergence Theorem, one can show that (2.5) holds for arbitrary
q-measurable functions f such that f 0 p q-a.e. (Indeed, consider the
truncation fn ptq : minpf ptq, nq and let n 8.) The convention is that if one side
of (2.5) is infinite then the other is infinite, too.
Now, given a measurable set A, we choose f so that f h  1A . In other words,
we consider
1A
f :
h

and apply the identity in (2.5). We obtain

pAq 

1
h
d.
h

The proof is complete with g : p1
hq{h.

2.2. REPRESENTATION THEOREMS FOR LINEAR FUNCTIONALS

45

Lec.15: 10/11

2.2.3. The dual of Lp . A version of the representation theorem for L2 , Corollary 2.2.3, holds in fact for all Lp spaces. In short, it states that Lp  Lp1 where p
and p1 are conjugate exponents as in Holders inequality, i.e.
1
1
 1, 1 p, p1 8,
p p1
and p1  8 if p  1. The rigorous statement is the following:

Theorem 2.2.5 (Lp  Lp1 ). Consider the space Lp  Lp p, , q with finite

of -finite measure , and where 1 p 8. Let p1 be the conjugate exponent of p.
(i) For every weight function g P Lp1 , integration with weight
Gpf q :

f g d,

P Lp

is a bounded linear functional on Lp , and its norm is }G}  }g }p1 .

(ii) Conversely, every bounded linear functional G P Lp can be represented as
integration with weight for some unique weight function g P Lp1 . Moreover,
}G}  }g}p1 .
We shall prove Theorem 2.2.5 for the particular case of spaces `p . The general
case is somewhat similar, and it relies on an application of Radon-Nykodim theorem.
For the spaces `p , Theorem 2.2.5 reads as follows:
Corollary 2.2.6 (`p  `p1 ). Let 1 p
exponent of p.
(i) For every y P `p1 , summation with weight
Gpxq :

xk yk ,

and let p1 be the conjugate

x P `p

k 1

is a bounded linear functional on `p , and its norm is }G}  }y }p1 .

(ii) Conversely, every bounded linear functional G P `p can be represented as
summation with weight for some unique weight y P `p1 . Moreover, }G}  }y }p1 .
p1

Proof. We will only prove the case where 1

 8 is an exercise.

(i) By H
olders inequality, we have


|Gpxq|  

p, p1 8;

the case p

 1,




xk yk  }x}p }y }p1 .

It follows that F is a bounded linear functional on `p , and }G} }y }p1 . To prove

the converse inequality, note that Holders inequality is sharp. Namely, for every
y P `p1 there exists x P `p such that


|Gpxq|  


xk yk   }x}p }y }p1 .

Indeed, one can check that this holds for x  pxk q defined as
1
xk  e
i Argpyk q |yk |p
1 .

For this x, it follows that }G} }y }p1 , so part (i) of the theorem is proved.
1The result can be extended (by decomposition) to -finite measures.

2.2. REPRESENTATION THEOREMS FOR LINEAR FUNCTIONALS

46

(ii) Consider G P `p . Let ek denote as usual the coordinate vectors in `p ,

that is ek  p0, . . . , 0, 1, 0, . . .q with k-th coordinate equal 1. By the linearity and
continuity of G we have

8

Gpxq  G

xk ek

xk Gpek q,

x  pxk q P `p .

k 1

k 1

We claim that the conclusion of (ii) follows for

Obviously, Gpxq 
y pnq :

yk ek

 pyk q : pGpek qq.

xk yk as required. To prove that y

and xpnq :

P `p1 , consider

1
e
i Argpyk q |yk |p
1 ek ,

n  1, 2, . . .

k 1

k 1

Then using the equality case of Holders inequality, we see that

n


}G}}xpnq }p |Gpxpnq q|  


xk yk   }xpnq }p }y pnq }p1 .

k 1

Canceling }xpnq }p on both sides, we conclude that

n
1{p1


|yk |p1
}ypnq }p1 

}G}.

k 1

Letting n 8 we conclude that y P `p1 and that }y }p1

actually have }y }p1  }G}. This completes the proof.

}G}.

By part (i), we


Exercise 2.2.7. [c0  `1 ] Prove that c0  `1 . The meaning of this is

the same as in Corollary 2.2.6, i.e. the functionals on c0 are given by
summation with weight from `1 .
2.2.4. The dual of C pK q. Finally, we state without proof the following characterization of bounded linear functionals on C pK q.
Theorem 2.2.8 (C pK q ). Consider the space C pK q where K is a compact
topological space.
(i) For every Borel regular signed measure2 on K, integration
Gpf q :

f d,

P C pK q

is a bounded linear functional on C pK q, and its norm is the total variation

}G}  ||pK q.
(ii) Conversely, every bounded linear functional G P C pK q can be represented
as integration with respect to a unique Borel regular signed measure on K.
Moreover, }G}  ||pK q.

2A signed measure is an extension of the notion of measure by allowing it to take on

negative values. By Hahn decomposition theorem, a signed measure can be uniquely represented
as

where and
are measures. The measure

is called the variation

of , and the value K is called the total variation of .

| |p q

| |

2.3. HAHN-BANACH THEOREM

47

2.2.5. Additional Exercises.

Exercise 2.2.9. [Span of point evaluation functionals] Compute the
norm of the following linear functional on C r0, 1s:
F pf q 

ak f ptk q

k 1

where ak are fixed scalars, and t1 , . . . , tn are fixed distinct points in r0, 1s.
Deduce that the linear span of point evaluation functionals t1 , . . . , tn
in C r0, 1s is isometric to `n1 .
2.3. Hahn-Banach theorem

Lec.16: 10/13

Hahn-Banach theorem allows one to extend continuous linear functionals f from

a subspace to the whole normed space, while preserving the continuity of f . HahnBanach theorem is a major tool in functional analysis. Together with its variants
and consequences, this result has applications in various areas of mathematics,
computer science, economics and engineering.
Let X be a normed space, and let X0 be a subspace of X. Consider a bounded
linear functional f0 defined on X0 , i.e. f0 P X0 . An extension of f0 to the whole
space X is a bounded linear functional f P X  whose restriction on X0 coincides
with f0 , i.e.
f |X0

 f0 |X ,
0

meaning that f0 px0 q  f pxq for all x0

P X0 .

Constructing extensions is a nontrivial problem because of the continuity (=boundedness) requirement for f .
Exercise 2.3.1. Show that if one does not require continuity of f , one
can construct f using a Hamel basis.
2.3.1. Extension by continuity. Before we state Hahn-Banach theorem, let
us address the simpler problem of extending a continuous linear functional from a
dense subspace to the whole space.
Proposition 2.3.2 (Extension by continuity). Let X0 be a dense subspace of a
normed space X. Then every functional f0 P X0 admits a unique extension f P X  .
Moreover, }f }  }f0 }.
Proof. Let x P X be arbitrary. By density, we can find a sequence pxn q X0
such that
xn x.
Then pf0 pxn qq is a Cauchy sequence, since

|f0 pxn q
f0 pxm q| }f0 } }xn
xm } 0

as n, m 8.

By completeness of R or C, the sequence converges. So we define

f pxq : lim f0 pxn q
n

and claim that f satisfies the conclusion of the proposition.

First of all, f is well defined, i.e. it depends only on x and not on the choice of
an approximating sequence pxn q. Indeed, if x1n x then

|f0 pxn q
f0 px1n q| }f0 } }xn
x1n } 0

2.3. HAHN-BANACH THEOREM

48

since the limits of xn and x1n coincide.

Next, f is a linear functional on X, since for xn

x and yn y we have
f pax by q  lim f0 paxn byn q  a lim f0 pxn q b lim f0 pyn q  af pxq bf py q.
n
n
n
Finally, f is a bounded linear functional. Indeed, for xn x we have
|f pxq|  lim
|f0 pxn q| }f0 } lim
}xn }  }f0 } }x}.
n
n
This shows that f P X  and }f } }f0 }. Note that the reverse inequality }f } }f0 }

trivially holds (for any extension f ). This completes the proof.

Example 2.3.3 (Lebesgue integral as an extension of Riemann integral). Let

X0 be the space of continuous functions C r0, 1s equipped with the norm

}f } 

1
0

|f ptq| dt

The Riemann integration

F0 pf q 

(Riemann integral).
1
0

f ptq dt

is clearly a bounded linear functional on X0 . Let X denote the completion of

C r0, 1s. One can define Lebesgue integral as the extension F P X  of F0 , and one
can define L1 r0, 1s : X.
Exercise 2.3.4. Check that this indeed gives an equivalent definition
of Lebesgue integral and of the space L1 r0, 1s.
2.3.2. Hahn-Banach theorem. Now we address the more difficult problem
of extending linear funcitonals from arbitrary subspaces. The result is the same as
for dense subspaces, except the extensions need not be unique.
Theorem 2.3.5 (Hahn-Banach theorem). Let X0 be a subspace of a normed
space X. Then every functional f0 P X0 admits an extension f P X  such that
}f }  }f0 }.
Proof. We will prove this result for spaces over R only.
Step 1. Extension by one dimension. Let us first assume that X0 is a hyperplane
in X, i.e. codim X0  1. Fix a vector z P X zX0 . Every vector x P X is then
uniquely represented as
x  az

Then an extension f has the form

f pxq  af pz q

x0 ,

a P R, x0

P X0 .

f px0 q  af pz q

f0 px0 q.

So f is determined by just one number

f pz q : C.

Without loss of generality we can assume that }f0 }  1 (by rescaling). We are
looking for an extension f such that }f }  1, which means that

|f pxq| }x},

This requirement is equivalent to

f pxq }x},

x P X.

x P X.

2.3. HAHN-BANACH THEOREM

49

(Why?) Expressing x and f as above, we write this equivalently as

(2.8)

f0 px0 q }az

aC

x0 },

a P R, x0

P X0 .

The desired extension f will be constructed if we are able to show that this inequality has solutions in C.
For a  0, inequality (2.8) trivially holds by the assumption that }f0 }  1. For
a 0, inequality (2.8) can be written as

x
x0 
0
,

f0
a
a

z

a P R , x0

P X0 .

For a 
b 0, inequality (2.8) can be written as

f0 xb1
 xb1
z, b P R , x1 P X1 .

Existence of a number C P R that satisfies both these inequalities is equivalent to
C

the inequality

sup

f0

b R , x 1 X1

x
1

 xb1
z aPR inf

, x PX
0



z

x
x0 
0
.

f0
a
a

Collecting the similar terms we see that this in turn is equivalent to the inequality
f0

x

x1 
z
b

x0 

a


 x1


z,

a, b P R ; x0 , x1

P X0 .

But this inequality is true; it follows from the fact that }f0 }  1, i.e.
f0

x

x1  x0
 a
b

x1 
,
b

and from the triangle inequality.

Step 2. Transfinite induction. This standard argument relies on Zorns Lemma 1.1.7
in a similar way to the proof of Proposition 1.1.6 on the existence of Hamel bases.
We consider the set of all extensions of f0 , precisely
(

Y X a subspace, g P Y  an extension of f0 , }g}  }f0 } .

The proof will be complete if we show that contains an element pY, g q with Y  X.


pY, gq :

X0

We introduce a partial order on by

pY1 , g1 q pY2 , g2 q if Y1 Y2 , g2 is an extension of g1 .


Every chain pY , g q has an upper bound pY, g q in , namely

Y :
Y , g pxq : g pxq if x P Y .

(Why is g a well defined bounded linear functional on Y ?) Therefore, by Zorns

Lemma 1.1.7, there exists a maximal element pY, g q in .
We claim that Y  X. Indeed, otherwise by Step 1 we could extend g to a
subspace larger than Y , which would contradict the maximality of pY, g q in . The
proof is complete.

Exercise 2.3.6. Deduce Hahn-Banach theorem for spaces X over C.

2.3. HAHN-BANACH THEOREM

50

Lec.17: 10/15

2.3.3. Supporting functionals. Hahn-Banach theorem has a variety of consequences, both analytic and geometric. One of the basic tools guaranteed by
Hahn-Banach theorem is the existence of a supporting functional f P X  for every
vector x P X.
Proposition 2.3.7 (Supporting functional). Let X be a normed space. For
every x P X there exists f P X  such that

}f }  1,

f pxq  }x}.

The functional f is called the supporting functional of x.

Proof. Consider the one-dimensional subspace X0
functional f0 P X0 by
f0 ptxq  t}x}, t P R or C.

Then }f0 }  1. An extension f

clearly satisfies the conclusion.

 Spanpxq, and define a

P X  of f0 guaranteed by Hahn-Banach theorem

Exercise 2.3.8. Consider a normed space X  pR2 , }  }q and a unit

vector x0 P X. Let f be a supporting functional of x0 . Interpret geometrically the level set tx : f pxq  1u as a tangent hyperplane for the unit
ball BX at point x0 .
Construct an example of a normed space for which the supporting
functional of x is not unique.

P X  is defined as
}f }  sup |f}pxx}q|
x0

Recall that the norm of a functional f

Generally it is not true that every functional f attains its norm on some vector x,
i.e. that the supremum above can be replaced by the maximum.
Exercise 2.3.9. Construct a bounded linear functional on C r0, 1s which
does not attain its norm.
However, every vector x does attain its norm on some functional f P X  ,
namely the supporting functional. This immediately follows from Proposition 2.3.7:
Corollary 2.3.10. For every vector x in a normed space X, one has

|f pxq|
}x}  max
f 0 }f }

where the maximum is taken over all non-zero functionals f

P X .

Hahn-Banach theorem implies that there are enough bounded linear functionals

P X  on every space X. One manifestation of this is the following:

Corollary 2.3.11 (X  separates the points of X). For every two vectors x1 
x2 in a normed space X, there exists a functional f P X  such that f px1 q  f px2 q.
Proof. The supporting functional f P X  of the vector x
satisfy f px1
x2 q  }x1
x2 }  0, as required.

 x1
x2

must


2.3. HAHN-BANACH THEOREM

51

2.3.4. Second dual space. Let X be a normed space as usual. The functionals f  are designed to act on vectors x P X via f : x f pxq. Vice versa, we
can say that vectors x P X act on functionals f P X  via
(2.9)

x:f

f pxq,

P X .

Thus a vector x P X can itself be considered as a function from X  to R.

This function (2.9) is clearly linear, so we may consider x as a linear functional
on X  . Also, the inequality

|f pxq| }x} }f }
shows that this functional is bounded, so
x P X 
and the norm of x as a functional is }x}X 
functional f P X  of x we see that actually

}x}.

Considering the supporting

}x}X   }x}.
We demonstrated that there exists a canonical embedding of X into into X  .
We summarize this as follows.
Theorem 2.3.12 (Second dual space). Let X be a normed space. Then X can
be considered as a linear subspace of X  . For this, a vector x P X is considered
as a bounded linear functional on X  via the action
x:f

f pxq,

P X .

Example 2.3.13. As we know from Section 2.2.3, c0

c
0

 `1 while `1  `8 , so

 `8 .

The space c0 of sequences converging to zero is indeed canonically embedded into

the larger space `8 of all bounded sequences (and with the same sup-norm).
Definition 2.3.14 (Reflexive spaces). A normed space X is called reflexive if
X   X (under the canonical embedding).
Example 2.3.15. As we know from Section 2.2.3, pLp q  Lp1 where 1 p 8
and p1 is the conjugate index of p. Therefore, Lp is a reflexive space for 1 p 8.
One can show that L1 and L8 are not reflexive spaces.
Proposition 2.3.16. Let X be a reflexive space. Then every functional f
attains its norm on X.
Proof. By reflexivity, the supporting functional of f is a vector x P X 
thus }x}  1 and f pxq  }f }, as required.

P X

 X,

The converse of Proposition 2.3.16 is also true. If every functional f P X  on

a Banach space X attains its norm, then X is reflexive. This is James theorem
(1971).

2.3. HAHN-BANACH THEOREM

52

2.3.5. Hahn-Banach theorem for sublinear functions. A quick inspection of the proof of Hahn-Banach theorem in Section 2.3.2 reveals that we have not
used all the norm axioms there. We used just these two positive homogeneity
and triangle inequality, precisely
(i) }ax}  a}x} for all x P X, a 0;
(ii) }x y } }x} }y } for all x, y P X.
Definition 2.3.17 (Sublinear function). Let X be a linear space. Functions

}  } : X r0, 8q that satisfy (i) and (ii) above are called sublinear functions.

It is an exercise to check that our proof of Hahn-Banach theorem is valid not

only for norms but for sublinear functions:
Theorem 2.3.18 (Hahn-Banach theorem for sublinear functions). Let X0 be a
subspace of a linear vector space X. Let }} be a sublinear function on X. Consider
a linear functional f on X such that
f0 pxq }x}

for all x P X0 .

Then f0 admits an extension f to the whole space X such that

f pxq }x}

for all x P X.

Exercise 2.3.19. Prove Theorem 2.3.18 by modifying the proof of

Hahn-Banach Theorem.
Lec.18: 10/20

Considering sublinear functionals instead of norms offers us more flexibility

in geometric applications. Sublinear functionals arise as Minkowski functionals of
convex sets.
Definition 2.3.20. A subset K of a linear vector space X is called absorbing
if
X
where tK

tK

t 0

 ttx : x P K u.

Exercise 2.3.21. Let K be a subset of a normed space X such that

0 P K. Then K is an absorbing set.

Proposition 2.3.22 (Minkowski functional). Let K be a absorbing convex subset of a linear vector space X such that 0 P K. Then Minkowski functional

}x}K  inf t 0 : x{t P K

is a sublinear functional on X.
Conversely, let }  } be a sublinear functional on a linear vector space X. Then
the sub-level set
K  tx P X : }x} 1u

is an absorbing convex set, and 0 P K.

Exercise 2.3.23. Prove Proposition 2.3.22.

Proposition 2.3.22 should be compared to a similar result for norms that we
proved earlier in Exercise 1.2.22.

2.3. HAHN-BANACH THEOREM

53

2.3.6. Separation of convex sets. Hahn-Banach theorem has some remarkable geometric implications, which are grouped together under the name of separation theorems. Under some mild topological requirements, these results guarantee
that two convex sets A, B can always be separated by a hyperplane. As we know
from Section 2.1.3, the hyperplanes correspond to the level sets of linear functionals
f . Therefore, we expect that a separation theorem for A, B would give us a linear
functional f and a number C such that
f paq C

f pbq,

a P A, b P B.

In this case, the sets A and B get separated by the hyperplane tx : f pxq  C u.
Let us start from the simpler case when one of the two sets is a point.
Theorem 2.3.24 (Separating a point from a convex set). Let K be an open
convex subset of a normed space X, and consider a point x0 R K. Then there exists
a functional f P X  , f  0, such that
f p xq f p x0 q ,

x P K.

Proof. Translating K if necessary, we can assume without loss of generality

that 0 P K. (Why?) By Exercise 2.3.21, K is an absorbing set. Therefore, by
Proposition 2.3.22, Minkowski functional }}K of K is a sublinear functional on X.
Since 0 P K and K is open, it contains a centered ball BX p0, rq for some radius
r 0 (see the figure). The set inclusion BX p0, rq K implies the inequality for
the Minkowski functionals:
1
}x} }x}K , x P K.
r
(Why?)

Figure 2.2. Separation of a point x0 from the set K by a functional f

Consider the one-dimensional subspace
X0

 Spanpx0 q

and define a linear functional f0 on X0 by

f0 ptxq  t}x},

t P R.

2.3. HAHN-BANACH THEOREM

54

}  }K on X0 , since for t 0 we have

f0 ptx0 q  }tx0 }K ;
f0 p
tx0 q 
tf px0 q 
t}x0 }K 0 }tx0 }K .

Then f0 is dominated by

By Hahn-Banach theorem for sublinear functionals (Theorem 2.3.18), f0 admits an

extension f onto the whole space X such that the domination is preserved, i.e.
f pxq }x}K ,

x P X.

To finish the proof, we need to check that f is bounded and that it separates
x0 from K as required. The boundedness follows from the inequality
1
f pxq }x}K }x}, x P X,
r
so f P X  . To check the separation, consider x P K. Since x0 R K, we have
f pxq }x}K

1 }x0 }  f0 px0 q  f px0 q.

This completes the proof.

Theorem 2.3.25 (Separation of open convex sets). Let A, B be disjoint open

convex subsets of a normed space X.
(i) Assume that A is open. Then there exists a functional f P X  and a number
C P R such that
f paq C

f pbq,

a P A, b P B.

(ii) If both A and B are open, then the stronger inequality holds:
f paq C

f pbq,

a P A, b P B.

Proof. (i) Consider the Minkowski difference set

K

 A
B : ta
b : a P A, b P B u.

The set K is open and convex. (Check!) Since A and B are disjoint, 0 P K.
Using Theorem 2.3.24, we obtain a functional f P X  , f  0 such that
f pa
bq f p0q  0,

a P A, b P B.

Hence f paq f pbq for all a P A, b P B, so letting C : supaPA f paq we obtain

f paq C

f pbq,

a P A, b P B.

Since A is open, by considering a small neighborhood of a in A (check!) we obtain

the strict inequality
f paq C f pbq, a P A, b P B.
(ii) This part is similar, and follows by considering small neighborhoods of a in
A and of b in B.

Exercise 2.3.26. Fill in the details in the proof of Theorem 2.3.25.
Corollary 2.3.27 (Separation of closed convex sets). Let A, B be disjoint
closed convex subsets of a normed space X. Assume B is compact. Then there
exists a functional f P X  and a number C P R such that
sup f paq inf f pbq.

a A

b B

2.3. HAHN-BANACH THEOREM

Proof. Let
r

55

 distpA, B q : aPA,infbPB }a
b}.

By the assumptions, r 0. (Why?) Therefore, the open r{3-neighborhoods Ar{3

of A and Br{3 of B are disjoint, open and convex sets. Applying Theorem 2.3.25,
we obtain a functional f P X  that separates the neighborhoods:
sup f paq inf f pbq.
bPBr{3

a A r {3

From this the conclusion easily follows. (How?)

Exercise 2.3.28. Fill in the details in the proof of Corollary 2.3.27.

Remark 2.3.29. Suppose that in Theorem 2.3.24, the set K is either open (i.e.
as stated) or closed. Then the strict separation holds:
f pxq f px0 q,

x P K.

Indeed, for open sets this follows from Theorem 2.3.25, while for closed sets this
follows from Corollary 2.3.27.
2.3.7. Convex sets are intersections of half-spaces.
Corollary 2.3.30. Every closed convex subset K of a normed space X is the
intersection of all (closed) half-spaces that contain K.
Recall that the half-space is what lies on one side of a hyperplane; therefore
half-spaces have the form
tx P X : f pxq au

for some f P X , a P R. See the picture illustrating Corollary 2.3.30.

Figure 2.3. Convex set K is the intersection of half-spaces

Proof. K is trivially contained in the intersection of the half-spaces that contain K. To prove the reverse inclusion, choose a point x0 R K and use Separation
Theorem 2.3.27 for A  K and B  tx0 u. We thus obtain a functional f P X 
such that
a : sup f pxq f px0 q.
It follows that the half-space tx
completes the proof.

x K

PX

: f pxq

au contains K

but not x0 . This



2.4. BOUNDED LINEAR OPERATORS

56

2.3.8. Additional exercises.

Exercise 2.3.31. [Closed convex sets that can not be strictly separated] Show that the compactness assumption in Corollary 2.3.27 is
essential. Construct two closed convex sets on the plane R2 that can not
be strictly separated.
Exercise 2.3.32. [Convex sets that can not be separated] Show that
the openness assumption in Theorem 2.3.25 is essential.
To this end, consider the linear space P of all polynomials in one
variable and with real coefficients. Let the subset A consist of polynomials with negative leading coefficient, and let the subset B consists of
polynomials with all non-negative coefficients. Show that A and B are
disjoint convex subsets of P, and that there does not exist a nonzero
linear functional f on P such that
f paq f pbq

for all a P A, b P B.

(Hint: assume that for some C P R one has f paq C f pbq, a P A, b P B;

deduce from 0 P B that C 0 and by considering monomials that C 0.
Exercise 2.3.33. [Functionals that annihilate a subspace] Let X0 be a
closed subspace of a normed space X. Prove that there exists a functional
f P X  such that
f pxq  0 for all x P X0 .
You may deduce this from Hahn-Banach theorem directly or from a
separation theorem.
2.4. Bounded linear operators
In this section, we shall study continuous linear operators T : X Y between
normed spaces X, Y . Linear functionals can be seen as a particular case of linear
operators with Y being the scalar field, R or C. So several results for linear operators
will be generalizations of those we have already seen for linear functionals; there will
be important differences though (e.g. a natural extension Hahn-Banach theorem
fails for linear operators).
2.4.1. Operator norm. Continuity and boundedness. The notions of
boundedness, continuity and norm of linear operators are similar to those for linear
functionals given in Section 2.1.2:
Definition 2.4.1 (Operator norm). A linear operator T : X Y acting
between normed spaces X and Y is called bounded if there exists a number C such
that
}T x} C }x} for all x P X.
The norm of T is the smallest constant C in this inequality, so it is defined as
}T }  sup }}Txx}}  sup }T x}.
x0
}x}1
Remark 2.4.2. We always have the inequality

}T x} }T } }x},

x P X.

Lec.19: 10/22

2.4. BOUNDED LINEAR OPERATORS

57

Exercise 2.4.3. [Composition of operators] If T : X Y and S : Y

are bounded linear operators between normed spaces, then ST : X
is a bounded linear operator, and

Z
Z

}ST } }T } }S }.
A version of Proposition 2.1.7 for linear functionals holds for linear operators,
and with a similar proof:
Proposition 2.4.4. Continuity and boundedness of linear operators are equivalent.
Exercise 2.4.5. Prove that a linear operator T : X Y is bounded
if and only if it maps sequences that converge to zero to bounded sequences.
2.4.2. Space of operators. Let X and Y be normed spaces. The space of all
bounded linear operators T : X Y equipped with the operator norm is denoted
LpX, Y q.
As an example, the dual space is a space of operators that map to scalars, i.e.
X

 LpX, Rq.

Proposition 2.4.6. LpX, Y q is a normed space. Moreover, if Y is a Banach

space then LpX, Y q is also a Banach space. In particular, the dual space X  is
always a Banach space, even if X is incomplete.
Proof. The norm axioms are straightforward. (Check!) To prove completeness, let Tn P LpX, Y q be a Cauchy sequence, i.e.

}Tn
Tm } 0, n, m 8.
Applying this to an arbitrary x P X and noting that
}Tn x
Tn x} }Tn
Tm } }x} 0, n, m 8
we see that pTn xq is a Cauchy sequence in Y . By the completeness of Y
Define the map T as

it converges.

T x : lim Tn x.
n

We claim that T is the limit of Tn in LpX, Y q, which would complete the proof.
It is easy to check that T : X Y is a linear operator. (Check!)
To show that T is bounded, we choose an arbitrary x P X and use continuity
of the norm:
}T x}  lim }Tn x} sup }Tn }}x}.
n

Since a Cauchy sequence is always bounded (why?), supn }Tn } 8. It follows that
T is a bounded linear operator, i.e. T P LpX, Y q.
It remains to show that Tn T in LpX, Y q, i.e. in the operator norm. Since
T is Cauchy, for every 0 there exists a N P N such that

}Tn
Tm } for n, m N.
Applying this to an arbitrary x P X we obtain
}Tn x
Tm x} }x} for n, m N.

2.4. BOUNDED LINEAR OPERATORS

Letting m 8, we get

}Tn x
T x} }x}

58

for n N.

Since x is arbitrary it follows that

}Tn
T } for n N.
This means that Tn T in LpX, Y q as required.

2.4.3. Operators on the finite-dimensional Euclidean space. Let X 

Y  `n2 ; recall that `n2 is the Euclidean space pRn , }  }2 q. As we know from linear
algebra, T can be identified with its n  n matrix ptij q, where
tij

 xT ej , ei y,

i, j

 1, . . . , n

and where pei q denotes the canonical basis of `n2 . This way, the i-th coordinate of
the vector T x can be computed as

pT xqi 

(2.10)

tij xj .

j 1

Proposition 2.4.7. Every linear operator T : `n2

}T } }T }HS

`n2 is bounded. Specifically,

where }T }HS denotes the Hilbert-Schmidt norm of T defined as

}T }HS 

n

|tij |2

1{2

i,j 1

Recall that we already encountered Hilbert-Schmidt (or Frobenius) norm of

matrices in (1.12).
Proof. Using (2.10) and applying Cauchy-Schwarz inequality, we obtain

}T x}22 

|pT xqi |2 

i 1

n 
n
2



tij xj 


i 1 j 1

n

n

i 1

|tij |2

n

j 1

The claim follows.

|xj |2  }T }2HS }x}22 .

j 1

2.4.4. Hilbert-Schmidt integral operators. A similar construction in function spaces L2 leads to the notion of Hilbert-Schmidt integral operators. To this
end, consider a function k pt, sq P L2 pr0, 1s2 q which we call the kernel. Define a
linear operator T : L2 r0, 1s L2 r0, 1s as
(2.11)

pT f qptq 

1
0

k pt, sqf psq ds.

We can view this definition as a continuous version of (2.10), where kernel k pt, sq
can be considered as a continuous version of matrix. The operator T defined this
way is called Hilbert-Schmidt integral operator with kernel k pt, sq.
Proposition 2.4.8. A Hilbert-Schmidt integral operator T : L2 r0, 1s L2 r0, 1s
with kernel k pt, sq P L2 pr0, 1s2 q is bounded. Specifically,

}T } }k}2 .

2.4. BOUNDED LINEAR OPERATORS

59

Proof. Our argument is a continuous version of the proof of Proposition 2.4.7.

Applying Cauchy-Schwarz and Fubini inequalities, we obtain
1

2
 1


k t, s f s ds
dt 

}T f } 
p qpq
0
0
 }k}22 }f }22 .
2
2

1

dt
0

|kpt, sq|

1

ds
0

|f psq|2 ds

The claim follows.

Exercise 2.4.9. Why is pT f qptq defined for almost all t?

Remark 2.4.10 (Fredholm integral equations). Hilbert-Schmidt integral operators arise from the study of Fredholm integral equations. The so-called homogeneous
Fredholm equation of the first kind is
g ptq 

1
0

k pt, sqf psq ds.

Given a kernel k pt, sq and the left hand side g ptq, the problem is to find the function
f psq.
Fredholm equations can be written as
Tf

g

where T is the corresponding Hilbert-Schmidt operator. Therefore, Fredholm equations are linear equations. They can be thought of as continuous versions of matrix
linear equations Ax  b, where A is an m  n matrix, b P Rm and x P Rn .
Remark 2.4.11. The particular measure space r0, 1s does not play any role in
the discussion above, and can be replaced with an arbitrary measure space.

2.4.5. Volterra operator and differential operators. Volterra operator

T : L2 r0, 1s L2 r0, 1s is defined as
(2.12)

pT f qptq 

t
0

f psq ds.

By Lebesgue differentiation theorem, T f is the antiderivative of f .

Volterra operator is clearly a particular case of Hilbert-Schmidt integral operator with kernel k pt, sq  1tstu , and is therefore a bounded linear operator.
Exercise 2.4.12. Compute the norm of Volterra operator.
One would naturally think about the inverse of Volterra operator, which should
be the differential operator
Df  f 1 .
Clearly D can only defined on an appropriate dense subspace of L2 , such as C 1 ,
the space of continuously differentiable functions.
However, D is an unbounded linear operator on L2 r
, s.3 To see this, consider the Fourier basis pek qkPZ of L2 r
, s, which we defined in Example 1.6.3
as
1
ek ptq  ? eikt .
2
3The interval
interval.

r
, s is chosen only for convenience; a similar result holds for an arbitrary

Lec.20: 10/25

2.4. BOUNDED LINEAR OPERATORS

Differentiating yields
Dek
Hence

 pikqek ,

60

P Z.

}Dek }2  k}ek }2  k,

P Z.

Letting k 8 implies that D is an unbounded operator.

This simple example suggests that various differential operators are unbounded
even on good function spaces. They may be studied through their inverses, which
are bounded integral operators.
2.4.6. Orthogonal projections and partial Fourier series. Let X0 be a
closed subspace of a Hilbert space X. Consider the orthogonal projection P in X
onto X0 . As we know from Section 1.5.2, P : X X is a linear map. Since P x
and x
P x are orthogonal vectors, we have

}x}2  }P x}2 }x
P x}2 ,

so

}P x} }x},

x P X,

x P X.

We have shown:
Proposition 2.4.13. The orthogonal projection P in a Hilbert space X onto a
closed subspace X0 is a bounded linear operator.
Exercise 2.4.14. Show that }P }=1.
As an example, consider the n-th partial sum of the Fourier series of a function

P L2 r
, s:

Sn f


k


n

xf, ek yek

where ek denotes the Fourier basis of exponentials (1.16). As we know from

Lemma 1.6.10, Sn is an orthogonal projection onto Spanpek qnk
n . This subspace
is finite-dimensional and thus closed. (Why?) So by Proposition 2.4.13, Sn is a
bounded linear operator in L2 r
, s.
A convenient and classical way to represent Sn is via convolution with Dirichlet
kernel. Indeed,
(2.13)

pSn f qptq  21

Dn pt
sqf psq ds 

1
pDn  f qptq
2

where
(2.14)

Dn pq 
k


n

pn 2 q
 sinsin
1

eik

is called the Dirichlet kernel.

Exercise 2.4.15. Prove this identity.
We see that Sn acts as a convolution with Dirichlet kernel Dn . In particular,
1
Sn is a Hilbert-Schmidt operator with Dirichlet kernel k pt, sq  2
Dn pt
sq.

2.4. BOUNDED LINEAR OPERATORS

61

2.4.7. Isomorphisms, isometries. An important class of bounded linear

operators is formed by isomorphisms. Recall that a transformation T : X Y
between topological spaces X and Y is called an isomorphism if T is invertible
and both T , T
1 are continuous. Spaces X and Y are called isomorphic if there
exists an isomorphism between them. So a linear map T : X Y between normed
spaces X and Y is an isomorphism if and only if T is invertible and T P LpX, Y q,
T
1 P LpY, X q.
Exercise 2.4.16. Show that a surjective linear map T : X Y between
normed spaces is an isomorphism if and only if there exist C, c 0 such
that
c}x} }T x} C }x} for all x P X.

In other words, an isomorphism preserves all distances up to a multiplicative

factor C {c. An isometry is an isomorphism T such that }T }  }T
1 }  1, i.e.

}T x}  }x}

for all x P X.

Isometries preserve all distances exactly.

Example 2.4.17. Isometries on a Hilbert space are commonly called unitary
operators. As an example, in Section 1.6.7 we constructed an isometry T between
any pair of infinite-dimensional separable Hilbert spaces X and Y .
Example 2.4.18. The right shift Rpx1 , x2 , . . .q  p0, x1 , x2 , . . .q on any classical
sequence space considered in Example 1.1.19 is not an isometry because R is not
surjective, although }Rx}  }x}. The left shift Lpx1 , x2 , . . .q  px2 , x3 , . . .q is
obviously not invertible but is bounded, and }L}  1. (Why?)
Exercise 2.4.19. [Isomorphisms preserve vital structures in normed
spaces] Show that an isomorphism between normed spaces maps open
sets to open sets, closed sets to closed sets, convegrent sequences to
convergent sequences, complete spaces to complete spaces.
Unfortunately, most infinite-dimensional Banach spaces are not isomorphic to
each other. For example, A. Pelczynski proved that among the spaces c0 , Lp r0, 1s
and `q , 1 p, q 8, there are exactly two isomorphic ones, L2 r0, 1s and `2 , see
[8].
However, as we know all separable Hilbert spaces are isomorphic (even isometric) to each other. We will also see in Section 3.1.5 that all finite-dimensional
Banach spaces of same dimension are isomorphic (but not isometric to each other).
The inverse mapping theorem of Section 3.1.2 below is a powerful tool to construct
isomorphisms.
2.4.8. Extensions and projections. We will now address the extension
problems for bounded linear operators. Similar to Section 2.3, we consider a normed
space X and its subspace X0 Consider an operator T0 P LpX0 , Y q where Y is some
normed space. An extension of T0 to the whole space X is an operator T P LpX, Y q
whose restriction on X0 coincides with T0 , i.e.
T |X0

 T0 |X .
0

As we know from Section 2.3, every bounded linear functional can be extended
from either dense or closed subspace to the whole space. For dense subspaces, we

2.4. BOUNDED LINEAR OPERATORS

62

can extend by continuity, while for closed subspaces the extension is guaranteed by
Hahn-Banach theorem.
For general linear operators, extension by continuity holds with the same proof
as in Proposition 2.3.2:
Proposition 2.4.20 (Extension by continuity). Let X0 be a dense subspace of
a normed space X, and Y be a Banach space. Then every operator T0 P LpX0 , Y q
admits a unique extension T P LpX, Y q. Moreover, }T }  }T0 }.
Unfortunately, extension from a closed subspace is not always possible, and
Hahn-Banach theorem does not generalize to bounded linear operators. There is
a simple geometric description of the situations when such extensions are possible.
To state it we need a general notion of projections in normed space (not necessarily
orthogonal).
Definition 2.4.21 (Projection). Let X0 be a closed subspace of a normed
space X. An operator P P LpX, X q is called a projection in X onto X0 if
(i) P pX q X0 ;
(ii) P x  x for all x P X0 , i.e. P |X0  IX0 .
Example 2.4.22. Any orthogonal projection in a Hilbert space is clearly a
projection in this sense. However, even in a Hilbert space there is a plenty of
non-orthogonal projections. (Construct one in a two-dimensional space.)
The following observation characterizes the subspaces from which extensions of
linear operators are possible.
Proposition 2.4.23 (Extensions of operators and projections). Let X0 be a
closed subspace of a normed space X. Then the following are equivalent:
(i) There exists a projection in X onto X0 . In this case we say that X0 is a
complemented subspace of X;
(ii) For every normed space Y , every operator T0 P LpX0 , Y q admits an extension
T P LpX, Y q.
Proof. Assume P is a projection in X onto X0 . Then for every operator

T0

P LpX0 , Y q, the operator T : T0 P P LpX0 , Y q is an extension.

Vice versa, consider the identity map I : X0 X0 . Its extension P : X X0

is clearly a projection in X onto X0 .

Since every closed subspace in a Hilbert space is complemented, the extension

problem in Hilbert spaces always has a positive solution.
Unfortunately, in general normed spaces there may be uncomplemented subspaces. Here is a synopsis of some of the known results, without proof:
Theorem 2.4.24 (Complemented subspaces). In what follows, X0 stands for
a closed subspace of a Banach space X.
(i) If either dim X0 8 or codim X 8 then X0 is complemented in X.
(ii) c0 is not complemented in `8 [10].
(iii) Every complemented subspace X0 of X is isomorphic to X if X is one of the
spaces `p , p P r1, 8s or c0 . These results are due to Pelczynski and Lindenstrauss [9, 6].
(iv) Every Banach space that is not isomorphic to a Hilbert space has an uncomplemented subspace. This is a theorem of Lindenstrauss and Tzafriri [7].

2.4. BOUNDED LINEAR OPERATORS

63

(v) There exists a Banach space X without non-trivial complemented subspaces

X0 (i.e. such that dim X0  codim X0  8). This was proved in 1993 by
Gowers and Maurey [5].
(vi) If X0 is isomorphic to `8 then it is complemented in X.
2.4.9. Adjoint operators. The concept of adjoint operator is a generalization of matrix transpose in linear algebra. Recall that if A  paij q is an n  n
matrix with complex entries, then the Hermitian transpose of A is the n  n matrix
A  paji q. The transpose thus satisfies the identity

xA x, yy  xx, Ayy,

(2.15)

x, y

P Cn .

Now we would like to extend this to a general definition of the adjoint T  for a
linear operator T : X Y acting between normed spaces X and Y .
T

Definition 2.4.25 (Adjoint operator). Let T

P LpY  , X  q is defined by
pT  f qpxq : f pT xq,

P LpX, Y q. The adjoint operator

P Y  , x P X.

In order to see a similarity with (2.15), we adopt the following alternative

notation for the action of functionals, one that resembles the inner product:

xf, xy : f pxq,

(2.16)

P X ,

x P X.

Notice that if X is a Hilbert space, this notation agrees with the inner product by
Riesz representation theorem (up to complex conjugation). In general, xf, xy does
not define an inner product since the arguments are taken from different spaces.
Then the definition of the adjoint reads as

xT  f, xy  xf, T xy,

P Y , x P X

and we see that this is a general form of (2.15).

Remark 2.4.26 (Adjoint operators on Hilbert spaces). For operators T on a
Hilbert space X  Y  H, the definition (2.4.25) of the adjoint operator T  takes
place with x, y denoting the inner product on H. This makes a small difference
the inner product is conjugate linear in the second argument, while the action
of functionals (2.16) is just linear. So, for operators on a Hilbert space one has
paT q  aT  for a P C, while the general definition of adjoint for Banach spaces
incurs paT q  aT  .

One point has not been justified in Definition 2.4.25, why T  is a bounded
linear operator. We shall prove this now:

P LpX, Y q, we have T  P LpY  , X  q, and

}T  }  }T }.

Proposition 2.4.27. For every T

Proof. Denoting as usual SX the unit sphere of X, and using notation (2.16),
we have

}T  }  sup }T  f }X   sup sup |xT  f, xy|  sup sup |xf, T xy|

xPS f PS 
f PS 
f PS  xPS
 sup }T x}Y (choosing f as a supporting functional of T x)
xPS
 }T }
Y

as required.

Lec. 21: 10/27

2.4. BOUNDED LINEAR OPERATORS

64

Exercise 2.4.28. Let T be the Hilbert-Schmidt integral operator with

kernel k pt, sq. Show that T  is also the Hilbert-Schmidt integral operator
with kernel k ps, tq.
Exercise 2.4.29. Let R and L denote the right and left shift operators
on `2 . Prove that R  L.
Exercise 2.4.30. (i) Let T P LpX, Y q and S P LpY, Z q. Show that
pST q  T  S  .
(ii) Let S, T P LpX, Y q and a, b P C. Show that paS bT q  a
S  bT  .


1
(iii) Let T P LpX, Y q be such that T
P LpY, X q. Show that pT
1 q 



1
pT q .
The kernel and image of bounded linear operators are in a duality relation.
To state it, we consider a generalization of the notion of orthogonal complement,
which we studied in Section 1.5.1 for Hilbert spaces.
Definition 2.4.31 (Annihilator). An annihilator of a subset A of a normed
space X is the set AK X  defined as
AK

 tf P X  : xf, xy  0 for all x P Au.

Proposition 2.4.32 (Duality of kernel and image). Let T

P LpX, Y q. Then

pIm T qK  ker T  .
Proof. Let f P Y  . Then f P ker T  means that T  f  0, which is equivalent
to xT  f, xy  xf, T xy  0 for all x P X, which means that f P pIm T qK .

Corollary 2.4.33. Let H be a Hilbert space, and T
orthogonal decomposition holds:

P LpH, H q.

Then the

 Im T ` ker T  .
By Proposition 2.4.32, we have pIm T qK  pIm T qK  ker T  .
H

Proof.
does the first identity hold?) By Proposition 1.5.7, the proof is complete.

(Why


Exercise 2.4.34. [Duality of kernel and image II] For a subset A X  ,

define the pre-anihilator as
AK
Let T

P LpX, Y q.

 tx P X : xf, xy  0 for all f P Au.

Prove that
ker T

Deduce that

 pIm T  qK .

pker T qK Im T  .

Give an example of a linear operator such that pker T qK

 Im T  .

2.4. BOUNDED LINEAR OPERATORS

65

2.4.10. Application: ergodic theory. Ergodic theorems allow one to compute space averages as time averages. Let us first state and prove a preliminary
form of von Neumanns ergodic theorem; its interpretation will follow.
Theorem 2.4.35 (von Neumann ergodic theorem). Let U be a unitary operator
on a Hilbert space H. Let P denote the orthogonal projection onto the invariant
subspace tx P X : U x  xu. Then, for all x P H, we have
N
1
1 n
U x  P x.
N 8 N
n0

lim

Proof. It suffices to prove the result for x P ker P and for x P Im P , because
then the result will follow for all x P H by the orthogonal decomposition H 
ker P ` Im P . (Check!)
For x P Im P the result is trivial because in this case U n x  U x  x and
P x  x. So let x P ker P . We will first find a convenient representation of ker P .
By definition, Im P  kerpI
U q  kerpI
U  q because for unitary operators,
U x  x if and only if U  x  x. (Check!) Therefore, using the duality between
kernels and images, Corollary 2.4.33, we have
ker P

 pkerpI
U  qqK  ImpI
U q.

Therefore, every x P ker P can be approximated arbitrarily well by vectors of the

form pI
U qy. For x  pI
U qy, we arrive at a telescoping sum
N
1
1 n
1
U x  p x
U N xq 0
N n0
N

This carries over to vectors x

The proof is complete.

as N

8.

P ImpI
U q by a simple approximation argument.

Exercise 2.4.36. Write down the approximation argument in this

proof.
Now we explain the implications of Theorem 2.4.35 for time and space averages.
Consider first the simple example of a discrete dynamical system studied by Weyl.
Let T denote the unit circle. We put a particle on the circle, and consider its
consecutive rotations by some fixed angle P p0, 2 q. Equivalently, we consider the
sequence
n mod 2, n  0, 1, 2, . . .

If {2 is rational then this sequence has finitely many values. If {2 is irrational,

then the number of values is infinite. Moreover, in the latter case simulations
suggest that the values become uniformly distributed for large N on the circle, see
the picture.
This observation was formalized by the Weyls ergodic theorem, which can be
stated as follows. For every number P p0, 2 q such that {2 is irrational, and
every measurable subset A r0, 2 s, one has
mod 2 P Au|
N
where denotes the Lebesgue measure.

(2.17)

|tn N :

pAq
2

as N

8.

2.4. BOUNDED LINEAR OPERATORS

66

Figure 2.4. Rotations by angle are equidistributed on the circle

Theorem 2.4.35 implies a very general form of Weyls ergodic theorem, for an
arbitrary ergodic measure-preserving transformation of a probability space (instead
of an irrational rotation of the circle).
Definition 2.4.37 (Ergodic transformation). Let p, , q be a probability
space. A transformation T : is called measure-preserving if
pT
1 pAqq  pAq

for all measurable subsets A .4 A one-to-one, measure preserving transformation T is ergodic if the only functions f P L2 p, , q which satisfy f pT q  f p q
for almost all P are the constant functions.
Exercise 2.4.38. Show that T is ergodic if and only if for all measurable subset A , T
1 pAq  A implies pAq  0 or pAq  1.
Theorem 2.4.39. Consider a measure-preserving, ergodic transformation T on
a probability space p, , q. For every f P L2 p, , q one has

N
1
1
n
lim
f pT q 
f d
N 8 N

n0

(2.18)

where the convergence is in the L2 norm.

Proof. To see a connection with ergodic Theorem 2.4.35, define an operator
U : L2 p, , q L2 p, , q by
Uf

 f  T,

i.e.

pU f qpq  f pT q,

P .

Since T is measure-preserving, U is a unitary operator. (Check!) Since T is ergodic,

the invariant subspace of U is the space of constants. The orthogonal projection
P in L2 p, , q onto the subspace of constants is the integral in the right side of
(2.18) (see Exercise 1.5.11). The claim then follows from Theorem 2.4.35.

Weyls ergodic theorem (2.17) is a partial case of Theorem 2.4.39 for being
the unit circle, T being the rotation of the circle by an irrational angle (why is T
ergodic?) and for f  1A .

p q  t P :

4Here T 1 A

P Au is the preimage of A under T .

2.4. BOUNDED LINEAR OPERATORS

67

2.4.11. Additional Exercises.

Exercise 2.4.40. [Integral operators in C r0, 1s] Consider the integral
operator T defined by (2.11) with kernel k pt, sq P C pr0, 1s2 q. Then T maps
continuous functions into continuous functions, i.e. T : C r0, 1s C r0, 1s.
Compute the norm of T .
Exercise 2.4.41. [Schurs test] Assume that a kernel function k pt, sq
satisfies
1

sup

Pr s

t 0,1

sup

Pr s

s 0,1

|kpt, sq| ds : M1 8,
|kpt, sq| dt : M2 8.

Show that the intergal operator (2.11) T : L2 r0, 1s

k pt, sq is bounded, and5
a
}T } M1 M2 .

L2 r0, 1s with kernel

Exercise 2.4.42. [Multiplication operator] Consider a multiplier function k ptq P C r0, 1s, and define a linear operator T : C r0, 1s C r0, 1s by

pT f qptq  kptq f ptq.

Show that T is a bounded linear operator and compute its norm.

Exercise 2.4.43. [Annihilators] Let A, B be subsets of a normed space
X. Prove the following:
(i) AK is a closed linear subspace of X  .
(ii) If A B then AK B K .
(iii) pA Y B qK  AK X B K . Give an example where pA X B qK  AK Y B K .
(iv) AK  pSpan AqK .
(v) pAqK  AK . (This property was used in the proof of Corollary 2.4.33.)
(vi) Suppose X0 is a closed linear subspace of X. Then X0K  t0u is
equivalent to X0  X.
Exercise 2.4.44. [Isomorphisms] Let X be a Banach space. Show that
the isomorphisms on X form an open subset of LpX, X q, and that the
inversion map T T
1 is continuous on this subset.
Exercise 2.4.45. [Sesquilinear forms] Let H be a Hilbert space. A
sesquilinear form on H is a function B : H  H C which is linear in the
first argument and conjugate-linear in the second argument, i.e.
B pa1 x1

B px, b1 y1

a2 x2 , y q  a1 B px1 , y q
b2 y2 q  b1 B px, y1 q

a2 B px2 , y q,

b2 B px, y2 q.

An example of a sesquilinear form is B px, y q  xT x, y y where T

P LpH, H q.

r s r s

5In fact, under the conditions of Schurs test, T is bounded as an operator L 0, 1

Lp 0, 1
p
for all p
1, . Indeed, for p
1 and p
this is an exercise. The result for intermediate p
follows at once by Riesz-Thorin theorem (which we do not discuss here).

P r 8s

8

2.4. BOUNDED LINEAR OPERATORS

68

Consider a sesquilinear form B px, y q which satisfies

|B px, yq| M }x}}y},

x, y

PH

for some number M . Show that there exists an operator T

with }T } M and such that
B px, y q  xT x, y y

for all x, y

P H.

P LpH, H q

CHAPTER 3

Main principles of functional analysis

Lec. 22: 10/29

In this chapter we shall study the three theorems that, together with HahnBanach theorem, form the main principles of functional analysis. Those are the
open mapping theorem, the uniform boundedness principle, and the closed graph
theorem.
3.1. Open mapping theorem
3.1.1. Statement and proof. This result was proved by S. Banach.
Theorem 3.1.1 (Open mapping theorem). Let X, Y be Banach spaces. Then
every surjective linear operator T P LpX, Y q is an open map, i.e. T maps open sets
in X to open sets in Y .
The proof of the open mapping theorem relies on Baire category theorem, which
states that every complete metric space M is a set of second category, i.e. M can
not be represented as a countable union of nowhere dense sets. Recall that a subset
A M is nowhere dense set if there is no neighborhood in X on which A is dense.
Equivalently, A is nowhere dense if the interior of the closure of A is empty.
The open mapping theorem states that for every open set U X, every y P T U
is an interior point of T U . We claim that it suffices to show this for U being the
unit ball BX and for y  0:
Claim 3.1.2. To prove the open mapping theorem, it suffices to find 0 such
that
(3.1)

T BX

BY .

Proof of Claim. Let U X be open and choose y P T U . We find x

such that y  T x. Since U is open, there exists 0 such that
U

PU

BX .

Applying T to both sides and using (3.1), we conclude that

TU

T px

BX q  y

T BX

so y is an interior point of T U .

BY ,


We will now prove the Claim. In view of application of Baire category theorem,
we represent

X
nBX .

n N

Therefore
Y

 TX 

n N
69

nT BX .

3.1. OPEN MAPPING THEOREM

70

By Baire category theorem, there exists n P N such that nT BX is not a nowhere

dense set. Thus T BX is not a nowhere dense set, i.e. its closure has nonempty
interior. So there exist y P Y and 0 such that
T BX
By symmetry, T BX

y

BY .

BY . Hence by convexity (check this!) we have

T BX

BY .

As we see, we have almost proved the Claim, except for the closure. Unfortunately, in general K D does not imply K D even for convex and symmetric
sets K, D in a Banach space. (Give a counterexample!) However, this is true for
perfectly convex sets, defined as follows.
Definition 3.1.3 (Perfectly convex set). A set K in a Banach space Y is called
perfectly
convex if for every
sequence pxk q8
k1 and every numbers k 0 such that
8
8


1,
one
has

x
P
K.
k
k
k
k1
k 1
Convex sets satisfy this property only for finite sequences pxk q. (Why?) Therefore, every perfectly convex set is convex, but not vice versa. (Give an example.)
Lemma 3.1.4 (Perfectly convex sets). Let K be a perfectly convex set in a
Banach space Y . If K BY for some 0, then K 2 BY .
Proof. Assume B : BY
assumption clearly implies that

K; we would like to show that

1
2B

K.

The

1
B,
2
for the right side is the -neighborhood of K in Y . Iterating this inclusion gives
B

K
K

1

1
1
K
B K
K
2
2
2



1
1
1
K
K
B K
2
4
2

Therefore1

1
K
2
By perfect convexity (check!), we have
B

1
1
B K
2
2
This proves the lemma.

1
K
4

1
K
4
1
K
8

1
B
4
1
K
2
1
K
8
1
K
16

1
K
4

1
B
8




   K.


Exercise 3.1.5. Verify the steps of the above proof where we used
Minkowski sums and series of sets.
Now we are ready to complete the proof of the open mapping theorem. By
Lemma 3.1.4, it suffices to show that K  T BX is perfectly convex. This is easy to

1
All sums and series of sets are in Minkowski sense. The sum of sets is defined as k Ak 
t k ak : ak P Ak u. The same for infinite sums (series), where we insist on the convergence of

ak .

3.1. OPEN MAPPING THEOREM

check. Indeed,
consider any sequence pT xk q
k such that k k  1. Then

(3.2)
provided that the series

T

k T xk

T BX

71

with xk

P BX , and numbers

k xk ,

k k xk converges. It indeed converges absolutely:





k xk 

k

k }xk }

 1.

By completeness of X, the series k k xk converges to a vector in BX . It follows

that the right side of (3.2) belongs to T BX  K, as required. This completes the
proof of the open mapping theorem.

3.1.2. Inverse mapping theorem. As an immediate consequence of the
open mapping theorem, we obtain:
Theorem 3.1.6 (Inverse mapping theorem). Let X, Y be Banach spaces. Then
every bijective linear operator T P LpX, Y q is an isomorphism, i.e. T
1 P LpY, X q.
Proof. The open mapping theorem states that the preimages of open sets
under T
1 are open, hence T
1 is a continuous map.

The inverse mapping theorem is often used to establish stability of solutions of
linear equations. Consider a linear equation in x in Banach space:
Tx  b

(3.3)

with T P LpX, Y q and b P Y .

Assume a solution x exists and is unique for every right hand side b. Then,
by inverse mapping theorem, the solution x  xpbq is continuous with respect to b.
In other words, the solution is stable under perturbations of the right hand side of
(3.3).
In case T is not injective (but is surjective) in the inverse mapping theorem,
one can still apply inverse mapping theorem to the injectivization of T :
Corollary 3.1.7 (Surjective operators are essentially quotient maps). Let
X, Y be Banach spaces. Then every surjective linear operator T P LpX, Y q is a
composition of a quotient map and an isomorphism. Specifically,
T  Tq,
where q : X

X { ker T

is the quotient map, T : X { ker T

is an isomorphism.

Proof. Let T be the injectivization of T constructed in Example 1.1.24. By

construction, T  Tq and T is injective. Since T is surjective, T is also surjective.
Therefore, by inverse mapping theorem T is an isomorphism, completing the proof.

3.1.3. Equivalent norms. Sometimes one wants to consider several norms
on the same space, such as }  }8 and }  }1 on C r0, 1s. In this case one naturally
needs to compare them.
Definition 3.1.8 (Equivalent norms). Two norms }  } and ~  ~ on a linear
vector space E are called equivalent if there exist C, c 0 such that
c}x} ~x~ C }x}

for all x P E.

Lec. 23: 11/1

3.1. OPEN MAPPING THEOREM

72

}  } and ~  ~ are equivalent if the identity operator

I : pE, }  }q pE, ~  ~q

In other words, the norms

(3.4)
is an isomorphism.

Exercise 3.1.9. Show that two norms on a linear vector space E are
equivalent if and only if they generate the same topology on E.
Proposition 3.1.10 (Domination and equivalence of norms). Consider two
norms }} and ~~ on a linear vector space E such that E is complete with respect
to both norms. Suppose that one norm dominates the other, i.e. one can find C
such that
~x~ C }x} for all x P E.
Then the two norms are equivalent.
Proof. The claim follows from inverse mapping theorem applied to the identity map (3.4).

Proposition 3.1.10 indicates that it is difficult to construct different good norms
on the same space. Either the space will be incomplete or the norms need to be
incomparable. This is a simple way to prove incompleteness of various spaces.
As an example, consider the norms }  }1 and }  }8 on C r0, 1s. On the one
hand, }  }1 }  }8 . On the other hand, the norms are not equivalent one can
easily construct functions with }f }1  1 and }f }8 arbitrarily large. (Do this.) By
Proposition 3.1.10, C r0, 1s must be incomplete with respect to one of these norms.
Since it is complete with its natural norm }}8 , it follows that C r0, 1s is incomplete
with respect to }  }1 .
This argument is flexible and applies to a whole range of norms. It implies that
there is essentially only one natural norm on C r0, 1s, namely the sup-norm }  }8 .
Exercise 3.1.11. [Direct sums of normed spaces] Let X and Y be
normed spaces and p P r1, 8s. Define the direct sum of X `p Y as the
Cartesian product X  Y equipped with the norm

}px, yq} : p}x}p }y}p q1{p if p 8, }px, yq} : maxp}x}, }y}q if p  8.

Show that X `p Y is a normed space, and all norms }px, y q}p , p P r1, 8s,

are equivalent to each other.

For this reason, the index p is usually omitted from notation, and
the space X ` Y is called the direct sum of X and Y .
3.1.4. Isomorphic embeddings. As we know, the kernel of every operator
T P LpX, Y q is always a closed subspace. The image of T may or may not be closed.
The following result characterizes operators with closed images.
Proposition 3.1.12 (Isomorphic embeddings). Let T P LpX, Y q be an operator
between Banach spaces X and Y . Then the following are equivalent:
(i) T is an isomorphic embedding, i.e. T acts as an isomorphism between spaces
X and Im T Y ;
(ii) T is injective and it has closed image;
(iii) T is bounded below, i.e. one can find c 0 such that

}T x} c}x}

for all x P X.

3.1. OPEN MAPPING THEOREM

73

Proof. (i) (ii). Recall that isomorphisms preserve completeness of spaces.

Since X is complete, it follows that the subspace Im T Y is complete. By
Exercise 1.3.2, Im T is closed. Injectivity of T is an obvious consequence of the
isomorphic embedding property.
(ii) (iii). Considering T as an operator from X to Im T we see that T is
injective and surjective. By inverse mapping theorem, T is an isomorphism.
(iii) (i). We have
c}x} }T x} C }x}

where C

 }T }. It follows that T

for all x P X

is an isomorphic embedding. (How?)

3.1.5. Finite dimensional normed spaces. Finite dimensional normed spaces

are the simplest examples of Banach spaces. As we will see now, they are all complete, all isomorphic (but not isometric) to each other, all of their subspaces are
closed and all linear operators on them are bounded.
Theorem 3.1.13. Every n-dimensional normed space X is isomorphic to the
Euclidean space `n2 . Consequently, all n-dimensional normed spaces are isomorphic
to each other.
Proof. We construct an isomorphism between X and `n2 which that identifies
a basis of X with the canonical basis of `n2 . To this end, consider a basis u1 , . . . , un
of X, and define the operator T : `n2 X by
Tx 

xk uk

for x  px1 , . . . , xn q P `n2 .

k 1

Boundedness of T follows from triangle and Cauchy-Schwarz inequalities:

}T x}

k 1

|xk | }uk }
1{2

n

|xk |2

n
1{2

}uk }2

1{2

 M }x},

k 1

k 1

2
where M 
.
k1 }uk }
Moreover, T is bijective because puk q is a basis of X. (Why?) By inverse
mapping theorem, T is an isomorphism.


Exercise 3.1.14. Instead of using the inverse mapping theorem, give

a proof based on the compactness of the unit sphere of `n2 .
Exercise 3.1.15. Any two n-dimensional normed spaces X and Y are
isomorphic but not necessarily isometric. Show that X and Y are isometric if and only if BY  T BX for some invertible linear transformation
T : X Y . Show that `22 is not isometric to `28 .
Remark 3.1.16 (Banach-Mazur distance). A quantitative notion of isomorphism is given by the Banach-Mazur distance dpX, Y q between two isomorphic
normed spaces X and Y . It is defined as
dpX, Y q  inf t}T } }T
1 } : T : X

is an isomorphismu.

A theorem of F. John (see e.g. [1]) is a quantitative form of Theorem 3.1.13. It

states that every n-dimensional normed space X satisfies
dpX, `n2 q

?n.

3.2. CLOSED GRAPH THEOREM

74

It follows that for every two n-dimensional normed spaces X and Y , one has
dpX, Y q n.

(Why?) E. Gluskin [4] proved in 1981 that this upper bound is asymptotically
sharp, i.e. there exists an absolute constant c 0 such that, for every n P N one
can construct n-dimensional normed spaces Xn and Yn with
dpXn , Yn q cn.

Corollary 3.1.17. (i) Every finite dimensional normed space is a Banach

space.
(ii) Every finite dimensional subspace of a normed space is closed.
(iii) Every linear operator on a finite dimensional normed space is bounded.
(iv) Every two norms on a finite dimensional normed space are equivalent.
Proof. (i) Since an n-dimensional normed space X is isomorphic to the complete space `n2 , X itself is complete.
(ii) Let Y be a finite-dimensional subspace of a normed space X. Then Y
is complete by part (i). Therefore Y is closed. (Indeed, if a sequence pyn q Y
converges to x P X then pyn q is Cauchy in Y , hence its limit x is in Y .)
(iii) By Proposition 2.4.7, every linear operator on `n2 is bounded, }T } }T }HS .
Since an n-dimensional normed space X is isomorphic `n2 , the same is true for linear
operators on X.
(iv) Let }  } and ~  ~ be two norms on X. By part (iii), the identity map
I : pX, }  }q pX, ~  ~q as well as its inverse are bounded. This completes the
proof.

Exercise 3.1.18. [Operators from finite dimensional spaces] Let X and
Y be normed spaces, and X be finite dimensional. Show that every
linear operator T : X Y is bounded. (Hint: identify X with `n2 by some
isomorphism, and show boundedness of T similarly to the argument of
Theorem 3.1.13.)
Lec. 24: 11/3

3.2. Closed graph theorem

3.2.1. Statement and proof. Closed graph theorem is an alternative way
to check whether a given linear operator is bounded. This result characterizes
bounded operators in terms of their graphs.
Definition 3.2.1 (Graph of an operator). Let T : X Y be a linear operator
between normed spaces X and Y . The graph of T is the following subset of the
direct sum2 X `1 Y :
(
pT q  px, T xq : x P X .
Clearly, pT q is a linear subspace of the normed space X `1 Y . The main result
of this section is that pT q is closed if and only if T is bounded.
Let us compare these two notions, boundedness (equivalently, continuity) of T
and having closed graph. T is continuous if and only if
(3.5)

xn

xPX

implies T xn

T x.

2The direct sum of normed spaces is studied in Exercise 3.1.11. Instead of X

1 Y , one can
choose to work with X p Y for any p
1, . As we know this defines the same topology, and
will not affect the arguments in this section.

P r 8s

3.2. CLOSED GRAPH THEOREM

75

In contrast, pT q is closed if and only if

(3.6)

xn

x P X and T xn y P Y

imply y

 T x.

It is clear from these two lines that continuity always implies the closed graph
property:
Proposition 3.2.2. Let T : X Y be a linear operator between normed spaces
X and Y . If T is bounded then pT q is closed.
The opposite statement is nontrivial and requires completeness of both spaces
X and Y :
Theorem 3.2.3 (Closed graph theorem). Let T : X Y be a linear operator
between Banach spaces X and Y . If pT q is closed then T is bounded.
Proof. The direct sum X `1 Y is a Banach space (Exercise 1.3.13). The graph
pT q is a closed linear subspace of X `1 Y , hence pT q is a Banach space itself.
Consider the linear operator
u : pT q X,

upx, T xq : x.

Then u is a bounded, surjective and injective linear operator between two Banach
spaces. (Check!) By the open mapping theorem, u
1 is bounded. This means that
there exists a number M such that

}x} }T x}  }px, T xq} M }x} for all x P X.

The inequality }T x} M }x} implies that T is bounded.

3.2.2. Interpretation and an example. Recalling the interpretation of continuity (3.5) and closed graph property (3.6), we can make advantage of the closed
graph property the extra assumption that T xn converges in Y . So, to check the
continuity of a linear operator T using the definition (3.5), one can always assume
for free that T xn converges. Checking continuity no longer requires proving that
the limit exists; it reduces to checking the consistency of the limits of xn and T xn .
As an example, consider the simplest differential operator
T

 dtd ,

T : C 1 r0, 1s C r0, 1s

where C 1 r0, 1s is considered as a subspace of C r0, 1s, i.e. with respect to sup-norm.
Lemma 3.2.4. The differential operator T has closed graph.

Proof. Let fn f and T fn  fn1 g in C r0, 1s, i.e. uniformly. Then, by

the theorem on differentiation under the limit,3 plimn fn q1  limn fn1 , i.e. g  T f .
This completes the proof.

Nevertheless, as we know the differential operator is unbounded. (Why?) This
does not contradict the closed graph theorem, because C 1 r0, 1s is not complete
under the sup-norm. If we consider C 1 r0, 1s under its natural norm }f }8 }f 1 }8
in which it is a Banach space (Exercise 1.3.12), then the differential operator will
obviously be bounded.

3The theorem on differentiation under the limit states that lim f

n n
that fn1 converges uniformly and fn t0 converges for some point t0 .

p q

q1  limn fn1

provided

3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS

76

3.2.3. Symmetric operators on Hilbert spaces. A remarkable application

of closed graph theorem is that the symmetry property of an operator always implies
boundedness:
Theorem 3.2.5 (Hellinger-Toeplitz). Let T : H
a Hilbert space H. Suppose that

xx, T xy  xT x, yy

(3.7)

for all x, y

be a linear operator on

P H.

Then T is bounded.
Proof. By the closed graph theorem, it suffices to check that the graph of T
is closed. To this end we choose convergent sequences xn x, T xn y in H. We
would like to show that y  T x. It suffices to show that

xz, yy  xz, T xy

for all z

P H.

(Why?) This follows by using continuity of the inner product and (3.7) twice:

xz, yy  lim
xz, T xn y  lim
xT z, xn y  xT z, xy  xz, T xy.
n
n
The proof is complete.

Hellinger-Toeplitz theorem identifies the source of considerable difficulties in

mathematical physics. Many natural operators, such as differential, satisfy the
symmetry condition (3.7) but are unbounded. Hellinger-Toeplitz theorem declares
that such operators can not be defined everywhere on the Hilbert space. For example, one can never define a useful notion of differentiation that would make all
functions in L2 differentiable.
This explains that working with unbounded operators one has to always keep
track of their domains. For example, a linear operator T on a Hilbert space H is
called symmetric if the domain of T is dense in H, and (3.7) holds. An example of
a symmetric operator is the differential operator on L2 r0, 1s
T

 i dtd

with domain
Dom T

 tf P L2 r0, 1s : f P C 1 r0, 1s, f p0q  f p1q  1u.

3.3. Principle of uniform boundedness

Lec.25: 11/5

3.3.1. Statement and proof. The principle of uniform boundedness is a

result due to Banach and Steinhaus.
Theorem 3.3.1 (Principle of uniform boundedness). Let X and Y be Banach
spaces. Consider a family of bounded linear operators T LpX, Y q. Assume that
T is pointwise bounded, i.e.
(3.8)

sup }T x} 8

T T

for every x P X.

Then T is uniformly bounded, i.e.

sup }T } 8.

T T

Note that the reverse direction is trivially true uniform boundedness implies
pointwise boundedness. (Why?)

3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS

77

Proof. For every x P X, define

M pxq  sup }T x}.

T T

Pointwise boundedness of T means that

X

Xn ,

where

Xn

 tx P X :

M pxq nu.

n N

Baire category theorem implies that one of Xn is not a nowhere dense subset of
X. Since Xn is closed (why?), Xn has nonempty interior. Summarizing, we have
shown that there exist n P N, x0 P X and 0 such that

x0 BX .
By symmetry of Xn , we also have Xn
x0 BX .
Xn

we have

Xn

Hence by convexity of Xn ,

BX .

(Check!) By definition of Xn this means that for x P X,

}x}
It follows that for every x P X,

T T

sup }T x}

n
}x}.

sup }T }

T T

This implies that

sup }T x} n.

implies

T T

as required.

Exercise 3.3.2. Check that the sub-level sets Xn in the proof above
are closed, convex and symmetric. (All these properties were used in
the argument).
Remark 3.3.3 (Principle of condensation of singularities). Banach and Steinhaus called their Theorem 3.3.1 the principle of condensation of singularities for the
following reason. Suppose a family T LpX, Y q is not uniformly bounded. This
means that the set of vectors tT x : x P BX , T P T u is unbounded. Theorem 3.3.1
states that T is not even pointwise bounded, so there exists one vector x P X with
unbounded trajectory tT x : T P T u. One can say that the unboundedness of the
family T is condensated in a single singularity vector x.
Remark 3.3.4 (Completeness). In the proof of Theorem 3.3.1, the completeness of only X was used. So the result still holds if X is a Banach space and Y is
a normed space.
3.3.2. Weak and strong boundedness. Principle of uniform boundedness
can be used to check whether a given set in a Banach space is bounded.
Corollary 3.3.5 (Weak and strong boundedness). Let A be a subset of a
Banach space X. Assume that A is weakly bounded, i.e.
sup |f pxq| 8

x A

for every f

P X .

3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS

78

Then A is (strongly) bounded, i.e.

sup }x} 8.

x A

Here again the reverse statement is trivially true (strong) boundedness trivially implies weak boundedness.
Proof. We embed X into X  using the canonical embedding that we studied
in Theorem 2.3.12. So we consider vectors x P A as bounded linear functionals on
X  acting as xpf q : f pxq, f P X  . Rewriting the weak boundedness assumption
as supxPA |xpf q| 8 for f P X  , we may understand this assumption as pointwise boundedness of the family A X   LpX  , Rq. The principle of uniform
boundedness implies that supxPA }x}X   supxPA }x}X 8, as required.

Remark 3.3.6. Using Corollary 3.3.5, one can weaken the assumption (3.8) in
the principle of uniform boundedness to the following one:
sup |f pT xq| 8 for every x P X, f

T T

P Y .

(Why?)
3.3.3. Application to convergence of Fourier series. A basic and classical question in Fourier analysis is when does Fourier series of a function f on an
interval converge to f ?
Hilbert space technique provides a complete answer to this question in the
space L2 . As we know from Theorem 1.6.20, the Fourier series of every function in
L2 r
, s converges to f in the L2 -norm.
In function spaces other than L2 , the answer to this problem is often nontrivial
and even negative. Unfortunately, such is the situation in the space of continuous
functions C r
, s. There exist continuous functions f whose Fourier series do
not converge in C r
, s (i.e. uniformly). This follows from a somewhat stronger
result, which in turn is a consequence of the principle of uniform boundedness:
Theorem 3.3.7 (Divergent Fourier series). There exists a function f
whose partial Fourier sums
n

pSn f qptq 
k


n

P C r
, s

fppk qeikt

forms an unbounded sequence of complex numbers at t  0. In particular, the

Fourier series of f is unbounded (hence divergent) at t  0.
Proof. Recall from (2.13) that the partial Fourier sums can be represented
via convolution with Dirichlet kernel Dn :

pSn f qptq  21

Dn pt
sqf psq ds,

where

Dn pq 

sinpn 21 q
.
sin 21

We are interested in the behavior of pSn f qp0q. These are obviously linear functionals
on C r
, s, which we denote
n pf q : pSn f qp0q 
(We used that Dn is an even function).

1
2

Dn psqf psq ds.

3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS

79

Since Dirichlet kernel Dn is continuous, n are bounded linear functionals on

C r
, s, and

}n }  21
|Dn psq| ds


(Why?)
On the other hand, evaluating these integrals by hand one can see that
(3.9)

}n } 8 as n 8.
Therefore, pn qnPZ is not a uniformly bounded family of

(see the exercise below).

linear functionals.
By principle of uniform boundedness, this family is not even pointwise bounded.
This means that there exists a function f P C r
, s such that the set of numbers
tn pf q : n P Nu is unbounded. This is exactly what we wanted to show.

Exercise 3.3.8. [L1 norm of Dirichlet kernel] Show that
1
2

|Dn psq| ds c log n

where c 0 is an absolute constant. This validates step (3.9) in the

argument above. Hint: show that the area under k-th bump of the graph
of the Dirichlet kernel is k (see the picture); as there are
 n bumps on
r
, s this will show that the total area of the bumps is nk1 1{k log n.

Figure 3.1. Dirichlet kernel

Remark 3.3.9 (Convergence of Fourier series). (i) Analyzing the proof of Theorem 3.3.7 one can show that continuous functions whose Fourier series converge pointwise are rare. Precisely, the set of such functions is of first Baire
category in C r
, s (i.e. it is a countable union of nowhere dense sets).
(ii) Nevertheless, for every continuous function f , and even for f P Lp r
, s,
p 1, Fourier series converges to f almost everywhere. So the set of points
of divergence is always small. This is a deep result of L. Carleson [2].
(iii) For functions in L1 , Carleson result is generally false. Kolmogorov (apparently
when he was 19-21!) constructed a function f P L1 r
, s whose Fourier series
diverges everywhere.
(iv) If f is differentiable at a point t, then Fourier series of f converges to f at t.
This is called Dirichlet-Dini condition.

3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS

80

3.3.4. Schauder bases. The notion of Hamel basis, which we studied in Section 1.1.4, has a serious drawback. In all infinite-dimensional Banach spaces, Hamel
bases are uncountable, see Exercise 3.3.22. This makes it difficult to use Hamel bases
in practice. There exists an alternative notion of basis, which is more tailored to
the needs of analysis:
Definition 3.3.10 (Schauder basis). A sequence pxk q8
k1 in a Banach space X
is called a Schauder basis of X if every vector x P X can be uniquely expressed as
a convergent series
x

(3.10)

ak xk

k 1

for some scalars ak .

Exercise 3.3.11. Show that only separable spaces X may have Schauder
bases.
If pxk q is a Schauder basis then the sequence pxk q is clearly linear independent
and complete (recall from Definition 1.6.15 that completeness means that Spanpxk q
is dense in X.) However, the basis property is considerably stronger than linear
independence and completeness. Completeness means that for arbitrary vector
x P X and arbitrary precision 0, one can find a linear combination of basis
elements that approximates x well:
(3.11)



x




ak xk  .

k 1

However, the coefficients ak  ak pq may depend on . The limit lim0 ak pq

generally does not exist, as will be clear from the next exercise. In contrast, the
basis property guarantees that one can achieve approximation (3.11) with the same
coefficients ak independent of by only increasing the number of terms n in the
linear combination.
Exercise 3.3.12. [Completeness and basis property] In a Hilbert space
H, find a complete and linearly independent sequence pxk q which is not a
Schauder basis. Hint: construct xk so that it converges to some nonzero
vector in H; show that such sequences are never Schauder bases.
Example 3.3.13 (Bases in sequence spaces). An orthogonal basis of a Hilbert
space is a Schauder basis. (Check the uniqueness of represenation of x!)
In sequence spaces `p , 1 p 8 and c0 , the canonical (coordinate) system
forms a Schauder basis. (Check!)
In `8 there is no Schauder basis because this space is not separable.
Example 3.3.14 (Bases in Lp ). One can show (albeit somewhat nontrivially)
that Fourier basis (1.16) and the trigonometric system each form Schauder bases
in Lp r
, s, 1 p 8.
The Haar system defined in Exercise 1.6.34 also forms a Schauder basis in
Lp r
, s, 1 p 8. (Check!)
In L8 r
, s there is no Schauder basis because this space is not separable.

Lec. 26: 11/8

3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS

81

Example 3.3.15 (Basis of the space of continuous functions). In C r0, 1s, the
natural candidates fail to be Schauder bases. The Fourier basis is not a Schauder
basis otherwise this would imply that the Fourier series of every continuous function would converge in C r0, 1s (why?), which would contradict Theorem 3.3.7.
The sequence of monomials 1, t, t2 , . . . is not a Schauder basis of C r0, 1s either.
(Prove this!)
The most known Schauder basis of C r0, 1s is the so-called Schauder system of
wavelets. Its mother wavelet ptq is obtained by integration of Haar mother wavelet,
i.e.
#
t
t,
t P r0, 1{2q
hpsq ds 
ptq 
1
t, t P r1{2, 1q
0
Then we consider the translates and dilates of the mother wavelet:
kl ptq  p2k t
lq,

 0, 1, 2, . . . ,

l  0, 1, 2, . . . , 2k
1.

Together with the constant function 1, the system of functions kl ptq is called the
Schauder system, see the picture. It forms a Schauder basis on C r0, 1s. (Check!)

Figure 3.2. The first few functions of the Schauder system

Exercise 3.3.16. [Verifying Schauder bases] Do all checks in the previous examples. For C r0, 1s, note that the linear span of Schauder system
consists of piecewise-linear functions on r0, 1s whose nodes are dyadic
points.
A major property of Schauder bases is the uniform bound on the partial sums:
Theorem 3.3.17 (Partial sums of a Schauder basis). Let pxk q be a Schauder
basis of a Banach space X. Then there exists a number M called the basis constant
of pxk q with the following property. The partial sums of the basis expansion (3.10)
of every x P X satisfy
n




ak xk 


M }x},

n  1, 2, . . .

k 1

Proof. Consider the sequence space

!

(3.12)

E : a  pak q8
k1 :

ak xk converges in X

k 1

with the norm

(3.13)

n


}a}E : sup 
P

n N

k 1




ak xk .

3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS

82

It is an exercise to check that E is a Banach space. We will show that X and E

are isomorphic.4
To this end, we consider the synthesis operator T : E X defined as

Ta 

ak xk .

k 1

By construction, T is a bounded linear operator:

}T a} }a}E .

Since pxk q is a Schauder basis, T is surjective and injective. By the inverse mapping
theorem, T is an isomorphism. Therefore one can find a number M such that

}a}E M }T a},

But this means that

n



sup 

n N

a P E.




ak xk  M }x},

x P X,

k 1

which we wanted to prove.

Exercise 3.3.18. [Space of coefficients] Let pxk q8

k1 be a sequence of

nonzero vectors in a Banach space X. Define the space of coefficients E

by (3.12) and (3.13). Prove that E is a Banach space. (This property was
used in the proof of Theorem 3.3.17.)
Consider the partial sums of a basis expansion (3.10):
Sn pxq :

ak xk .

k 1

By Theorem 3.3.17, we see that Sn is a projection5 in X onto Spanpx1 , . . . , xn q,

and that Sn are uniformly bounded (by the basis constant):
sup }Sn }  M

n N

8.

Also, the coefficients ak  ak pxq of the basis expansion (3.10) are obviously
linear functionals on X. They are called biorthogonal functionals of the basis pxk q
and denoted xk , i.e.
xk pxq  ak .
With this notation, the basis expansion of x P X looks as
x

xk pxqxk

k 1

This resembles the Fourier series with respect to orthogonal bases in a Hilbert
space, except now we discuss this in general Banach spaces.
Corollary 3.3.19 (Biorthogonal functionals). The biorthogonal functionals

pxk q of a Schauder basis pxk q are uniformly bounded:

sup }xk } }xk } 8.
P

k N

p q

4For an orthonormal basis x

k in a Hilbert space X, this statement means that X is isomorphic to E `2 . We have proved this in Section 1.6.7.
5To recall the notion of projection in a Banach space, see Definition 2.4.21.

3.4. COMPACT SETS IN BANACH SPACES

83

Proof. We estimate the k-th term of basis expansion as

}xk pxqxk }  }Sk pxq
Sk
1 pxq} }Sk pxq} }Sk
1 pxq} 2M }x}
where M is the basis constant. On the other hand, }xk pxqxk }  |xk pxq| }xk }. This
clearly completes the proof.

Example 3.3.20. The biorthogonal functionals for an orthonormal basis in a

Hilbert space coincide with the basis vectors, i.e. xk  xk . The biorthogonal
functionals for the canonical (coordinate) basis of `p , 1 p 8 are the coordinate
vectors in `q , where p and q are adjoint indices.
Remark 3.3.21 (Basis problem). All classical separable Banach spaces are
known to have Schauder bases. However, there exist separable Banach spaces that
have no Schauder bases. The first constructions of such spaces was proposed by
Enflo [3] as a negative solution to Banachs basis problem.

3.3.5. Additional Exercises.

Exercise 3.3.22. [Hamel bases are uncountable] Show that a Hamel
basis of an infinite-dimensional Banach space X is always uncountable.
(This is the reason why Hamel bases are rarely used in practice). Hint:
first prove that every finite-dimensional subspace of X is nowhere dense;
then use Baire category theorem.
3.4. Compact sets in Banach spaces
Lec. 27: 11/10

Compactness is a useful substitute of finite dimensionality. We assume that the

reader is familiar with the notion of compactness from a basic course in topology,
but we will give a brief review.
3.4.1. Review of compactness. By definition, a subset A of a topological
space Xis compact if every open cover of A contains a finite sub-cover.
Precisely,
n
if A U for some collection of open sets U , then A k1 Uk for some
sub-collection. Some basic properties of compact sets are:
(i) compact subsets of a Hausdorff space are closed;
(ii) closed subsets of compact sets are compact;
(iii) images of compact sets under continuous maps are compact;
(iv) continuous functions on compact sets are uniformly continuous and they attain
their maxima and minima.
A set A is called precompact if its closure A is compact.
In metric spaces X, a useful description of compact sets A can be given in
terms of -nets. Recall that a subset N is an -net of A for some 0 if for every
x P A there exists y P N such that dpx, y q . Equivalently, N is an -net of A if
A can be covered by balls of radius centered at points in N .
Theorem 3.4.1 (Compactness in metric spaces). For a subset A of a metric
space X, the following are equivalent:
(i) A is precompact;
(ii) Every sequence pxn q in A has a Cauchy subsequence (which furthermore converges in X if X is complete);

3.4. COMPACT SETS IN BANACH SPACES

84

(iii) For every 0, there exists a finite -net of A.

As a consequence, precompact sets in metric spaces are always bounded. The
converse is true in all finite dimensional normed spaces:
Theorem 3.4.2 (Heine-Borel). A subset A of finite dimensional normed space
X is precompact if and only if A is bounded.
Actually, the classical Heine-Borel theorem is the statement for the specific
space X  `n2 . But as we know, all finite dimensional normed spaces X are isomorphic to `n2 (Theorem 3.1.13), so the general result is also true.
3.4.2. Compactness in infinite-dimensional normed spaces. In infinitedimensional normed spaces, Heine-Borel theorem fails. For example, an orthonormal basis pxk q of `2 is a bounded set but it is not
? precompact, because it does not
have a convergent subsequence (as }xk
kj }  2 for k  j).
Compact sets are almost finite dimensional. This heuristics, which is made
rigorous in the following result, underlies many arguments in analysis:
Lemma 3.4.3 (Approximation by finite dimensional subspaces). A subset A of
a normed space X is precompact if and only if A is bounded and, for every 0,
there exists a finite dimensional subspace Y of X which forms an -net of A.
Proof. Necessity. Let A be precompact. Choose a finite -net N of A; then
the subspace Y : SpanpN q is finite-dimensional and forms an -net of A.
Sufficiency. Since A is bounded, A rBX for some finite radius r. Since Y is
an -net of A, it follows that pr qBY is also an -net of A. (Check!) Further, since
Y is finite-dimensional, the set pr qBY is precompact by Heine-Borel theorem.
So we have found a precompact -net of A. Therefore A itself is precompact.
(Why?)

By Heine-Borel theorem, the unit ball BX of a finite-dimensional normed space
X is compact. This never holds in infinite dimensions:
Theorem 3.4.4 (F. Riesz). The unit ball BX of an infinite-dimensional normed
space X is never compact.
Proof. Suppose BX is compact. By approximation Lemma 3.4.3, we can find
a finite dimensional subspace Y of X which forms an 12 -net of BX , i.e.
1
for all x P BX .
2
Since X is infinite dimensional and Y is finite dimensional, the quotient space
X {Y is nonzero. So we can find a coset rxs P X {Y with }rxs}  0.9. Since by
definition }rxs}  inf xPrxs }x}, we can further choose a representative x P rxs such
that }x} 1. Summarizing, distpx, Y q  }rxs}  0.9 and x P BX . This contradicts
(3.14) and completes the proof.

(3.14)

distpx, Y q

The next useful result states that pointwise convergence of operators implies
uniform convergence on compacta. We say that a sequence of operators Tn P
LpX, Y q between normed spaces X and Y converges pointwise to some T P LpX, Y q
if
Tn x T x for all x P X.

3.4. COMPACT SETS IN BANACH SPACES

85

In contrast, we say that Tn converges uniformly to T on a subset A X if

}Tn x
T x} n 0

where n

for all x P A,

0 is some sequence of numbers (that does not depend on x).

Theorem 3.4.5 (Convergence on compacta). Let X, Y be Banach spaces, and

Tn , T P LpX, Y q. Assume that Tn T pointwise. Then Tn T uniformly on
every precompact subset A X.

Proof. Since the sequence pTn q is pointwise convergent, it is pointwise bounded.

Therefore, by the principle of uniform boundedness, pTn q is uniformly bounded, i.e.
there exists a number M such that

}Tn } M

for all n.

Let 0 be arbitrary, and choose a finite -net N of A. Since the set N is

finite, Tn T uniformly on N . (Why?) So there exists a number N such that

}Tn y
T y} for all n N and y P N .

For every x P A, we choose y P N with }x
y } . Then
}Tn x
T x}  }pTn
T qx} }pTn
T qy} }pTn
T qpx
yq}
p}Tn } }T }q }x
y} 2M .
Summarizing, we have proved that for every 0 there exists N such that }Tn x

T x} p1 2M q for all n N and x P A. This means that Tn T uniformly on
A.

3.4.3. Compactness criteria in various spaces. There is a useful criterion

of compactness in spaces with Schauder basis (which covers all classical spaces).
Theorem 3.4.6 (Compactness in spaces with basis). Let X be a Banach space
with a Schauder basis pxk q. A subset A X is precompact if and only if A is
bounded and the basis expansions of vectors x P A converge uniformly, i.e.




xk x xk 


pq

 }x
Sn x} n 0

for all x P A,

k n

0 is some sequence of numbers (that does not depend on x).

Proof. Necessity. Since pxk q is a Schauder basis, this means that the projections Sn I pointwise (where I is the identity operator in X). By Theorem 3.4.5,
Sn I uniformly on A, as required.
Sufficiency. Assume that }x
Sn x} for all x P A. Since Sn x P ImpSn q, this
implies that ImpSn q forms an -net of A. But ImpSn q has finite dimension n. We

where n

conclude that A is precompact by Lemma 3.4.3.

Corollary 3.4.7 (Compactness in `p ). A subset A `p , p P r1, 8q is precompact if and only if A is bounded and has uniformly decaying tails, i.e.

|ak |p n 0

for all a  pak q P A,

k n

where n

0 is some sequence of numbers (that does not depend on a).

Proof. The claim follows immediately by applying Theorem 3.4.6 for the
canonical basis of `p .


3.5. WEAK TOPOLOGY

86

Example 3.4.8. The Hilbert cube is the following subset of `2 :

A  tpak q P `2 : |ak |

1
for all k
k

P Nu.

By Corollary 3.4.7, the Hilbert cube is compact.

Exercise 3.4.9. [Compactness in c0 ] Prove that a subset A c0 is
precompact if and only if there exists a vector b P c0 that majorizes all
vectors a P A, i.e.

|ak | bk

for all k

P N.

(Hint: use Theorem 3.4.6).

Finally, we mention without proof compactness criteria in two other spaces,
C ra, bs and L1 .
Theorem 3.4.10 (Arzela-Ascoli). A subset A C ra, bs is precompact if and
only if A is bounded and equicontinuous, i.e. for every 0 there exists 0
such that

|s
t|

|f psq
f ptq|

implies

for all f

P A.

Example 3.4.11. Arzela-Ascoli theorem implies that the set of differentiable

functions f with }f 1 }8 1 is compact in C r0, 1s.

Theorem 3.4.12 (Compactness in L1 ). A subset A L1 r0, 1s is precompact if

and only if A is bounded and uniformly bounded on average, i.e. for every 0
there exists 0 such that

| |

implies
0

|f pt

q
f ptq|

for all f

P A.

3.4.4. Additional Exercises.

Exercise 3.4.13. [Stability of compactness] Prove that compactness in
normed spaces is stable under linear operations:
(i) If A, B are precompact sets in a normed space, then Minkowski
sum A B is precompact;
(ii) If A is a precompact subset of X and T P LpX, Y q then T pAq is a
precompact set in Y .
Exercise 3.4.14. [Convex hull of compact sets]
(i) Show that the convex hull of a precompact set in a normed space
is a precompact set.
(ii) Show that the closure of the convex hull of a compact set in a
normed space is compact.
(iii) Construct an example showing that the convex hull of a compact
set in a normed space does not need to be compact.

3.5. WEAK TOPOLOGY

87

3.5. Weak topology

Every normed space X is a metric space, with the metric given by dpx, y q 
}x
y}, x, y P X. This topology on X is called strong topology . Thus a sequence
xn x converges (strongly) in X if

}xn
x} 0.

(3.15)

In addition to the strong topology, X carries a different topology called weak

topology. We are going to study the weak topology in this section.
3.5.1. Weak convergence.
Definition 3.5.1 (Weak convergence). A sequence pxk q in a normed space X
converges weakly to a vector x P X if
f pxk q f pxq

The weak convergence is denoted xk

for every f

x.

P X .

Strong convergence (3.15) clearly implies weak convergence. The converse is

generally not true:
Example 3.5.2. An orthonormal system pxk q in a Hilbert space X converges
weakly to zero, and it clearly does not converge strongly. Indeed, by Riesz representation theorem the weak convergence to zero is equivalent to

xxk , xy 0

for every x P X.

This indeed follows from Bessels inequality

|xxk , xy|2 }x}2 .

k 1

Even though weak convergence is generally strictly weaker than strong convergence, there are several useful ties between weak and strong properties. Weak
convergence clearly implies weak boundedness, which in turn implies strong boundedness by a consequence to the principle of uniform boundedness (Corollary 3.3.5):
Proposition 3.5.3. Weakly convergent sequences in Banach spaces are bounded.

Moreover, we have a good control of the weak limit, given in the next two
results.
w

x in a normed space then }x} lim inf n }xn }.

Proof. Let f P X  be a supporting functional of x, i.e. }f }  1, f pxq  }x}.
Then f pxn q }xn } for all n. Taking lim inf of both sides, we conclude that
lim inf }xn } lim inf f pxn q  f pxq  }x}
n
n

Proposition 3.5.4. If xn

as required.

Lemma 3.5.5 (Mazur). Let xk

p q

x in a normed space, then6 x P convpxk q.

6Recall that conv A is the smallest convex set containing A, see Exercise 1.2.24.

Lec. 28: 11/12

3.5. WEAK TOPOLOGY

88

Proof. Suppose x R K : convpxk q. Using a separation theorem (Corollary refclosed convex separation), we can separate the closed convex set K from the
point txu. Namely, there exists a functional f P X  such that
sup f py q f pxq.

Since xk

P K, this implies that

y K

sup f pxk q f pxq,

k

which contradicts weak convergence.

3.5.2. Criteria of weak convergence. Some known criteria of weak convergence in classical normed spaces rely on the following tool.
Lemma 3.5.6 (Testing weak convergence on a dense set). Let X be a normed
w
space and A X  be a dense set. Then xk

x in X if and only if pxk q is bounded

and
f pxk q f pxq for every f P A.
Proof. Nesessity follows by Proposition 3.5.3.
Sufficiency will be proved by a standard approximation argument. Consider
arbitrary g P X  and 0. By density, we can choose f P A such that }g
f } .
Then
lim sup |g pxk
xq| lim sup |f pxk
xq|
k

lim sup |pg
f qpxk
xq|

n

0 }g
f } lim supp}xk } }x}q M

k
where M  supk }xk } }x} 8 by the boundedness assumption. Since 0
is arbitrary, we conclude that lim supk |g pxk
xq|  0. Hence g pxk q g pxq as

required.

Theorem 3.5.7 (Weak convergence in c0 and `p ). Let X  c0 or X  `p ,

w
p P p1, 8q. Then xk

x in X if and only if the sequence pxk q is bounded and

converges to x pointwise, i.e.
xk piq xpiq

for every i P N.

Proof. Necessity. If xn
x then by applying coordinate functionals ei P X 

(i.e. those acting as ei pxq  xpiq) we see that xk piq xpiq as required.
Sufficiency. We are given that pxk q is bounded and that f pxk q f pxq for
every coordinate functional f  ei . By linearity, we get f pxk q f pxq for every
f P Spanpei q8
i1 .
On the other hand, the representation theorems (Corollary 2.2.6 and Exercise 2.2.7) state that X   `1 if X  c0 and X   `p1 if X  `p . The functionals
ei P X  get identified with the coordinate vectors p0, . . . , 0, 1, 0, . . .q, which shows

that Spanpei q8
i1 is dense in X . (Why?)
The proof is finished by applying Lemma 3.5.6 to A  Spanpei q8

i1 .
w

Exercise 3.5.8. Consider the sequence xk  p1, . . . , 1, 0, 0, . . .q (with k

ones) in `8 . Use Mazurs lemma to show that xk does not converge
weakly. Deduce that the criterion of weak convergence in `p , p P p1, 8q,
given in Theorem 3.5.7 fails for `8 . (There is no useful criterion of weak
convergence in `8 .

3.5. WEAK TOPOLOGY

89

Exercise 3.5.9. State and prove a similar criterion of weak convergence in spaces with Schauder basis.
A similar criterion of weak convergence holds in spaces of continuous functions.
Theorem 3.5.10 (Weak convergence in C pK q). Let K be a compact topological
w
space. Then xk

x in C pK q if and only if the sequence of functions xk ptq is

uniformly bounded7 and converges to xptq pointwise, i.e.
xk ptq xptq

for every t P K.

Proof. Necessity. Boundedness of pxk q follows from weak convergence as

before. Pointwise convergence follows by applying point evaluation functionals
t P C pK q (acting as t pxq  xptq).
Sufficiency. We need to show that f pxk q f pxq for all f P C pK q . By
representation Theorem 2.2.8, this is equivalent to claiming that

(3.16)
K

xn d

x d
K

for every Borel regular signed measure . On the other hand, our assumptions are
that the sequence of functions xn ptq is uniformly bounded and it converges to xptq
pointwise. The Lebesgue dominated convergence theorem implies (3.16).

A similar criterion of weak convergence holds in Lp spaces. However, it does
not make sense to consider the values of functions x P Lp in individual points.
Instead, we shall consider integrals of xptq over short intervals.
Theorem 3.5.11 (Weak convergence in Lp ). Let p P p1, 8q. A sequence xk
x in Lp r0, 1s if and only if the sequence pxk q is bounded in Lp and
b
a

xk ptq dt

b
a

xk ptq dt

for every interval ra, bs r0, 1s.

Proof. One notices that the set of characteristic functions 1ra,bs ptq for ra, bs
(Why?) The
argument is finished similarly to Theorem 3.5.7.


r0, 1s spans the set of step functions, which is dense in pLp q  Lp1 .
Remark 3.5.12. The same criterion holds for Lp pRq. (Why?)

Example 3.5.13 (Sliding bumps). A good example of weakly convergent but

strongly divergent sequence of functions is formed by a sliding bump in Lp pRq,
p P p1, 8q. Consider a function x P Lp pRq with compact support. Then the
sequence xk ptq  xpt
k q converges weakly to zero by Theorem 3.5.11.
Remark 3.5.14 (Weak convergence in `1 ). The weak convergence in `1 is unusual. (Note that this space is not covered by Theorem 3.5.7.) The weak and strong
convergence in `1 are equivalent. This result is due to I. Schur, and is called Schur
property of `1 .
Exercise 3.5.15. [Weak convergence in finite dimensional spaces] Show
that all finite-dimensional normed spaces X have Schur property, so the
weak and strong convergence in X coincide.

3.5. WEAK TOPOLOGY

90

Lec. 29: 11/15

3.5.3. Weak topology. Now we broaden the picture and study the weak
topology on X which defines weak convergence. This way, in addition to weak
convergence, we could be able to study other weak properties, such as weak boundedness, weak compactness and so on.
Definition 3.5.16 (Weak topology). The weak topology on a normed space X
is defined as the weakest topology in which all maps f P X  (i.e. f : X R) are
continuous.
Equivalently, the base of the weak topology is given by the cylinders, which are
the sets of the form

tx P X : |fk px
x0 q| , k  1, . . . , N u
where x0 P X, fk P X  , 0, and n P N. So, these cylinders form a local base of

weak topology at point x0 .

To distinguish the two natural topologies on X, the norm topology is sometimes

called strong.
Exercise 3.5.17. Check the equivalence of the two ways to define the
weak topology in the definition above.
Remark 3.5.18. The cylinders are clearly open in the strong topology. Hence
the weak topology is weaker than the strong topology.
Remark 3.5.19. In infinite-dimensional spaces X, the cylinders are rather large
as they contain subspaces of finite codimension tx P X : fk px
x0 q  0, k 
1, . . . , N u. This shows that in infinite dimensions, weakly open sets are unbounded.
Exercise 3.5.20. Prove that in an infinite dimensional normed space
X, weak topology is strictly weaker than the strong topology. Why
does not this contradict Schur property of X  `1 mentioned in Remark 3.5.14?
Nevertheless, some weak and strong properties are equivalent. For example,
weak boundedness and strong boundedness are equivalent. This follows from the
principle of uniform boundedness, see Corollary 3.3.5. (Check that what we called
weak boundedness is indeed boundedness in the weak topology).
Also, weak closedness and strong closedness are equivalent for convex sets:
Proposition 3.5.21 (Weak closedness). Let K be a convex set in a normed
space X. Then K is weakly closed if and only if K is (strongly) closed.
Proof. Necessity is trivial.
Sufficiency. Assume K is closed and convex. By Corollary intersection of halfspaces to Hahn-Banach theorem, K is the intersection of the closed half-spaces that
contain K. Each closed half-space has the form
Af,a

 tx P X : f pxq au

for some f P X  and a P R. Hence Af,a is weakly closed.8 The intersection K of

the closed half-spaces is therefore automatically weakly closed.


} }8 8

7That is, sup x

.
k
k
8Indeed, A
f,a is the preimage of the closed subset

which is continuous by the definition of weak topology.

p
8, as under the function f : X R,

3.6. WEAK TOPOLOGY. BANACH-ALAUGLUS THEOREM

91

Remark 3.5.22. Convexity assumption is critical in Proposition 3.5.21. Otherwise the result would claim that the weak and strong topologies are equivalent,
which is false.
3.6. Weak topology. Banach-Alauglus theorem
On X  , there are two natural weaker topologies. The weak topology that
we already considered makes all functionals in X  continuous functions on X  .
The other topology, called weak topology, is only concerned with continuity of
functionals that come from X X  .
3.6.1. Weak convergence.
Definition 3.6.1 (Weak convergence). Let X be a normed space. A sequence
of functionals pfk q in X  weak converges to a functional f P X  if
fk pxq f pxq
The weak convergence is denoted fk

for every x P X.

w f .

In other words, the weak convergence of functionals is pointwise convergence.

While weak convergence of functionals fk P X  is tested on all functionals from
X  , weak convergence of fk is tested on the subset X X  . Therefore, weak
convergence implies weak convergence in X  . Of course, for reflexive spaces, weak
and weak convergence coincide.
Example 3.6.2 (Weak convergence of measures). In probability theory, one
says that a sequence of regular Borel measures n on R converges weakly to a Borel
regular measure if

f dn

f d

for every f

P C pRq.

Assume that the measures n and are compactly supported, say on an interval
ra, bs. By the representation theorem for pC ra, bsq , Theorem 2.2.8, this convergence
is nothing different from
w
n in pC ra, bsq .
Summarizing, the weak convergence of measures in probability theory is actually
the weak convergence of measures acting as linear functionals on C ra, bs.
Example 3.6.3 (Dirac delta function). Recall that we understand Dirac delta
function ptq as the point evaluation functional at zero, see Example 2.1.3. Equivalently, Dirac delta function may be identified with the probability measure on R
with the only atom at the orgigin. Therefore Dirac delta function is the weak limit
of uniform measures on r
n1 , n1 s as n 8.
This gives a natural way to approximate Dirac delta function ptq (which does
not exists as a function on R) by genuine functions n ptq, which are the probability
distribution functions of the uniform measures on r
n1 , n1 s, see the picture.

3.6. WEAK TOPOLOGY. BANACH-ALAUGLUS THEOREM

92

Figure 3.3. Approximation of Dirac delta function ptq by functions n ptq

3.6.2. Weak topology. Banach-Alaoglus theorem. Similarly to the
Definition 3.5.16, we will consider the weak topology on X  .

Definition 3.6.4 (Weak topology). Let X be a normed space. The weak

topology on X  is defined as the weakest topology in which point evaluation maps
f f pxq from X  to R are continuous for all points x P X.
Equivalently, the base of the weak topology is given by the cylinders, which are
the sets of the form

tf P X  : |pf
f0 qpxk q| , k  1, . . . , N u
where f0 P X  , xk P X, 0, and n P N. So, these cylinders form a local base of
weak topology at f0 .

Looking at the first part of this definition and recalling that x P X X  ,

we see that weak topology is weaker than the weak topology on X  . However, for
reflexive spaces the two topologies are of course equivalent.
The main result on weak topology is Alaoglus theorem. It allows one to bring
back to life compactness arguments in infinite-dimensional normed spaces X, even
though the unit ball of such X is always not compact.
Theorem 3.6.5 (Banach-Alaoglu). For every normed space X, the closed unit
ball BX  is weak compact.
The proof will be based on Tychonoff s theorem that states that the product of
any collection of compact topological spaces is compact. Let us briefly recall this
result.
Consider a collection of pX q P of any number
(countable or uncountable) of

topological spaces X . The Cartesian product P X can be equipped with the

product topology whose base is formed by the sets of the form
"

A : A is open in X ; all but finitely many of A equal X .

Tychonoffs theorem states that if each X is compact then

in the product topology.

P X is compact

Proof of Banach-Alaoglus theorem. We shall embed BX  into the product space of intervals
K :

x X

r
}x}, }x}s 

f :X

R : |f pxq| }x} for all x P X

3.6. WEAK TOPOLOGY. BANACH-ALAUGLUS THEOREM

93

equipped with the product topology.

This is the weakest topology in which the point evaluation maps f f pxq from
K to R are continuous for all x P X. (Check!) We identify a functional f P BX 
with the element of the product space pf pxqqxPX P K. With this identification,
the weak topology on BX  coincides with the product topology on K. (Why?)
Therefore, this identification is a homeomorphic embedding of BX  into K.
It remains to check that BX  is a weak closed subset of K; the proof will
then be finished by Tychonofffs theorem. This is simple. Indeed, note that BX 
consists of the linear functions in K. So we can represent
BX 

Bx,y,a,b ,

where Bx,y,a,b

P K : f pax

by q  af pxq bf py q .

x,y X, a,b R

Each set Bx,y,a,b is the preimage of the weak closed set t0u under the map f
f pax by q
af pxq
bf py q which, as we know, is continuous in the product topology.9
Therefore all sets Bx,y,a,b are weak closed, and so is their intersection BX  . This
completes the proof.


3.6.3. Universality of space C pK q. As an application of Banach-Alaoglus

theorem, we will show that the space of continuous functions C pK q is universal in
the sense that it contains every Banach space X as a subspace.10
Theorem 3.6.6 (Universality of C pK q). Every Banach space X can be isometrically embedded into C pK q for some compact topological space K.

Proof. Let K : BX  equipped with weak topology. By Banach-Alaoglus

theorem, K is indeed compact. We define the embedding X C pK q by associating
every x P X the point evaluation function
(3.17)

xpf q : f pxq,

P K.

Recall that the point evaluation function is indeed in C pK q by the definition of

weak topology. The map defined by (3.17) is linear by construction. Finally, this
map is an isometric embedding; indeed

|f pxq|  }x}X ,
}x}C pK q  f PKmax
B 
X

where the last inequality uses a consequence of Hahn-Banach theorem, Corollary 2.3.10.

Exercise 3.6.7. [Universality of `8 ] Show that `8 is a universal space
for all separable Banach spaces. In other words, show that every separable Banach space X isometrically embeds into `8 .
Hint: Consider a dense subset pxk q8
k1 of SX , choose supporting functionals fk P SX  of xk , and define the embedding X `8 by x pfk pxqq8
k1 .

9Recall that the point evaluation maps are continuous in the product topology.
10A little disclaimer is that the compact topological space K may depend on X; otherwise

the result is false for spaces X of too large cardinality.

Lec. 30: 11/17

CHAPTER 4

Compact operators. Elements of spectral theory

4.1. Compact operators
Compact operators form an important class of bounded linear operators. On
the one hand, they are almost finite rank operators (in the same way as compact
sets are almost finite dimensional). So compact operators do share some properties of finite rank operators, which facilitates their study. On the other hand, the
class of compact operators is wide enough to include integral and Hilbert-Schmidt
operators, which are important in many applications.
4.1.1. Definition. Integral operators. Throughout this section, X, Y will
denote normed spaces.
Definition 4.1.1 (Compact operators). A linear operator T : X Y is called
compact if it maps bounded sets in X to precompact sets in Y . The set of compact
operators is denoted K pX, Y q.
Exercise 4.1.2. Show that T is compact if and only if it maps BX to
a precompact set in Y .
Since precompact sets are bounded, compact operators are always bounded,
i.e. K pX, Y q LpX, Y q.

Example 4.1.3. Every finite rank operator1 T P LpX, Y q is compact. Indeed,

T pBX q is a bounded subset of a finite dimensional normed space Im T Y , so
T pBX q is precompact by Heine-Borel theorem.
The next example is one of the main motivation to study compact operators.
Proposition 4.1.4 (Integral operators are compact). Consider the integral
operator T : C r0, 1s C r0, 1s defined as

pT f qptq 

1
0

k pt, sqf psq ds

with kernel k pt, sq P C pr0, 1s2 q. Then T is a compact operator.

Proof. We need to show that K : T pBC r0,1s q is a precompact subset of

C r0, 1s, see Exercise 4.1.2. By Arzela-Ascoli Theorem 3.4.10, this would follow
from (uniform) boundedness and equicontinuoity of the set K.
The (uniform) boundedness of K follows from the boundedness of T . (Why?)
To prove equicontinuity, we let 0 and choose 0 such that

|t1
t2 |

implies

|kpt1 , sq
f pt2 , sq|

1T is a finite rank operator if dim Im T

8.
94

for all s P r0, 1s.

4.1. COMPACT OPERATORS

95

(We can do this by continuity of the kernel k pt, sq.) Now, for every f
obtain by triangle inequality that

 T f t1


T f t2 

p qp q
p qp q

1
0

P BC r0,1s , we

|kpt1 , sq
f pt2 , sq| |f psq| ds

as |f psq| 1 for all s. This shows that the set K is equicontinuous, and therefore
precompact.

Exercise 4.1.5. Show that Volterra operator (2.12) is compact on
C r0, 1s, even though its kernel is discontinuous. See Exercise 4.1.19 for a
more general result.
4.1.2. Basic properties of compact operators.
Proposition 4.1.6 (Properties of K pX, Y q). (i) The set of compact operators K pX, Y q is a closed linear subspace of LpX, Y q.
(ii) K pX, Y q is an operator ideal. This means that if T P K pX, Y q then the
compositions ST and T S are both compact for every bounded linear operator
S.
Proof. (i) Linearity follows from the observation that the Minkowski sum of
two precompact sets is precompact (see exercise below).
Closedness. Consider a sequence Tn P K pX, Y q such that Tn T in LpX, Y q;
we want to prove that T P K pX, Y q. Let 0 and choose n P N such that
}Tn
T } . This means that

}Tn x
T x} for every x P BX .

This shows that Tn pBX q is a precompact -net of T pBX q.
T pBX q is itself precompact. (Why?)
(ii) is straightforward and is left as an exercise.

Since is arbitrary,


Exercise 4.1.7. Prove that Minkowski sum A B of two precompact

subsets A, B of a normed space is a precompact set.
Corollary 4.1.8 (Isomorphisms are not compact). Let X be an infinite dimensional normed space. Then the identity operator on X is not compact. More
generally, any isomorphism T : X Y is not compact.
Proof. For the identity operator on X, the result follows from F. Rieszs
Theorem 3.4.4 on non-compactness of BX . As for the general statement, if an
isomorphism T : X Y were compact then the identity operator T
1 T would also
be compact by Proposition 4.1.6, which would be a contradiction.

As we know, finite rank operators are compact (Example 4.1.3). More generally,
since K pX, Y q is closed, it follows that any operator that can be approximated by
finite rank operators is also compact:
Corollary 4.1.9 (Almost finite rank operators are compact). Suppose a linear
operator T : X Y can be approximated by finite rank operators Tn P LpX, Y q,
i.e.
}Tn
T } 0 as n 8.
Then T is compact.


4.1. COMPACT OPERATORS

96

Exercise 4.1.10. Fix a sequence of real numbers pk q8

k1 , and define
the linear operator T : `2 `2 by
T x  pk xk q8
k1 .

For what multiplier sequences pk q is the operator T (a) well defined?

(b) bounded? (c) compact?
4.1.3. Hilbert-Schmidt operators. This is a most frequently used class of
compact operators in Hilbert spaces. As we will see, it covers the class of HilbertSchmidt integral operators discussed in Section 2.4.4.
Definition 4.1.11 (Hilbert-Schmidt operators). Let H be a separable Hilbert
space, and let pxk q be an orthonormal basis of H. A linear operator T : H H is
called a Hilbert-Schmidt operator if

}T xk }2 8.

k 1

The quantity

}T }HS :

}T xk }

1{2

k 1

is called the Hilbert-Schmidt norm of T .

Exercise 4.1.12. For operators in finite dimensional spaces T : Cm
C , (equivalently, for m  n matrices), we already gave a definition of
Hilbert-Schmidt norm in (1.12). Verify that the two definitions do agree.
n

Proposition 4.1.13. The definition of Hilbert-Schmidt operator and of the

Hilbert-Schmidt norm does not depend on the choice of an orthonormal basis of H.

Proof. Assume that k }T xk }2 8 for some orthonormal basis pxk q of H.

Using Parsevals identity twice, we obtain
(4.1)

}T xk }2 

|xT xk , xj y|2 

k,j

|xxk , T  xj y|2 

k,j

}T  xj }2 .

Let px1k q be another orthonormal basis of H. Then a similar argument gives

}T  xj }2 

j,k

|xx1k , T  xj y|2 

|xT x1k , xj y|2 

j,k

}T x1k }2 .

This completes the proof.

Remark 4.1.14. As a byproduct of the proof, we obtained in (4.1) that

}T  }HS  }T }HS .

Lec. 31: 11/19

As we know from Proposition 2.4.7, linear operators on finite-dimensional

spaces satisfy }T } }T }HS . The same holds in general:
Proposition 4.1.15. Hilbert-Schmidt operators are bounded, and }T } }T }HS .

4.1. COMPACT OPERATORS

97

Proof. Let T be a Hilbert-Schmidt operator on a Hilbert space H, and pxk q

be an orthonormal basis of H. It suffices to prove that the restriction of T on the
dense subspace Spanpxk q of H is bounded and has norm at most }T }HS ; the result
would then follow by extension by continuity (Proposition
2.4.20).

So let x P Spanpxk q, which means that x  k ak xk for some scalars ak (finite

sum). Then using triangle inequality and Cauchy-Schwarz inequality, we obtain


}T x}  




ak T xk 

|ak | }T xk }

1{2

}T xk }2

1{2

 }x} }T }HS .

|ak |2

This completes the proof.

An important example of Hilbert-Schmidt operators are integral operators on

L2 , which we introduced in Section 2.4.4.
Proposition 4.1.16 (Hilbert-Schmidt integral operators). Consider the integral operator T : L2 r0, 1s L2 r0, 1s defined as
1

pT f qptq  kpt, sqf psq ds

0
with kernel k pt, sq P L2 pr0, 1s2 q. Then T is a Hilbert-Schmidt operator, and
}T }HS  }k}2 .
Proof. We will view the integral in the definition of T as the inner product
of f with the kernel k. Specifically, consider the function Kt psq  k pt, sq; then

pT f qptq  xKt , f y for every t P r0, 1s.

Let us fix some orthonormal basis pxk q of L2 r0, 1s. Then
}T } 
2
HS

} 

T xk 22

k
1

1
k

|xKt , xk y|2 dt

|pT xk qptq|

dt 

1
k

|xKt , xk y|2 dt

(by monotone convergence theorem)

 }Kt }22 dt (by Parsevals identity)

0
 }k}22 (by definition of Kt and Fubini theorem).
This completes the proof.

4.1.4. Compactness of the adjoint operator. Recall the basic duality

property for bounded linear operators: if T P LpX, Y q then T  P LpY  , X  q and
}T  }  }T }. A similar duality principle holds for compact operators:
Theorem 4.1.17 (Schauder). Let X and Y be Banach spaces. If T
then T  P K pY  , X  q.

P K pX, Y q

P Y  , we are seeking a bound on

}T  f }X   sup |pT  f qpxq|  sup |f pT xq|  sup |f pyq|

Proof. Given f
(4.2)

x BX

x BX

y K

where K : T pBX q. (Taking the closure here is justified by continuity of f ).

We shall interpret the identity (4.2) in topological terms. Indeed, we know that
K is compact, and we need to prove that G : T  pBY  q is precompact in X  . Let

4.2. FREDHOLM THEORY

98

us embed the subset G X  into C pK q and use Arzela-Ascoli theorem. Namely,

we define the embedding U : G C pK q by
U pT  f q : f |K

(making some selection of f ; it does not matter in which way). Then identity (4.2)
implies that
}T  f }X   }f |K }C pK q for every f P Y  ,
which shows that U is an isometric (thus homeomorphic) embedding.
Now, U pGq is (uniformly) bounded in C pK q as

}T  f }X  }T  } }f }Y  }T } for every f P BY  .
Moreover, U pGq is equicontinuous. Indeed, for every f P BY  and for y1 , y2 P K
we have


f

|K py1 q
f |K py2 q  |f py1
y2 q| }f }X  }y1
y2 } }y1
y2 }.

Arzela-Ascoli theorem completes the proof.

Remark 4.1.18 (For future). Consider proving the reverse direction in Schauders
theorem. Also consider proving that compact operators map weak Cauchy sequences to strongly convergent.

4.1.5. Additional Exercises.

Exercise 4.1.19. [Compactness of integral operators] Consider an intergal operator T with kernel k pt, sq : r0, 1s2 R which satisfies the following:
(i) for each s P r0, 1s, the function ks ptq  k pt, sq is integrable in t;
(ii) the map s ks is a continuous map from r0, 1s to L1 r0, 1s.

Show that the integral operator T is compact in C r0, 1s.

4.2. Fredholm theory

Lec. 32: 11/22

Fredholm theory studies operators of the form identity plus compact. They
are conveniently put in the form I
T where I is the identity operator on some
Banach space X and T P K pX, X q.
Fredholm theory is motivated by two applications. One is for solving linear
equations x
T x  b, and in particular integral equations (T being an integral
operator). Another related application is in spectral theory, where the spectrum of
T consists of numbers for which the operator I
T is invertible. We will discuss
both applications in detail later.
4.2.1. Closed image.
I

Theorem 4.2.1. Let X be a Banach space and T

has closed image.

T

P K pX, X q.

Then operator

4.2. FREDHOLM THEORY

99

Proof. The argument relies on the characterization of injective operators with

closed image, Proposition 3.1.12. So we consider the operator A  I
T and its
it suffices to show that A is
injectivization A : X { ker A X. Since Im A  Im A,
bounded below.
Suppose the contrary, that there exists cosets rxk s with }rxk s}  1 and such
that
Arxk s 0.
We can find representatives xk P rxk s with, say, }xk } 2 and such that
distpxk , ker Aq  }rxk s}  1

but Axk

 Arxk s 0.

So we have Axk  xk
T xk 0. By compactness of T , we can assume (passing

to a subsequence if necessary) that T xk z for some z P X. It follows that
xk z. Therefore Axk Az; but we already know that Axk 0, thus Az  0.
So z P ker A. Furthermore, since xk z it follows that distpxk , ker Aq 0. This
contradiction completes the proof.

4.2.2. Fredholm alternative. We now state and prove a partial case of the
so-called Fredholm alternative. The more general theorem is given in Exercise 4.2.4.
Theorem 4.2.2 (Fredholm alternative). Let X be a Banach space and T
K pX, X q. Then operator A  I
T is injective if and only if A is surjective.

Proof. Necessity. Assume that A is injective but not surjective. Consider the
subspaces of X
Yn : ImpAn q, n  0, 1, . . .
Then
Y0 Y1 Y2   
is a chain of proper inclusions. Indeed, the first inclusion X ImpAq is proper by
assumption; the claim follows by induction. (Check this!)
Furthermore, Yn are closed subspaces of X. Indeed, by Newtons binomial
expansion we see that An  pI
T qn has the form A  I
T1 for some compact
operator T1 , so the claim follows from Theorem 4.2.1.
By Hahn-Banach theorem (see Exercise 2.3.33) we can find functionals
fn

P Yn

}fn }  1,

such that

fn

P YnK 1 .

We can extend fn , again by Hahn-Banach theorem, so that fn P X  . We are

going to show that the sequence pT  fn q has no convergent subsequences. This
will contradict the compactness of T  and, by Schauders Theorem 4.1.17, the
compactness of T .
To this end, let us fix n m and compute the pairwise distances
dn,m : }T  fn
T  fm }  }T  pfn
fm q}  }pI

So
dn,m

sup

x BYn

|xT  fn
T  fm , xy| 

sup

x BYn

T q pfn
fm q

fm
fn }.

|xfn
fm , pI
T qxy xfm
fn , xy|.

Now, pI
T qx  Ax P Yn 1 while fn
fm P YnK 1 by construction, so xfn
fm , pI

T qxy  0. Further, x P Yn while fm P YnK , so xfm , xy  0. Therefore
dn,m

sup

x BYn

|xfn , xy|  1

4.3. SPECTRUM OF A BOUNDED LINEAR OPERATOR

100

by construction. It follows that the terms of the sequence pT  fn q are pairwise

separated, so there can not be any convergent subsequence. This completes the
proof of the necessity direction.
Sufficiency will follow from a duality argument. We use the relations
(4.3)

pIm AqK  ker A , pker AqK Im A

which we proved in Proposition 2.4.32 and Exercise 2.4.34. So, assume that A 
I
T is surjective. Then A  I
T  is injective by (4.3). Since T  is compact
by Schauders theorem, the first part of the proof gives that A is surjective. This
implies that A is injective by (4.3). The proof is complete.

Remark 4.2.3 (Compactness is essential). Fredholm alternative does not hold
for non-compact operators in general. For example, the right shift operator in `2
is injective but not surjective; the left shift operator in `2 is surjective but not
injective.
The name Fredholm alternative is explained by the following application to
solving linear equations of the form
x
T x  b
where T P K pX, X q, P C, b P X. One is interested in existence and uniqueness of
solution. Theorem 4.2.2 states that exactly one the following statements holds for
every  0:
either the homogeneous equation x
T x  0 has a nontrivial
solution,
or the inhomogeneous equation x
T x  b has a solution
for every b; this solution is automatically unique.

This alternative is particularly useful

for studying integral equations, since
1
for the integral operator pT f qptq  0 k pt, sqf psq ds, the homogeneous Fredholm
equation is
f ptq

1
0

k pt, sqf psq ds  0,

while the inhomogeneous Fredholm equation (of second kind) is

f ptq

1
0

k pt, sqf psq ds  bptq.

4.2.3. Additional exercises.

Exercise 4.2.4. [General Fredholm alternative] Let X be a Banach
space and T P K pX, X q. Show that operator A  I
T satisfies
dim ker A  dim ker A

 codim Im A  codim Im A .

4.3. SPECTRUM OF A BOUNDED LINEAR OPERATOR

101

4.3. Spectrum of a bounded linear operator

Lec. 33: 11/24

Studying linear operators through their spectral properties is a powerful approach in analysis and mathematical physics. Recall from linear algebra that the
spectrum of a linear operator T acting on Cn consists of the eigenvalues of T ,
which are the numbers P C such that T x  x for some nonzero vector x P Cn ;
such x are called the eigenvectors of T . Eigenvalues always exist by the fundamental theorem of algebra, as they are the roots of the characteristic polynomial
detpT
I q  0. There are at most n eigenvalues of T , or one can say exactly
n counting multiplicities. Eigenvectors corresponding to different eigenvalues are
linearly independent.2
4.3.1. Examples and definition of spectrum. In infinite-dimensional normed
spaces, the spectrum is a richer concept than in finite-dimensional spaces. Let us
illustrate the difference on two examples.
Example 4.3.1 (Uncoutnable number of eigenvalues). Consider the differential
operator
d
T 
dt

acting, for example, on C 1 pCq. To compute the spectrum of T , we solve the ordinary
differential equation u1  u. The solution has the form
uptq  Cet .
Therefore, every P C is an eigenvalue of T .

Example 4.3.2 (No eigenvalues). Consider a multiplication operator on L2 r0, 1s

acting as

pT f qptq  tf ptq.

Suppose is an eigenvalue of T with eigenvector f

following identity holds
tf ptq  f ptq
It follows that f

 0. Therefore, T

P L2 r0, 1s. This means that the

for all t P r0, 1s.

has no eigenvalues.

Definition 4.3.3 (Spectrum). Let X be a normed space and T P LpX, X q. A

number P C is called a regular point if T
I is invertible as a bounded linear
operator,3 i.e. pT
I q
1 P LpX, X q. All other are called spectrum points. The
set of all regular points is denoted pT q and is called the resolvent set of T . The
set of all spectrum points is denoted pT q and is called the spectrum of T .
2Recall however that the eigenvalues do not need to form a basis of Cn . The dimension of
the span of eigenvectors corresponding to a given eigenvalue (the eigenspace) may be strictly
 less

1
than the multiplicity of that root. This happens, for example, for the Jordan block T
.
0


An orthonormal basis of eigenvectors exists if and only if T is normal, i.e. T T
TT .
3In the future, we will often say invertible instead of invertible as a bounded linear
operator.

4.3. SPECTRUM OF A BOUNDED LINEAR OPERATOR

102

4.3.2. Classification of spectrum. For operators T acting on a finite dimensional space, the spectrum consists of eigenvalues of T . In infinite dimensions,
this is not true, as there are various reasons why T
I may be non-invertible.
These reasons are listed in the following definition:
Definition 4.3.4 (Classification of spectrum). Let X be a normed space and

P LpX, X q.

(i) The point spectrum p pT q is the set of all eigenvalues of T , i.e. the numbers
P C satisfying
kerpT
I q  0.

(ii) The continuous spectrum c pT q is the set of all P C such that

kerpT

(iii)

I q  0 and ImpT
I q is dense in X.

The residual spectrum r pT q is the set of all P C such that
kerpT
I q  0 and ImpT
I q is not dense in X.

So the spectrum of T can be expressed as a disjoint union

pT q  p pT q Y c pT q Y r pT q.
Let us now compute and classify the spectrum of come basic linear operators.
Example 4.3.5 (Diagonal operator on `2 ). Let us fix some sequence k
in Czt0u, and consider the operator T on `2 defined as

8
T ppxk q8
k1 q  pk xk qk1 .

8

1 y  yk
As pT
I qx  ppk
qxk q8
k1 , we have pT
I q
k
k1 . It follows


1
that pT
I q
is a bounded operator is and only if is not in the closure of
tk u8k1 , which is tk u8k1 Y t0u.
All k are clearly the eigenvalues of T as T ek  k ek for the canonical basis
pek q of `2 . 0 is not an eigenvalue since T is injective (as all k  0). So 0 is either
in continuous or residual spectrum. Now, Im T is dense in `2 (why?), so 0 is in the
continuous spectrum. Our conclusion is:
p pT q  tk u8
k 1 ,

c pT q  t0u,

r pT q  H.

Example 4.3.6 (Multiplication operator on L2 ). Let us consider the same

multiplication operator as in Example 4.3.2, i.e.

pT f qptq  tf ptq.
As pT
I qf ptq  pt
qf ptq, we have
(4.4)
pT
I q
1 yptq  t
1 yptq.
If R r0, 1s then the function t
1 is bounded, thus pT
I q
1 is a bounded operator.
Therefore such are regular points. Conversely, if P r0, 1s then t
1 R L2 r0, 1s
because of the non-integrable singularity at 0. Hence T
I is not invertible (at
y ptq  1). Hence all such are regular points. Therefore, pT q  r0, 1s.
As we noticed in Example 4.3.2, T has no eigenvalues. It follows from (4.4)
that ImpT
I q is dense in L2 r0, 1s. (Check!) Our conclusion is:
p pT q  H,

c pT q  r0, 1s,

r pT q  H.

4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS.

103

Remark 4.3.7. If Dirac delta function ptq was a genuine function in L2 , then
its translates ptq : pt
q would be the eigenvectors of the multiplication
operator on L2 :
T 
and would be the eigenfunctions of T . The situation would be similar to the
discrete multiplication operator from Example 4.3.5.
Example 4.3.8 (Shift operator). Consider the right and left shift operators on
`2 , acting on a vector x  px1 , x2 , . . .q as
Rpxq  p0, x1 , x2 , . . .q,

Lpxq  px2 , x3 , . . .q.

Since R is clearly injective but Im R is not dense in `2 (why?), 0 is in the residual

spectrum of R. One can show that
p pRq  H,

c pRq  t P C : ||  1u,

p pLq  t P C : || 1u,

r pRq  t P C : || 1u;

c pRq  t P C : ||  1u,

r pRq  H.

Exercise 4.3.9. Prove the claims about the spectra of shift operators
made in Example 4.3.8.
4.4. Properties of spectrum. Spectrum of compact operators.
Throughout this section, X denotes a Banach space and T

P LpX, X q.

Lec. 34: 11/29

4.4.1. Resolvent operator. Spectrum is bounded. Studying the spectrum of T is convenient via the so-called resolvent operator:
Definition 4.4.1 (Resolvent operator). To each regular point P pT q we
associate the operator
Rpq  pT
I q
1 .
Rpq is called the resolvent operator of T . So the resolvent is a function R : pT q
LpX, X q.
The resolvent operator can be computed in terms of series expansion involving
T . This technique is based on the following simple lemma:
Lemma 4.4.2 (Von Neumann). Consider an operator S P LpX, X q such that
}S } 1. Then I
S is invertible, and it can be expressed as a convergent series in
LpX, X q:
8

pI
S q
1  S k , }pI
S q
1 } 1
1}S } .

Proof. The series
}S } 1. Furthermore,

k 0

k
 S converges absolutely because }S }
k

k 0

pI
S q

Sk

k 0

as telescoping series. (Check!) Finally,

}pI
S q
1 }

Sq  I

k 0

k 0

This completes the proof.

S k pI

}S }k while

}S }k 1
1}S } .


4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS.

104

Proposition 4.4.3. The spectrum pT q is a bounded set. Specifically,

pT q t P C : || }T }u.

Proof. The claim is that || }T } implies P pT q, i.e. the operator T
I

is invertible. We express T
I  p 1 T
I q. Since } 1 T } 1, von Neumanns
Lemma 4.4.2 implies that the operator in question is indeed invertible.

Furthermore, writing the series expansion of the inverse of T
I
according to von Neumanns lemma, we immediately obtain:


pI
1 T q

Proposition 4.4.4 (Bound on the resolvent). If || }T } then

Rpq 

k
1 T k ,

k 1

}Rpq} ||
1 }T } .

4.4.2. Resolvent identity. Spectrum is closed.

Proposition 4.4.5 (Resolvent identity). For all , P pT q we have
Rpq
Rpq  p
qRpqRpq.

Proof. In the scalar case, the following identity clearly holds:

1
1





x
x

px
qpx
q
for all x, y P C, ,  x. It is an exercise to generalize it to the operator-valued
case, which yields the resolvent identity. (Do this!)

Solving the resolvent identity for Rpq we obtain for , P pT q that

(4.5)

p
qRpq
1 Rpq
The regular set pT q is an open set. Equivalently, the spec

Rpq  I

Corollary 4.4.6.
trum pT q is a closed set.

Proof. Let P pT q. Von Neumanns lemma implies that the operator I

p
qRpq is invertible for all in a suitably small neighborhood of . So for such

, the right hand side of (4.5) defines a bounded linear opearator. One can check
that in this case identity (4.5) holds (do this!) and therefore P pT q.

4.4.3. Resolvent is an analytic function. Spectrum is nonempty. The
proof of Corollary 4.4.6 gives us a bit more information about the resolvent than we
have noticed. Let us go fo back to identity (4.5) and write the series expansion of
the inverse of p
qRpq
I according to von Neumanns lemma. We immediately
obtain:
Corollary 4.4.7 (Resolvent expansion). The resolvent Rpq is an analytic
operator-valued function on its domain pT q. Specifically, Rpq can be expressed as
a convergent power series in a small neighborhood of any point P pT q:
(4.6)

Rpq 

p
qk
1 Rpqk .

k 1

Remark 4.4.8. It follows that for every functional f P LpX, X q , the function
f pRpqq is a usual (i.e. complex-valued) analytic function on pT q.
Theorem 4.4.9. The spectrum pT q is a nonempty set.

4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS.

105

Proof. We shall deduce this result from Liouvilles theorem in complex analysis.4 To this end, assume that pT q  H, hence pT q  C and the resolvent Rpq
is an entire function (i.e. analytic on the whole complex plane).
Claim. Rpq is also bounded function on C with Rpq 0 as 8.
Indeed, by Proposition 4.4.4, Rpq is a bounded in the annulus || 2}T } and
vanishes at infinity. Since Rpq is a continuous function by Corollary 4.4.7, Rpq is
also bounded in the disc || 2}T }.
Claim. By Liouvilles theorem, Rpq  0 everywhere.
Indeed, we fix a functional f P LpX, X q and apply the usual Liouvilles theorem for the bounded entire function f pRpqq. It follows that f pRpqq is constant,
and since it must vanish at infinity it is zero everywhere. The claim follows.
The last claim contradicts the fact that Rpq is an invertible operator.

Summarizing our findings, we can state that the spectrum of every bounded
linear operator is a nonempty compact subset of C.

4.4.4. Spectral radius. The spectrum of any operator T P LpX, X q is a

bounded set by Proposition 4.4.3, and moreover we have a quantitative bound
|| }T } for all P pT q. This bound is not always sharp, and we will try to come
up with a sharp bound.
Definition 4.4.10. The spectral radius of an operator T
as

P LpX, X q is defined

rpT q  max || : P pT q .

Theorem 4.4.11 (Gelfands formula). For every operator T

on a Banach space X, one has

P LpX, X q acting

rpT q  lim }T n }1{n

 inf
}T n }1{n .
n
Exercise 4.4.12. Clearly rpT q }T }n1{n }T }, so Gelfands formula is
an improvement upon Proposition 4.4.3. Give an example where rpT q
}T }.
n

Remark 4.4.13. Gelfands formula gives an asymptotics for the growth of

operator powers }T n }  rpT qn .

Proof. Upper bound. Let P pT q. One can easily show that n P pT n q,

see Exercise 4.4.27.5 Therefore, using Proposition 4.4.3, we have |n | }T n }, so
|| }T n }1{n . We conclude that
rpT q inf }T n }1{n .
n

Lower bound will be obtained as an application of the theory of Laurent series

in complex analysis.6 Consider the function f pRpqq where Rpq is the resolvent of
T and f P LpX, X q is an arbitrary functional. Recall two facts:
4Liouvilles theorem states that an entire (i.e. analytic on C) and bounded function f : C

C
is constant everywhere.
5This is a partial case of the spectral mapping theorem we will study later.
6Specifically, we shall use the following theorem of compex analysis. Consider a Laurent
series

p q

f z

ak pz
z0 qk .
k
8

Lec. 35: 12/1

4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS.

106

(i) f pRpqq is an analytic function on the annulus || rpT q by Corollary 4.4.4;

(ii) f pRpqq is represented by a convergent Laurent series
f pRpqq 

(4.7)

k
1 f pT k q

k 1

in the smaller annulus || }T }.

By the theory of convergence of Laurent series (outlined in the footnote), the
series (4.7) converges in the larger annulus || rpT q. So the terms of the series
are bounded for such :


sup 
n
1 f pT n q 8.
n

This means that the sequence p
n
1 T n ) is weakly bounded in LpX, X q. By a

consequence of the principle of uniform boundedness (Corollary 3.3.5), this sequence
is (strongly) bounded, i.e.


sup 
n
1 T n  : K
n

8.

Taking n-th root and rearranging the terms, we obtain }T n }1{n K 1{n 1 1{n for
all n. It follows that lim supn }T n }1{n ||. Since this happens for all such that
|| rpT q, we have proved that
lim sup }T n }1{n
n

rpT q.

So, putting this together with the upper bound, we have proved that
rpT q inf }T n }1{n
n

limninf }T n }1{n lim sup }T n }1{n rpT q.

n

This completes the proof.

4.4.5. Spectrum of compact operators. As compact operators are proxies

of finite rank operators, one is able to fully classify their spectrum. First of all, for
every T P K pX, X q one has
0 P pT q
since T is not invertible by Corollary 4.1.8.
Exercise 4.4.14. Construct three examples of compact operators for
which 0 is in the point, continuous, and residual spectrum respectively.
Theorem 4.4.15 (Point spectrum of compact operators). Let T P K pX, X q be
a compact operator on a normed space X. For every 0 there exists a finite
number of linearly independent eigennvectors corresponding to eigenvalues k with
|k | .
Consequently, the point spectrum p pT q is at most countable, and it lies in a
sequence that converges to zero.
It also follows that each eigenvalue k of T has finite multiplicity, i.e. dim kerpT

k I q 8.
There exist unique radii r, R

P R X t8u such that the Laurent series converges in the annulus

A  t z P C : r | z
z0 | R u

and diverges outside the closure of A. Moreover, there exists at least one point on the inner
boundary z C : z z0
r of A and at least one point on the outer boundary z C :
z z0
R of A such that f z can not be analytically continues to those points.

t P
|
| u

|
| u
pq

t P

4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS.

107

Proof. Clearly, the second and third claims of the theorem follow from the
first one (why?). So, assume the contrary, that there exist 0 and an infinite
sequence of linearly independent vectors pxk q8
k1 such that

 k xk , where |k | .
Consider the subspaces En  Spanpxk qnk1 ; then E1 E2   
T xk

is a sequence of

proper inclusions. Therefore we can choose vectors

P En , }yn }  1, distpyn , En
1 q 12 .

(Why? Think about En {En
1 .)
yn

Figure 4.1. Construction of subspaces En and vectors yn

We will show that the sequence pT yn q8
n1 contains no Cauchy subsequences,
which will contradict compactness of T . To this end, we express yn as a linear
combination
yn

pnq

a k xk

 apnnq xn

un
1 ,

where un
1

P En
1 .

k 1

Then

 n apnnq xn

 T un
1 P En
1 .
Now we are ready to estimate }T yn
T ym } for n m. Since T ym P Em En
1 ,
T yn

vn
1 ,

where vn
1

we obtain

}T yn
T ym }  }n apnnq xn wn
1 } where wn
1 P En
1

 }n yn zn
1 } where wn
1 P En
1
|n | distpyn , En
1 q 2 .
It follows that pT yn q8
n1 contains no Cauchy subsequences as claimed.
is complete.

The proof


Proposition 4.4.16 (Classification of spectrum of compact operators). Let

P K pX, X q be a compact operator on a Banach space X. Then

pT q  p pT q Y t0u.

Proof. As we already noticed in the beginning of this section, 0 P pT q. Let

now P pT q,  0. Fredholm alternative (Theorem 4.2.2) states that either
T
I is not injective (in which case P p pT q) or T
I is both injective and
surjective. In the latter case T
I is invertible by the inverse mapping theorem,
which means that R pT q. The proof is complete.


4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS.

108

4.4.6. Spectrum of unitary operators. In this subsection, H denotes a

Hilbert space.
Definition 4.4.17. An operator U P LpH, H q is called a unitary operator if U
is a bijective isometry on H. The latter means that U is bijective and

}U x}  }x}

for all x P H.

Example 4.4.18. Examples of unitary operators include:

(i) operators on Cn and Rn given by n  n unitary complex matrices and orthogonal real matrices; in particular rotations, symmetries, and permutations of
coordinates in Cn and Rn ;
(ii) right shift R on `2 (but not left why?)
(iii) an isometry between any pair of separable Hilbert spaces established in Theorem 1.6.30.
Remark 4.4.19. A unitary operator U preserves all pairwise distances, i.e.
Moreover, by polarization identity 1.4.19, U also preserves
the inner products:

}U x
U y}  }x
y}.

xU x, U yy  xx, yy

for all x, y

P H.

P LpH, H q is unitary if and only if

U  U  U U   I,
i.e. if and only if U is invertible and U
1  U  .
This identity is analogous to the one for unit complex numbers, zz  zz  1.
Proposition 4.4.20. An operator U

Proof. Necessity. If U is unitary then xU  U x, y y  xU x, U y y  xx, y y. Since

this holds for arbitrary x, y P H, it follows that U  U  I. Similarly one proves
that U U   I.
Sufficiency. Since U is invertible it is bijective. The isometry property follows
because }U x}2  xU x, U xy  xU  U x, xy  xx, xy  }x}2 . The proof is complete.

Proposition 4.4.21 (Spectrum of unitary operators). The spectrum of a unitary operator U P LpH, H q lies on the unit circle:
pU q t P C : ||  1u.

Proof. The isometry property implies that }U }  }U
1 }  1. Therefore the

spectral radius is rpU q }U } 1 by Proposition spectrum bounded. On the
other hand, if || 1 then the operator U
1 pU
I q  I
U
1 is invertible by
von Neumanns Lemma 4.4.3 since }U
1 }  || 1. It follows that U
I is
invertible.

Exercise 4.4.22. Show that eigenvectors of a unitary operator U that
correspond to distinct eigenvalues are orthogonal.

4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS.

109

4.4.7. Additional exercises. In the following two exercises, one can work
over R. Similar results hold over C. The only difference is that for Hilbert spaces,
one has to take complex conjugation in appropriate places (which ones?), see Remark 2.4.26.
Exercise 4.4.23. [Spectrum of adjoint I] Let T P LpX, X q. Prove that
pT  q  pT q. Here the bar stands for complex conjugation rather than
for closure.
Exercise 4.4.24. [Spectrum of adjoint II] Let T P LpX, X q
(i) Prove that if P p pT q and R p pT  q then P r pT  q. (Hint: use
the duality relations from Proposition 2.4.32 and Exercise 2.4.34
for the operator T
I.)
(ii) Prove that
r pT q p pT  q r pT q Y p pT q.
Deduce that if X is reflexive, then r pT  q p pT q. Deduce that
self-adjoint bounded linear operators in Hilbert space do not have
residual spectrum.
Exercise 4.4.25. [General multiplication operator on L2 ] Consider a
general multiplication operator T acting on L2 r0, 1s as

pT f qptq  gptqf ptq

where g

P L8 r0, 1s is some fixed multiplier function.

Exercise 4.4.26. [Invertibility of a composition] Let S, T P LpX, X q.

Prove that the operator ST is invertible if and only if both S and T are
invertible.
Exercise 4.4.27. [Partial case of spectral mapping theorem] Let T P
LpX, X q. Prove that P pT q implies n P pT n q. (Hint: (i) factor T n

n I  S pT
I q for some S P LpX, X q. (ii) Show that for U, V P LpX, X q,
the operator U V is invertible if and only if both U and V are invertible.)
Exercise 4.4.28. [Spectrum of projections] Compute the spectrum of
a projection P P LpX, X q on a Banach space X.

CHAPTER 5

Self-adjoint operators on Hilbert space

Lec. 36: 12/03

Throughout this chapter, H will denote a Hilbert space, and we will study
bounded self-adjoint operators T on H.
5.1. Spectrum of self-adjoint operators
5.1.1. Definition and examples. Let T be a bounded linear operator on a
Hilbert space, i.e. T P LpH, H q. Recall from Section 2.4.9 that the adjoint operator
T  P LpH, H q is defined by xT  x, y y  xx, T y y for x, y P H.
Definition 5.1.1. An operator T
i.e.

P LpH, H q is called self-adjoint if T   T ,

xT x, yy  xx, T yy,

x, y

P H.

Example 5.1.2. Examples of self-adjoint operators include:

(i) linear operators on Cn given by Hermitian matrices paij q, i.e. such that aij 
aji ;
1
(ii) integral operators pT f qptq  0 k ps, tqf psq ds on L2 r0, 1s with Hermitian symmetric kernels, i.e. such that k ps, tq  k pt, sq;
(iii) orthogonal projections P on H. (Why?)
Every bounded linear operator can be decomposed into two self-adjoint operators:
Lemma 5.1.3. Every operator A P LpH, H q can be uniquely represented as
AT

iS

P LpH, H q are self-adjoint operators.

Proof. If A  T iS then A  T
iS. Solving these two equations we see


that the lemma holds with T  A 2A and S  A
2iA .

where T, S

Exercise 5.1.4. Prove that the set of self-adjoint operators forms a

closed linear subspace in LpH, H q.
5.1.2. The quadratic form and the norm of a self-adjoint operator.
It is convenient to study self-adjoint operators T P LpH, H q through the quadratic
form
f pxq  xT x, xy, x P H.
One sees immediately that this quadratic form is real-valued, i.e. f pxq P R for
all x P H.1

y  xx, T xy  xT x, xy.

1This follows from the identity T x, x

110

5.1. SPECTRUM OF SELF-ADJOINT OPERATORS

111

Furthermore, the quadratic form f pT q determines the operator T uniquely.

This follows from the polarization identity that relates bilinear and quadratic forms:
(5.1)

xT x, yy  41 f px

y q
f px
y q

if px

iy q
if px
iy q

This is a slight generalization of the polarization identity of Proposition 1.4.19,

where T  I. (Prove it!)
The norm of T can be conveniently computed from the quadratic form f pT q:
Proposition 5.1.5 (Norm of a self-adjoint operator). For every self-adjoint
operator T P LpH, H q, one has

}T } 

sup

x SH

|xT x, xy|.

Proof. The lower bound follows by definition of the operator norm:

}T } 

sup }T x}  sup

x SH

x,y SH

|xT x, yy|

sup

x SH

|xT x, xy| : M.

It remains to show that the inequality here is actually the identity. To this end, we
note that
sup |xT x, y y|  sup RexT x, y y

x,y SH

x,y SH

and use the real part of polarization identity (5.1):


1
RexT x, y y  xT px y q, x y y
xT px
y q, x
y y
4


M4 }x y}2 }x
y}2 (by the definition of M )


M4 2}x}2 2}x}2 (by the parallelogram law)
M (as }x}  }y}  1).
This completes the proof.

5.1.3. Criterion of spectrum points. We would like to study the spectrum

of self-adjoint operators T P LpH, H q. An easy observation is that all eigenvalues
of T must be real, that is
p pT q R.
Indeed, if is an eigenvalue with an eigenvector x then xT x, xy  xx, xy  xx, xy
which must be the same as xx, T xy  xx, xy  xx, xy. This shows that  , so
P R.
We will soon prove that the whole spectrum of T is real, i.e. pT q R, and
moreover we will compute the smallest interval containing pT q. Let us start with
ruling out the residual spectrum:
Proposition 5.1.6 (No residual spectrum). Let T
operator. Then r pT q  H.

P LpH, H q be a self-adjoint

Proof. Let P r pT q. This means that kerpT
I q  0 while ImpT
I q is

not dense in H. Since is not an eigenvalue, is not an eigenvalue either (recall
that all eigenvalues of T must be real). Using this and the duality relation (2.4.32),
we obtain that pImpT
I qqK  kerpT
I q  kerpT
I q  0. It follows that
ImpT
I q is dense in H, which is a contradiction.


5.1. SPECTRUM OF SELF-ADJOINT OPERATORS

112

Proposition 5.1.7 (Invertibility criterion). Let T P LpH, H q be a self-adjoint

operator. T is invertible2 if and only if T is bounded below, i.e. there exists c 0
such that
}T x} c}x} for all x P H.

Proof. If T is invertible then T is bounded below with c  1{}T
1 }. To prove

the converse, assume that T is bounded below. Then by the criterion of isomorphic
embedding (Proposition 3.1.12), T is injective and Im T is closed in H. On the
other hand, since 0 R r pT q  H, injectivity of T implies that Im T is dense in H.
It follows that Im T  H. So T is injective and surjective, thus T is invertible by
the inverse mapping theorem.


I, we immediately obtain
Corollary 5.1.8 (Criterion of spectrum points). Let T P LpH, H q be a selfadjoint operator. Then P pT q if and only if the operator T
I is not bounded
Applying this result for the operator T

below.

Remark 5.1.9 (Approximate point spectrum). A number P pT q for which

T
I is not bounded below is called an approximate eigenvalue of T , and the
set of all approximate eigenvalues is called the approximate point spectrum of T .
Corollary 5.1.8 states that for self-adjoint operators, the whole spectrum is the
approximate point spectrum.
The reason for the name approximate is the following. If is an eigenvalue
then pT
I qx  0 for some x P SH . If is an approximate eigenvalue then
pT
I qx can be made arbitrarily close to zero for some x P SH . So, eigenvalues
of T form the point spectrum p pT q while the approximate eigenvalues of T form
the continuous spectrum c pT q.
5.1.4. The spectrum interval. Now we compute the tightest interval that
contains the spectrum of a self-adjoint operator T . This interval can be computed
from the quadratic form of T :
Theorem 5.1.10 (Spectrum interval). Let T P LpH, H q be a self-adjoint operator. Then
(i) The spectrum of T is real, and moreover pT q rm, M s where
m  inf xT x, xy,

x SH

sup xT x, xy.

x SH

P pT q.
(i) Let P Czrm, M s; since the interval is closed we have
d : distp, rm, M sq 0.

(ii) The endpoints m, M

Proof.

Given x P SH , we use Cauchy-Schwarz to obtain the lower bound



}pT
I qx} xpT
I qx, xy  xT x, xy
 d

where the last inequality follows because xT x, xy P rm, M s by definition. We have
shown that T
I is bounded below. By the criterion of spectrum points (Proposition 5.1.8), we conclude that R pT q.
(ii) Let us show that M P pT q; the claim for m can be proved similarly. (Do
this!) Without loss of generality we can assume that 0 m M (This follows by

P LpH, H q.

2As usual, by being invertible we mean continuously invertible, i.e. T 1

5.2. SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS

113

Figure 5.1. Distance from to the spectrum interval rm, M s is d.

a translation argument, namely by considering T
mI instead of T . Check this!)
So }T }  M .
Let us choose a sequence of vectors xn P SH so that xT xn , xn y M . Then
@

}pT
M I qxn }2  pT
M I qxn , pT
M I qxn

 }T xn }2
2M xT xn , xn y M 2 }xn }2 .
Now, }T xn }2 }T }2  M 2 , xT xn , xn y M and }xn }2  1. It follows that
lim sup }pT
M I qxn }2 M 2
2M 2 M 2  0
so T
M I is not bounded below. Therefore M P pT q. The proof is complete. 
As a consequence of this result, the spectral radius rpT q of a self-adjoint operator equals }T }, so Proposition 4.4.3 is tight and Gelfands formula is useless for
self-adjoint operators:

Corollary 5.1.11 (Spectral radius). Let T P LpH, H q be a self-adjoint operator. Then

rpT q  max ||  }T }.

Pp q

Proof. By the properties of the spectrum interval in Theorem 5.1.10,

rpT q  maxp|m|, |M |q  }T }
as claimed.

5.1.5. Additional Exercises.

Exercise 5.1.12. Consider a self-adjoint operator P
that P 2  P . Prove that P is an orthogonal projection.

P LpH, H q

such

5.2. Spectral theorem for compact self-adjoint operators

Lec.37: 12/06

Compact self-adjoint operators on a Hilbert space H are proxies of Hermitian

matrices. As we know from linear algebra, every Hermitian matrix has diagonal
form in some orthonormal basis of Cn . Equivalently, such for such a matrix there
exists an orthonormal basis of Cn consisting of the eigenvectors. In this section, we
generalize this fact to infinite dimensions, for all compact self-adjoint operators on
H.

5.2. SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS

114

5.2.1. Invariant subspaces.

Proposition 5.2.1 (Eigenvectors orthogonal). Let T P LpH, H q be a selfadjoint operator. Then its eigenvectors corresponding to distinct eigenvalues are
orthogonal.

 1 x1 and T x2  2 x2 then
1 xx1 , x2 y  xT x1 , x2 y  xx1 , T x2 y  2 xx1 , x2 y.

Proof. If T x1

(In the last identity we used that 2 is always real, so there is no conjugation). It
follows that if 1  2 then xx1 , x2 y  0 as claimed.

Definition 5.2.2 (Invariant subspace). A subspace E of H is called an invariant subspace of T if T pE q E.
Example 5.2.3. Every eigenspace of T is invariant. More generally, the linear
span of any subset of eigenvectors of T is an invariant subspace.
One of the most well known open problems in functional analysis is the invariant subspace problem. It asks whether every operator T P LpH, H q has a proper
invariant subspace (i.e. different from t0u and H).
Proposition 5.2.4. Let T P LpH, H q be self-adjoint. If E
subspace of T then E K is also an invariant subspace of T .

H is an invariant

Proof. Let x P E K ; we need check that T x P E K . So let us choose y

arbitrarily. Then xT x, y y  xx, T y y  0 since x P E K and y P E so T y P E.

PE

5.2.2. Spectral theorem. The following result is known as the Hilbert-Schmidt

theorem.
Theorem 5.2.5 (Spectral theorem for compact self-adjoint operators). Let T
be a compact self-adjoint linear operator on a separable Hilbert space H. Then there
exists an orthonormal basis of H consisting of eigenvectors of H.
Proof. Let us first prove that T has at least one eigenvector.
By Proposition 4.4.16,
pT q  p pT q Y t0u.

If pT q  0 then p pT q  H, so T has an eigenvector. If pT q  0 then by

Corollary 5.1.11 we have }T }  rpT q  0, so T  0 and every vector in H is an
eigenvector of T .
We will complete the proof by induction. Consider the family of all orthonormal
sets in H consisting of eigenvectors of T . All such sets are at most countable since
H is separable. By Zorns lemma, this family has a maximal element pk q8
k 1 .
(Check!) It remains to show that E : Spanpk q  H.
Suppose E  H. Since E is an invariant subspace of T (check!), E K  t0u is
also an invariant subspace of T by Proposition 5.2.4. So we can use the first part of
the proof for the restriction T |E K which is a compact self-adoint operator on E K .
It follows that T |E K (and thus T itself) has an eigenvector in E K . This contradicts
the maximality of pk q. The proof is complete.


5.2. SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS

115

5.2.3. Diagonalization. Spectral Theorem 5.2.5 allows us to always represent compact self-adoint operators T P LpH, H q in a diagonal form, similarly to the
one for Hermitian matrices.
Let pk q be an orthonormal basis of eigenvectors of T . Then T k  k where
n are the eigenvalues. We can identify the space H with `2 by identifying pk q
with the canonical basis pek q of `2 (recall Section 1.6.7). With this identification,
T becomes a multiplication operator acting on `2 as T ek  k ek ; equivalently

8
T ppxk q8
k1 q  pk xk qk1 .

We see that T now has a quite simple form, which we studied in Example 4.3.5.
In literature, one comes across various forms of spectral Theorem 5.2.5. We
mention two of them. Let as before pk q denote an orthonormal basis of eigenvectors
of T with corresponding eigenvalues k . Orthogonal basis expansion gives
x

xx, k yk ,

x P H.

Applying operator T for both sides and using that T k

Tx 

n xx, n yn ,

 k k we obtain that

x P H.

This is sometimes written as

(5.2)

k k b k

where k b k P LpH, H q denotes the elementary tensor, which is the rank-one

orthogonal projection in H onto the span of the vector k .
Furthermore, by Theorem 4.4.15 all eigenvalues k have finite multiplicity. So,
grouping in (5.2) the tensors k b k that correspond to the same k , we obtain
the following decomposition of T :
(5.3)

k Pk

where Pk denotes the orthogonal projection in H onto the (finite-dimensional)

eigenspace corresponding to the eigenvalue k .
Remark 5.2.6 (Normal operators). One can show that spectral Theorem 5.2.5
holds also for compact normal linear operators T on H. Recall that T is normal
if T  T  T T  . The only difference between the spectral properties of self-adjoint
and normal operators is that the eigenvalues of normal operators need not be real.
5.2.4. Separation of variables. We illustrate Spectral Theorem 5.2.5 with
a purely analytic consequence. The following result shows how one can separate
variables of a general function k pt, sq. It is also due to Hilbert and Schmidt.

Theorem 5.2.7 (Separation of variables). Consider a function k pt, sq P L2 pr0, 1s2 q

such that k pt, sq  k ps, tq. There exists an orthonormal basis pn q of L2 r0, 1s and
numbers k 0 such thath
k pt, sq 

n 1

n n ptqn psq.

The convergence of this series is understood in L2 pr0, 1s2 q.

5.3. POSITIVE OPERATORS. CONTINUOUS FUNCTIONAL CALCULUS

116

Proof. Consider the integral operator pT f qptq  0 k pt, sqf psq ds on L2 r0, 1s.
Let pn q be an orthonormal basis of its eigenvectors. Then the functions

 n ptqm psq, n, m  1, 2, . . .
form an orthonormal basis of L2 pr0, 1s2 q. (Check!)
Let us write the basis expansion of our function in L2 pr0, 1s2 q:

k
xk, nm ynm .
nm

n,m

Now we compute the coefficients

xk, nm y 


11
0
1

k pt, sqn ptqm psq dt ds

1
0

k pt, sqm psq ds n ptq dt (by Fubini theorem)

 pT m qptqn ptq dt  xT m , n y  m xm , n y


#0

m ,
0,

Therefore
k

nm
n  m.

 xk, nn ynn 

n n ptqn psq

as claimed.

5.2.5. Additional exercises.

Exercise 5.2.8. [Spectral theorem for general compact operators] Let
T be a compact linear operator on a separable Hilbert space H. Show
that there exist an orthonormal basis pk q of H, an orthonormal system
pk q in H, and a sequence of numbers k 0, k 0 such that
T

n k b k .

The numbers k are called singular values of T and the vectors k and
k are called left (resp. right) singular vectors of T .
(Hint: Choose pk q to be an orthonormal basis of eigenvectors of
T  T . Write the basis expansion of x P H and apply T to both sides.)
5.3. Positive operators. Continuous functional calculus
Lec. 38: 12/8

It is a powerful idea to work with linear operators by analogy with complex

numbers. For example, both numbers and operators can be added and multiplied,
so both C and LpH, H q are algebras (over C). Moreover, complex conjugate corresponds to operator adjoint, which makes LpH, H q a  -algebra. Operator algebras
is a rich topic which we omit in this course.
In this section, we develop the analogy between numbers and operators in the
following way. We introduce a partial order on the set of self-adjoint operators
on T P LpH, H q, and we define an operator f pT q P LpH, H q for every continuous

5.3. POSITIVE OPERATORS. CONTINUOUS FUNCTIONAL CALCULUS

117

function f : C C. In other words, we develop what is called the functional

calculus of operators.
5.3.1. Positive operators.

P LpH, H q is called positive3 if

for all x P H.

Definition 5.3.1. A self-adjoint operator T

xT x, xy 0

Positive operators are generalizations of non-negative numbers (which correspond to operators on one-dimensional space C1 ).
Example 5.3.2. Examples of positive operators include:
(i) T 2 for every self-adjoint T P LpH, H q, as xT 2 x, xy  xT x, T xy 0;
(ii) Hermitian matrices with non-negative eigenvalues;
(iii) More generally, compact self-adjoint operators on H with non-negative eigenvalues. (Why?)
Definition 5.3.3 (Partial order). For self-adjoint operators S, T
we shall say that S T if T
S 0.

P LpH, H q,

This defines a partial order on LpH, H q.

Let us restate Theorem 5.1.10 on the spectrum interval in these new terms:

Theorem 5.3.4 (Spectrum interval). Let T P LpH, H q be a self-adjoint operator. Let m, M be the smallest and the largest numbers such that

T M I.
Then pT q rm, M s and m, M P pT q.
mI

As an immediate corollary, T is positive if and only if its spectrum is positive:

Corollary 5.3.5. Let T
and only if pT q r0, 8q.

P LpH, H q be a self-adjoint operator.

Then T

0 if


5.3.2. Polynomials of an operator. We start to develop a functional calculus for self-adjoint operators T P LpH, H q. We begin by defining polynomials of T ,
then we extend the definition to continuous functions of T by approximation. Working with polynomials is straightforward, and the result of this subsection remain
valid for every bounded linear operator T on a general Banach space X.
a0

Definition 5.3.6 (Polynomials of an operator). Consider a polynomial pptq 

a1 t    an tn . For an operator T P LpH, H q, we define
ppT q  a0 I

a1 T

   an T n .

If T is self-adjoint operator then ppT q is also self-adjoint. (Check!) Moreover,

for two polynomials f and g, one has

paf

bg qpT q  a  f pT q

b  g pT q,

pf gqpT q  f pT qgpT q,

f pT q  f pT  q.

This last property states in other words that for a fixed T P LpH, H q, the map
p ppT q is an -algebra homomorphism from P rts into LpH, H q.
The following example may serve us as a test case for many future results.
3In linear algebra, positive operators are called positive semidefinite.

5.3. POSITIVE OPERATORS. CONTINUOUS FUNCTIONAL CALCULUS

118

Example 5.3.7. Let T be a self-adjoint linear operator on an n-dimensional

Hilbert space. In an orthonormal basis of eigenvectors, T can be identified with the
n  n diagonal matrix
T  diagp1 , . . . , n q
where k are the eigenvalues of T . Then for every polynomial pptq we have
ppT q  diagppp1 q, . . . , ppn qq.

(Check!) This example can be generalized for all compact self-adjoint operators T
on a general Hilbert space H. (Do this!)
5.3.3. Spectral mapping theorem for polynomials.
Lemma 5.3.8 (Invertibility). Let pptq be a polynomial and T P LpH, H q. Then
the operator ppT q is invertible if and only if pptq  0 for all t P pT q.

Proof. Let us factorize pptq  an pt
t1 q    pt
tn q where tk are the roots of

pptq. Then ppT q  an pT
t1 I q    pT
tn I q. By Exercise 4.4.26, ppT q is invertible
if and only if all factors T
tk I are invertible. This in turn is equivalent to the fact
that all roots tk R pT q.

The spectrum of a polynomial ppT q can be easily computed from the spectrum
of T :
T

Theorem 5.3.9 (Spectral mapping theorem). Let pptq be a polynomial and

P LpH, H q. Then4

pppT qq  pp pT qq.

Proof. For every complex number , we have P pppT qq if and only if the
operator ppT q
I  pp
qpT q is not invertible. By the invertibility Lemma 5.3.8,
this is equivalent to the condition that pp
qptq  0 for some t P pT q, which
means that pptq  for some t P pT q. The latter is equivalent to P pp pT qq. 
Using the spectral mapping theorem, one can in particular easily compute the
norms of operator polynomials:
Corollary 5.3.10 (Operator norm of polynomials). Let pptq be a polynomial
and T P LpH, H q be a self-adjoint operator. Then

}ppT q}  tmax
|pptq|.
PpT q

This result generalizes the identity rpT q  }T } for the spectral radius of selfadjoint operators T proved in Corollary 5.1.11.
Proof. Let us apply Corollary 5.1.11 for the operator ppT q. Then spectral
mapping theorem yields

}ppT q}  rpppT qq  tPmax

|t|  tPpmax
|t|  smax
|ppsq|
pppT qq
ppT qq
PpT q
as claimed.

p p qq : tpptq : t P pT qu.

4Here we use the notation p T

5.3. POSITIVE OPERATORS. CONTINUOUS FUNCTIONAL CALCULUS

119

5.3.4. Continuous functions of an operator. Let T P LpH, H q be a selfadjoint operator, and f ptq be a continuous function on pT q. We would like to
define f pT q P LpH, H q. To this end, we use Weierstrass approximation theorem,5
and we find polynomials pn ptq such that
pn ptq f ptq

(5.4)

uniformly on pT q.

This suggests us to define f pT q as the limit of operator polynomials pn pT q:

Definition/Proposition 5.3.11 (Continuous functions of an operator).
(i) The sequence pn pT q converges in LpH, H q to a limit that we call f pT q.
(ii) The operator f pT q P LpH, H q is self-adjoint, and it does not depend on the
choice of the approximating polynomials pn .
Proof. (i) By completeness of LpH, H q it suffices to check that ppn pT qq is a
Cauchy sequence in the operator norm. Computing the operator norm with the
help of Corollary 5.3.10 and using (5.4), we obtain the desired conclusion that

}pn pT q
pm pT q}  }ppn
pm qpT q}  tmax

|pp
p qptq| 0
PpT q n m

as n, m 8.

(ii) Since the operators pn pT q are self-adjoint, and the self-adjoint operators
form a closed subset of LpH, H q (Exercise 5.1.4), f pT q is also self-adjoint. Furthermore, repeating the estimate in part (i), one sees that for any other approximating
sequence of polynomials qn ptq one has }pn pT q
qn pT q} 0 as n 8. It follows
that the limit f pT q must be the same whether one chooses pn pT q or qn pT q as an
approximating sequence.

By passing to the limit in the corresponding properties for polynomials, one
sees that for two polynomials f and g we have

paf

bg qpT q  a  f pT q

b  g pT q,

pf gqpT q  f pT qgpT q,

f pT q  f pT  q.

(Check!) This property states in other words that for a fixed T P LpH, H q, the map
f f pT q is an -algebra homomorphism from C p pT qq into LpH, H q.
Example 5.3.12. Consider an invertible self-adjoint operator T P LpH, H q;
then pT q P rm, M s with m 0. Consider the function f ptq  1{t, which is
continuous on rm, M s. Then f pT q  T
1 (Check!) In other words, we have the
remarkable identity of reciprocal and inverse:
1
T

 T
1 .
Lec. 39: 12/10

5.3.5. Spectral mapping theorem. We will now generalize the spectral

mapping Theorem 5.3.9 from polynomials to continuous functions of an operator.
It is based on the straightforward generalization of the invertibility Lemma 5.3.8:
Lemma 5.3.13 (Invertibility). Let T P LpH, H q be a self-adjoint operator and
f P C p pT qq. Then the operator f pT q is invertible if and only if f ptq  0 for all
t P pT q.
5Formally, since Weierstrass theorem is typically stated for functions on an interval, we first
extend f t to a continuous function on some interval m, M
T (for example, the spectral
interval). Such extension can be done e.g. by Tietze extension theorem.

pq

s p q

5.3. POSITIVE OPERATORS. CONTINUOUS FUNCTIONAL CALCULUS

120

Proof. Sufficiency. If f ptq  0 for all t P pT q then 1{f P C p pT qq, so 1{f pT q

is the inverse of T (see Example 5.3.12).
Necessity. Assume that f pt0 q  0 for some t0 P pT q. Choose a sequence
of polynomials pn ptq such that pn ptq f ptq uniformly on spT q. Without loss
of generality, we can assume that pn pt0 q  f pt0 q  0. (Justify this by a slight
translation!) Then pn pT q f pT q in LpH, H q.
On the other hand, the invertibility Lemma 5.3.8 for polynomials, the operators
pn pT q are not invertible. Since the non-invertible operators form a closed subset of
LpH, H q (see Exercise 2.4.44), it follows that f pT q is not invertible. This completes
the proof.

Now the spectral mapping theorem follows from invertibility Lemma 5.3.13 by
the same argument as the corresponding result for polynomials, Theorem 5.3.9:
Theorem 5.3.14 (Spectral mapping theorem). Let T
adjoint operator and f P C p pT qq. Then
pf pT qq  f p pT qq.

P LpH, H q

be a self-

This gives a simple way to create positive operators:

Corollary 5.3.15. Let T P LpH, H q be a self-adjoint operator and f

If f ptq 0 for all t P pT q then f pT q 0.

P C ppT qq.

Proof. By Corollary 5.3.5 it suffices to check that pf pT qq r0, 8q. The

latter follows from the spectral mapping theorem, as pf pT qq  f p pT qq r0, 8q.

Exercise 5.3.16. Prove the converse statement in Corollary 5.3.15.

It follows that the algebra homomorphism from C p pT qq into LpH, H q preserves

not only addition and multiplication, but also the order:
f ptq g ptq for all t P pT q

implies

f pT q g pT q.

Exercise 5.3.17. [Further properties of continuous functions of an operator] Let T, S P LpH, H q be self-adjoint operators and f, g P C p pT qq.
Prove that:
(i) f pT q  f pT q ;
(ii) }f pT q}  suptPpT q |f ptq| (this generalizes Corollary 5.3.10, and follows
by the same argument);
(iii) If operators S and T commute then operators f pS q and g pT q commute. (Check this for polynomials and pass to a limit.)
5.3.6. Square root of an operator. Consider a positive
? self-adjoint operator
T P LpH, H q. Then pT q?
r0, 8q. The function f ptq  t is continuous on r0, 8q,
so we can define
?
? 2 f pT q  T .
Since
p
? property implies that p T q2  T .
? tq  t, the algebra homomorphism
Since t 0, Corollary 5.3.15 implies that T is a positive self-adjoint operator.
Summarizing, we have proved the following (except uniqueness):
Proposition 5.3.18 (Square root of an operator). For every positive selfadjoint
operator T P LpH, H q, there exists a unique positive self-adjoint operator
?
T P LpH, H q such that
?
p T q2  T.

5.3. POSITIVE OPERATORS. CONTINUOUS FUNCTIONAL CALCULUS

Exercise 5.3.19. Prove the uniqueness of

121

T.

As an application of operator square root, we will now prove the following

result, whose formulation has nothing to do with functional calculus
Proposition 5.3.20. Let S, T P LpH, H q be a pair of commuting self-adjoint
operators. If S 0 and T 0 then ST 0.
For compact operators S, T this would follow because in this case one can find
a common basis of eigenvectors in H. (Do this!) In general, I dont know any proof
that wont involve functional calculus.

Proof. We want to show that xST x, xy 0 for every x P H.?Since ?S ?

and
T are communing self-adjoint operators (see Exercise 5.3.17), and ST  S T .
(Why?) So we have

? ? ? ?
? ? ? ?
? ? ? ?
xST x, xy  x S S T T x, xy  x T S  S T x, xy  x S T x, S T xy 0

as required.

5.3.7. Modulus of an operator. Now consider an arbitrary operator T P

LpH, H q. Then T  T is a positive self-adjoint operator (check!) so it has a unique
positive square root. Hence we define

|T | :

T T ,

P LpH, H q

which we call the modulus of T . This generalizes the concept of modulus of complex
numbers,
?
|z|  z z, z P C.

Example 5.3.21. (i) For a diagonal operator T  diagp1 , . . . , n q on Cn ,

one has |T |  diagp|1 |, . . . , |n |q. (Check!)
(ii) For the multiplication operator pT f qptq  g ptqf ptq on L2 r0, 1s, one has p|T |f qptq 
|gptq|f ptq. (Check!)
(iii) For the right shift operator R on `2 , we saw that R  L (the left shift), so
|R|  I.

P LpH, H q and vector x P H, one has

}|T |x}  }T x}.

Lemma 5.3.22. For every operator T

Proof. This is a matter of straightforward computation:

}|T |x}2  x|T |x, |T |xy  x|T |2 x, xy  xT  T x, xy  xT x, T xy  }T x}2 .


Lec. 40: 12/13

5.3.8. Polar decomposition. Lemma 5.3.22 motivates us to consider a map

U : |T |x T x,

x P H.

The following properties are obvious:

(i) U is well defined;
(ii) U is a linear operator.;
(iii) U is an isometry, i.e. }U y }  }y } for all y P Imp|T |q (by Lemma 5.3.22);
(iv) ImpU q  ImpT q.
We have proved the following result (except uniqueness):

5.4. BOREL FUNCTIONAL CALCULUS. SPECTRAL THEOREM FOR SELF-ADJOINT OPERATORS

122

Theorem 5.3.23 (Polar decomposition). For every operator T P LpH, H q,

there exists a unique bijective linear isometry U P LpImp|T |q, ImpT qq such that

 U |T |.
The uniqueness of U follows easily: T x  U |T |x means that U
T x, thus U is uniquely determined on Imp|T |q.
T

takes |T |x to

Theorem 5.3.23 generalizes the polar decomposition in the complex plane. The
latter states that every z P C can be represented as
z

 ei Argpzq |z|.

Here ei Argpzq is a unit scalar (generalized by U ), and |z | is the modulus of z (generalized by |T |).
Remark 5.3.24. In general, U can not be extended to a bijective linear isometry
on the whole space H. Indeed, if T is the right shift on `2 then |T |  I, so the
polar decomposition yields U  T . Although U  T is defined on the whole `2 , its
image is not even dense in `2 , so U is not bijective on `2 .
However, for invertible operators T , it is possible to extend U to a bijective
isometry on the whole space:
Theorem 5.3.25 (Polar decomposition for invertible operators). For every operator T P LpH, H q, there exists a unique unitary operator U P LpH, H q such that
T

 U |T |.

Proof.
?  Since T is invertible, T  is invertible, so T  T is invertible, and finally
|T |  T T is invertible. (Why? Use Example 4.4.26.) Therefore ImpT q 
Imp|T |q  H, and the claim follows from Theorem 5.3.23.

Remark 5.3.26. This second form of the polar decomposition holds also for
all normal operators (those with T  T  T T  ), compact
scalar operators, and
generally for all operators for which dim kerpT q  dim kerpT  q.
5.4. Borel functional calculus. Spectral theorem for self-adjoint
operators
In this section, we extend functional calculus to bounded Borel functions of
operators. This is done primarily to define the spectral projections, which are
indicator functions of an operator. Once we have spectral projections, we formulate
the spectral theorem for general (not necessarily compact) self-adjoint operators.
As usual, T P LpH, H q will denote a fixed self-adjoint operator on a Hilbert
space H. Let us fix an interval rm, M s which contains the spectrum pT q, e.g. the
spectrum interval.
5.4.1. Borel functional calculus. We consider the linear space of bounded
Borel complex-valued functions on rm, M s; denote this space B rm, M s. We would
like to define f pT q for f P B rm, M s, so that this extends the definition of f pT q for
continuous functions f P C rm, M s. The difficulty is that Borel functions are only
pointwise (but not uniform) limits of continuous functions.
Theorem 5.4.1 (Borel functional calculus). Let T P LpH, H q be a self-adjoint
operator on a Hilbert space H. For each f P B rm, M s one can define a self-adjoint
operator f pT q P LpH, H q such that

5.4. BOREL FUNCTIONAL CALCULUS. SPECTRAL THEOREM FOR SELF-ADJOINT OPERATORS

123

(i) paf bg qpT q  a  f pT q b  g pT q, pf g qpT q  f pT qg pT q, f pT q  f pT  q. In

other words, the map f f pT q is an -algebra homomorphism from B rm, M s
into LpH, H q.
(ii) If fn , f P B rm, M s satisfy supn }fn }8 8 and fn f pointwise then
fn pT q f pT q pointwise in LpH, H q.
(iii) }f pT q} }f }8 .
(iv) If T and S commute then f pT q and g psq commute for f, g P B rm, M s.

Proof. Construction of f pT q. For a fixed pair of vectors x, y

linear functional Fx,y on C rm, M s by
Fx,y pf q : xf pT qx, y y,

P H, define a

P C rm, M s.

This functional is bounded:

|Fx,y pf q| }f pT q}}x}}y} }f }8 }x}}y},

where the last inequality follows by the spectral mapping Theorem 5.3.14. We have
shown that
Fx,y P pC rm, M sq and }Fx,y } }x}}y }.
By representation Theorem 2.2.8 for C pK q , there exists a unique Borel regular
signed measure x,y on rm, M s with total variation |x,y |prm, M sq }x}}y }, and
such that

xf pT qx, yy 

f pq dx,y pq,

P C rm, M s.

We extend this to bounded Borel functions by defining

pBf qpx, yq :

M
m

f pq dx,y pq,

P Brm, M s.

(Since x,y is a Borel measure, the integral is defined.) One quickly checks using
the definition of x,y that pBf qpx, y q is linear in x and conjugate linear in y. So B
is what is called a sesquilinear form. Moreover, this form is bounded:

|pBf qpx, yq| }f }8  |x,y |prm, M sq }f }8 }x}}y}.

It follows (see Exercise 2.4.45) that B is given by a linear operator that we shall
call f pT q P LpH, H q acting as

pBf qpx, yq  xf pT qx, yy, f P Brm, M s, x, y P H,

and that }f pT q} }f }8 . We thus defined f pT q for all bounded Borel functions f ,

and we verified (iii).

(ii) with weak convergence. Let us try to prove (ii). By the dominated convergence theorem, for every x, y P H we have

xfn pT qx, yy 

M
m

fn pq dx,y pq

M
m

f pq dx,y pq  xf pT qx, y y.

This establishes a somewhat weaker conclusion than in (ii), namely that fn pT qx

f pT qx weakly for all x P H. We will prove the actual statement (ii) in a minute.
(i). All these identities hold for continuous functions f , and they are preserved
under pointwise limits. As every Borel function is a pointwise limit of continuous
functions, the conclusion follows by applying (ii) with weak convergence. (Check!)
The same argument proves (iv), and it also establishes xthat f pT q is a self-adjoint
operator.

5.4. BOREL FUNCTIONAL CALCULUS. SPECTRAL THEOREM FOR SELF-ADJOINT OPERATORS

124

(ii) completed. Now we complete the proof of (ii) by showing that fn pT qx

fn pT qx in H. To this end, we bound

pointwise. This completes the proof.

}pfn
f qpT qx}2  pfn
f qpT qx, pfn
f qpT qx

@
D
 ppfn
f qpT qq2 x, x
@
D
 ppfn
f q2 pT qqx, x (by part (i), we have pg2 qpT q  pgpT qq2 )
0
where the last line follows by part (ii) with weak convergence, as pfn
f q2 0
Exercise 5.4.2. Show that if f ptq 0 for all t then f pT q 0.

5.4.2. Spectral measures. Let us fix a self-adjoint operator T P LpH, H q.

In the proof of Borel functional calculus, we constructed Borel regular measures
x,y pT q for every pair of vectors x, y P H. They are called spectral measures for T .
We showed that spectral measures satisfy the identity
(5.5)

xf pT qx, yy 

f pq dx,y pq

valid for every bounded Borel function f P B rm, M s. In particular, the bilinear
form of T can be reproduced using spectral measures as
(5.6)

xT x, yy 

M
m

dx,y pq.

Exercise 5.4.3. Compute the spectral measures for the diagonal matrix T  diagp1 , . . . , n q acting as an operator on Cn .
Exercise 5.4.4. Let T be a multiplication operator in L2 r0, 1s by a
function g P L8 r0, 1s. Show that for f P B r0, 1s, the operator f pT q is the
multiplication operator in L2 r0, 1s by the function f pg ptqq.
5.4.3. Spectral projections. Let E be a Borel subset of rm, M s, and we
consider the indicator function
1E ptq 

1, t P E
0, t R E.

We shall consider the operator PE  1E pT q P LpH, H q.

Since 1E ptq2  1E ptq and 1E ptq is real valued, the homomorphism property
(iii) in Theorem 5.4.1 yields that PE is an orthogonal projection in H (see Exercise 5.1.12).
The projections PE are called spectral projections associated with T .
Exercise 5.4.5. Show that if E1
ImpE2 q.

E2

then PE1

PE

and ImpE1 q

Proposition 5.4.6 (Projection-valued measure). The spectral projections PE

associated with an operator T have the following properties:
(i) PH  0, Prm,M s  I;

5.4. BOREL FUNCTIONAL CALCULUS. SPECTRAL THEOREM FOR SELF-ADJOINT OPERATORS

125

(ii) For every decomposition rm, M s

has
I

 8k1 Ek into disjoint sets Borel Ek , one

8

PE n ,

n 1

where the pointwise convergence of the series.

These properties are strikingly similar to the axioms of a Borel probability
measure on rm, M s. However, the usual probability measures are functions : E
pE q with values in r0, 1s, while PE takes values in LpH, H q, more specifically in the
operator interval between 0 and I. For this reason, the assignment P : E PE
for Borel subsets E rm, M s is called a projection-valued measure for T .
The projection-valued measure can be reduced to the usual measure in a simple
way. For x P H, the assignment E xPE x, xy is clearly a usual Borel measure
on rm, M s. This follows from Proposition 5.4.6. Moreover, (5.5) shows that this is
nothing else than the spectral measure x,x :

xP pE qx, xy  xPE x, xy 

M
m

1E ptq dx,x pq  x,x pE q.

For this reason, the projection-valued measure P itself, rather than x,x , is often
called the spectral measure associated with the operator T .
5.4.4. Spectral theorem for self-adjoint operators.
Theorem 5.4.7 (Spectral theorem). Let T
on a Hilbert space H. Then

8

P LpH, H q be a self-adjoint operator

dP

where P is the projection-valued measure associated with T . The integral in fact

can be taken over a finite interval rm, M s containing the spectrum of T .
The integration with respect to the operator-valued measure P is understood
in the sense that

xT x, xy 

xdP x, xy;

As we noted, xdP x, xy is just the spectral measure x,x , so the last integral is the
usual Lebesgue integral.
Proof. With this remark, Theorem 5.4.7 is a reformulation of a partial case
of (5.6):

xT x, xy 

M
m

dx,x pq.

Theorem 5.4.7 should be compared to the spectral Theorem 5.3 for compact
self-adjoint operators T . According to this theorem, T can be decomposed into the
sum

T 
k Pk
k

where k are the eigenvalues of T and Pk denotes the orthogonal projection in H

onto the eigenspace corresponding to the eigenvalue k , see (5.3).
As we know, for general operators T (not necessarily compact), the spectrum
of T may no longer be countable. So the sum is replaced by the integral in Theorem 5.4.7.

Bibliography
[1] K. Ball, An elementary introduction to modern convex geometry. Flavors of geometry, 158,
Math. Sci. Res. Inst. Publ., 31, Cambridge Univ. Press, Cambridge, 1997.
[2] L. Carleson, On convergence and growth of partial sums of Fourier series, Acta Mathematica 116 (1966), 135157.
[3] P. Enflo, A counterexample to the approximation problem in Banach spaces., Acta Math.
130 (1973), 309317.
[4] E. Gluskin, The diameter of the Minkowski compactum is roughly equal to n, Funktsional.
Anal. i Prilozhen. 15 (1981), 7273.
[5] W. T. Gowers, B. Maurey, The unconditional basic sequence problem, J. Amer. Math. Soc.
6 (1993), 851874
[6] J. Lindenstrauss, Joram, On complemented subspaces of m, Israel J. Math. 5 (1967), 153
156.
[7] J. Lindenstrauss, L. Tzafriri, On the complemented subspaces problem, Israel J. Math. 9
(1971), 263269.
[8] J. Lindenstrauss, L. Tzafriri, Classical Banach spaces. I and II. Springer-Verlag, , 1977,
1979.
[9] A. Pelczkynski, Projections in certain Banach spaces, Studia Math. 19 (1960), 209228.
[10] R. S. Phillips, On linear transformations, Trans. Amer. Math. Soc. 48 (1940), 516541.

126

Index

Essentially bounded, 5
Extension
of bounded linear functionals, 47
of bounded linear operators, 61

Absorbing set, 52
Adjoint operator, 63
Angle between vectors, 21
Annihilator, 64
approximate, 112

Fourier
basis, 29, 80
coefficients, 31
series, 28, 31
Fourier basis, 80
Frame, 36
Fredholm
alternative, 99
integral equation, 59
theory, 98

Banach space, 15
Banach-Alaoglus theorem, 91
Banach-Mazur distance, 73
Basis
constant, 81
Hamel, 2
orthonormal, 32
Schauder, 79
Bessels inequality, 31
Biorthogonal functionals, 82

Gelfands formula, 105

Gram-Schmidt orthogonalization, 33
Graph of an operator, 73

Cauchy-Schwarz inequality, 20
Closed graph theorem, 73
Codimension, 4
Compact
operator, 94
set, 83
compact operators, 106
Completion, 18
Convex
combination, 14
hull, 14
Convexity, 9

Haar system, 36, 80

Hellinger-Toeplitz theorem, 75
Hermite polynomials, 37
Hilbert cube, 85
Hilbert space, 25
Hilbert-Schmidt
operator, 23, 58
theorem, 114
Hyperplane, 41

Differential operator, 59, 74

Dirac delta function, 39, 91
Direct sum of normed spaces, 13, 71
Dirichlet kernel, 60
Dual space, 41
of `p , 45
of C K , 46
of Lp , 45
of a Hilbert space, 42
second dual, 51

Image, 5
closed, 72, 98
Injectivization, 6
Inner product, 19
Integral operator, 58, 94, 97, 100
Inverse mapping theorem, 70
Isometry, 61
Isomorphic embedding, 72
Isomorphism, 61, 70

p q

Kernel, 5
Embedding, 6
Equicontinuity, 85
Ergodic theory, 64
Ergodic transformation, 66

Legendre polynomials, 34
Linear functional, 39
bounded, 40
127

INDEX

Linear operator, 5
bounded, 56
norm, 56
Measure-preserving transformation, 66
Minkowski functional, 14, 52
Minkowski inequality, 11
Modulus of an operator, 121
Net, 83
Norm, 7
equivalent, 71
Normed space, 7
Open map, 68
Open mapping theorem, 68
Operator norm, 56
Orthogonal complement, 26
orthogonal decomposition, 28
orthogonal projection, 28
Orthogonal series, 30
Orthogonal system, 29
Orthogonality principle, 26
Parallelogram law, 24
Parsevals identity, 32
Perfectly convex set, 69
Point evaluation functional, 39
Pointwise
bounded family of operators, 76
convergence of operators, 84
Polar decomposition, 122
Polarization identity, 24, 111
Positive operator, 117
Precompact set, 83
Principle of uniform boundedness, 76
Projection, 62
onto convex sets, 28
orthogonal, 26, 28, 60
Projection-valued measure, 125
Pythagorean inequality, 20
Quotient space, 4, 6, 12, 15
Rademacher system, 37
Radon-Nikodym theorem, 43
Reflexive spaces, 51
Regular point, 101
Resolvent
identity, 104
operator, 103
set, 101
Riesz representation theorem, 42
Schauder basis, 79
Schauder system, 80
Schur property, 89
Self-adjoint operator, 110
Semi-norm, 14
Separable space, 35

128

Separation of convex sets, 53

Series in normed spaces, 16
Sesquilinear form, 67, 123
Shift operator, 6, 61, 103
Singular values and vectors, 116
Spectral
measure, 124
projection, 124
radius, 105, 113
theorem, 114, 125
Spectral mapping theorem, 118, 120
Spectrum
continuous, 102
interval, 112, 117
of a bounded linear operator, 101
point, 102
residual, 102
Square root of an operator, 120
Strong
topology, 86
Sublinear function, 52
Subspace, 2, 6, 12, 15
complementary, 6
complemented, 62
invariant, 114
Supporting functional, 50
Triangle inequality, 7
Tychonoffs theorem, 92
Uniform
convergence of operators, 84
Uniformly
bounded family of operators, 76
Unitary operator, 61, 108
Universal space C K , 92

p q

Volterra operator, 59, 95

Walsh system, 37
Weak
boundedness, 77
closedness, 90
convergence, 86
topology, 89
Weak
convergence, 90
topology, 91
Youngs inequality, 22
Zorns lemma, 3