Chapter 2

Functional Analysis: Normed &

Inner Product Spaces

We start with definitions of some algebraic structures.

2.1 Some important algebraic structures

2.1.1 Groups
By a group, we mean a pair (G, ∗), where G is a non-empty set and ∗ a binary operation
applying on any a, b ∈ G, satisfying the following properties:

G1 : G is closed under ∗. That is, for any pair a, b ∈ G, we have a ∗ b ∈ G.

G2 : G is associative under ∗. That is, for any triplet a, b, c ∈ G, we have (a∗b)∗c = a∗(b∗c).

G3 : There exists e ∈ G (the identity element of G) such that ∀a ∈ G, we have a∗e = e∗a = a.

G4 : For any a ∈ G, there exists a0 ∈ G (the inverse of a) such that a ∗ a0 = a0 ∗ a = e.

Remark. If in addition to satisfying G1 − G4 , the set G satisfies the commutative law, i.e.
any pair a, b ∈ G =⇒ a ∗ b = b ∗ a. then, G is called the abelian group.

Note: If no ambiguity arises, we usually omit ∗ from (G, ∗) and simply denote the group
with G.

Example 2.1.1. 1. The sets Z, Q, R and C of integers, rationals, real numbers and com-
plex numbers respectively, under the usual additional of numbers, are abelian groups.
√
−1± 3ι
2. The three cube roots of unity i.e. 1, ω, ω 2 where ω = 2 and ω 3 = 1, form a group
under multiplication.
√
3. The four fourth roots of unity i.e. ±1, ±ι where ι = −1, form a group under multi-
plication.

1
 4. For a nonempty set X, let F = {f | f : X → R, and | f |≤ M }. Define ‘addition’ in F
by:
For f, g ∈ F, we put (f + g)(x) = f (x) + g(x) , x ∈ X.
Then, (F, +) is an abelian group with 0 : X → R defined by 0(x) = 0, for all x ∈ X,
as the identity and, for any f ∈ F, the function (−f ) : X → R defined by (−f )(x) =
−f (x), as the inverse of f .

2.1.2 Rings
A triplet (R, +, ·) where ‘+’ and ‘·’ are binary operations of addition and multiplication
respectively, and R is a nonempty set, is said to be a ring if:

R1 : (R, +) is an abelian group.

R2 : R is associative under ‘·’. That is, for a triplet a, b, c ∈ R, we have a · (b · c) = (a · b) · c.

R3 : R satisfies the right and left distributive laws; that is, for all a, b, c ∈ R,

(b + c) · a = b · a + c · a (right distributive law)

a · (b + c) = a · b + a · c (left distributive law)

If, in (R, +, ·), we have a · b = b · a for all a, b ∈ R, then R is called the commutative ring.

Example 2.1.2. 1. Under the usual addition and multiplication of numbers, the sets
(Z, +, ·), (Q, +, ·), (R, +, ·) and (C, +, ·) are rings.

2. Let F = {f | f : X → R, and | f |≤ M } with addition and multiplication in F defined

above. Then, (F, +, ·) is a commutative ring.

2.1.3 Fields
A commutative ring (F, +, ·) is said to be a field if the nonempty set F form a group under
multiplication.

Example 2.1.3. The rings (Q, +, ·), (R, +, ·) and (C, +, ·) are example of fields.

2.1.4 Vector spaces

Let V be a nonempty set and F be a field. The set V is said to be a vector space over F if:

V1 : There is a binary operation ‘+’ called addition, such that (V, +) is an abelian group.

V2 : In V , a scalar multiplication a·v is defined for each a ∈ F and v ∈ V such that a·v ∈ V
and

(a) a · (v1 + v2 ) = a · v1 + a · v2
(b) (a + b) · v = a · v + b · v

2
 (c) (ab).v = a(b · v)
(d) 1 · v = v

for all a, b ∈ F and v, v1 , v2 ∈ V and 1 is the multiplicative identity of F .

Remark. A vector space V over a field F is usually written as V (F ).

Example 2.1.4.
1. The rings Q, R and C are all vector space over themselves.
2. R(Q) and C(R) are vector spaces over the fields Q of rationals and R of reals respectively.
( ! ) ! !
2 x x1 y1
3. Let R = : x, y ∈ R . For a ∈ R and x = ,y= ∈ R2 , we define
y x2 y2
the operation of addition and scalar multiplication as follows:
! ! ! ! !
x1 a · x1 2 x1 y1 x 1 + y1
a.x = a. = ∈R , x+y = + = ∈ R2 .
x2 a · x2 x2 y2 x 2 + y2

Under these operations, the set R2 is a vector space over R. A linear space is a vector space
over F , where F ∈ {R, C}.

2.2 Normed spaces

Let N be a linear space over the field F , where F ∈ {R, C}. A norm on N is a function
k.k : N → R satisfying the following conditions:

N1 : For all x ∈ N , kxk ≥ 0 where kxk = 0 ⇐⇒ x = 0.

N2 : kaxk = |a|.kxk for all a ∈ F and x ∈ N .

N3 : kx + yk ≤ kxk + kyk for all x, y ∈ N . (the triangle inequality)

The pair (N, k.k), where N is a linear space and k.k on N , is called a normed spaced.
Elements of N (resp. F ) are called vectors (resp. scalars).

Remark. Note that k − xk = kxk from N2 .

Example 2.2.1. 1. The linear spaces R and C over themselves are normed spaces
under the usual modulus norm. Thus, for any x ∈ R,

kxk = |x|

and similarly for z ∈ C,

kzk = |z|
are norms on R and C respectively.

3
 2. The space Rn is a normed space over R under the norm defined as follows:
For x = (x1 , x2 , . . . , xn ) ∈ Rn , we put
v
u n
uX
kxk = t |xi |2 (∗)
i=1

To show Rn a normed space with norm (∗), we need to satisfy (N1 − N3 ):

N1 : For 1 ≤ i ≤ n, |xi |2 ≥ 0. This implies that

v
X n u n
2
uX
|xi | ≥ 0 =⇒ t |xi |2 ≥ 0 =⇒ kxk ≥ 0.
i=1 i=1
s
n n
|xi |2 = 0 ⇐⇒ |xi |2 = 0 ⇐⇒ |xi | = 0, ∀i.
P P
Moreover, kxk = 0 ⇐⇒
i=1 i=1
This implies that x = 0.
s s s
n n n
2 2 2 |xi |2 = |a|.kxk
P P P
N2 : kaxk = |axi | = |a| |xi | = |a|
i=1 i=1 i=1

N3 : For x = (x1 , x2 , . . . , xn ), y = (y1 , y2 , . . . , yn ), we use Minkowski’s inequality

v
u n
uX
kx + yk = t |xi + yi |2
i=1
v v
u n u n
uX uX
≤ t 2
|xi | + t |yi |2
i=1 i=1
= kxk + kyk.

The space Rn together with (∗) is a normed space.

3. The space `p : This space consists of all sequences x = {xn }, xn ∈ R with the norm
defined by v
u∞ ∞
uX X
kxk = tp
|xi |p , |xi |p < ∞
i=1 i=1

Here N3 follows again by Minkowski’s inequality.

4. The space `n : This space is simply Rn , but with norm
n
kxk = sup |xi |.
i=1

N1 : For any x ∈ `n , kxk ≥ 0 and

n
kxk = 0 ⇐⇒ sup |xi | = 0
i=1
⇐⇒ xi = 0, for 1 ≤ i ≤ n
⇐⇒ x = 0.

4
 n n
N2 : kaxk = sup |axi | = |a| sup |xi | = |a|.kxk
i=1 i=1

N3 : For x = (x1 , x2 , . . . , xn ), y = (y1 , y2 , . . . , yn ), we use Minkowski’s inequality

n n n
kx + yk = sup |xi + yi | ≤ sup |xi | + sup |yi | (the triangle inequality)
i=1 i=1 i=1
= kxk + kyk.

5. The space `∞ , c and c0 : The space `∞ consists of all bounded sequences with norm as
follows:
For x = {xn } ∈ `∞ , with |xn | < M, ∀n and some M ∈ R, we put
∞
kxk = sup |xi |.
i=1

Then `∞ with this norm is a normed space.

The space c (resp. c0 ) is a subspace of `∞ and consists of all convergent (resp. converging
to 0) are normed spaced with same norm as defined for `∞ .
6. The space C[a, b] consists of all continuous real-valued functions defined on [a, b]. With
norm defined below:
kf k = sup |f (x)|,
x∈[a,b]

the space C[a, b] is normed space.

2.2.1 Banach spaces (Complete normed spaces)

Let N be a normed space. A sequence {xn } in N is said to be a Cauchy sequence in N if for
any  > 0, there exists n0 ∈ N such that

∀m, n ≥ n0 =⇒ kxm − xn k < .

A sequence {xn } in N is said to be a convergent to x ∈ N if for any  > 0, there exists n0 ∈ N

such that
∀n ≥ n0 =⇒ kxn − xk < .
We then call x the limit of {xn } and write

x = lim xn .
n→∞

A normed space N is said to be complete if every Cauchy sequence is convergent in N . A

complete normed space is said to be a Banach space.

Example 2.2.2. 1. The normed spaces R and C of real and complex number are
Banach spaces.

5
 2. The space Rn is also a Banach space. Here we prove that Rn is complete.
Let {x(p) } be a Cauchy sequence in Rn ,
 
(p) (p)
x(p) = x1 , x2 , . . . , x(p)
n , p = 1, 2, . . .

Then, given any  > 0, there exists n0 such that

v
u n
(p) (q)
uX (p) (q)
∀p, q ≥ n0 =⇒ kx − x k = t |xi − xi |2 < 
i=1

Hence,
(p) (q)
∀p, q ≥ n0 =⇒ |xi − xi | ≤ kx(p) − x(q) k < 
(p) (p)
So, for each i, {xi } is a Cauchy sequence in R. Since R is complete, {xi } converges
to a real number xi , say 1 ≤ i ≤ n. But this implies that for  already chosen, there
exists pi ∈ N such that
(p) 
∀p ≥ pi =⇒ |xi − xi | < √ (2.1)
n
(p)
Take x = (x1 , x2 , . . . , xn ), where xi = limp→∞ xi . Then, x ∈ Rn . We show that
limp→∞ x(p) = x. Let p0 = max{p1 , p2 , . . . , pn }
v
u n
(p)
uX (p)
∀p ≥ p0 ≥ n0 =⇒ kx − xk = t |xi − xi |2 < 
i=1

by (2.1). Hence, {x(p) } converges to x ∈ Rn , as required. Thus, Rn is complete and,

thus, a Banach space.

3. The spaces `p , p ≥ 1, `n , `∞ , c, c0 , and C[a, b] defined in Section 2.2 are all Banach
spaces.

2.2.2 Some incomplete spaces

1. The space Q of rational numbers, as a subspace of R, under the usual operation of
addition and multiplication by a scalar a ∈ Q is a normed space with norm kqk = |q|.
√
Take a sequence of the decimal approximation of 2 as
√
{xn } = {1, 1.4, 1.41, 1.414, . . .} → 2.
√
The sequence {xn } converges to 2 which is not a rational.

2. Consider the subspace P of all polynomials

P (n) (x) = p0 + p1 x + . . . + pn xn , x ∈ [a, b], pi ∈ R

6
 of finite degree. Note that P is a subspace of C[a, b]. The norm in P is the induced
norm on C[a, b] as follows:

kP (n) k = sup |p0 + p1 x + . . . + pn xn |

x∈[a,b]

P is not complete. Here the sequence of polynomials

x2 xn
P (0) (x) = 1, P (1) (x) = 1 + x, . . . , P (n) (x) = 1 + x + + ... +
2! n!
is Cauchy because

kP (n) − P (m) k = sup |P (n) (x) − pm (x)| → 0, as m, n → ∞

x∈[a,b]

n
xi
= ex . However,
P
since lim
n→∞ i=0 i

x2
lim P (n) (x) = 1 + x + + . . . = ex ∈
/ P.
n→∞ 2!

2.3 Inner product spaces

Let V be a linear space over a field F where F ∈ {R, C}. An inner product in V is a function

h·, ·i : V × V → F

(so that, with each pair x, y ∈ V , a scalar hx, yi is associated) satisfying the following condi-
tions:

(i) hx, xi ≥ 0 and hx, xi = 0 ⇐⇒ x = 0, x ∈ V.

(ii) hx + y, zi = hx, zi + hy, zi, ∀x, y, z ∈ V

(iii) hax, yi = ahx, yi, ∀x, y ∈ V, a ∈ F

(iv) hy, xi = hx, yi, ∀x, y ∈ V (the Hermitian property)

The pair (V, h·, ·i) is called an inner product space. Note that if V = R, then hy, xi = hx, yi.
Some consequences of the definition are as follows:

(a) For all x, y, z ∈ V and a, b ∈ F , we have hax + by, zi = ahx, zi + bhy, zi. Moreover,
h0, zi = h0 · x, zi = 0 · hx, zi = 0.

(b) For all x, y ∈ V, a ∈ F , we have hx, ayi = hay, xi = ahy, xi = ahy, xi = ahx, yi.

(c) hx, ay + bzi = ahx, yi + bhx, zi.

7
Let (V, h·, ·i) be an inner product space. The function k · k : V → R+ defined by
p
kxk = hx, xi, x ∈ V (2.2)

gives a norm on V . Thus, every inner product space is, in fact, a normed space. An inner
product space V is called a Hilbert space if V is complete with respect to the norm (2.2).

Example 2.3.1. 1. The space Rn of real n-tuples is an inner product space with inner
n
xi yi , where x = (x1 , . . . , xn ), y = (y1 , . . . , yn ) ∈ Rn .
P
product hx, yi =
i=1
n
The norms induced in Rn by the inner product hx, yi =
P
xi yi is
i=1
q
kxk = x2i , x ∈ Rn . (2.3)

We showed in Example 2.2.2(2) that Rn together with norm (2.3) is complete. Thus,
(Rn , h·, ·i) is a Hilbert space.

2. The space C[a, b] of all real-values continuous function defined on [a, b], together with
inner product
Zb
hf, gi = f (t) · g(t)dt
a
is an inner product space.
n
3. The space `2 of all real sequences x = {xi } such that |xi |2 < ∞ is an inner product
P
i=1
space under the inner product defined by:
∞
X
hx, yi = xi yi , y = {yi } ∈ `2
i=1

The space `2 is also a Hilbert space.

4. The space Pn of all polynomials of degree ≤ n defined over [a, b] with inner product

Zb
hu(x), v(x)i = u(x) · v(x)dx, x ∈ [a, b], a < b,
a

is an inner product space.

5. The space Mn of all n × n matrices over R with inner product

hA, Bi = tr(AB T ),

is an inner product space, where B T is the transpose of B. Use the properties of trace
n Pn
and tr(AAT ) = a2ij , A = (aij ) to show it an inner product.
P
i=1 j=1

8
Theorem 2.3.1 (Cauchy-Schwarz Inequality). For any x, y in an inner product space V , we
have
|hx, yi| ≤ kxk · kyk

Two vectors x, y in an inner-product space V are said to be orthogonal if hx, yi = 0.