MAT02B2 / MAT2B20

6.1 Inner Products

Recall that the dot product was defined as follows:

If u = (u1 , . . . , un ) and v = (v1 , . . . , vn ) are vectors in Rn , then the dot
product of u and v is denoted by u • v and is defined by

u • v = u1 v1 + u2 v2 + · · · + un vn .

Some of the algebraic properties of the dot product include: If u, v and w

are vectors in Rn , and if k is a scalar, then

1. u • v = v • u,

2. (u + v) • w = u • w + v • w,

3. k(u • v) = (ku) • v,

4. v • v ≥ 0 and v • v = 0 if, and only if, v = 0.

Rn equipped with the dot product is the prototype example of the type of
spaces that we will develop and study in this section. The algebraic prop-
erties of the dot product listed above leads us to the following more general
definition.

General Inner Products

An inner product on a real vector space V is a function that associates

a real number hu, vi with each pair of vectors in V in such a way that the
following axioms are satisfied for all vectors u, v and w in V and all scalars
k:

1. hu, vi = hv, ui

2. hu + v, wi = hu, wi + hv, wi

3. hku, vi = k hu, vi

4. hv, vi ≥ 0 and hv, vi = 0 if, and only if, v = 0.

A real vector space with an inner product is called a real inner product
space.

1
Theorem 0.1 (Theorem 6.1.2). If u, v and w are vectors in a real inner
product space V , and if k is a scalar, then:
(i) 0, v = v, 0 = 0.

(ii) hu, v + wi = hu, vi + hu, wi.

(iii) hu, v − wi = hu, vi − hu, wi.

(iv) hu − v, wi = hu, wi − hv, wi.

(v) k hu, vi = hu, kvi.

Proof. We shall utilize the four axioms in the definition of an inner product.
(i) By property 1 we have
0, v = v, 0 .
Moreover, since 0 = 0v, it follows from property 3 that

0, v = h0v, vi = 0 hv, vi = 0.

(ii) By property 1 and property 2 we have

hu, v + wi = hv + w, ui
= hv, ui + hw, ui
= hu, vi + hu, wi .

(iii) By (ii), property 1 and property 3 we have

hu, v − wi = hu, vi + hu, −wi

= hu, vi + h−w, ui
= hu, vi − hw, ui
= hu, vi − hu, wi .

(iv) By property 1 and (iii), we obtain

hu − v, wi = hw, u − vi
= hw, ui − hw, vi
= hu, wi − hv, wi .

(v) By property 1 and property 3, we have

k hu, vi = k hv, ui = hkv, ui = hu, kvi .

2
This completes the proof.
As mentioned earlier, Rn equipped with the dot product is our prototype
example of a real inner product space.

The dot product on Rn is also commonly called the Euclidean inner

product (or standard inner product).

We call Rn with the Euclidean inner product Euclidean n-space.

Weighted Euclidean Inner Products

If w1 , w2 , . . . , wn are positive real numbers, which we will call weights, and

if u = (u1 , . . . , un ) and v = (v1 , . . . , vn ) are vectors in Rn , then
hu, vi = w1 u1 v1 + w2 u2 v2 + . . . + wn un vn
defines an inner product on Rn that we call the weighted Euclidean inner
product with weights w1 , w2 , . . . , wn .

Example 1
Consider the weighted Euclidean inner product on R3 with weights 2, 1, 3.
(a) Show that this function satisfies the axioms of an inner product.
(b) Let u = (1, −2, 1) and v = (4, 1, 0). Compute hu, vi and hu, ui.
solution:

(a) Let u = (u1 , u2 , u3 ), v = (v1 , v2 , v3 ) and x = (x1 , x2 , x3 ) be vectors in

R3 and let k be any scalar. Note that the function under consideration
is given by
hu, vi = 2u1 v1 + u2 v2 + 3u3 v3 .
We shall verify the four axioms in the definition of an inner product:
1. hu, vi
= 2u1 v1 + u2 v2 + 3u3 v3
= 2v1 u1 + v2 u2 + 3v3 u3
= hv, ui
2. hu + v, xi
= 2(u1 + v1 )x1 + (u2 + v2 )x2 + 3(u3 + v3 )x3
= (2u1 x1 + u2 x2 + 3u3 x3 ) + (2v1 x1 + v2 x2 + 3v3 x3 )
= hu, xi + hv, xi

3
 3. hku, vi
= 2(ku1 )v1 + (ku2 )v2 + 3(ku3 )v3
= k(2u1 v1 + u2 v2 + 3u3 v3 )
= k hu, vi
4. Certainly,
hu, ui = 2u21 + u22 + 3u23 ≥ 0.
(Note the importance of positive weights here.)
Moreover, if u = 0, then u1 = u2 = u3 = 0, and so,

hu, ui = 2u21 + u22 + 3u23 = 0.

On the other hand, if hu, ui = 0, then 2u21 + u22 + 3u23 = 0. Hence,

u1 = u2 = u3 = 0, and so, u = 0. Thus, we conclude that
hu, ui = 0 if and only if u = 0.

This shows that the function defines an inner product.

(b) With u = (1, −2, 1) and v = (4, 1, 0), we have

hu, vi = 2(1)(4) + (−2)(1) + 3(1)(0) = 6

and
hu, ui = 2(1)2 + (−2)2 + 3(1)2 = 9.

The next example illustrates why it is important that the weights w1 , . . . , wn

in
hu, vi = w1 u1 v1 + w2 u2 v2 + . . . + wn un vn
are all positive.

Example 2
Let u = (u1 , u2 ) and v = (v1 , v2 ) be in R2 and define hu, vi as

hu, vi = −u1 v1 + u2 v2 .

Show that this operation is not an inner product on R2 .

Solution:
Let u = (1, 1). Then
hu, ui = −(1)2 + (1)2 = 0,
however, u 6= 0. This shows that axiom 4 fails and proves that the given
operation does not define an inner product.

4
Remark. In order to prove that a given operation defines an inner product,
one has to verify that all four axioms in the definition are satisfied for all
vectors u, v and w and all scalars k. However, to show that a given operation
does not define an inner product, it is enough to show that one of the axioms
fails. Typically, this last step can be done by giving a specific example which
shows that it does not hold (see the solution to Example 2 above).

We shall now consider a few more examples of real inner product spaces.

Example 3
   
u1 u2 v1 v2
Let U = and V = .
u3 u4 v3 v4
Show that hU, V i = tr (U T V ) defines an inner product on the vector space
M22 .

Solution:
Let U , V and W be matrices in M22 and let k be any scalar. We shall verify
the axioms in the definition of an inner product. (You might want to recap
a few properties of the transpose and trace of a matrix.)

1. hU, V i
= tr(U T V )
= tr[(U T V )T ] since (tr(A) = tr(AT ))
= tr(V T U ) = hV, U i.

2. hU + V, W i
= tr((U + V )T W )
= tr(U T W + V T W )
= tr(U T W ) + tr(V T W )
= hU, W i + hV, W i.

3. hkU, V i
= tr((kU )T V )
= tr(kU T V )
= ktr(U T V )
= k hU, V i .

5
    
u1 u2 v1 v2
4. If U = and V = , then observe that
u3 u4 v3 v4
 T  !
u1 u2 v1 v2
hU, V i = tr(U T V ) = tr
u3 u4 v3 v4
    
u1 u3 v1 v2 u1 v1 + u3 v3 ∗
= tr = tr
u2 u4 v3 v4 ∗∗ u2 v2 + u4 v4
= (u1 v1 + u3 v3 ) + (u2 v2 + u4 v4 )
= u1 v1 + u2 v2 + u3 v3 + u4 v4 . (0.1)

Note that we have essentially taken a “dot product” of our two matrices
here (we have multiplied corresponding entries together and summed
up the results). This representation of the formula will allow us to
easily prove this axiom. The details are as follows. Certainly,

hU, U i = tr(U T U ) = u21 + u22 + u23 + u24 ≥ 0.

Moreover, if O denotes the zero matrix, then

U = O ⇔ u1 = u2 = u3 = u4 = 0
⇔ u21 + u22 + u23 + u24 = 0
⇔ hU, U i = 0.

This shows that all the axioms are satisfied and proves that the operation
defines an inner product.

Example 4 [Standard inner product on P2 (and more generally on Pn )]

Let p = a0 + a1 x + a2 x2 and q = b0 + b1 x + b2 x2 . Show that

hp, qi = a0 b0 + a1 b1 + a2 b2

defines an inner product on P2 .

Solution:
Let p = a0 + a1 x + a2 x2 , q = b0 + b1 x + b2 x2 and r = c0 + c1 x + c2 x2 be vectors
in P2 and let k be any scalar. In particular we note that

p + q = (a0 + b0 ) + (a1 + b1 )x + (a2 + b2 )x2 and kp = ka0 + ka1 x + ka2 x2 .

We shall verify the four axioms in the definition of an inner product:

6
 1. hp, qi
= a0 b 0 + a1 b 1 + a2 b 2
= b 0 a0 + b 1 a1 + b 2 a2
= hq, pi

2. hp + q, ri
= (a0 + b0 )r0 + (a1 + b1 )r1 + (a2 + b2 )r2
= a0 r0 + b0 r0 + a1 r1 + b1 r1 + a2 r2 + b2 r2
= (a0 r0 + a1 r1 + a2 r2 ) + (b0 r0 + b1 r1 + b2 r2 )
= hp, ri + hq, ri

3. hkp, qi
= (ka0 )b0 + (ka1 )b1 + (ka2 )b2
= k(a0 b0 + a1 b1 + a2 b2 )
= k hp, qi

4. Certainly,
hp, pi = a20 + a21 + a22 ≥ 0.
Moreover, if p = 0, then a0 = a1 = a2 = 0, and so,

hp, pi = a20 + a21 + a22 = 0.

On the other hand, if hp, pi = 0, then a20 + a21 + a22 = 0. Hence,

a0 = a1 = a2 = 0, and so, p = 0. Thus, we conclude that hp, pi = 0 if
and only if p = 0.

This shows that the function defines an inner product.

For completeness sake we mention here that the standard inner product on
Pn is defined by
hp, qi = a0 b0 + a1 b1 + · · · + an bn
for each p = a0 + a1 x + · · · + an xn and q = b0 + b1 x + · · · + bn xn in Pn .
In addition to the standard inner product above, there are other interesting
inner products on Pn . One of these is discussed in the next example.

Example 5 [The evaluation inner product on Pn ]

Let p = a0 +a1 x+· · ·+an xn and q = b0 +b1 x+· · ·+bn xn and let x0 , x1 , . . . , xn
be distinct real numbers (called sample points). Show that
n
X
hp, qi = p(xj )q(xj ) = p(x0 )q(x0 ) + p(x1 )q(x1 ) + · · · + p(xn )q(xn )
j=0

7
defines an inner product on Pn called the evaluation inner product at
x0 , x1 , . . . , xn .

Solution:
Let p = p(x), q = q(x) and r = r(x) be vectors in Pn and let k be any scalar.
In particular we note that

p + q = p(x) + q(x) and kp = kp(x).

We shall prove that the given operation satisfies the four axioms in the defi-
nition of an inner product:

1.
n
X
hp, qi = p(xj )q(xj )
j=0
Xn
= q(xj )p(xj )
j=0
= hq, pi .

2.
n
X
hp + q, ri = (p(xj ) + q(xj )) r(xj )
j=0
n
X
= (p(xj )r(xj ) + q(xj )r(xj ))
j=0
n
X n
X
= p(xj )r(xj ) + q(xj )r(xj )
j=0 j=0
= hp, ri + hq, ri .

3.
n
X
hkp, qi = (kp(xj ))q(xj )
j=0
Xn
= k p(xj )q(xj )
j=0
= k hp, qi .

8
 4. Certainly,
n
X
hp, pi = p(xj )2 = p(x0 )2 + p(x1 )2 + · · · + p(xn )2 ≥ 0.
j=0

Moreover, if p = 0, then p(x) = 0 for all x ∈ R. Hence,

p(x0 ) = p(x1 ) = · · · = p(xn ) = 0,
and so,
n
X
hp, pi = p(xj )2 = p(x0 )2 + p(x1 )2 + · · · + p(xn )2 = 0.
j=0

On the other hand, if hp, pi = 0, then

p(x0 ) = p(x1 ) = · · · = p(xn ) = 0.
Since the sample points x0 , x1 , . . . , xn are all distinct, it follows that
p = p(x) ∈ Pn is an (at most) nth degree polynomial with at least
n + 1 distinct roots. However, recall that a non-constant polynomial of
degree n or less has at most n distinct roots. Thus, we conclude that
p = p(x) must be the zero polynomial, i.e. p(x) = 0 for all x ∈ R, that
is, p = 0. Together we have therefore shown that
p = 0 ⇔ hp, pi = 0.

This proves that the given operation defines an inner product.

Note the importance of using n + 1 distinct sample points in the definition

of the evaluation inner product on Pn . If we use fewer points or non-distinct
points, axiom 4 will fail. Please remember this subtle but extremely impor-
tant point.

Norm and Distance

If V is a real inner product space, then the norm (or length) of a vector v
in V is denoted by kvk and is defined by
p
kvk = hv, vi.
The distance between two vectors is denoted by d(u, v) and is defined by
p
d(u, v) = ku − vk = hu − v, u − vi.
A vector of norm 1 is called a unit vector.

9
Theorem 0.2 (Theorem 6.1.1). If u and v are vectors in a real inner product
space V , and if k is a scalar, then:

(i) kvk ≥ 0 with equality if, and only if, v = 0.

(ii) kkvk = |k|kvk.

(iii) d(u, v) = d(v, u).

(iv) d(u, v) ≥ 0 with equality if, and only if, u = v.

Proof. (i) From axiom 4 it follows that hv, vi ≥ 0 and hv, vi = 0 if and
only if v = 0. Consequently,
p
kvk = hv, vi ≥ 0

and p
v = 0 ⇔ hv, vi = 0 ⇔ hv, vi = 0 ⇔ kvk = 0.

(ii) Using the properties of an inner product, we have

p
kkvk = hkv, kvi
p
= k hv, kvi
p
= k 2 hv, vi
√ p
= k 2 hv, vi
= |k|kvk.

(iii) From (ii) we obtain that

d(u, v) = ku − vk = k − (v − u)k = | − 1|kv − uk = kv − uk = d(v, u).

(iv) This property follows from (i) if we observe that d(u, v) = ku − vk.
Indeed, then
d(u, v) = ku − vk ≥ 0
and
d(u, v) = 0 ⇔ ku − vk = 0 ⇔ u − v = 0 ⇔ u = v.
This completes the proof.
Example 6
Consider the weighted Euclidean inner product on R3 with weights 2, 1, 3.
Let u = (1, −2, 1) and v = (4, 1, 0). Compute:

10
 (a) kvk

(b) d(u, v)

Solution:
p p √
(a) kvk = hv, vi = 2(4)2 + (1)2 + 3(0)2 = 33.

(b) Since u − v = (−3, −3, 1),

p
d(u, v) = ku − vk = hu − v, u − vi
p
= 2(−3)2 + (−3)2 + 3(1)2
√
= 30.

Example 7
Consider the inner product

hU, V i = tr (U T V )

on M22 .    
1 2 −2 −1
Let U = and V = . Determine
0 −1 1 3

(a) hU, V i

(b) kU k

(c) d(U, V )

Solution: Recall from (0.1) that

hA, Bi = tr(AT B) = a1 b1 + a2 b2 + a3 b3 + a4 b4

for all matrices

   
a1 a2 b1 b2
A= and B = in M2 2.
a3 a4 b3 b4

Using this, we obtain:

(a) hU, V i = (1)(−2) + (2)(−1) + (0)(1) + (−1)(3) = −7.

p p √
(b) kU k = hU, U i = (1)2 + (2)2 + (0)2 + (−1)2 = 6.

11
 (c) Since 
3 3
U −V = ,
−1 −4
it follows that
p
d(U, V ) = kU − V k = hU − V, U − V i
p
= (3)2 + (3)2 + (−1)2 + (−4)2
√
= 35.

Note that it is perfectly correct to use the definition involving the transpose
and the trace above directly; for instance, for part (a) we have
 T  !
1 2 −2 −1
hU, V i = tr(U T V ) = tr
0 −1 1 3
  
1 0 −2 −1
= tr
2 −1 1 3
 
(1)(−2) + (0)(1) ∗
= tr
∗∗ (2)(−1) + (−1)(3)
= (1)(−2) + (0)(1) + (2)(−1) + (−1)(3)
= −7.
Example 8
Consider the evaluation inner product
hp, qi = p(−1)q(−1) + p(0)q(0) + p(1)q(1)
on P2 . Let p = p(x) = 2 − x + x2 and q = q(x) = 3 − x2 . Determine
(a) hp, qi
(b) kpk
(c) d(p, q)
Solution: By direct calculation, we have
p(−1) = 4, p(0) = 2, p(1) = 2
and
q(−1) = 2, q(0) = 3, q(1) = 2.
Moreover, since p − q = (p − q)(x) = p(x) − q(x), it follows that
(p − q)(−1) = 2, (p − q)(0) = −1, (p − q)(1) = 0.
Hence,

12
 (a) hp, qi = (4)(2) + (2)(3) + (2)(2) = 18.
p p √
(b) kpk = hp, pi = (4)2 + (2)2 + (2)2 = 24.
p p √
(c) d(p, q) = kp − qk = hp − q, p − qi = (2)2 + (−1)2 + (0)2 = 5.

Matrix inner products

Let u and v be vectors in Rn that are expressed in column form, and let A
be an invertible n × n matrix. If u • v is the Euclidean inner product on Rn ,
then
hu, vi = Au • Av = (Av)T Au = v T AT Au
also defines an inner product — called the inner product on Rn gener-
ated by A. An inner product on Rn generated by an (invertible) matrix will
sometimes be called a matrix inner product on Rn .

It is extremely important that the matrix A above is invertible. This is to

ensure that the operation actually satisfies the four axioms in the definition
of an inner product. Can you see which axiom will fail if A is not invertible?
(This question forms part of the tutorial on §6.1.)

Example 9
2
 hu,vi = 9u1 v1 + 4u2 v2 is the inner product on R generated
(a) Show that
3 0
by A = .
0 2

(b) Use this inner product to compute hu, vi if u = (−3, 2) and v = (1, 7).

Solution:

(a) Let u = (u1 , u2 ) and v = (v1 , v2 ) be any vectors in R2 . Then

     
3 0 u1 3 0 v1
Au • Av = •
0 2 u2 0 2 v2
   
3u1 3v1
= •
2u2 2v2
= 9u1 v1 + 4u2 v2
= hu, vi .

Since u and v were arbitrary, this shows that the given inner product
is generated by the matrix A.

13
 (b) With u = (−3, 2) and v = (1, 7) we have

hu, vi = 9(−3)(1) + 4(2)(7) = 29.

Note that part (a) of Example 9 is very suggestive. Indeed, without too much
thought we see that all weighted Euclidean inner products are generated by
diagonal matrices. To see this, consider the weighted Euclidean inner product
on Rn with weights w1 , . . . , wn . If u = (u1 , . . . , un ) and v = (v1 , . . . , vn ) are
vectors in Rn , then
√    √  
w1 0 0 ··· 0 u1 w1 0 0 ··· 0 v1
√ √
 0 w2 0 ··· 0   u2   0
    w2 0 ··· 0   v2 
 
 √ √ 
 0
 0 w3 · · · 0   u3  •  0
    0 w3 · · · 0   v3 
 
 .. .. .. ... ..   ..   .. .. .. ... ..   .. 
 . . . .  .   . . . .  . 
√ √
0 0 0 ··· wn un 0 0 0 ··· wn vn
√  √ 
w 1 u1 w1 v1
 √w2 u2   √w2 v2 
√
 w3 u3   √w3 v3 
  
=  • 
 ..   .. 
 .   . 
√ √
w n un wn vn
= w1 u1 v1 + w2 u2 v2 + w3 u3 v3 + · · · + wn un vn .

This shows that the weighted Euclidean inner product on Rn with weights
w1 , . . . , wn is generated by the matrix
√ 
w1 0 0 ··· 0
√
 0 w2 0 ··· 0 
 √ 
A= 0
 0 w3 · · · 0  .
 .. .. .. . . .. 
 . . . . . 
√
0 0 0 ··· wn

So the class of weighted Euclidean inner products is a special subclass of the

class of matrix inner products.

Example 10
√
Let kuk = 1, kvk = 2, kwk = 3, hu, vi = −1, hu, wi = 0 and hv, wi = 3.
Compute:
hv + w, 2u − vi .

14
Solution:
By using the properties of an inner product, we find

hv + w, 2u − vi = hv, 2u − vi + hw, 2u − vi
= hv, 2ui − hv, vi + hw, 2ui − hw, vi
= 2 hv, ui − hv, vi + 2 hw, ui − hw, vi
= 2 hu, vi − kvk2 + 2 hu, wi − hv, wi
= 2(−1) − (2)2 + 2(0) − (3)
= −9.

15