MSML604 Homework 3

Peeyush Dyavarashetty
21 February 2025

1 Question 1
1.1 Question 1.a
 
x1
For any x = ∈ R2 ,
x2
  
 5 8 x1
xT Ax = x1 x2

8 13 x2
 
 5x1 + 8x2
= 5x21 + 16x1 x2 + 13x22

=⇒ x1 x2
8x1 + 13x2
p
−16x2 ± 256x22 − 260x22 −16x2 ± 2x2 i
=⇒ x1 = =
2 2
We can observe that the roots for x1 are imaginary. This means that the
parabola does not have any real roots. Therefore, the value 5x21 + 16x1 x2 +
13x22 > 0, implying that xT Ax > 0, which says that A is positive definite.

1.2 Question 1.b

We know about Eigen decomposition that if A ∈ Rn×n had n linearly inde-
pendent eigenvectors vi for i ∈ [n], then A can be factorized as A = V ΛV −1
Finding eigenvalues and eigenvectors for A. The eigenvalues are

det(A − λI) = 0
5−λ 8
=⇒ =0
8 13 − λ
=⇒ (5 − λ)(13 − λ) − 64 = 0
=⇒ λ2 − 18λ + 1 = 0
√
18 ± 182 − 4 √
=⇒ λ = =9±4 5
2
Theorem: Suppose that S ∈ Rn×n is a symmetrical real matrix, then n eigen-
vectors vi of S are orthogonal when eigenvalues are distinct. We know that

1
eigenvalues are distinct. Therefore we can write A = V ΛV −1 = V ΛV T . Now,
the eigenvectors are
    
5 8 v11 v
Av1 = λ1 v1 =⇒ = λ1 11
8 13 v12 v12
√ √
=⇒ 5v11 + 8v12 = (9 − 4 5)v11 & 8v11 + 13v12 = (9 − 4 5)v12
√
1− 5
=⇒ v12 = v11
2 p√
 
√ 1

1
 √ 5 + 1
2 5p
=⇒ v1 = v11 1−√5 =⇒ v1 =  −1 √ 
2 √ √ 5−1
2 5

    
5 8 v21 v
Av2 = λ2 v2 =⇒ = λ2 21
8 13 v22 v22
√ √
=⇒ 5v21 + 8v22 = (9 + 4 5)v21 & 8v21 + 13v22 = (9 + 4 5)v22
p√
√
 
√ 1√ 5−1
1+ 5
=⇒ v22 = v21 =⇒ v2 =  21 5 p√ 
2 √√ 5+1
2 5

Now, A can be written as

 √√ √√   √√ √√ 
√ 5+1
√ √ 5−1
√
√ √ 5+1 −
√ √5−1
0 √  √ 2 √5
 
9−4 5
A = V ΛV T =  √2√ 5 √√2 5  √√2 5 

 √
0 9+4 5
 
−
√ √5−1 √ 5+1
√ √ 5−1
√ √ 5+1
√
2 5 2 5 2 5 2 5
(1)

Now, we know that

 √√ √√   √√ √√ 
−
√ 5+1
√ √ 5−1
√ √ 5+1
√ √ √5−1
V V T =  √2√ 5 √√2 5   √√ √√2 5 
 2 5
=I (2)

− 5−1 5+1 5−1
√√ √√ √√ √ 5+1
√
2 5 2 5 2 5 2 5
 √√ √√   √√ √√ 
−
√ 5+1
√ √ √5−1 √ 5+1
√ √ 5−1
√
V T V =  √√2 5 √√2 5    √√
2 5 √√2 5 =I (3)
 
5−1 5+1 − 5−1
√√ √√ √√ √ 5+1
√
2 5 2 5 2 5 2 5

1.3 Question 1.c

√√ √√ √ √
Consider v1 = √ 5+1
√ and v2 = √ 5−1
√ and λ1 = 9 − 4 5 and λ = 9 + 4 5.
2 5 2 5
Now,
1 √ √
v1 v2 = √ λ2 = 161 ± 72 5 λ−1 = 9 ∓ 4 5 =⇒ λ1 = λ−1
2
5

2
 From equations 1, 2 and 3
A2 = AA = V ΛV T V ΛV T = V ΛΛV T = V Λ2 V T
Verification:
    
25 8 5 8 89 144
A = =
8 13 8 13 144 233

v1 v2 λ21 0 v1 −v2
   
2 T
VΛ V =
−v2 v1 0 λ22 v2 v1
v1 v2 λ1 v1 −λ21 v2
  2 
=
−v2 v1 λ22 v2 λ22 v1
λ21 v12 + λ22 v22 −λ21 v1 v2 + λ22 v1 v2
 
=
−λ21 v1 v2 + λ22 v1 v2 λ22 v12 + λ21 v22
 
89 144
= = A2
144 233

A−1 = (V ΛV T )−1 = (V T ) − 1Λ− 1V −1 = (V −1 )−1 Λ−1 V T = V Λ−1 V T

Verification:
 
−1 13 −8
A =
−8 5

   
−1 T v1 v2 λ2 0 v1 −v2
VΛ V =
−v2 v1 0 λ 1 v2 v1
λ2 v1 + λ1 v22
2
 
−λ2 v1 v2 + λ1 v1 v2
=
−λ2 v1 v2 + λ1 v1 v2 λ1 v12 + λ2 v22
 
13 −8
= = A−1
−8 5

2 Question 2
Given Sn+ is a set of n × n positive semidefinite matrixes. Now to prove Sn+ is
convex, we need to prove that the line between any 2 matrices A, B ∈ Sn+ should
belong to Sn+ .
For that, we know for any matrix x ∈ Rn , xT Ax ≥ 0 and xT Bx ≥ 0.
Now, for a matrix aA + (1 − a)B, where a ∈ [0, 1],
xT (aA + (1 − a)B)x = axT Ax + (1 − a)xT Bx ≥ 0
as a ≥ 0 is a constant and 1 − a ≥ 0 since a ∈ [0, 1], implying axT Ax ≥ 0 and
(1 − a)xT Bx ≥ 0, This implies that aA + (1 − a)B ∈ Sn+ . This means that the
line segment passing any 2 matrices A and B lie in Sn+ concluding that Sn+ is
convex.

3
3 Question 3
Given,

||x − a||2 ≤ ||x − b||2

=(x − a)T (x − a) ≤ (x − b)T (x − b)
=xT x − xT a − aT x + aT a ≤ xT x − xT b − bT x + bT b
= − 2aT x + aT a ≤ −2bT x + bT b
=2(b − a)T x ≤ bT b − aT a

Now, we can say that c = 2(b − a)T and d = bT b − aT a and {x ∈ Rn |||x − a||2 ≤
||x − b||2 } is of form {x ∈ Rn |cT x ≤ d}, which is a halfspace.

4 Question 4
We know that the function f is convex iff
• dom f is convex

• Hessian matrix of f , represented by Hf is positive semidefinite

∂2f ∂2f
" #
∂x21 ∂x1 ∂x2
Hf = ∂2f ∂2f
∂x2 ∂x1 ∂x22

To prove that R++ is a convex set, consider 2 numbers a, b ∈ R++ where a > b,
consider c = θa + (1 − θ)b where θ ∈ [0, 1]. Now, Since c = b + θ(a − b) > 0 since
a − b > 0. This implies that R++ is a convex set.

4.1 Question 4.a

1
Given f (x1 , x2 ) = x1 x2 , where 0 < x1 , x2 < ∞. We know that the domain is
convex. Now,
∂f −1 ∂f −1
= 2 =
∂x1 x1 x2 ∂x2 x1 x22
∂2f 2 ∂2f 2 ∂2f ∂2f 1
= 3 = = = 2 2
∂x21 x1 x2 ∂x22 x1 x32 ∂x1 x2 ∂x2 x1 x1 x2

Using these equations, we get

" 2 1
#  2 
x31 x2 x21 x22 1 2x2 x1 x2
Hf = 1 2 = 3 3
x21 x22 x1 x32 x1 x2 1 x2
x 2x21

4
  
v1
To check whether this hessian matrix is positive semidefinite, consider v = ,
v2
where v1 , v2 ∈ R and try to find v T Hf v
 2  
1 2x2 x1 x2 v1
v T Hf v = v 1
 
v2
x31 x32 x1 x2 2x21 v2
1
=⇒ v T Hf v = (2x22 v12 + 2x1 x2 v1 v2 + 2x21 v22 )
x31 x32
2
Consider a = x2 v1 and b = x1 v2 , then we will have v T Hf v = x31 x32
(a2 + b2 + ab).
Now, to find whether the roots of a2 + ab + b2 are real, we use quadratic formula
b2 − 4ac ≥ 0

b2 − 4ac = b2 − 4(1)(b2 ) = −3b2 < 0

Since the roots are imaginary, we can say that v T Hf v ≥ 0 since x1 , x2 ∈ R++
(In fact, Hf is positive definite). Therefore, the function f (x1 , x2 ) is convex
function.

5 Question 4.b
x1
Given f (x1 , x2 ) = x2 , where 0 < x1 , x2 < ∞. We know that the domain is
convex. Now,
∂f 1 ∂f −x1
= = 2
∂x1 x2 ∂x2 x2
∂2f 2
∂ f 2x1 ∂2f ∂2f −1
=0 = 3 = = 2
∂x21 ∂x22 x2 ∂x1 x2 ∂x2 x1 x2

Using these equations, we get

−1
" #
0
 
x22 1 0 −x2
Hf = −1 2x1 = 3
x22 x32 x2 −x2 2x1
 
v1
To check whether this hessian matrix is positive semidefinite, consider v = ,
v2
where v1 , v2 ∈ R and try to find v T Hf v
  
T
  1 0 −x2 v1
v Hf v = v 1 v2 3
x2 −x 2 2x1 v2
2v2
=⇒ v T Hf v = (x1 v2 − x2 v1 )
x32

There exists v such that v T Hf v < 0. This implies that the Hf is not positive
semidefinite matrix. Therefore, f is not convex function.