Scribd
Upload
230 views

Assignment 11

This document contains the solutions to an assignment on machine learning. It addresses questions about the necessary conditions for a probability density function, properties of a Gaussian Mixture Model (GMM) including the number of clusters k, values of πi for different values of k, issues that could arise from a non-Gaussian data distribution, and how to update πk in the EM algorithm. It also notes that K-nearest neighbors is a special case of GMM where the covariance is εI and πk = 1/k.

Uploaded by

prakash subramanian
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
230 views

Assignment 11

This document contains the solutions to an assignment on machine learning. It addresses questions about the necessary conditions for a probability density function, properties of a Gaussian Mixture Model (GMM) including the number of clusters k, values of πi for different values of k, issues that could arise from a non-Gaussian data distribution, and how to update πk in the EM algorithm. It also notes that K-nearest neighbors is a special case of GMM where the covariance is εI and πk = 1/k.

Uploaded by

prakash subramanian
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Assignment 11

Introduction to Machine Learning


Prof. B. Ravindran
1. What are the necessary conditions for
k
X
P (xn ) = πk P (xn |θk )
1

to be a density? (Select all that apply)


(a) 0 ≤ πk ≤ 1
P
(b) k ≤ k πk
(c) P (xi , xj |θk ) = P (xi |θk )P (xj |θk )
P
(d) i πi = 1

Sol. (a), (d)


Refer to the lecture.

Based on the density estimation of a GMM given below, answer questions 2-4.

2. What is value of k?
(a) 3
(b) 4
(c) 5
(d) 6
Sol. (b)
Equal to the number of clusters you can see in the figure.
3. What is the minimum value of k ′ ̸= k, where k is from previous question, for which you will
get a very similar density estimation?

1
(a) 3
(b) 4
(c) 5
(d) 6
Sol. (a)
Should be clear from the image.
4. (2 marks) Assume equal πi for each gaussian model after convergence as in Q2. What would
(approximately) be πi ’s for the model you’ll get with k ′ as in Q3?
(a) [0.33, 0.33, 0.17, 0.17]
(b) [0.2, 0.2, 0.6]
(c) [0.25, 0.25, 0.5]
(d) [0.2, 0.2, 0.2, 0.4]
Sol. (c)
For k = 4, πi ’s = [0.25, 0.25, 0.25, 0.25]
For k = 3, two of the clusters are merged =⇒ πi ’s = [0.25, 0.25, 0.5]

For a set of points (given in orange), the density estimation of a GMM is given below. Based
on this, answer questions 5 and 6.

5. What is the problem evident in the image?

(a) πi ’s are too big


(b) The clusters are not sampled from a gaussian distribution.
(c) The GMM has not converged yet.
(d) There is no problem

Sol. (b)
We would have have gotten a nice density estimate if the clusters were sampled from a gaussian.

2
6. What can be done to get a better fit?
(a) Increase k
(b) Use a better initialisation
(c) Learn for more iterations
(d) There is no problem
Sol. (a)
You can increase k to allow the model to fit a larger number of gaussians to approximate a
clearly non-gaussian distribution.
An example is shown below with k=8.

7. What does soft clustering mean in GMMs?


(a) There may be samples that are outside of any cluster boundary.
(b) The updates during maximum likelihood are taken in small steps, to guarantee conver-
gence.
(c) It restricts the underlying distribution to be gaussian.
(d) Samples are assigned probabilities of belonging to a cluster.
Sol. (d)
Refer to lecture.
8. What is the update for πk in EM algorithm for GMM?
PN
(m) n=1 γ(znk )|v(m−1)
(a) πk = N −1
PN
(m) n=1 γ(znk )|v(m)
(b) πk = N
PN
(m) n=1 γ(znk )|v(m−1)
(c) πk = N
PN
(m) n=1 γ(znk )|v(m)
(d) πk = N −1

Sol. (c)
Refer to lecture.

3
9. KNN is a special case of GMM with the following properties: (Select all that apply)
1
(a) γi = i
(2πϵ)1/2
e− 2ϵ
(b) Covariance = ϵI
(c) µi = µj ∀i, j
1
(d) πk = k

Sol. (b), (d)


Refer to lecture.

You might also like