Scribd
Upload
34 views

S6, S7, S8 CS - U4 Getter Setter EM Algorithm

The document discusses getters and setters in Python object-oriented programming and the EM algorithm. It explains that getters and setters are used in Python classes to add validation logic when getting or setting values and to avoid direct access of private variables. It then describes the EM algorithm as an iterative method used to find maximum likelihood estimates for latent variable models. It consists of an expectation step (E-step) where missing values are estimated, and a maximization step (M-step) where parameter values are updated, repeating until convergence. The EM algorithm is applied in problems like Gaussian mixture models, clustering, and other domains.

Uploaded by

yenop
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
34 views

S6, S7, S8 CS - U4 Getter Setter EM Algorithm

The document discusses getters and setters in Python object-oriented programming and the EM algorithm. It explains that getters and setters are used in Python classes to add validation logic when getting or setting values and to avoid direct access of private variables. It then describes the EM algorithm as an iterative method used to find maximum likelihood estimates for latent variable models. It consists of an expectation step (E-step) where missing values are estimated, and a maximization step (M-step) where parameter values are updated, repeating until convergence. The EM algorithm is applied in problems like Gaussian mixture models, clustering, and other domains.

Uploaded by

yenop
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 32

Computational Statistics

Unit -4
S7- Getting values & setting
values
S8- EM algorithm
Getting values & setting values
• the main purpose of using getters and setters in
object-oriented programs is to ensure data
encapsulation.
• Private variables in python are not actually hidden
fields like in other object oriented languages.

Getters and Setters in python are often used when:


• We use getters & setters to add validation logic
around getting and setting a value.
• To avoid direct access of a class field i.e. private
variables cannot be accessed directly or modified
by external user.

2
Getting values & setting values
# Python program showing a use
# of get() and set() method in
# normal function

class User:
def __init__(self, age = 0):
self._age = age

# getter method
def get_age(self):
return self._age

3
# setter method
EM algorithm

4
EM algorithm

5
EM algorithm
• What is an EM algorithm?
• The Expectation-Maximization (EM) algorithm is
defined as
– the combination of various unsupervised machine
learning algorithms
– Used to determine the local maximum likelihood
estimates (MLE) or maximum a posteriori estimates
(MAP) for unobservable variables in statistical models.
• Further, it is a technique to find maximum
likelihood estimation when the latent variables
are present. It is also referred to as the latent
variable model.

6
EM algorithm
• A latent variable model consists of both
observable and unobservable variables where
observable can be predicted while unobserved
are inferred from the observed variable.
• These unobservable variables are known as
latent variables.

7
EM algorithm
• EM algorithm is the combination of various
unsupervised ML algorithms, such as the k-means
clustering algorithm.
• Being an iterative approach, it consists of two
modes.
• In the first mode, we estimate the missing or
latent variables. Hence it is referred to as
the Expectation/estimation step (E-step).
• The other mode is used to optimize the
parameters of the models so that it can explain
the data more clearly. The second mode is known
as the maximization-step or M-step.

8
EM algorithm

Expectation step (E - step): It involves the estimation (guess) of all missing values in
the dataset so that after completing this step, there should not be any missing
value.
Maximization step (M - step): This step involves the use of estimated data in the
E-step and updating the parameters.
Repeat E-step and M-step until the convergence of the values occurs.
EM algorithm
Steps in EM Algorithm
The EM algorithm is completed mainly in 4 steps, which include Initialization Step, Expectation
Step, Maximization Step, and convergence Step. These steps are explained as follows:
EM algorithm
Steps in EM Algorithm
The EM algorithm is completed mainly in 4 steps, which include Initialization Step,
Expectation Step, Maximization Step, and convergence Step. These steps are
explained as follows:
1st Step: The very first step is to initialize the parameter values. Further, the system is
provided with incomplete observed data with the assumption that data is obtained
from a specific model.
2nd Step: This step is known as Expectation or E-Step, which is used to estimate or
guess the values of the missing or incomplete data using the observed data. Further,
E-step primarily updates the variables.
3rd Step: This step is known as Maximization or M-step, where we use complete data
obtained from the 2nd step to update the parameter values. Further, M-step primarily
updates the hypothesis.
4th step: The last step is to check if the values of latent variables are converging or not.
If it gets "yes", then stop the process; else, repeat the process from step 2 until the
convergence occurs.
Gaussian Mixture Model (GMM)
Gaussian Mixture Model (GMM)
Gaussian Mixture Model (GMM)
Gaussian Mixture Model (GMM)

https://www.kaggle.com/code/charel/learn-by-example-expectation-maximization/notebook
Gaussian Mixture Model (GMM)

https://www.kaggle.com/code/charel/learn-by-example-expectation-maximization/notebook
Gaussian Mixture Model (GMM)
The Gaussian Mixture Model or GMM is defined as a
mixture model that has a combination of the unspecified
probability distribution function.

GMM also requires estimated statistics values such as mean


and standard deviation or parameters.

It is used to estimate the parameters of the probability


distributions to best fit the density of a given training dataset.

Although there are plenty of techniques available to estimate


the parameter of the Gaussian Mixture Model (GMM),
the Maximum Likelihood Estimation is one of the most
popular techniques among them.
Gaussian Mixture Model (GMM)
Let's understand a case where we have a dataset with
multiple data points generated by two different processes.

However, both processes contain a similar Gaussian


probability distribution and combined data.

Hence it is very difficult to discriminate which distribution a


given point may belong to.
Gaussian Mixture Model (GMM)
The processes used to generate the data point represent a
latent variable or unobservable data.

In such cases, the Estimation-Maximization algorithm is one


of the best techniques which helps us to estimate the
parameters of the gaussian distributions.

In the EM algorithm, E-step estimates the expected value for


each latent variable, whereas M-step helps in optimizing
them significantly using the Maximum Likelihood Estimation
(MLE).
Further, this process is repeated until a good set of latent
values, and a maximum likelihood is achieved that fits the
data.
Gaussian Mixture Model (GMM)
Gaussian Mixture Models
The main assumption of these mixture models is that there
are a certain number of Gaussian distributions, and each of
these distributions represents a cluster.

Hence, a Gaussian Mixture model tries to group the


observations belonging to a single distribution together.

Gaussian Mixture Models are probabilistic models which use


the soft clustering approach for distributing the observations
in different clusters i.e, different Gaussian distribution.
Gaussian Mixture Model (GMM)
Gaussian Mixture Models
For Example, the Gaussian Mixture Model of 2 Gaussian
distributions
We have two Gaussian distributions- N(𝜇1, 𝜎12) and N(𝜇2, 𝜎22)
Here, we have to estimate a total of 5 parameters:
𝜃 = ( p, 𝜇1, 𝜎12,𝜇2, 𝜎22)
where p is the probability that the data comes from the
first Gaussian distribution and 1-p that it comes from the
second Gaussian distribution.
Then, the probability density function (PDF) of the mixture
model is given by:
g(x|𝜃) = p g1(x| 𝜇1, 𝜎12) + (1-p)g2(x| 𝜇2, 𝜎22 )
Objective: To best fit a given probability density by finding
𝜃 = ( p, 𝜇 , 𝜎 2,𝜇 , 𝜎 2) through EM iterations.
Gaussian Mixture Model (GMM)
Gaussian Mixture Model (GMM)
Gaussian Mixture Model (GMM)
Gaussian Mixture Model (GMM)
Gaussian Mixture Model (GMM)
Gaussian Mixture Model (GMM)
Gaussian Mixture Model (GMM)
Gaussian Mixture Model (GMM)
EM algorithm
Applications of EM algorithm
The primary aim of the EM algorithm is to estimate the missing data in
the latent variables through observed data in datasets. The EM
algorithm or latent variable model has a broad range of real-life
applications in machine learning. These are as follows:
• The EM algorithm is applicable in data clustering in machine learning.
• It is often used in computer vision and NLP (Natural language
processing).
• It is used to estimate the value of the parameter in mixed models
such as the Gaussian Mixture Model and quantitative genetics.
• It is also used in psychometrics for estimating item parameters and
latent abilities of item response theory models.
• It is also applicable in the medical and healthcare industry, such as in
image reconstruction and structural engineering.
• It is used to determine the Gaussian density of a function.
EM algorithm
Advantages of EM algorithm
• It is very easy to implement the first two basic steps of the
EM algorithm in various machine learning problems, which
are E-step and M- step.
• It is mostly guaranteed that likelihood will enhance after
each iteration.
• It often generates a solution for the M-step in the closed
form.
Disadvantages of EM algorithm
• The convergence of the EM algorithm is very slow.
• It can make convergence for the local optima only.
• It takes both forward and backward probability into
consideration. It is opposite to that of numerical
optimization, which takes only forward probabilities.
Thank

You might also like