Scribd
Upload
17 views

Lecture 15-ParameterEstimation

Uploaded by

saramukhopadhyay
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
17 views

Lecture 15-ParameterEstimation

Uploaded by

saramukhopadhyay
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 16

Parameter Estimation

Sunita Sarawagi
CS 215. Fall 2024
So far..
• Computing probabilities of outcomes given a fixed distribution.

• Distributions were given to us as a function..

• Functions had parameters with fixed values


Today’s class

How to determine the values of the parameters.

Parameters differ based on the task and application. These are not fixed
like the speed of light.
The setup for parameter estimation in real-life
• Step 1: A real-life problem:
1. Estimating the probability that at least two out of four servers will be alive next day
2. The probability that stock price will rise by 10% in the next week
3. The expected number of clicks on an advertisement in the next 3 hours
• Step 2: Model the problem: Choose a functional form of the uncertainty.
1. Binomial?

2. Gaussian?

3. Poisson?
The setup for parameter estimation in real-life
Step 3: Collect a training sample by observing over several days.
1. Sample server failure data observed over 3 days

2. Stock price change over a 10 days

3. Number of clicks on the ad over the last 20 hour

• Step 4: Estimate the unknown parameters using the training sample


The overall setup in parameter estimation
• Given: a density or distribution function with parameters
• Given: sample:
• The i-th sample is a random variable assumed to be independently identically
distributed as per the unknown
• Find

• Since D is a finite sample, we cannot really know the actual . Best we


can do is obtain an estimate of
• We will denote the estimate as
• Goodness of estimate will be discussed later.
Types of estimators
• Maximum likelihood: sample D is all you got.

• Bayesian estimation: in addition to sample, we got prior beliefs.


Maximum Likelihood Estimation
• If were known we could have calculated the probability of getting the N
outcomes in from the distribution as

• Likelihood refers to the above function. Often denoted as

• Maximum likelihood estimator:


• Choose the parameter for which the above likelihood is maximized
Finding that maximizes likelihood
• Use log-likelihood instead of likelihood to convert products into sums

• Maximum likelihood estimator

Solved using numerical optimization methods applying calculus.


MLE for Bernoulli

Data sample: D
Examples: MLE for Poisson
MLE for Gaussian
• Most difficult question: what distribution to use to model accidents in a city?

• Binomial? Will need to know total number of drivers


• Gaussian?
• Poisson?

You might also like