Scribd
Upload
254 views10 pages

Parzen Windowing

The document discusses Parzen window density estimation, a non-parametric method for estimating an unknown probability density function from a finite data sample. It describes how Parzen windows work by placing a kernel function, such as a Gaussian, over each data point and estimating the density as the average of the kernels. The width of the kernel determines the degree of smoothing. An example calculates the Parzen density estimate at a point using a Gaussian kernel for a sample of 5 data points. Graphs show the individual Gaussian kernels and the final estimated density function.

Uploaded by

Avishek Chowdhury
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
254 views10 pages

Parzen Windowing

The document discusses Parzen window density estimation, a non-parametric method for estimating an unknown probability density function from a finite data sample. It describes how Parzen windows work by placing a kernel function, such as a Gaussian, over each data point and estimating the density as the average of the kernels. The width of the kernel determines the degree of smoothing. An example calculates the Parzen density estimate at a point using a Gaussian kernel for a sample of 5 data points. Graphs show the individual Gaussian kernels and the final estimated density function.

Uploaded by

Avishek Chowdhury
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 10

Parzen windows

Probability density function (pdf) The mathematical denition of a continuous probability function, p(x), satises the following properties: 1. The probability that x is between two points a and b P (a < x < b) =
b a

p(x)dx

2. It is non-negative for all real x. 3. The integral of the probability function is one, that is

p(x)dx = 1

The most commonly used probability function is Gaussian function (also known as Normal distribution) 1 (x c)2 p(x) = exp 2 2 2
1

where c is the mean, 2 is the variance and is the standard deviation.


0.8 0.7

0.6

0.5

0.4

0.3

0.2

0.1

0.1

0.2 5

Figure: Gaussian pdf with c = 0, = 1. Extending to the case of a vector x, we have non-negative p(x) with the following properties: 1. The probability that x is inside a region R P = 2. The integral of the probability function is one, that is p(x)dx = 1
2

p(x)dx

0.2

0.15

0.1

0.05

0 2 1 0 1 2 2 1 1 0 2

Figure: 2D Gaussian pdf.

Density estimation Given a set of n data samples x1,..., xn, we can estimate the density function p(x), so that we can output p(x) for any new sample x. This is called density estimation. The basic ideas behind many of the methods of estimating an unknown probability density function are very simple. The most fundamental techniques rely on the fact that the probability P that a vector falls in a region R is given by P =
R

p(x)dx

If we now assume that R is so small that p(x) does not vary much within it, we can write P =
R

p(x)dx p(x)

where V is the volume of R.


4

dx = p(x)V

On the other hand, suppose that n samples x1,..., xn are independently drawn according to the probability density function p(x), and there are k out of n samples falling within the region R, we have P = k/n Thus we arrive at the following obvious estimate for p(x), k/n p(x) = V Parzen window density estimation Consider that R is a hypercube centered at x (think about a 2-D square). Let h be the length of the edge of the hypercube, then V = h2 for a 2-D square, and V = h3 for a 3-D cube.
5

h ( x 1 h/2, x 2+h/2 ) ( x 1 +h/2, x 2 +h/2 )

( x 1 h/2, x 2 h/2 )

( x 1 +h/2, x 2 h/2 )

Introduce

xi x ( )= h

1 0

|xik xk | 1/2, h

k = 1, 2 otherwise

which indicates whether xi is inside the square (centered at x, width h) or not. The total number k samples falling within the region R, out of n, is given by

xi x k= ( ) h i=1
6

The Parzen probability density estimation formula (for 2-D) is given by k/n p(x) = V 1 n 1 xi x = ( ) 2 n i=1 h h
( xih x ) is called a window function. We can generalize the idea and allow the use of other window functions so as to yield other Parzen window density estimation methods. For example, if Gaussian function is used, then (for 1-D) we have

1 n 1 (xi x)2 p(x) = exp n i=1 2 2 2 This is simply the average of n Gaussian functions with each data point as a center. needs to be predetermined.
7

Example: Given a set of ve data points x1 = 2, x2 = 2.5, x3 = 3, x4 = 1 and x5 = 6, nd Parzen probability density function (pdf) estimates at x = 3, using the Gaussian function with = 1 as window function. Solution: 1 (x1 x)2 exp 2 2 1 (2 3)2 exp = 2 2 = 0.2420

(x2 x)2 1 exp 2 2 1 (2.5 3)2 = exp 2 2 = 0.3521

1 (x3 x)2 exp 2 2 1 (x4 x)2 exp 2 2 1 (x5 x)2 exp 2 2 so

= 0.3989

= 0.0540

= 0.0044

p(x = 3) = (0.2420 + 0.3521 + 0.3989 +0.0540 + 0.0044)/5 = 0.2103

The Parzen window can be graphically illustrated next. Each data point makes an equal contribution to the nal pdf denoted by the solid line.

0.6

0.5

0.4

0.3

p(x)

0.2

0.1

0.1

0.2

2 x

Figure above: The dotted lines are the Gaussian functions centered at 5 data points.
0.6

0.5

0.4

0.3

p(x)

0.2

0.1

0.1

0.2

2 x

Figure above: The Parzen window pdf function sums ups 5 dotted line.
10

You might also like