Scribd
Upload
30 views

Probability: by Zhichun Li

This document discusses key concepts in probability and stochastic processes. It defines random variables, probability density functions, moments, joint and conditional probability, and commonly encountered distributions. It also covers the central limit theorem, properties of stochastic processes like stationarity and autocorrelation, and issues relevant to modeling internet measurements like arrival processes and the impact of short-tailed vs. long-tailed distributions.

Uploaded by

Deepa Shree
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
30 views

Probability: by Zhichun Li

This document discusses key concepts in probability and stochastic processes. It defines random variables, probability density functions, moments, joint and conditional probability, and commonly encountered distributions. It also covers the central limit theorem, properties of stochastic processes like stationarity and autocorrelation, and issues relevant to modeling internet measurements like arrival processes and the impact of short-tailed vs. long-tailed distributions.

Uploaded by

Deepa Shree
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 25

Probability

By Zhichun Li
Notations
Random variable & CDF
• Definition: is the outcome of a random event
or experiment that yields a numeric values.
• For a given x, there is a fixed possibility that
the random variable will not exceed this value,
written as P[X<=x].
• The probability is a function of x, known as
FX(x). FX(.) is the cumulative distribution
function (CDF) of X.
PDF & PMF
• A continuous random variable has a probabilit
y density function (PDF) which is:

• The possibility of a range (x1,x2] is

• For a discrete random variable. We have a disc


rete distribution function, aka. possibility mass
ive function.
Moment
• The expected value of a continuous random
variable X is defined as

• Note: the value could be infinite (undefined). The mean of X is


its expected value, denote as X
• The nth moment of X is:
Variability of a random variable
• Mainly use variance to measure:

• The variance of X is also denote as: X


• Variance is measured in units that are the
square of the units of X; to obtain a quantity in
the same units as X one takes the standard
deviation:
Joint probability
• The joint CDF of X and Y is:

• The covariance of X and Y is defined as:

• Covariance is also denoted:


• Two random variable X and Y are independent
if:
Conditional Probability
• For events A and B the conditional probability
defined as:

• The conditional distribution of X given an


event denoted as:

• It is the distribution of X given that we know


that the event has occurred.
Conditional Probability (cont.)
• The conditional distribution of a discrete
random variable X and Y

• Denote the distribution function of X given


that we happen to know Y has taken on
the value y.
• Defined as:
Conditional Probability (cont.)
• The conditional expectation of X given an
event:
Central Limit Theorem
• Consider a set of independent random variable
X1,X2, … XN, each having an arbitrary
probability distribution such that each
distribution has mean  and variance 
• When

With parameter  and variance 


Commonly Encountered Distributions
• Some are specified in terms of PDF, others in terms of CDF.
In many cases only of these has a closed-form expression
Stochastic Processes
• Stochastic process: a sequence of random
variables, such a sequence is called a
stochastic process.
• In Internet measurement, we may encounter a
situation in which measurements are presented
in some order; typically such measurements
arrived.
Stochastic Processes
• A stochastic process is a collection of random
variables indexed on a set; usually the index
denote time.
• Continuous-time stochastic process:

• Discrete-time stochastic process:


Stochastic Processes
• Simplest case is all random variables are indep
endent.
• However, for sequential Internet measurement,
the current one may depend on previous ones.
• One useful measure of dependence is the autoc
ovariance, which is a second-order property:
Stochastic Processes
• First order to n-order distribution can characterize the
stochastic process.
– First order:
– Second order:

• Stationary
– Strict stationary

For all n,k and N


Stochastic Processes
• Stationary
– Wide-sense Stationary (weak stationary)
• If just its mean and autocovariance are invariant with ti
me.
Stochastic Processes
• Measures of dependence of stationary process
– Autocorrelation: normalized autocovarience

– Entropy rate
• Define entropy:

• Joint entropy:
Stochastic Processes
• Measures of dependence of stationary process
– Entropy rate
• The entropy per symbol in a sequence of n symbols

• The entropy rate


Special Issues in the Internet
• Relevant Stochastic Processes
– Arrivals: events occurring at specific points of tim
e
– Arrival process: a stochastic process in which succ
essive random variables correspond to time instant
s of arrivals:

• Property: non-decreasing & not stationary


– Interarrival process (may or may not stationary)
Special Issues in the Internet
• Relevant Stochastic Processes
– Timeseries of counts
• Fixed-size time intervals and counts how many arrivals
occur in each time interval. For a fixed time interval T, t
he yields where:

• T called timescale
• Can use an approximation to the arrival process by maki
ng additional assumption (such as assuming Poisson)
• A more compact description of data
Short tails and Long tails
“In the case of network measurement large
values can dominate system performance,
so a precise understanding of the
probability of large values is often a prime
concern”
• As a result we care about the upper tails of a
distribution
• Consider the shape of
Short tails and Long tails
• Declines exponentially if exists >0, such that:

– AKA. Short-tailed or light-tailed


– Decline as fast as exponential or faster.
• Subexponential distribution

– A long tail
– The practical result is that the samples from such distributio
ns show extremely large observations with non-negligible fr
equency
Short tails and Long tails
• Heavy-tailed distribution:
– a special case of the subexponential distributions
– Asymptotically approach a hyperbolic (power-law)
shape
– Formally:

– Such a distribution will have a PDF also follow a p


ower law:
Short tails and Long tails
• A comparison of a short-tailed and a long-
tailed distribution

You might also like