Probability Cheatsheet

Yahya Almardeny
[email protected]

June 2019

1
1 Overview 5. Bayes Theorem: P (A|B) = P (B|A)×P P (B)
(A)
. It
gives the probability of A based on prior knowl-
Probability provides measures for reasoning the like- edge of B.
lihood that an event will occur in an experiment.

2 Terminology 5 Probability Distributions

1. Experiment: Procedure that yields one of the 1. Probability Mass Function (PMF): Gives
possible outcomes. E.g. Tossing a coin. the probability a discrete R.V X takes on the
value x: P (X = x).
2. Sample Space S: A set of all possible outcomes
of an experiment. E.g. S = {Head, Tail} when
tossing a fair coin.

3. Event E: Set of outcomes of an experiment.

Note that if the Event contains only one out-
come, it is then called an elementary event. E.g.
The event that coin is Head.

3 Random Variable R.V

A variable that its possible values are the outcomes 2. Probability Density Function (PDF):
of an experiment. It has two types: Gives the probability a continuous R.V X takes
on the value x: P (X = x) which means how
1. Discrete R.V: Take one countable number of
dense the probability of X near x, and it will
distinct values. E.g 1, 2, 3, 4
be the definite Integrals between two points.
2. Continuous R.V: Take infinite number of pos-
sible values (usually measurements). E.g. all
interval values in the rage [1 - 2].

4 Probability of Events
1. Marginal probability: Simply P (A) which
means ”unconditional probability”. And for
each outcome s, it should satisfy: 0 ≤ P (s) ≤ 1
and ΣP (si ) = 1.

2. Addition Rule: P (A ∪ B) = P (A) + P (B) −

P (A ∩ B) where ∪ means OR, ∩ means AND.
3. Cumulative Density Function (CDF):
3. Joint (Compound) Probability Rule: Gives the probability a R.V X is less than or
equals the value x: P (X ≤ x). It is cumulative
(a) Independent Events: P (A ∩ B) = P (A) ×
because it adds the probabilities up to x.
P (B).
(b) Dependent Events: P (A ∩ B) = P (A) × (a) For Discrete R.V: It is the summation of
P (B|A). where P (B|A) read: Probability previous probabilities Σp(xi ).
of B given A had happened.
(b) RFor Continuous R.V: It is the integral
P (A∩B) x
4. Conditional Probability: P (A|B) = P (B) −∞ p(x)dx

1
6 Expected Value of R.V • 10% Rule: We assume independence if the
sample ≤ 10% of the population.
Intuitively, it is the long-run average value of repeti-
• PMF: P (X = x) = (kn )px (1 − p)n−x which
tions of the same experiment it represents (i.e. what
means the probability of k successes in n
outcome to expect on long run)
trials.
1. For a Discrete R.V: It is the average of R.V • E(X) = µX = n.p where p is the probabil-
values based on their associated probabilities ity of success per trial.
E(X) = Σ(xi × p(xi )). p p
• σX = var(X) = n.p(1 − p).
2. RFor a Continuous R.V: E(X) = • The more trials we do, the more Binomial
∞
−∞ xf (x)dx where f (x) is probability density becomes Normal distribution.
function.
2. Bernoulli Distribution: Simply a Binomial
Distribution with just one trial.
7 Combining Random Variables
3. Geometric Distribution: How many trials
If we have two independent random variables X and until success?. It is similar to Binomial dist.
Y , we can add or subtract to get the total or the but here we do not have fixed number of trials
difference of their means (expected values) and vari- because we do not know ahead how many trials
ances: T = X ± Y ⇒ µT = µX ± µY and σT2 = until we get the desired outcome:
σX2 ± σ2 .
Y
• PMF: P (X = x) = (1 − p)k−1 p.
• E(X) = µX = p1 .
8 The Law of Large Numbers q
• σX = var(X) = 1−p
p
p2
.
Let X be R.V of a population where its expected
value (i.e Mean) is E(X). Let Xn = x1 +x2n+...xn be 4. Hypergeometric Distribution: It is a dis-
the mean of n samples ∈ X. As n → ∞ : Xn → crete probability distribution that describes the
E(X). Which means as we do more experiments, the probability of k successes in n draws, without
outcomes become closer and closer to the outcomes replacement, from a finite population of size N
of the probability theory. that contains exactly K objects. In contrast,
the binomial distribution describes the proba-
bility of k successes in n draws with replace-
9 Statistical Distributions ment.
1
5. Poisson Distribution: It is a discrete proba-
1. Binomial Distribution: How many success in bility distribution that predicts rare events that
finite number of trials: are independent of one another and occur with
a known constant rate λ. It is usually used in-
• Made up of independent trials. stead of Binomial dist. if the number of trials is
• Each trial has one of two discrete outcomes very high (fixed interval of time) and the prob-
(success or failure). ability (occurrence) of each is relatively low:
λx −λ
• Fixed number of trials • PMF: P (x) = x! e .
• Probability of success in each trial is con- • Mean is E(X) = λ. and Variance is λ.
stant.
6. Normal Distribution (a.k.a Gaussian): It
• E.g. X = number of heads after 10 flips of is a continuous probability distribution that has
a coin. the notation X ∼ N (µ, σ 2 ). It is symmetric
1
Included because it is related directly to probability and has a bell shape where most of the values

2
 are near to the mean and follows the 68–95–99.7 in finance, for elapsed times, or during Poisson
rule (a.k.a empirical rule). This rule states that processes. In other words, it is a generalization
68.27%, 95.45% and 99.73% of the values lie of both the exponential and chi-squared distri-
within one, two and three standard deviations butions.
of the mean, respectively:
10. Exponential Distribution: Special case of
2
• PDF: P (x) =
2
√1 e−(x−µ) /2σ . Gamma Distribution. It is a continuous ana-
σ 2π
logue of the geometric distribution and often
2 .
• Mean is E(X) = µX . and Variance is σX used to model the time elapsed between events.
Formally, it is the probability distribution of the
7. Log-Normal Distribution: It is a continuous
time between events in a process where events
probability distribution of a random variable
occur continuously and independently at a con-
whose logarithm is normally distributed.Because
stant average rate (i.e. Poisson process):
the values in a log-normal distribution are pos-
itive, they create a right-skewed curve which • PDF: P (X = x) = λe−λx
is important in determining which distribution • E(X) = µX = λ−1 = β
is appropriate to use in investment decision- 2 = λ−2 = β 2
• σX
making. One of the most common applications
of log-normal distributions is in the analysis 11. Chi-Square Distribution: It is a special case
of stock prices. A log-normal distribution can of the Gamma distribution. It is a continuous
be translated to a normal distribution and vice distribution of a sum of the squares of k inde-
versa using associated logarithmic calculations. pendent standard normal deviates (where k is
known as degree of freedom). A standard nor-
8. Uniform Distribution: It is a probability dis- mal deviate is a random sample from the stan-
tribution that has constant probability. dard normal distribution.
k −x
(a) Discrete Uniform Dist.: It takes one of a 1 x 2 −1 e 2
• PDF: P (X = x) = k where
finite n possible values: 2 2 Γ( K
2
)

1
Γ is the gamma function
• PMF: P (X = x) = n • E(X) = µX = k
(a+b)
• E(X) = µX = 2 2 = 2k
• σX
2 = (b−a+1)2 −1
• σX 12 12. Student’s t-Distribution: It is a probability
(b) Continuous Uniform Dist.: It takes values distribution used to estimate population param-
within a specified range a to b: eters when the sample size is small and standard
• PDF: deviation is unknown. It is mainly used by ap-
( plying Student’s t-test for assessing the statis-
1
if x ∈ [a, b]
b−a tical significance of the difference between two
P (X = x) =
0 otherwise sample means, and in linear regression analysis.
• PDF:
• E(X) = µX = 21 (a + b)
Γ( ν+1 ) x2 ν+1
2 = 1 (b − a)2
• σX 12
√ 2 ν (1 + )− 2
νπΓ( 2 ) ν
9. Gamma Distribution: It is a family of a con- where ν is the number of degrees of free-
tinuous, positive-only, unimodal distributions dom and Γ is the gamma function.
that encode the time required for alpha events • E(X) = µX = 0 f or ν > 1, otherwise
to occur in a Poisson process with mean arrival undefined.
time of beta. These distributions are useful in 2 = ν f or ν > 2, ∞ f or 1 < ν ≤ 2,
• σX ν−2
real-life where something has a natural mini-
otherwise undefined.
mum of 0. For example, it is commonly used