Stats 1 Formulae

Statistics 1 Formulae
Measures of Central Tendency
Mean
Continuous data
n
∑ xi
i=1
x̄ =
n

n total number of observations

xi each i observation
th

Continuous grouped data

n
∑ fi mi
i=1
x̄ =
∑ fi

mi each group's middle value

fi frequency of each group

Discrete data
m
∑ fi xi
i=1
x̄ =
∑ fi

xi each discrete value

fi frequency of each discrete value

Adding a constant
ȳ = x̄ + c

x̄ is old mean
ȳ is new mean
c is constant value added in each x or overall mean x̄
i
Multiplying a constant
ȳ = x̄ ⋅ c

x̄ is old mean
ȳ is new mean
c is constant value multiplied with each x or overall mean x̄
i

Median

Important

Data must be sorted

Odd frequency n
n + 1
xi =
2

n is frequency
xi is median value

Even frequency n
n n+1
+
2 2
xi =
2

n is frequency
xi is median value

Adding a constant
yi = xi + c

x̄ is old median
ȳ is new median
 c is constant value added in each x or overall median
i

Multiplying a constant
yi = xi ⋅ c

x̄ is old median
ȳ is new mean
c is constant value multiplied in each x or overall median x̄
i

Mode

Most occured value

If data has 2 modes, then it's bimodal.
If data has more than 2 modes, then it's multimodal.

Adding a constant
y i = mode + c

c is constant added to all terms

yi is new mode

Multiplying a constant
y i = mode ⋅ c

c is constant value multiplied by all terms

yi is new mode

Measures of Dispersion
It tells how much our data varies and spread.

Range
 Difference b/w maximum and minimum value

Range = max − min

Variance

It means how far data is spread in a data set.

The units will be squared, hence we can't measure the quantity.
It is calculated by sum of squared deviation
Deviation = x i − μ

Population variance
n 2
∑ (x i − μ)
2 i=1
σ =
n

n is total no. of observations

μ is mean of all observations

Sample variance
n 2
∑ (x i − μ)
2 i=1
σ =
n − 1

n is total no. of observations

μ is mean of all observations

Adding a constant

Adding a constant does not effect the variance.

As constant will be added to all terms as well as mean, so it cancelled out.

Multiplying a constant
2 2
σ = var ⋅ c

var is old variance

 c is the constant value being multiplied by each term.
As we are squaring the deviations, hence we need to square the constant also.

Standard Deviation

It also measures how much the data is dispersed.

But, we can use it as it's calculated with same quantity.
It is calculated by adding up all squared deviations and square root it.

Note

Standard deviation is the square root of variance.

√ 2
σ = σ

Population Standard Deviation

n 2
∑ (x i − μ)
√ i=1
σ =
n

xi each observation
n is total no. of observations
μ is mean of all observations

Sample Standard Deviation

n 2
∑ (x i − μ)
√ i=1
σ =
n − 1

xi each observation
n is total no. of observations
μ is mean of all observations
Adding a constant

Same as variance, adding a constant does not change the standard deviation.
As constant will be added to all terms and mean, hence it cancelled out.

Multiplying a constant

√ 2
σ = std ⋅ c

std is old standard deviation

c is constant value being multiplied by each term.
As we are squaring the deviations, and then taking total's root, same we are
doing here.

Percentiles

Percentiles tells us how much data is distributed on a k percentile. th

Data should be sorted in ascending order.

Even frequency
p n (n + 1)
percentile = ⋅ +
100 2 2

n is the total number of observations

p is k percentile.
th

Odd frequency
p n
percentile = ⋅
100 2

2
should be rounded to next integer.

Qaurtile
 1. Q1 - Lower (first) = 25% tile
2. Q2 - Middle (Median) = 50% tile
3. Q3 - Upper (Third) = 75% tile

Quartile range

1. Min - Q1
2. Q1 - Q2
3. Q2 - Q3
4. Q3 - Max

InterQuartile Range (IQR)

I QR = Q3 − Q1

Detecting outliers
Lower = Q1 − 1.5 ⋅ I QR

U pper = Q3 + 1.5 ⋅ I QR

Association b/w two categorical variables

Contingency table
Association b/w two numerical variables
Covariance

It quantifies the strength of the linear association b/w two numerical variables.
It shows how two variables are different.

Important

Covariance is always measures b/w −∞ and ∞

−∞ ≤ Cov(x, y) ≤ ∞

Population covariance
N
∑ (x i − x̄) (y i − ȳ)
i=1
Cov(x, y) =
N

N is the total number of observations

x is first variable / column
y is second variable / column

Sample covariance
n−1
∑ (x i − x̄) (y i − ȳ)
i=1
Cov(x, y) =
n − 1

n is the total number of observations

 x is first variable / column
y is second variable / column

Correlation

It is used to interpret the linear association b/w two numerical variables.

It shows how two variables are related.

It derives from covariance.

Correlation is always measures b/w −1 and 1

−1 ≤ r ≤ 1

Pearson correlation

We are using pearson correlation to find a linear relationship b/w two

variables.

Sample correlation
cov(x, y)
r =
Sx Sy

n
∑ (x i − x̄) (y i − ȳ)
i=1
r =
n 2 n 2
√∑ (x i − x̄) ⋅ √∑ (y i − ȳ)
i=1 i=1

cov(x, y) is covariance of x and y

x̄ is x variable mean.
ȳ is y variable mean.
n is total number of observations.
Sx is standard deviation of x variable/column.
 Sy is standard deviation of y variable/column.

Population correlation
cov(x, y)
ρ =
Sx Sy

N
∑ (x i − x̄) (y i − ȳ)
i=1
ρ =
N 2 N 2
√∑ (x i − x̄) ⋅ √∑ (y i − ȳ)
i=1 i=1

cov(x, y) is covariance of x and y

x̄ is x variable mean.
ȳ is y variable mean.
N is total number of observations.
Sx is standard deviation of x variable/column.
Sy is standard deviation of y variable/column.

Point Bi-serial Correlation

¯ − Y
Y ¯
0 1
ρ = √P P
0 1
Ss

We need to assign a value of 1 or 0 to each observation, depending on

whether the observation is in the first group or the second group.
Y0 is the mean of the observations in the first group.
Y1 is the mean of the observations in the second group.
Ss is the sample standard deviation of the entire sample.
P0 is the proportion of observations in the first group.
P1 is the proportion of observations in the second group.

Scatter plot
In scatter plot we usually measure 4 things:

1. Direction (upwards, downwards)

2. Curve (linear, quadratic, cubic)
 3. Clustered or spreaded
4. Outliers

Fitting a line
A linear regression has a equation:
y = mx + c

m is slope
c is y-intercept

2
R

It measures the goodness of fit of regression line.

It's the error metric.
Ranges from 0 ≤ R 2
≤ 1

Permuation and Combination

Permutation Formula
The number of permutations of n objects when p if them are of one kind and
rest distinct is given by:

n!
n
Pr =
(n − r)!

The number of permutations of n objects where p is of one kind, p is of

1 2

another kind and p is of k kind is given by:

k
th

n!

p 1 ! ∗ p 2 !∗. . . p k !

Circular Permutation: Clockwise and Anti-clockwise

 We fix one element and then we have n − 1 elements to arrange in n − 1 ways.

(n − 1)!

If number of elements arranged in clockwise direction and anti-clockwise

direction are same then:

(n − 1)!

Combination Formula
In general, each combination of r objects from n objects can give rise to r!
arrangements.
The number of possible combinations of r objects from collection of n objects
is denoted by:

n!
n
Cr =
r!(n − r)!

Also

n! n!
n
= = C (n−r)
r!(n − r)! (n − r)!r!

In other words, selecting r objects from n objects is the same as rejecting n − r

objects from n objects.

n
Cn = 1 and n
C0 = 1 for all values of n.

n n+1 n+1
Cr = C r−1 + Cr ; 1 ≤ r ≤ n

Probability
Conditional Probability
 The probability of an event A given that another event B has already occurred
is called conditional probability.
It is denoted by P (A|B).

P (A ∩ B)
P (A|B) =
P (B)

Multiplication Rule

The probability of two events A and B occurring together is given by:

P (A ∩ B) = P (A) ⋅ P (B|A)

The probability of three evennts A, B and C occuring together is given by:

P (A ∩ B ∩ C) = P (A) ⋅ P (B|A) ⋅ P (C|A ∩ B)

P (A ∩ B ∩ C)
P (C|A ∩ B) =
P (A ∩ B)

Independent Events
In case of independent events, the probability of two events occurring
together is given by:

P (A|B) = P (A)

In other words, the probability of A occurring given that B has occurred is

equal to the probability of A occurring irrespective of whether B has occurred
or not.

P (A ∩ B) = P (A) ⋅ P (B)

Therefore the probability of A and B are independent of each other.

Independence of more than two events

If A, B, C are independent events, then:

 P (A ∩ B ∩ C) = P (A) ⋅ P (B) ⋅ P (C)

Law of Total Probability

Let E and F be two events such that E ∩ F = ∅ .
c
P (E) = P (E ∩ F ) + P (E ∩ F )

or

c c
P (E) = P (E|F ) ⋅ P (F ) + P (E|F ) ⋅ P (F )

Bayes' Theorem
P (B ∩ A)
P (B|A) =
P (B)

or

P (A|B) ⋅ P (B)
P (B|A) =
c
P (B) ⋅ P (A|B) + P (B ) ⋅ P (A|B)

Suppose that events F

Random Experiment
A random experiment is an experiment whose outcome is not known in
advance. It is an experiment whose outcome is determined by chance.
The outcome of a random experiment is called a random variable.
Sample Space: The set of all possible outcomes of a random experiment is
called the sample space of the experiment.
Sample space is denoted by S .
Suppose a random experiment of throwing a die is performed. The
sample space of the experiment is given by:

S = {1, 2, 3, 4, 5, 6}

The sample space of a random experiment of tossing two coins is given

by:
 S = {(H , H ), (H , T ), (T , H ), (T , T )}

Discrete Random Variable

A random variable that can take on at most a countable number of possible

values is said to be a Discrete Random Variable.
Thus, any random variable that can tkae on only a finite number or
countably infinite number of values is a discrete.

Continuous Random Variable

There also exists a random variable that can take on an uncountably infinite
number of values. Such a random variable is called a Continuous Random
Variable.

Probability Mass Function

The probablity mass function p(x) is positive for at most a coutntable number
values of x. That is, if Xmust assume one of the values x , x , . . . , x then:
1 2 n

p(x i ) ≥ 0 for all i and p(x i) = 0 for all x ≠ x .

i

Since p(x) is a probability, it must satisfy the following:

\inf in

∑ p(x i ) = 1

i=1

Any x i should not be ≥ to 0.

The graph from the probablity mass function can take many shapes. The graph
can be skewed positively or negatively. It can be symmetric or asymmetric. It
can be unimodal or multimodal. It can be discrete or continuous. It can also be
uniform or non-uniform.

Cumulative Distribution Function

The cumulative distribution function F (x) is defined as:

F (x) = P (X ≤ x)
 The cumulative distribution function F (x) is a non-decreasing function of x.

Expected Value of a Random Variable

Let X be a Discreet Random variable taking values x 1, x2 , . . . , xn with
probabilities p , p , . . . , p respectively.
1 2 n

The Expectation of X is denoted by E(X) and is defined as:

n

E(X) = ∑ x i ⋅ p i

i=1

Suppose we want to find out the expectation of a random variable X that takes
on values x , x , . . . , x with probabilities p , p , . . . , p respectively.
1 2 n 1 2 n

We can find the expectation by multiplying each value of X by its

probability and then adding the results.
The expectation of a random variable is the weighted average of the
possible values of the random variable.

E(g(X)) = ∑ g(x i )P (X = x i )

If the function g(x) is multiplied by a constant c, then the expectation of the

function is also multiplied by the same constant.

E(cg(X)) = cE(g(X))

If we add a constant c to the random variable X, then the expectation of the

random variable is also increased by the same constant.

E(X + c) = E(X) + c

The expected value of the sum of random variables is equal to the sum of the
individual expected values.

E(X + Y ) = E(X) + E(Y )

Example
X −1 0 1
X −1 0 1

P (X) 0.2 0.5 0.3

Let Y = g(X) = X
2
. What is E(Y )
n
E(Y ) = ∑ yi ⋅ pi
i=1

E(Y ) = [1 ⋅ 0.2] + [0 ⋅ 0.5] + [1 ⋅ 0.3]

E(Y ) = 0.2 + 0.3 = 0.5

X −2 2 5

P (X) 0.25 0.3334 0.41667

Calculate the expected value of (2X + 1) . 2

n
E(Y ) = ∑ yi ⋅ pi
i=1

E(Y ) = [(2 ⋅ −2 + 1) ⋅ 0.25] + [(2 ⋅ 2 + 1) ⋅ 0.3334] + [(2 ⋅ 5 + 1) ⋅ 0.41667]

$E(Y) = 0.75 + 1.6667 + 4.5833

Variance of a Random Variable

Let's denote expected value of a random variable X by the greek letter μ.
Let X be a random variable with the expected value μ. Then the variance of X
denoted by V ar(X) is defined as:
2
V ar(X) = E(X − μ)

or

2 2
V ar(X) = E(X ) − μ

2
E(X) = √ V ar(X) + μ

In other words, the variance of random variable X measures the square of the
difference of the random variable from its mean, μ, on the average.

Example

Random Experiment: Roll a die once.

 Sample Space: S = {1, 2, 3, 4, 5, 6}

Random Variable: X = number of spots on the die |X | 1 | 2 | 3 | 4 | 5 | 6 | | --- |

--- | --- | --- | --- | --- | --- | |X | 1 | 4 | 9 | 16 | 25 | 36 | |P (X) | 1/6 | 1/6 | 1/6 | 1/6
2

| 1/6 | 1/6 | |E(X) | 1/6 | 2/6 | 3/6 | 4/6 | 5/6 | 6/6 | |E(X ) | 1/6 | 4/6 | 9/6 | 2

16/6 | 25/6 | 36/6 |

1 2 3 4 5 6 21
E(X) = + + + + + = = 3.5
6 6 6 6 6 6 6

2 1 4 9 16 25 36 91
E(X ) = + + + + + = = 15.17
6 6 6 6 6 6 6

2 2 2
V ar(X) = E(X ) − μ = 15.17 − 3.5 = 15.17 − 12.25 = 2.92

Bernoulli Random Variable

A random variable that takes on either the value 1 or 0 is called a Bernoulli
Random Variable.
Let X be a Bernoulli random variable with p being the probability of success.
Then the probability mass function of X is:

p if x = 1
p(x) = {
1 − p if x = 0

V ar(X) = p(1 − p)

Discrete Uniform random variable

Let X be a random variable that is equally likely to occur on any of the n

possible values x , x , . . . , x . Then X is said to be a Discrete Uniform Random
1 2 n

Variable.
2
(n − 1)
V ar(X) =
12

Variance of a Random Variable when mulplied by a constant

2
V ar(cX) = c V ar(X)

Variance of a Random Variable when added to a constant

V ar(X + c) = V ar(X)
Collorary
2
V ar(aX + b) = a V ar(X)

Variance of the sum of two random variables

This is applicable only when the two random variables are independent.

E(X + Y ) = E(X) + E(Y )

Variance of sum of many independent random variables

V (X 1 + X 2 . . . X k ) = V (X 1 ) + V (X 2 )+. . . +V (X k )

or

k k

V (∑ X i ) = ∑ V (X i )

i=1 i=1

Standard Deviation of a Random Variable

SD(X) = √ V ar(X)

Hence, the standard deviation is a positive square root of variance.

Standard deviation is in the same units as the random variable.

Standard Deviation adding or being multiplied by a constant

Multplied by a constant
2
SD(cX) = √ c V ar(X) = c√ V ar(X)

or

SD(cX) = cSD(X)

Added to a constant

The standard deviation of a random variable is not affected by adding a

constant to the random variable.

SD(X + c) = SD(X)
Collorary
SD(aX + b) = aSD(X)

2
SD(aX + b) = √ a V ar(X) = a√ V ar(X)

Bernoulli Distribution

X is a binomial random variable with praameters n and p that represents the

number of successes in n independent Bernoulli Trails, when each trial is a
succees witha probablity p. X takes the values 0, 1, 2, . . . , n with the probablity.
n i n−i
P (X = i) = Ci ⋅ p ⋅ (1 − p)

We can predict the skewness of the distribution graph of X by the value of p.

If p < 0.5 , then the distribution is skewed to the right.

If p = 0.5 , then the distribution is symmetric.
If p > 0.5 , then the distribution is skewed to the left.

The Probablity Distribution of Bernoulli Random Variable is

X 0 1

P (X) 1 − p p

E(X) = 0 ⋅ (1 − p) + 1 ⋅ p = p

Example : Tossing a coin

S = {H H H , H H T , H T H , H T T , T H H , T H T , T T H , T T T }

Success: H and Failure: T

X is a random variable which counts the number of heads in toe tosses of a
coin. n = 3, p = 0.5 | X | 0 | 1 | 2 | 3 | | --- | --- | --- | --- | --- | | P (X) | 0.125 | 0.375 |
0.375 | 0.125 |
 3 3 3−3
P (X = 3) = C 3 ⋅ 0.5 ⋅ (0.5) = 0.125

Variance of a Bernoulli Random Variable

2
V (X) = p(1 − p) = p − p

The largest variance occurs when p = 0.5. In happens when the success and
failure are equally likely. In other words the most uncertain outcome is when
the probability of success and failure are equal.

Independent and Identically Distributed Random Variables

Two random variables are said to be Independent and Identically Distributed if

they are independent and have the same probability distribution.
A collection of random variables is said to be Independent and Identically
Distributed if each random variable in the collection is independent and has
the same probability distribution.

Expectation of Binomial Random Variable

E(X) = np

Variance of Binomial Random Variable

V (X) = np(1 − p)

Standard Deviation of Binomial Random Variable

SD(X) = √ np(1 − p)

Hypergeometric Distribution
Let X be the number of items of type 1, then the probablity mass function of
the discrete random variable, X, is called the hypergeometric distribution and
is of the form:
m N −m
Ci ⋅ C n−i
P (X = i) =
N
Cn
Example: Choosing balls without replacement

A bag consists of 7 balls of which 4 are white and 3 are black. A student
randomly samples two balls without replacement. Let X be the number of
black balls selected.
Here, N = 7, m = 3, n = 2

X has the values 0, 1, 2

the probability mass function of X is:
3 7−3
C0 ⋅ C 2−0 12
P (X = 0) = =
7
C2 42

3 7−3
C1 ⋅ C 2−1 24
P (X = 1) = =
7
C2 42

3 7−3
C2 ⋅ C 2−2 6
P (X = 2) = =
7
C2 42

Expectation of Hypergeometric Random Variable

n ⋅ m
E(X) =
N

Variance of Hypergeometric Random Variable

m N − m
V (X) = n ⋅ ⋅
N N

The variance of the hypergeometric distribution is symmetric when both m

and n are equal to N
or
2
=
m

N
. 1

Poisson Distribution
The Poisson probablity distribution gives the probablity of a number of ecents
occurring in a fixed interval of time or space.
We assume that these events happen with a known average rate, λ, and
independently of the time since the last event.
Let X be the number of events in a given interval. Then the probability mass
function of the discrete random variable, X, is called the Poisson distribution
 and is of the form:
−λ x
e ⋅ λ
P (X = i) =
x!

Things to note about the graphs of Poisson Distribution

If the value of λ is very small, the the graph is skewed to the right.
As the value of λ increases, the graph becomes more symmetric.

Poisson as Binomial Approximation

Define "success" as exactly one event happening in a short interval of length δt

.
The n events happening in interval of length t can be viewed as n successes
happening in n intervals of length δt, with each one of them being an
Independent and Identical trails.
Hence the problem can be viewed as

λ λ n − λ
n x n−x
Bin(n, p = ) = C1 ⋅ ( ) ⋅ ( )
n n n

Expectation of Poisson Random Variable

The expectation of a Poisson random variable is equal to the value of λ itself.

E(X) = λ

Variance of Poisson Random Variable

The variance of a Poisson random variable is equal to the value of λ.

V (X) = λ

Probablity Density Function

Every Continuous Random Variable has a Probablity Density Function.
 That probablity distribution cruve of a continuous random variable is also
called the Probablity Density Function. It is denoted by f (x).

Area under the Probablity Density Function

Consider any two points a and b where a < b.

The probablility that X assumes a value that lies between a and b is given by
the area under the probablity density function between a and b.
b

P (a < X < b) = ∫ f (x) dx

a

Properties of the Probablity Density Function

The area under the probablity distribution curve of a continuous random

variable between any two points a and b is always between 0 and 1.
The area under the whole probablity distribution curve is always equal to 1.
∞

∫ f (x) dx = 1
−∞

1 ≥ ∫ f (x) dx ≥ 0
a

Expectation of a Continuous Random Variable

∞
E(X) = ∫ _−∞ xf (x) dx

Variance of a Continuous Random Variable

∞
2
V (X) = ∫ (x − E(X)) f (x) dx
−∞

Cumulative Distribution Function

For a continuous random variable, the Cumulative Distribution Function is the
integral of the Probablity Density Function.
 x

F (x) = ∫ f (x) dx
−∞

Uniform Distribution
A A random variable has the standard uniform distribution with minimum 0
and maximum 1 if its probability density function is given by

1 if 0 ≤ x ≤ 1
{
0 otherwise

The standard uniform distribution plays an important role in random variate

generation.
f (x) ≤ 0, f orx ∈ R

∞ 1
∫ f (x) dx = ∫ f (x)dx = 1
−∞ 0

Expectation of Uniform Random Variable

a + b
E(X) =
2

Variance of Uniform Random Variable

2
(b − a)
V (X) =
12

Expoential Distribution
A continuous random variable whose probablity density function is given, for
some λ > 0, by

−λx
λe x ≥ 0
f (x) = {
0 otherwise

These are called exponential random varaible or, called exponentially

distributed random variables with parameter λ.

Expectation of Exponential Random Variable

 1
E(X) =
λ

Variance of Exponential Random Variable

1
V (X) =
2
λ

Standart Deviation of Exponential Random Variable

1
σ = √ V (X) =
λ

Mean of Exponential Random Variable

1
μ = E(X) =
λ

Contributions:
Week 5-12 by Kabir Maniar

Week 1-4 Contributions by Parampreet Singh