Chapter 1: Probability concepts

Part III - Common distributions

Simon Fraser University

ECON 333
Summer 2023

Chapter 1 Probabilities: Common distributions

Disclaimer

I do not allow this content to be published without my consent.

All rights reserved ©2023 Thomas Vigié

Chapter 1 Probabilities: Common distributions

Outline

Discrete distributions
Uniform distribution
Bernoulli distribution
Continuous distributions
Normal distribution
Chi square distribution
Student distribution
F distribution
Suggested reading: Chapters 2.4 in Stock and Watson

Chapter 1 Probabilities: Common distributions

Discrete distributions

Chapter 1 Probabilities: Common distributions

The uniform distribution
Some variables follow common distributions functions
Distribution functions are characterized by some parameters. Once we know
these parameters, we can compute any probability for that r.v. Such
parameters can be the expectation, variance, or other measures depending on
the distributions
Definition: Uniform distribution
The uniform distribution, denoted U {x1 , x2 , ..., xN } is a distribution that
assigns the same probability to each outcome. It is characterized by the values
the r.v. takes. If the r.v. takes N values, then P(X = xi ) = 1/N for any
i = 1, ..., N . we formally note X ∼ U {x1 , x2 , ..., xN }.

Example: The number obtained after throwing a 6-faced die is a r.v. that
follows a U {1, 2, 3, 4, 5, 6} distribution and the probability of getting any of
these numbers is 1/6
Chapter 1 Probabilities: Common distributions
The Bernoulli distribution
Definition: Bernoulli distribution
The Bernoulli distribution is the distribution of a discrete random variable which
takes the value 1 with probability p and the value 0 with probability 1 − p.
If X follows a Bernoulli distribution with parameter p, then E[X] = p and
V[X] = p(1 − p).

Random variables formed after a qualitative variable which can take 2 values
(Man/Woman, Heads/Tails, Yes/No) follow a Bernoulli distribution
Example: A coin toss. Say Heads = 1, Tails = 0. If the coin is fair, then
p = 0.5. Then E[X] = 0 × 0.5 + 1 × 0.5 = 0.5 and V[X] = 0.25 (try it!)
What if the coin is bent and p = 0.3?
If we repeat the experiment n times, the number of successes is a r.v. that
follows a Binomial distribution (for instance, getting Heads k times out of n
coin tosses)

Chapter 1 Probabilities: Common distributions

Continuous distributions

Chapter 1 Probabilities: Common distributions

Common distributions: Continuous random variables

As for discrete r.v., some continuous r.v. follow some recognizable distributions
A distribution in this case is described by a probability density function
and a cumulative distribution function
There are many of them:
Uniform distribution (same as before, but there is a continuum of values the r.v.
can take)
Exponential distribution (describes the probability that something happens after
some time)
Gamma distribution (general distribution that encompasses many known ones)
In this lecture:
The normal distribution (describes many r.v., by far the most important one)
Chi-square distribution
The Student distribution
The F distribution

Chapter 1 Probabilities: Common distributions

The normal distribution

Also called Gaussian distribution after Carl Friedrich Gauss (genius

mathematician)
Invented by Gauss and Laplace (French) almost simultaneously, but
independently
Important parameters: The mean µ and the variance σ 2 . They are enough to
characterize the full probability distribution so it is denoted X ∼ N (µ, σ 2 ).
2
Equation of the density function: φ(x) = √ 1 2 exp ( − (x−µ) 2σ 2 )
2πσ
No need to remember the density function expression

Chapter 1 Probabilities: Common distributions

The normal probability density function

φ(x)
P(X ≤ x0 ) = Φ(x0 )

φ(x0 )

φ(x)

x0 x

Figure 1: Standard normal probability density function

Chapter 1 Probabilities: Common distributions

The normal pdf: Different means, same standard deviation

N (0, 1)
N (1, 1)

Figure 2: Standard normal probability density functions

Chapter 1 Probabilities: Common distributions

The normal pdf: Same means, different standard deviation

N (0, 2)
N (0, 1)

Figure 3: Standard normal probability density functions

Chapter 1 Probabilities: Common distributions

The normal pdf: Different means and standard deviations

N (0, 1)
N (1, 2)

Figure 4: Standard normal probability density functions

Chapter 1 Probabilities: Common distributions

Normal distribution properties

Properties of the normal distribution

Symmetric and centered around the mean, so the mean, median (value m
such that P(X < m) = P(X > m) = 50%) and mode (value that appears
the most often. It corresponds to the maximum of the pmf and pdf for discrete
and continuous r.v. respectively) are the same
The sum of normal r.v. is itself a normal r.v., i.e. if X and Y are normal r.v.,
then Z = aX + bY (with a and b being two constants) is also a normal r.v.
(careful when computing the expectation and variance though! See their
properties)
Its density is the “Bell curve”: Higher probability of being around the mean
than around the extremes

Chapter 1 Probabilities: Common distributions

Computing the probabilities of a normal distribution

Once we have the pdf φ(X), we need to integrate it from −∞ to the value x
we are interested in
The cumulative distribution function is
Rx 1 (x−µ)2
Φ(x) = −∞ √ exp ( − 2σ2 )dx
2πσ 2
Pretty fancy. . . But there is an easier way!
Values have been recorded for the N (0, 1) distribution (mean of 0, standard
deviation of 1)
So we need to go from a general N (µ, σ 2 ) to N (0, 1)

Chapter 1 Probabilities: Common distributions

Normal distribution: Standardizing

Let X ∼ N (µ, σ 2 ) (µ and σ 2 are given to you)

Remove the mean. Then X − µ has an average of 0
(E[X − µ] = E[X] − µ = µ − µ = 0)
Divide X − µ by σ to get Z = X−µ σ
Z has a variance of 1 now since
V[Z] = V[ X−µ
σ
] = σ12 V[X − µ] = σ12 V[X] = σ12 × σ 2 = 1

Chapter 1 Probabilities: Common distributions

Normal distribution: Standardizing

If we want to compute P(X ≤ x), we note that

X −µ x−µ
P(X ≤ x) = P( ≤ )
σ σ

So we have P(X ≤ x) = P(Z ≤ x−µ σ

)
x−µ
Compute σ , and look for it in the Z table (i.e. the X ∼ N (0, 1) table)

Chapter 1 Probabilities: Common distributions

Normal distribution: Example

Let X be the weight of a person. Assume X ∼ N (155, 121)

What is P(X ≤ 155)? 50% since 155 is the average!
But what is P(X ≤ 170)?
P(X ≤ 170) = P(Z ≤ 170−155
√
121
) = P(Z ≤ 1.36)
Locate the number 1.36 in the Z table: 0.9131
So there is a 91.31% chance that the next person we meet weighs less or equal
to 170 lbs

Chapter 1 Probabilities: Common distributions

Normal distribution: Example (cont’d)
Standard normal distribution: Cumulative distribution function

φ(x)
P(X ≤ x0 ) = Φ(x0 )

φ(x)
x0 x

0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

0.0 0.5000 0.5040 0.5080 0.5120 0.5160 0.5199 0.5239 0.5279 0.5319 0.5359

0.1 0.5398 0.5438 0.5478 0.5517 0.5557 0.5596 0.5636 0.5675 0.5714 0.5753

0.2 0.5793 0.5832 0.5871 0.5910 0.5948 0.5987 0.6026 0.6064 0.6103 0.6141

0.3 0.6179 0.6217 0.6255 0.6293 0.6331 0.6368 0.6406 0.6443 0.6480 0.6517

0.4 0.6554 0.6591 0.6628 0.6664 0.6700 0.6736 0.6772 0.6808 0.6844 0.6879

0.5 0.6915 0.6950 0.6985 0.7019 0.7054 0.7088 0.7123 0.7157 0.7190 0.7224

0.6 0.7257 0.7291 0.7324 0.7357 0.7389 0.7422 0.7454 0.7486 0.7517 0.7549

0.7 0.7580 0.7611 0.7642 0.7673 0.7704 0.7734 0.7764 0.7794 0.7823 0.7852

0.8 0.7881 0.7910 0.7939 0.7967 0.7995 0.8023 0.8051 0.8078 0.8106 0.8133

0.9 0.8159 0.8186 0.8212 0.8238 0.8264 0.8289 0.8315 0.8340 0.8365 0.8389

1.0 0.8413 0.8438 0.8461 0.8485 0.8508 0.8531 0.8554 0.8577 0.8599 0.8621

1.1 0.8643 0.8665 0.8686 0.8708 0.8729 0.8749 0.8770 0.8790 0.8810 0.8830

1.2 0.8849 0.8869 0.8888 0.8907 0.8925 0.8944 0.8962 0.8980 0.8997 0.9015

1.3 0.9032 0.9049 0.9066 0.9082 0.9099 0.9115 0.9131 0.9147 0.9162 0.9177

1.4 0.9192 0.9207 0.9222 0.9236 0.9251 0.9265 0.9279 0.9292 0.9306 0.9319

1.5 0.9332 0.9345 0.9357 0.9370 0.9382 0.9394 0.9406 0.9418 0.9429 0.9441

1.6 0.9452 0.9463 0.9474 0.9484 0.9495 0.9505 0.9515 0.9525 0.9535 0.9545

1.7 0.9554 0.9564 0.9573 0.9582 0.9591 0.9599 0.9608 0.9616 0.9625 0.9633

1.8 0.9641 0.9649 0.9656 0.9664 0.9671 0.9678 0.9686 0.9693 0.9699 0.9706

1.9 0.9713 0.9719 0.9726 0.9732 0.9738 0.9744 0.9750 0.9756 0.9761 0.9767

2.0 0.9772 0.9778 0.9783 0.9788 0.9793 0.9798 0.9803 0.9808 0.9812 0.9817

2.1 0.9821 0.9826 0.9830 0.9834 0.9838 0.9842 0.9846 0.9850 0.9854 0.9857

2.2 0.9861 0.9864 0.9868 0.9871 0.9875 0.9878 0.9881 0.9884 0.9887 0.9890

2.3 0.9893 0.9896 0.9898 0.9901 0.9904 0.9906 0.9909 0.9911 0.9913 0.9916

2.4 0.9918 0.9920 0.9922 0.9925 0.9927 0.9929 0.9931 0.9932

0.9936 0.9934
Chapter 1 Probabilities: Common distributions
Normal distribution: Example (cont’d)

What about P(X ≤ 145)?

P(X ≤ 145) = P(Z ≤ 145−155
√
121
) = P(Z ≤ −0.91)
That one is not on the table. . .
But we can use the symmetry of the normal distribution!
P(Z ≤ −0.91) = P(Z ≥ 0.91) = 1 − P(Z ≤ 0.91)
since P(Z ≤ 0.91) = 0.8186, P(Z ≤ −0.91) = 1 − 0.8186 = 0.1914

Chapter 1 Probabilities: Common distributions

The normal probability density function

φ(x)
P(X ≤ x0 ) = Φ(x0 )

φ(x0 ) φ(x0 )

φ(x) φ(x)

x0 x −x0 x

Figure 5: Symmetry of the normal distribution

Chapter 1 Probabilities: Common distributions

The Chi-squared distribution

The Chi(“kai”)-squared distribution, denoted χ2ν , is the distribution of a sum of

the squares of ν (“nu”) independent standard normal random variables
ν is the only parameter we need to know, and is called the degrees of
freedom (or df)
Take X ∼ N (0, 1) and Y ∼ N (0, 1)
Then Z = X 2 + Y 2 ∼ χ2ν , and ν = 2. So Z ∼ χ22 as we add two standard
normal r.v. that are squared
It is not typically seen in real life random events, but rather it is used in
statistical computations involving squared normal r.v. More on this later

Chapter 1 Probabilities: Common distributions

The Chi-squared distribution (cont’d)

This distribution also has a probability table, but instead of representing the
probabilities in the table, we represent the quantiles (also called
percentiles), i.e. the x-axis values (e.g. for the normal distribution, the
quantile associated to a probability of 50%, denoted q50 , is 0)
The reason is that in inference, we start with the probability, and look for the
corresponding value on the x-axis, i.e. the corresponding quantile (it is the
reverse reasoning as when we look for a probability)
Tu use it: Locate the probability you are looking for, locate the degrees of
freedom and look at the corresponding quantile
Example: Find x such that P(X < x) = 95% (or P(X > x) = 5%) if
X ∼ χ26

Chapter 1 Probabilities: Common distributions

The Chi-squared distribution quantiles table
χ2 distribution: Quantiles

x0 x

ν/α 0.995 0.990 0.975 0.950 0.900 0.100 0.050 0.025 0.010 0.005

1 0.000 0.000 0.001 0.004 0.016 2.706 3.841 5.024 6.635 7.879

2 0.010 0.020 0.051 0.103 0.211 4.605 5.991 7.378 9.210 10.597

3 0.072 0.115 0.216 0.352 0.584 6.251 7.815 9.348 11.345 12.838

4 0.207 0.297 0.484 0.711 1.064 7.779 9.488 11.143 13.277 14.860

5 0.412 0.554 0.831 1.145 1.610 9.236 11.070 12.833 15.086 16.750

6 0.676 0.872 1.237 1.635 2.204 10.645 12.592 14.449 16.812 18.548

7 0.989 1.239 1.690 2.167 2.833 12.017 14.067 16.013 18.475 20.278

8 1.344 1.646 2.180 2.733 3.490 13.362 15.507 17.535 20.090 21.955

9 1.735 2.088 2.700 3.325 4.168 14.684 16.919 19.023 21.666 23.589

10 2.156 2.558 3.247 3.940 4.865 15.987 18.307 20.483 23.209 25.188

11 2.603 3.053 3.816 4.575 5.578 17.275 19.675 21.920 24.725 26.757

12 3.074 3.571 4.404 5.226 6.304 18.549 21.026 23.337 26.217 28.300

13 3.565 4.107 5.009 5.892 7.042 19.812 22.362 24.736 27.688 29.819

14 4.075 4.660 5.629 6.571 7.790 21.064 23.685 26.119 29.141 31.319

15 4.601 5.229 6.262 7.261 8.547 22.307 24.996 27.488 30.578 32.801

16 5.142 5.812 6.908 7.962 9.312 23.542 26.296 28.845 32.000 34.267

17 5.697 6.408 7.564 8.672 10.085 24.769 27.587 30.191 33.409 35.718

18 6.265 7.015 8.231 9.390 10.865 25.989 28.869 31.526 34.805 37.156

19 6.844 7.633 8.907 10.117 11.651 27.204 30.144 32.852 36.191 38.582

20 7.434 8.260 9.591 10.851 12.443 28.412 31.410 34.170 37.566 39.997

21 8.034 8.897 10.283 11.591 13.240 29.615 32.671 35.479 38.932 41.401

22 8.643 9.542 10.982 12.338 14.041 30.813 33.924 36.781 40.289 42.796

23 9.260 10.196 11.689 13.091 14.848 32.007 35.172 38.076 41.638 44.181

24 9.886 10.856 12.401 13.848 15.659 33.196 36.415 39.364 42.980 45.559

25 10.520 11.524 13.120

14.611
Chapter 116.473 Probabilities:
34.382 37.652 40.646 44.314
Common 46.928
distributions
The Student distribution

Invented by an engineer (William Sealy Gosset) working for Guinness (the beer)
who named the distribution Student after his pen name
The Student distribution is similar to the Normal distribution in that it is a
bell curve as well
If the standard deviation in a normally distributed r.v. is unknown however, we
need to estimate it (more on this in the next chapter)
Because of that, the resulting distribution is not exactly normal, as we don’t
use the true standard deviation
Like the χ2 distribution, it has degrees of freedom corresponding to the
sample size used to estimate the standard deviation minus one, so n − 1 df if
the sample has n observations
Like the χ2 distribution, the Student distribution tables show percentiles, not
probabilities

Chapter 1 Probabilities: Common distributions

The Student distribution quantiles table
Student distribution: Quantiles

qα x

ν/α 0.995 0.990 0.975 0.950 0.900 0.100 0.050 0.025 0.010 0.005
1 -63.657 -31.821 -12.706 -6.314 -3.078 3.078 6.314 12.706 31.821 63.657
2 -9.925 -6.965 -4.303 -2.920 -1.886 1.886 2.920 4.303 6.965 9.925
3 -5.841 -4.541 -3.182 -2.353 -1.638 1.638 2.353 3.182 4.541 5.841
4 -4.604 -3.747 -2.776 -2.132 -1.533 1.533 2.132 2.776 3.747 4.604
5 -4.032 -3.365 -2.571 -2.015 -1.476 1.476 2.015 2.571 3.365 4.032
6 -3.707 -3.143 -2.447 -1.943 -1.440 1.440 1.943 2.447 3.143 3.707
7 -3.499 -2.998 -2.365 -1.895 -1.415 1.415 1.895 2.365 2.998 3.499
8 -3.355 -2.896 -2.306 -1.860 -1.397 1.397 1.860 2.306 2.896 3.355
9 -3.250 -2.821 -2.262 -1.833 -1.383 1.383 1.833 2.262 2.821 3.250
10 -3.169 -2.764 -2.228 -1.812 -1.372 1.372 1.812 2.228 2.764 3.169
11 -3.106 -2.718 -2.201 -1.796 -1.363 1.363 1.796 2.201 2.718 3.106
12 -3.055 -2.681 -2.179 -1.782 -1.356 1.356 1.782 2.179 2.681 3.055
13 -3.012 -2.650 -2.160 -1.771 -1.350 1.350 1.771 2.160 2.650 3.012
14 -2.977 -2.624 -2.145 -1.761 -1.345 1.345 1.761 2.145 2.624 2.977
15 -2.947 -2.602 -2.131 -1.753 -1.341 1.341 1.753 2.131 2.602 2.947
16 -2.921 -2.583 -2.120 -1.746 -1.337 1.337 1.746 2.120 2.583 2.921
17 -2.898 -2.567 -2.110 -1.740 -1.333 1.333 1.740 2.110 2.567 2.898
18 -2.878 -2.552 -2.101 -1.734 -1.330 1.330 1.734 2.101 2.552 2.878
19 -2.861 -2.539 -2.093 -1.729 -1.328 1.328 1.729 2.093 2.539 2.861
20 -2.845 -2.528 -2.086 -1.725 -1.325 1.325 1.725 2.086 2.528 2.845
21 -2.831 -2.518 -2.080 -1.721 -1.323 1.323 1.721 2.080 2.518 2.831
22 -2.819 -2.508 -2.074 -1.717 -1.321 1.321 1.717 2.074 2.508 2.819
23 -2.807 -2.500 -2.069 -1.714 -1.319 1.319 1.714 2.069 2.500 2.807
24 -2.797 -2.492 -2.064 -1.711 -1.318 1.318 1.711 2.064 2.492 2.797
25 -2.787 -2.485 -2.060
Chapter 1 -1.708 -1.316
Probabilities: Common2.787
1.316 1.708 2.060 2.485
distributions
The F distribution

The F distribution (also called Fisher-Snedecor after Ronald Fisher and George
W. Snedecor) is very important for inference (see F test statistics later)
It is similar to the χ2 distribution, but has two different degrees of freedom
It is denoted F ν1 ,ν2
If X1 ∼ χ2ν1 and X2 ∼ χ2ν2 , then

X1 /ν1
∼ F ν1 ,ν2
X2 /ν2

Because it has two degrees of freedom, a table will report the

quantiles/percentiles for one probability only (one df in rows, one df in columns)

Chapter 1 Probabilities: Common distributions

The F distribution quantiles table
F distribution: Quantiles for α = 0.05

α = 5%

qα x

ν2 /ν1 1 2 3 4 5 6 7 8 9 10

1 161.448 199.500 215.707 224.583 230.162 233.986 236.768 238.883 240.543 241.882

2 18.513 19.000 19.164 19.247 19.296 19.330 19.353 19.371 19.385 19.396

3 10.128 9.552 9.277 9.117 9.013 8.941 8.887 8.845 8.812 8.786

4 7.709 6.944 6.591 6.388 6.256 6.163 6.094 6.041 5.999 5.964

5 6.608 5.786 5.409 5.192 5.050 4.950 4.876 4.818 4.772 4.735

6 5.987 5.143 4.757 4.534 4.387 4.284 4.207 4.147 4.099 4.060

7 5.591 4.737 4.347 4.120 3.972 3.866 3.787 3.726 3.677 3.637

8 5.318 4.459 4.066 3.838 3.687 3.581 3.500 3.438 3.388 3.347

9 5.117 4.256 3.863 3.633 3.482 3.374 3.293 3.230 3.179 3.137

10 4.965 4.103 3.708 3.478 3.326 3.217 3.135 3.072 3.020 2.978

11 4.844 3.982 3.587 3.357 3.204 3.095 3.012 2.948 2.896 2.854

12 4.747 3.885 3.490 3.259 3.106 2.996 2.913 2.849 2.796 2.753

13 4.667 3.806 3.411 3.179 3.025 2.915 2.832 2.767 2.714 2.671

14 4.600 3.739 3.344 3.112 2.958 2.848 2.764 2.699 2.646 2.602

15 4.543 3.682 3.287 3.056 2.901 2.790 2.707 2.641 2.588 2.544

16 4.494 3.634 3.239 3.007 2.852 2.741 2.657 2.591 2.538 2.494

17 4.451 3.592 3.197 2.965 2.810 2.699 2.614 2.548 2.494 2.450

18 4.414 3.555 3.160 2.928 2.773 2.661 2.577 2.510 2.456 2.412

19 4.381 3.522 3.127 2.895 2.740 2.628 2.544 2.477 2.423 2.378

20 4.351 3.493 3.098 2.866 2.711 2.599 2.514 2.447 2.393 2.348

21 4.325 3.467 3.072 2.840 2.685 2.573 2.488 2.420 2.366 2.321

22 4.301 3.443 3.049 2.817 2.661 2.549 2.464 2.397 2.342 2.297

23 4.279 3.422 3.028 2.796 2.640 2.528 2.442 2.375 2.320 2.275

24 4.260 3.403 3.009 2.776 2.621 2.508 2.423 2.355 2.300 2.255

25 4.242 3.385
Chapter 1
2.991 2.759
Probabilities: Common distributions
2.603 2.490 2.405 2.337
2.236 2.282
Summary

Some random variables follow well known distributions

Most don’t! They might look similar to some known distributions, but are not
as simple in terms of formulas
Each distribution is characterized by some parameters (know which for each
distribution covered here)
For the continuous distributions used in this course, there are tables listing
probabilities
In exams, you will have the tables and will have to use them
Make sure you distinguish quantiles (or percentiles) and probabilities
Either could be asked, either could be given to compute the other!

Chapter 1 Probabilities: Common distributions