18.

600: Lecture 21
Joint distributions functions

Scott Sheffield

MIT
Outline

Distributions of functions of random variables

Joint distributions

Independent random variables

Examples
Outline

Distributions of functions of random variables

Joint distributions

Independent random variables

Examples
Distribution of function of random variable

I Suppose P{X a} = FX (a) is known for all a. Write

Y = X 3 . What is P{Y 27}?
Distribution of function of random variable

I Suppose P{X a} = FX (a) is known for all a. Write

Y = X 3 . What is P{Y 27}?
I Answer: note that Y 27 if and only if X 3. Hence
P{Y 27} = P{X 3} = FX (3).
Distribution of function of random variable

I Suppose P{X a} = FX (a) is known for all a. Write

Y = X 3 . What is P{Y 27}?
I Answer: note that Y 27 if and only if X 3. Hence
P{Y 27} = P{X 3} = FX (3).
I Generally FY (a) = P{Y a} = P{X a1/3 } = FX (a1/3 )
Distribution of function of random variable

I Suppose P{X a} = FX (a) is known for all a. Write

Y = X 3 . What is P{Y 27}?
I Answer: note that Y 27 if and only if X 3. Hence
P{Y 27} = P{X 3} = FX (3).
I Generally FY (a) = P{Y a} = P{X a1/3 } = FX (a1/3 )
I This is a general principle. If X is a continuous random
variable and g is a strictly increasing function of x and
Y = g (X ), then FY (a) = FX (g 1 (a)).
Distribution of function of random variable

I Suppose P{X a} = FX (a) is known for all a. Write

Y = X 3 . What is P{Y 27}?
I Answer: note that Y 27 if and only if X 3. Hence
P{Y 27} = P{X 3} = FX (3).
I Generally FY (a) = P{Y a} = P{X a1/3 } = FX (a1/3 )
I This is a general principle. If X is a continuous random
variable and g is a strictly increasing function of x and
Y = g (X ), then FY (a) = FX (g 1 (a)).
I How can we use this to compute the probability density
function fY from fX ?
Distribution of function of random variable

I Suppose P{X a} = FX (a) is known for all a. Write

Y = X 3 . What is P{Y 27}?
I Answer: note that Y 27 if and only if X 3. Hence
P{Y 27} = P{X 3} = FX (3).
I Generally FY (a) = P{Y a} = P{X a1/3 } = FX (a1/3 )
I This is a general principle. If X is a continuous random
variable and g is a strictly increasing function of x and
Y = g (X ), then FY (a) = FX (g 1 (a)).
I How can we use this to compute the probability density
function fY from fX ?
I If Z = X 2 , then what is P{Z 16}?
Outline

Distributions of functions of random variables

Joint distributions

Independent random variables

Examples
Outline

Distributions of functions of random variables

Joint distributions

Independent random variables

Examples
Joint probability mass functions: discrete random variables

I If X and Y assume values in {1, 2, . . . , n} then we can view

Ai,j = P{X = i, Y = j} as the entries of an n n matrix.
Joint probability mass functions: discrete random variables

I If X and Y assume values in {1, 2, . . . , n} then we can view

Ai,j = P{X = i, Y = j} as the entries of an n n matrix.
I Lets say I dont care about Y . I just want to know
P{X = i}. How do I figure that out from the matrix?
Joint probability mass functions: discrete random variables

I If X and Y assume values in {1, 2, . . . , n} then we can view

Ai,j = P{X = i, Y = j} as the entries of an n n matrix.
I Lets say I dont care about Y . I just want to know
P{X = i}. How do I figure that out from the matrix?
Answer: P{X = i} = nj=1 Ai,j .
P
I
Joint probability mass functions: discrete random variables

I If X and Y assume values in {1, 2, . . . , n} then we can view

Ai,j = P{X = i, Y = j} as the entries of an n n matrix.
I Lets say I dont care about Y . I just want to know
P{X = i}. How do I figure that out from the matrix?
Answer: P{X = i} = nj=1 Ai,j .
P
I

Similarly, P{Y = j} = ni=1 Ai,j .

P
I
Joint probability mass functions: discrete random variables

I If X and Y assume values in {1, 2, . . . , n} then we can view

Ai,j = P{X = i, Y = j} as the entries of an n n matrix.
I Lets say I dont care about Y . I just want to know
P{X = i}. How do I figure that out from the matrix?
Answer: P{X = i} = nj=1 Ai,j .
P
I

Similarly, P{Y = j} = ni=1 Ai,j .

P
I

I In other words, the probability mass functions for X and Y

are the row and columns sums of Ai,j .
Joint probability mass functions: discrete random variables

I If X and Y assume values in {1, 2, . . . , n} then we can view

Ai,j = P{X = i, Y = j} as the entries of an n n matrix.
I Lets say I dont care about Y . I just want to know
P{X = i}. How do I figure that out from the matrix?
Answer: P{X = i} = nj=1 Ai,j .
P
I

Similarly, P{Y = j} = ni=1 Ai,j .

P
I

I In other words, the probability mass functions for X and Y

are the row and columns sums of Ai,j .
I Given the joint distribution of X and Y , we sometimes call
distribution of X (ignoring Y ) and distribution of Y (ignoring
X ) the marginal distributions.
Joint probability mass functions: discrete random variables

I If X and Y assume values in {1, 2, . . . , n} then we can view

Ai,j = P{X = i, Y = j} as the entries of an n n matrix.
I Lets say I dont care about Y . I just want to know
P{X = i}. How do I figure that out from the matrix?
Answer: P{X = i} = nj=1 Ai,j .
P
I

Similarly, P{Y = j} = ni=1 Ai,j .

P
I

I In other words, the probability mass functions for X and Y

are the row and columns sums of Ai,j .
I Given the joint distribution of X and Y , we sometimes call
distribution of X (ignoring Y ) and distribution of Y (ignoring
X ) the marginal distributions.
I In general, when X and Y are jointly defined discrete random
variables, we write p(x, y ) = pX ,Y (x, y ) = P{X = x, Y = y }.
Joint distribution functions: continuous random variables

I Given random variables X and Y , define

F (a, b) = P{X a, Y b}.
Joint distribution functions: continuous random variables

I Given random variables X and Y , define

F (a, b) = P{X a, Y b}.
I The region {(x, y ) : x a, y b} is the lower left quadrant
centered at (a, b).
Joint distribution functions: continuous random variables

I Given random variables X and Y , define

F (a, b) = P{X a, Y b}.
I The region {(x, y ) : x a, y b} is the lower left quadrant
centered at (a, b).
I Refer to FX (a) = P{X a} and FY (b) = P{Y b} as
marginal cumulative distribution functions.
Joint distribution functions: continuous random variables

I Given random variables X and Y , define

F (a, b) = P{X a, Y b}.
I The region {(x, y ) : x a, y b} is the lower left quadrant
centered at (a, b).
I Refer to FX (a) = P{X a} and FY (b) = P{Y b} as
marginal cumulative distribution functions.
I Question: if I tell you the two parameter function F , can you
use it to determine the marginals FX and FY ?
Joint distribution functions: continuous random variables

I Given random variables X and Y , define

F (a, b) = P{X a, Y b}.
I The region {(x, y ) : x a, y b} is the lower left quadrant
centered at (a, b).
I Refer to FX (a) = P{X a} and FY (b) = P{Y b} as
marginal cumulative distribution functions.
I Question: if I tell you the two parameter function F , can you
use it to determine the marginals FX and FY ?
I Answer: Yes. FX (a) = limb F (a, b) and
FY (b) = lima F (a, b).
Joint density functions: continuous random variables

I Suppose we are given the joint distribution function

F (a, b) = P{X a, Y b}.
Joint density functions: continuous random variables

I Suppose we are given the joint distribution function

F (a, b) = P{X a, Y b}.
I Can we use F to construct a two-dimensional probability
density function? RPrecisely, is there a function f such that
P{(X , Y ) A} = A f (x, y )dxdy for each (measurable)
A R2 ?
Joint density functions: continuous random variables

I Suppose we are given the joint distribution function

F (a, b) = P{X a, Y b}.
I Can we use F to construct a two-dimensional probability
density function? RPrecisely, is there a function f such that
P{(X , Y ) A} = A f (x, y )dxdy for each (measurable)
A R2 ?

I Lets try defining f (x, y ) = x y F (x, y ). Does that work?
Joint density functions: continuous random variables

I Suppose we are given the joint distribution function

F (a, b) = P{X a, Y b}.
I Can we use F to construct a two-dimensional probability
density function? RPrecisely, is there a function f such that
P{(X , Y ) A} = A f (x, y )dxdy for each (measurable)
A R2 ?

I Lets try defining f (x, y ) = x y F (x, y ). Does that work?
I Suppose first that A = {(x, y ) : x a, b}. By definition of
F , fundamental theorem of calculus, fact that F (a, b)
vanishes
R b R a as either a or b tends Rto , we indeed find
b
x y F (x, y )dxdy = y F (a, y )dy = F (a, b).
Joint density functions: continuous random variables

I Suppose we are given the joint distribution function

F (a, b) = P{X a, Y b}.
I Can we use F to construct a two-dimensional probability
density function? RPrecisely, is there a function f such that
P{(X , Y ) A} = A f (x, y )dxdy for each (measurable)
A R2 ?

I Lets try defining f (x, y ) = x y F (x, y ). Does that work?
I Suppose first that A = {(x, y ) : x a, b}. By definition of
F , fundamental theorem of calculus, fact that F (a, b)
vanishes
R b R a as either a or b tends Rto , we indeed find
b
x y F (x, y )dxdy = y F (a, y )dy = F (a, b).
I From this, we can show that it works for strips, rectangles,
general open sets, etc.
Outline

Distributions of functions of random variables

Joint distributions

Independent random variables

Examples
Outline

Distributions of functions of random variables

Joint distributions

Independent random variables

Examples
Independent random variables

I We say X and Y are independent if for any two (measurable)

sets A and B of real numbers we have

P{X A, Y B} = P{X A}P{Y B}.

Independent random variables

I We say X and Y are independent if for any two (measurable)

sets A and B of real numbers we have

P{X A, Y B} = P{X A}P{Y B}.

I Intuition: knowing something about X gives me no

information about Y , and vice versa.
Independent random variables

I We say X and Y are independent if for any two (measurable)

sets A and B of real numbers we have

P{X A, Y B} = P{X A}P{Y B}.

I Intuition: knowing something about X gives me no

information about Y , and vice versa.
I When X and Y are discrete random variables, they are
independent if P{X = x, Y = y } = P{X = x}P{Y = y } for
all x and y for which P{X = x} and P{Y = y } are non-zero.
Independent random variables

I We say X and Y are independent if for any two (measurable)

sets A and B of real numbers we have

P{X A, Y B} = P{X A}P{Y B}.

I Intuition: knowing something about X gives me no

information about Y , and vice versa.
I When X and Y are discrete random variables, they are
independent if P{X = x, Y = y } = P{X = x}P{Y = y } for
all x and y for which P{X = x} and P{Y = y } are non-zero.
I What is the analog of this statement when X and Y are
continuous?
Independent random variables

I We say X and Y are independent if for any two (measurable)

sets A and B of real numbers we have

P{X A, Y B} = P{X A}P{Y B}.

I Intuition: knowing something about X gives me no

information about Y , and vice versa.
I When X and Y are discrete random variables, they are
independent if P{X = x, Y = y } = P{X = x}P{Y = y } for
all x and y for which P{X = x} and P{Y = y } are non-zero.
I What is the analog of this statement when X and Y are
continuous?
I When X and Y are continuous, they are independent if
f (x, y ) = fX (x)fY (y ).
Sample problem: independent normal random variables

I Suppose that X and Y are independent normal random

variables with mean zero and variance one.
Sample problem: independent normal random variables

I Suppose that X and Y are independent normal random

variables with mean zero and variance one.
I What is the probability that (X , Y ) lies in the unit circle?
That is, what is P{X 2 + Y 2 1}?
Sample problem: independent normal random variables

I Suppose that X and Y are independent normal random

variables with mean zero and variance one.
I What is the probability that (X , Y ) lies in the unit circle?
That is, what is P{X 2 + Y 2 1}?
I First, any guesses?
Sample problem: independent normal random variables

I Suppose that X and Y are independent normal random

variables with mean zero and variance one.
I What is the probability that (X , Y ) lies in the unit circle?
That is, what is P{X 2 + Y 2 1}?
I First, any guesses?
I Probability X is within one standard deviation of its mean is
about .68. So (.68)2 is an upper bound.
Sample problem: independent normal random variables

I Suppose that X and Y are independent normal random

variables with mean zero and variance one.
I What is the probability that (X , Y ) lies in the unit circle?
That is, what is P{X 2 + Y 2 1}?
I First, any guesses?
I Probability X is within one standard deviation of its mean is
about .68. So (.68)2 is an upper bound.
2 2 1 r 2 /2
f (x, y ) = fX (x)fY (y ) = 1 e x /2 1 e y /2 = 2 e
I
2 2
Sample problem: independent normal random variables

I Suppose that X and Y are independent normal random

variables with mean zero and variance one.
I What is the probability that (X , Y ) lies in the unit circle?
That is, what is P{X 2 + Y 2 1}?
I First, any guesses?
I Probability X is within one standard deviation of its mean is
about .68. So (.68)2 is an upper bound.
2 2 1 r 2 /2
f (x, y ) = fX (x)fY (y ) = 1 e x /2 1 e y /2 = 2 e
I
2 2
I Using
R1 polar coordinates, we want 1
1 r 2 /2 2
0 (2r ) 2 e dr = e r /2 0 = 1 e 1/2 .39.
Outline

Distributions of functions of random variables

Joint distributions

Independent random variables

Examples
Outline

Distributions of functions of random variables

Joint distributions

Independent random variables

Examples
Repeated die roll

I Roll a die repeatedly and let X be such that the first even
number (the first 2, 4, or 6) appears on the X th roll.
Repeated die roll

I Roll a die repeatedly and let X be such that the first even
number (the first 2, 4, or 6) appears on the X th roll.
I Let Y be the the number that appears on the X th roll.
Repeated die roll

I Roll a die repeatedly and let X be such that the first even
number (the first 2, 4, or 6) appears on the X th roll.
I Let Y be the the number that appears on the X th roll.
I Are X and Y independent? What is their joint law?
Repeated die roll

I Roll a die repeatedly and let X be such that the first even
number (the first 2, 4, or 6) appears on the X th roll.
I Let Y be the the number that appears on the X th roll.
I Are X and Y independent? What is their joint law?
I If j 1, then

P{X = j, Y = 2} = P{X = j, Y = 4}

= P{X = j, Y = 6} = (1/2)j1 (1/6) = (1/2)j (1/3).

Repeated die roll

I Roll a die repeatedly and let X be such that the first even
number (the first 2, 4, or 6) appears on the X th roll.
I Let Y be the the number that appears on the X th roll.
I Are X and Y independent? What is their joint law?
I If j 1, then

P{X = j, Y = 2} = P{X = j, Y = 4}

= P{X = j, Y = 6} = (1/2)j1 (1/6) = (1/2)j (1/3).

I Can we get the marginals from that?
Continuous time variant of repeated die roll

I On a certain hiking trail, it is well known that the lion, tiger,

and bear attacks are independent Poisson processes with
respective values of .1/hour, .2/hour, and .3/hour.
Continuous time variant of repeated die roll

I On a certain hiking trail, it is well known that the lion, tiger,

and bear attacks are independent Poisson processes with
respective values of .1/hour, .2/hour, and .3/hour.
I Let T R be the amount of time until the first animal
attacks. Let A {lion, tiger, bear} be the species of the first
attacking animal.
Continuous time variant of repeated die roll

I On a certain hiking trail, it is well known that the lion, tiger,

and bear attacks are independent Poisson processes with
respective values of .1/hour, .2/hour, and .3/hour.
I Let T R be the amount of time until the first animal
attacks. Let A {lion, tiger, bear} be the species of the first
attacking animal.
I What is the probability density function for T ? How about
E [T ]?
Continuous time variant of repeated die roll

I On a certain hiking trail, it is well known that the lion, tiger,

and bear attacks are independent Poisson processes with
respective values of .1/hour, .2/hour, and .3/hour.
I Let T R be the amount of time until the first animal
attacks. Let A {lion, tiger, bear} be the species of the first
attacking animal.
I What is the probability density function for T ? How about
E [T ]?
I Are T and A independent?
Continuous time variant of repeated die roll

I On a certain hiking trail, it is well known that the lion, tiger,

and bear attacks are independent Poisson processes with
respective values of .1/hour, .2/hour, and .3/hour.
I Let T R be the amount of time until the first animal
attacks. Let A {lion, tiger, bear} be the species of the first
attacking animal.
I What is the probability density function for T ? How about
E [T ]?
I Are T and A independent?
I Let T1 be the time until the first attack, T2 the subsequent
time until the second attack, etc., and let A1 , A2 , . . . be the
corresponding species.
Continuous time variant of repeated die roll

I On a certain hiking trail, it is well known that the lion, tiger,

and bear attacks are independent Poisson processes with
respective values of .1/hour, .2/hour, and .3/hour.
I Let T R be the amount of time until the first animal
attacks. Let A {lion, tiger, bear} be the species of the first
attacking animal.
I What is the probability density function for T ? How about
E [T ]?
I Are T and A independent?
I Let T1 be the time until the first attack, T2 the subsequent
time until the second attack, etc., and let A1 , A2 , . . . be the
corresponding species.
I Are all of the Ti and Ai independent of each other? What are
their probability distributions?
More lions, tigers, bears

I Lion, tiger, and bear attacks are independent Poisson

processes with values .1/hour, .2/hour, and .3/hour.
More lions, tigers, bears

I Lion, tiger, and bear attacks are independent Poisson

processes with values .1/hour, .2/hour, and .3/hour.
I Distribution of time Ttiger till first tiger attack?
More lions, tigers, bears

I Lion, tiger, and bear attacks are independent Poisson

processes with values .1/hour, .2/hour, and .3/hour.
I Distribution of time Ttiger till first tiger attack?
I Exponential tiger = .2/hour. So P{Ttiger > a} = e .2a .
More lions, tigers, bears

I Lion, tiger, and bear attacks are independent Poisson

processes with values .1/hour, .2/hour, and .3/hour.
I Distribution of time Ttiger till first tiger attack?
I Exponential tiger = .2/hour. So P{Ttiger > a} = e .2a .
I How about E [Ttiger ] and Var[Ttiger ]?
More lions, tigers, bears

I Lion, tiger, and bear attacks are independent Poisson

processes with values .1/hour, .2/hour, and .3/hour.
I Distribution of time Ttiger till first tiger attack?
I Exponential tiger = .2/hour. So P{Ttiger > a} = e .2a .
I How about E [Ttiger ] and Var[Ttiger ]?
I E [Ttiger ] = 1/tiger = 5 hours, Var[Ttiger ] = 1/2tiger = 25
hours squared.
More lions, tigers, bears

I Lion, tiger, and bear attacks are independent Poisson

processes with values .1/hour, .2/hour, and .3/hour.
I Distribution of time Ttiger till first tiger attack?
I Exponential tiger = .2/hour. So P{Ttiger > a} = e .2a .
I How about E [Ttiger ] and Var[Ttiger ]?
I E [Ttiger ] = 1/tiger = 5 hours, Var[Ttiger ] = 1/2tiger = 25
hours squared.
I Time until 5th attack by any animal?
More lions, tigers, bears

I Lion, tiger, and bear attacks are independent Poisson

processes with values .1/hour, .2/hour, and .3/hour.
I Distribution of time Ttiger till first tiger attack?
I Exponential tiger = .2/hour. So P{Ttiger > a} = e .2a .
I How about E [Ttiger ] and Var[Ttiger ]?
I E [Ttiger ] = 1/tiger = 5 hours, Var[Ttiger ] = 1/2tiger = 25
hours squared.
I Time until 5th attack by any animal?
I distribution with = 5 and = .6.
More lions, tigers, bears

I Lion, tiger, and bear attacks are independent Poisson

processes with values .1/hour, .2/hour, and .3/hour.
I Distribution of time Ttiger till first tiger attack?
I Exponential tiger = .2/hour. So P{Ttiger > a} = e .2a .
I How about E [Ttiger ] and Var[Ttiger ]?
I E [Ttiger ] = 1/tiger = 5 hours, Var[Ttiger ] = 1/2tiger = 25
hours squared.
I Time until 5th attack by any animal?
I distribution with = 5 and = .6.
I X , where X th attack is 5th bear attack?
More lions, tigers, bears

I Lion, tiger, and bear attacks are independent Poisson

processes with values .1/hour, .2/hour, and .3/hour.
I Distribution of time Ttiger till first tiger attack?
I Exponential tiger = .2/hour. So P{Ttiger > a} = e .2a .
I How about E [Ttiger ] and Var[Ttiger ]?
I E [Ttiger ] = 1/tiger = 5 hours, Var[Ttiger ] = 1/2tiger = 25
hours squared.
I Time until 5th attack by any animal?
I distribution with = 5 and = .6.
I X , where X th attack is 5th bear attack?
I Negative binomial with parameters p = 1/2 and n = 5.
More lions, tigers, bears

I Lion, tiger, and bear attacks are independent Poisson

processes with values .1/hour, .2/hour, and .3/hour.
I Distribution of time Ttiger till first tiger attack?
I Exponential tiger = .2/hour. So P{Ttiger > a} = e .2a .
I How about E [Ttiger ] and Var[Ttiger ]?
I E [Ttiger ] = 1/tiger = 5 hours, Var[Ttiger ] = 1/2tiger = 25
hours squared.
I Time until 5th attack by any animal?
I distribution with = 5 and = .6.
I X , where X th attack is 5th bear attack?
I Negative binomial with parameters p = 1/2 and n = 5.
I Can hiker breathe sigh of relief after 5 attack-free hours?
Buffons needle problem

I Drop a needle of length one on a large sheet of paper (with

evenly spaced horizontal lines spaced at all integer heights).
Buffons needle problem

I Drop a needle of length one on a large sheet of paper (with

evenly spaced horizontal lines spaced at all integer heights).
I Whats the probability the needle crosses a line?
Buffons needle problem

I Drop a needle of length one on a large sheet of paper (with

evenly spaced horizontal lines spaced at all integer heights).
I Whats the probability the needle crosses a line?
I Need some assumptions. Lets say vertical position X of
lowermost endpoint of needle modulo one is uniform in [0, 1]
and independent of angle , which is uniform in [0, ]. Crosses
line if and only there is an integer between the numbers X
and X + sin , i.e., X 1 X + sin .
Buffons needle problem

I Drop a needle of length one on a large sheet of paper (with

evenly spaced horizontal lines spaced at all integer heights).
I Whats the probability the needle crosses a line?
I Need some assumptions. Lets say vertical position X of
lowermost endpoint of needle modulo one is uniform in [0, 1]
and independent of angle , which is uniform in [0, ]. Crosses
line if and only there is an integer between the numbers X
and X + sin , i.e., X 1 X + sin .
I Draw the box [0, 1] [0, ] on which (X , ) is uniform.
Whats the area of the subset where X 1 sin ?