6.

041 Probabilistic Systems Analysis Coursework

6.431 Applied Probability
– Quiz 1 (October 12, 12:05-12:55pm) 17%
• Staff:
– Quiz 2 (November 2, 7:30-9:30pm) 30%
– Lecturer: John Tsitsikli s
– Final exam (scheduled by registrar) 40%
• Pick up and read course information handout – Weekly homework (best 9 of 10) 10%
• Turn in recitation and tutorial scheduling form – Attendance/participation/enthusiasm in 3%
(last sheet of course information handout) recitations/tutorials
• Pick up copy of slides • Collaboration policy described in course info handout

• Text: Introduction to Probability, 2nd Edition,
D. P. Bertsekas and J. N. Tsitsiklis, Athena Scientific, 2008
Read the text!

LECTURE 1 Sample space Ω

• Readings: Sections 1.1, 1.2 • “List” (set) of possible outcomes
• List must be:
Lecture outline – Mutually exclusive

• Probability as a mathematical framework – Collectively exhaustive

for reasoning about uncertainty • Art: to be at the “right” granularity

• Probabilistic models
– sample space
– probability law
• Axioms of probability
• Simple examples

1
 Sample space: Discrete example Sample space: Continuous example

• Two rolls of a tetrahedral die Ω = {(x, y) | 0 ≤ x, y ≤ 1}

– Sample space vs. sequential description
y
1,1
1 1,2
1,3
1
1,4
4 2
3
Y = Second
roll
3
2
1 x
1

1 2 3 4
4
X = First roll 4,4

Probability axioms Probability law: Example with finite sample space

• Event: a subset of the sample space 4

• Probability is assigned to events
Y = Second 3
roll
2
Axioms:
1
1. Nonnegativity: P(A) ≥ 0
1 2 3 4
2. Normalization: P(Ω) = 1
X = First roll
3. Additivity: If A ∩ B = Ø, then P(A ∪ B) = P(A) + P(B)
• Let every possible outcome have probability 1/16

• P({s1, s2, . . . , sk }) = P({s1}) + · · · + P({sk }) – P((X, Y ) is (1,1) or (1,2)) =

= P(s1) + · · · + P(sk ) – P({X = 1}) =

– P(X + Y is odd) =
• Axiom 3 needs strengthening
• Do weird sets have probabilities? – P(min(X, Y ) = 2) =

2
 Discrete uniform law Continuous uniform law

• Let all outcomes be equally likely • Two “random” numbers in [0, 1].
y
• Then,
1
number of elements of A
P(A) =
total number of sample points

1 x
• Computing probabilities ≡ counting
• Uniform law: Probability = Area
• Defines fair coins, fair dice, well-shuffled decks
– P(X + Y ≤ 1/2) = ?

– P( (X, Y ) = (0.5, 0.3) )

Probability law: Ex. w/countably infinite sample space

• Sample space: {1, 2, . . .}
– We are given P(n) = 2−n, n = 1, 2, . . .
– Find P(outcome is even)
Remember!
p

1/2
• Turn in recitation/tutorial scheduling form now

1/4 • Tutorials start next week

…..
1/8
1/16

1 2 3 4

1 1 1 1
P({2, 4, 6, . . .}) = P(2) + P(4) + · · · = + 4 + 6 + ··· =
22 2 2 3

• Countable additivity axiom (needed for this calculation):

If A1, A2, . . . are disjoint events, then:
P(A1 ∪ A2 ∪ · · · ) = P(A1) + P(A2) + · · ·

3
 LECTURE 2 Review of probability models

• Readings: Sections 1.3-1.4 • Sample space Ω

– Mutually exclusive
Collectively exhaustive
Lecture outline
– Right granularity
• Review • Event: Subset of the sample space
• Conditional probability
• Allocation of probabilities to events
• Three important tools: 1. P(A) ≥ 0
– Multiplication rule 2. P(Ω) = 1

– Total probability theorem 3. If A ∩ B = Ø,

then P(A ∪ B) = P(A) + P(B)
– Bayes’ rule
3’. If A1, A2, . . . are disjoint events, then:
P(A1 ∪ A2 ∪ · · · ) = P(A1) + P(A2) + · · ·

• Problem solving:
– Specify sample space
– Define probability law
– Identify event of interest
– Calculate...

Conditional probability Die roll example

A 4

Y = Second 3
B roll
2

1
• P(A | B) = probability of A,
1 2 3 4
given that B occurred
– B is our new universe X = First roll

• Definition: Assuming P(B) =

$ 0, • Let B be the event: min(X, Y ) = 2
P(A ∩ B) • Let M = max(X, Y )
P(A | B) =
P(B)
P(A | B) undefined if P(B) = 0 • P(M = 1 | B) =

• P(M = 2 | B) =

4
 Models based on conditional Multiplication rule
Multiplication rule
probabilities
PP(A∩∩B
(A C) =
B ∩ C) = PP(A) P(B | A)
(A)·P A)P (C ||A
· P(C A∩∩ B)
B)
• Event A: Airplane is flying above
Event B: Something registers on radar
screen
U U
P(B | A)=0.99 A B P(C | A B) U U
A B C
P(B | A)
P(Bc | A)=0.01
P(A)=0.05
A
P(Bc | A)
A Bc C
U U
P(A)
Bc
U
A
P(Ac)=0.95
A Bc Cc
U U
P(B | Ac)=0.10

P(Bc | Ac)=0.90 P(Ac)

Ac
P(A ∩ B) =

P(B) =

P(A | B) =

Total probability theorem Bayes’ rule

• Divide and conquer • “Prior” probabilities P(Ai)

– initial “beliefs”
• Partition of sample space into A1, A2, A3
• We know P(B | Ai) for each i
• Have P(B | Ai), for every i
• Wish to compute P(Ai | B)
A1 – revise “beliefs”, given that B occurred
B
A1
B
A2 A3

• One way of computing P(B): A2 A3

P(B) = P(A1)P(B | A1)

+ P(A2)P(B | A2)
+ P(A3)P(B | A3) P(Ai ∩ B)
P(Ai | B) =
P(B)
P(Ai)P(B | Ai)
=
P(B)
P(Ai)P(B | Ai)
= !
j P(Aj )P(B | Aj )

5
 LECTURE 3 Models based on conditional
probabilities
• Readings: Section 1.5
• 3 tosses of a biased coin:
• Review P(H) = p, P(T ) = 1 − p
• Independence of two events p HHH

• Independence of a collection of events p

HHT
1- p
p HTH
Review p 1- p

1- p HTT
P(A ∩ B) p THH
P(A | B) = , assuming P(B ) > 0
P(B) 1- p p
1- p THT
• Multiplication rule: p TTH
1- p
P(A ∩ B) = P(B) · P(A | B) = P(A) · P(B | A) 1- p TTT

• Total probability theorem:

P(B ) = P(A)P(B | A) + P(Ac)P(B | Ac) P(T HT ) =

• Bayes rule: P(1 head) =

P(Ai)P(B | Ai)
P(Ai | B) =
P(B) P(first toss is H | 1 head) =

Independence of two events Conditioning may affect independence

• “Defn:” P(B | A) = P(B) • Conditional independence, given C,

is defined as independence
– “occurrence of A
under probability law P( · | C)
provides no information
about B’s occurrence”
• Assume A and B are independent
• Recall that P(A ∩ B) = P(A) · P(B | A)

• Defn: P(A ∩ B) = P(A) · P(B) C

A
• Symmetric with respect to A and B
B
– applies even if P(A) = 0
– implies P(A | B) = P(A) • If we are told that C occurred,
are A and B independent?

6
Conditioning may affect independence Independence of a collection of events

• Two unfair coins, A and B: • Intuitive definition:

P(H | coin A) = 0.9, P(H | coin B) = 0.1
Information on some of the events tells
choose either coin with equal probability
us nothing about probabilities related to
0.9 the remaining events
0.1
0.9 – E.g.:
Coin A
0.9 P(A1 ∩ (Ac2 ∪ A3) | A5∩Ac6) = P(A1 ∩ (Ac2 ∪ A3))
0.5 0.1
0.1
• Mathematical definition:
0.1 Events A1, A2, . . . , An
0.5 0.1
0.9
are called independent if:
Coin B 0.1
0.9
P(Ai∩Aj ∩· · ·∩Aq ) = P(Ai)P(Aj ) · · · P(Aq )
0.9 for any distinct indices i, j, . . . , q,
(chosen from {1, . . . , n})
• Once we know it is coin A, are tosses
independent?
• If we do not know which coin it is, are
tosses independent?
– Compare:
P(toss 11 = H)
P(toss 11 = H | first 10 tosses are heads)

Independence vs. pairwise The king’s sibling

independence
• The king comes from a family of two
• Two independent fair coin tosses children. What is the probability that
– A: First toss is H his sibling is female?
– B: Second toss is H
– P(A) = P(B) = 1/2

HH HT

TH TT

– C: First and second toss give same

result
– P(C) =
– P(C ∩ A) =
– P(A ∩ B ∩ C) =
– P(C | A ∩ B) =
• Pairwise independence does not
imply independence

7
 LECTURE 4 Discrete uniform law

• Readings: Section 1.6 • Let all sample points be equally likely

• Then,
Lecture outline number of elements of A |A|
P(A) = =
total number of sample points |Ω|
• Principles of counting
• Just count. . .
• Many examples
– permutations
– k-permutations
– combinations
– partitions

• Binomial probabilities

Basic counting principle Example

• r stages • Probability that six rolls of a six-sided die

• ni choices at stage i all give different numbers?

– Number of outcomes that

make the event happen:

– Number of elements
in the sample space:

• Number of choices is: n1n2 · · · nr

– Answer:

• Number of license plates

with 3 letters and 4 digits =

• . . . if repetition is prohibited =

• Permutations: Number of ways

of ordering n elements is:

• Number of subsets of {1, . . . , n} =

8
 Combinations Binomial probabilities
�n �
• : number of k-element subsets
k • n independent coin tosses
of a given n-element set
– P(H) = p
• Two ways of constructing an ordered
sequence of k distinct items:
• P(HT T HHH) =
– Choose the k items one at a time:
n!
n(n−1) · · · (n−k+1) = choices
(n − k)! • P(sequence) = p# heads(1 − p)# tails
– Choose k items, then order them
(k! possible orders)
�
P(k heads) = P(seq.)
• Hence: k−head seq.
�n � n!
· k! =
k (n − k)! = (# of k−head seqs.) · pk (1 − p)n−k
�n � n! �n �
= = pk (1 − p)n−k
k k!(n − k)! k

n � �
� n
=
k=0
k

Coin tossing problem Partitions

• event B: 3 out of 10 tosses were “heads”. • 52-card deck, dealt to 4 players

– Given that B occurred, • Find P(each gets an ace)
what is the (conditional) probability
that the first 2 tosses were heads? • Outcome: a partition of the 52 cards
– number of outcomes:
• All outcomes in set B are equally likely:
52!
probability p3(1 − p)7
13! 13! 13! 13!
– Conditional probability law is uniform • Count number of ways of distributing the
four aces: 4 · 3 · 2
• Number of outcomes in B:
• Count number of ways of dealing the
• Out of the outcomes in B, remaining 48 cards
how many start with HH? 48!
12! 12! 12! 12!
• Answer:
48!
4·3·2
12! 12! 12! 12!
52!
13! 13! 13! 13!

9
 LECTURE 5 Random variables

• Readings: Sections 2.1-2.3, start 2.4 • An assignment of a value (number) to

every possible outcome
Lecture outline
• Mathematically: A function
• Random variables from the sample space Ω to the real
numbers
• Probability mass function (PMF)
– discrete or continuous values
• Expectation
• Can have several random variables
• Variance
defined on the same sample space

• Notation:
– random variable X
– numerical value x

Probability mass function (PMF) How to compute a PMF pX (x)

– collect all possible outcomes for which
• (“probability law”, X is equal to x
“probability distribution” of X) – add their probabilities
– repeat for all x
• Notation:
• Example: Two independent rools of a
pX (x) = P(X = x) fair tetrahedral die
= P({ω ∈ Ω s.t. X(ω) = x})
F : outcome of first throw
! S: outcome of second throw
• pX (x) ≥ 0 x pX (x) = 1 X = min(F, S)

• Example: X=number of coin tosses

until first head 4

– assume independent tosses, 3

P(H) = p > 0 S = Second roll
2
pX (k) = P(X = k)
1
= P(T T · · · T H)
= (1 − p)k−1p, k = 1, 2, . . . 1 2 3 4

F = First roll
– geometric PMF
pX (2) =

10
 Binomial PMF Expectation
• Definition:
• X: number of heads in n independent $
coin tosses E[X] = xpX (x)
x
• P(H) = p
• Interpretations:
• Let n = 4 – Center of gravity of PMF
– Average in large number of repetitions
pX (2) = P(HHT T ) + P(HT HT ) + P(HT T H) of the experiment
+P(T HHT ) + P(T HT H) + P(T T HH) (to be substantiated later in this course)

= 6p2(1 − p)2 • Example: Uniform on 0, 1, . . . , n

"4#
= p2(1 − p)2 pX(x )
2

1/(n+1)
In general:

pX (k) =
"n#

k
pk (1−p)n−k , k = 0, 1, . . . , n ...
0 1 n- 1 n x

1 1 1
E[X] = 0× +1× +· · ·+n× =
n+1 n+1 n+1

Properties of expectations Variance

• Let X be a r.v. and let Y = g(X) $

Recall: E[g(X)] = g(x)pX (x)
$ x
– Hard: E[Y ] = ypY (y) ! 2
y • Second moment: E[X 2] = x x pX (x)
$
– Easy: E[Y ] = g(x)pX (x)
x • Variance
% &
• Caution: In general, E[g(X)] %= g(E[X]) var(X) = E (X − E[X])2

$
= (x − E[X ])2pX (x)
x

Prop erties: If α, β are constants, then: = E[X 2] − (E[X])2

• E[α] =
Properties:
• E[αX] = • var(X) ≥ 0

• E[αX + β] = • var(αX + β) = α2var(X)

11
 LECTURE 6 Review
• Random variable X: function from
• Readings: Sections 2.4-2.6
sample space to the real numbers

Lecture outline • PMF (for discrete random variables):

pX (x) = P(X = x)
• Review: PMF, expectation, variance • Expectation:
!
• Conditional PMF E[X] = xpX (x)
x
• Geometric PMF !
E[g(X)] = g(x)pX (x)
• Total expectation theorem x

• Joint PMF of two random variables E[αX + β] = αE[X] + β

" #
• E X − E[X] =

$ %
var(X) = E (X − E[X])2
!
= (x − E[X])2pX (x)
x
= E[X 2] − (E[X])2
&
Standard deviation: σX = var(X)

Random speed Average speed vs. average time

• Traverse a 200 mile distance at constant • Traverse a 200 mile distance at constant
but random speed V but random speed V
pV (v ) 1/2 1/2 pV (v ) 1/2 1/2

1 200 v 1 200 v

• d = 200, T = t(V ) = 200/V • time in hours = T = t(V ) =

'
• E[T ] = E[t(V )] = v t(v)pV (v) =
• E[V ] =
• E[T V ] = 200 =
" E[T ] · E[V ]

• var(V ) = • E[200/V ] = E[T ] "= 200/E[V ].

• σV =

12
 Conditional PMF and expectation Geometric PMF
• X: number of independent coin tosses
• pX|A(x) = P(X = x | A)
until first head
!
• E[X | A] = xpX |A(x) pX (k) = (1 − p)k−1p, k = 1, 2, . . .
x

∞
! ∞
!
pX (x ) E[X] = kpX (k) = k(1 − p)k−1p
k=1 k=1

• Memoryless property: Given that X > 2,

1/4
the r.v. X − 2 has same geometric PMF
p
pX (k) pX |X>2(k)

p(1-p)2
p
1 2 3 4 x
... ...
• Let A = {X ≥ 2} 1 k 3 k
pX- 2|X>2(k)
pX|A(x) =
p
E[X | A] =
...
1 k

Total Expectation theorem Joint PMFs

• Partition of sample space • pX,Y (x, y) = P(X = x and Y = y)

into disjoint events A1, A2, . . . , An
y

A1 4 1/20 2/20 2/20

B
3 2/20 4/20 1/20 2/20

2 1/20 3/20 1/20

A2 A3 1 1/20

1 2 3 4 x

P(B) = P(A1)P(B | A1)+· · ·+P(An)P(B | An) !!

• pX,Y (x, y) =
pX (x) = P(A1)pX |A1 (x)+· · ·+P(An)pX |An (x) x y
!
E[X] = P(A1)E[X | A1]+· · ·+P(An)E[X | An] • pX (x) = pX,Y (x, y)
y

• Geometric example: pX,Y (x, y)

• pX |Y (x | y) = P(X = x | Y = y) =
A1 : {X = 1}, A2 : {X > 1} pY (y)
!
E[X] = P(X = 1)E[X | X = 1] • pX |Y (x | y) =
+P(X > 1)E[X | X > 1] x

• Solve to get E[X] = 1/p

13
 LECTURE 7 Review

• Readings: Finish Chapter 2

pX (x) = P(X = x)
Lecture outline
pX,Y (x, y) = P(X = x, Y = y)
• Multiple random variables
pX |Y (x | y) = P(X = x | Y = y )
– Joint PMF
– Conditioning
– Independence !
pX (x) = pX,Y (x, y)
• More on expectations y

• Binomial distribution revisited pX,Y (x, y) = pX (x)pY |X (y | x)

• A hat problem

Independent random variables Expectations

!
pX,Y,Z (x, y, z) = pX (x)pY |X (y | x)pZ |X,Y (z | x, y) E[X] = xpX (x)
x
!!
• Random variables X, Y , Z are
E[g(X, Y )] = g(x, y)pX,Y (x, y)
x y
independent if:
" #
pX,Y,Z (x, y, z) = pX (x) · pY (y) · pZ (z) • In general: E[g(X, Y )] #= g E[X], E[Y ]
for all x, y, z
y • E[αX + β] = αE[X] + β
4 1/20 2/20 2/20
• E[X + Y + Z] = E[X] + E[Y ] + E[Z]
3 2/20 4/20 1/20 2/20

2 1/20 3/20 1/20 • If X, Y are independent:

1 1/20
– E[XY ] = E[X]E[Y ]
1 2 3 4 x

– E[g(X)h(Y )] = E[g(X)] · E[h(Y )]

• Independent?

• What if we condition on X ≤ 2
and Y ≥ 3?

14
 Variances Binomial mean and variance

• Var(aX) = a2Var(X) • X = # of successes in n independent

trials
• Var(X + a) = Var(X)
– probability of success p
n
! "n#
• Let Z = X + Y . E[X] = k pk (1 − p)n−k
k=0
k
If X, Y are independent:

Var(X + Y ) = Var(X) + Var(Y ) 1, if success in trial i,
• Xi =
0, otherwise

• Examples:
• E[Xi] =
– If X = Y , Var(X + Y ) =

– If X = −Y , Var(X + Y ) = • E[X] =

– If X, Y indep., and Z = X − 3Y ,
Var(Z) = • Var(Xi) =

• Var(X) =

The hat problem Variance in the hat problem

• n people throw their hats in a box and • Var(X ) = E[X 2] − (E[X])2 = E[X 2] − 1
then pick one at random.
– X: number of people who get their own
! !
hat X2 = Xi2 + XiXj
i i,j :i=j
#
– Find E[X]

• E[Xi2] =

1, if i selects own hat
Xi =
0, otherwise.
P(X1X2 = 1) = P(X1 = 1)·P(X2 = 1 | X1 = 1)
• X = X1 + X2 + · · · + Xn
=
• P(Xi = 1) =

• E[Xi] =

• Are the Xi independent? • E[X 2 ] =

• E[X] = • Var(X) =

15
 LECTURE 8 Continuous r.v.’s and pdf’s

• Readings: Sections 3.1-3.3 • A continuous r.v. is described by a

probability density function fX

Lecture outline
fX(x)
Sample Space
• Probability density functions

• Cumulative distribution functions

a b x
Event {a < X < b }
• Normal random variables

! b
P(a ≤ X ≤ b) = fX (x) dx
a

! ∞
fX (x) dx = 1
−∞

! x+δ
P(x ≤ X ≤ x + δ) = fX (s) ds ≈ fX (x) · δ
x

!
P(X ∈ B) = fX (x) dx, for “nice” sets B
B

Means and variances Cumulative distribution function

! ∞ (CDF)
• E[X] = xfX (x) dx
−∞
! ∞ ! x
• E[g(X)] = g(x)fX (x) dx FX (x) = P(X ≤ x) = fX (t) dt
−∞ −∞
! ∞
• 2 =
var(X ) = σX (x − E[X ])2fX (x) dx
−∞ fX(x ) CDF

• Continuous Uniform r.v.

fX (x )

a b x a b x

• Also for discrete r.v.’s:

$
a b x FX (x) = P(X ≤ x) = pX (k)
k≤x
• fX (x) = a≤x≤b 3/6

2/6
• E[X] = 1/6

! b" #
2 = a+b 2 1 (b − a)2 1 2 4 x 1 2 4 x
• σX x− dx =
a 2 b−a 12

16
 Mixed distributions Gaussian (normal) PDF

• Schematic drawing of a combination of 1 2

• Standard normal N (0, 1): fX (x) = √ e−x /2
a PDF and a PMF 2π
1/2
Normal PDF fx (x) Normal CDF FX(x)
1
0.5

-1 0 1 2 x -1 0 1 2 x

• E[X] = var(X) = 1
0 1/2 1 x0
• General normal N (µ, σ 2):
• The corresponding CDF: 1 2 2
fX (x) = √ e−(x−µ) /2σ
σ 2π
FX (x) = P(X ≤ x)
CDF

• It turns out that:

1
E[X] = µ and Var(X) = σ 2.
3/4
• Let Y = aX + b

1/4 – Then: E[Y ] = Var(Y ) =

– Fact: Y ∼ N (aµ + b, a2σ 2)
1/2 1 x

Calculating normal probabilities The constellation of concepts

• No closed form available for CDF

– but there are tables
(for standard normal) pX (x) fX (x)
X −µ FX (x)
• If X ∼ N (µ, σ 2), then ∼ N( ) E[X], var(X)
σ
• If X ∼ N (2, 16): pX,Y (x, y) fX,Y (x, y)
" # pX |Y (x | y) fX |Y (x | y)
X −2 3−2
(X
PSec. 3) =Random
3.3 ≤ Normal P Variables ≤ = CDF(0.25)
155
4 4
.00 .01 .02 .03 .04 .05 .06 .07 .08 .09
0.0 .5000 .5040 .5080 .5120 .5160 .5199 .5239 .5279 .5319 .5359
0.1 .5398 .5438 .5478 .5517 .5557 .5596 .5636 .5675 .5714 .5753
0.2 .5793 .5832 .5871 .5910 .5948 .5987 .6026 .6064 .6103 .6141
0.3 .6179 .6217 .6255 .6293 .6331 .6368 .6406 .6443 .6480 .6517
0.4 .6554 .6591 .6628 .6664 .6700 .6736 .6772 .6808 .6844 .6879
0.5 .6915 .6950 .6985 .7019 .7054 .7088 .7123 .7157 .7190 .7224
0.6 .7257 .7291 .7324 .7357 .7389 .7422 .7454 .7486 .7517 .7549
0.7 .7580 .7611 .7642 .7673 .7704 .7734 .7764 .7794 .7823 .7852
0.8 .7881 .7910 .7939 .7967 .7995 .8023 .8051 .8078 .8106 .8133
0.9 .8159 .8186 .8212 .8238 .8264 .8289 .8315 .8340 .8365 .8389
1.0 .8413 .8438 .8461 .8485 .8508 .8531 .8554 .8577 .8599 .8621
1.1 .8643 .8665 .8686 .8708 .8729 .8749 .8770 .8790 .8810 .8830
1.2 .8849 .8869 .8888 .8907 .8925 .8944 .8962 .8980 .8997 .9015
1.3 .9032 .9049 .9066 .9082 .9099 .9115 .9131 .9147 .9162 .9177
1.4 .9192 .9207 .9222 .9236 .9251 .9265 .9279 .9292 .9306 .9319
1.5 .9332 .9345 .9357 .9370 .9382 .9394 .9406 .9418 .9429 .9441
1.6 .9452 .9463 .9474 .9484 .9495 .9505 .9515 .9525 .9535 .9545
1.7 .9554 .9564 .9573 .9582 .9591 .9599 .9608 .9616 .9625 .9633
1.8 .9641 .9649 .9656 .9664 .9671 .9678 .9686 .9693 .9699 .9706
1.9 .9713 .9719 .9726 .9732 .9738 .9744 .9750 .9756 .9761 .9767
2.0 .9772 .9778 .9783 .9788 .9793 .9798 .9803 .9808 .9812 .9817
2.1 .9821 .9826 .9830 .9834 .9838 .9842 .9846 .9850 .9854 .9857
2.2 .9861 .9864 .9868 .9871 .9875 .9878 .9881 .9884 .9887 .9890
2.3 .9893 .9896 .9898 .9901 .9904 .9906 .9909 .9911 .9913 .9916
2.4 .9918 .9920 .9922 .9925 .9927 .9929 .9931 .9932 .9934 .9936
2.5 .9938 .9940 .9941 .9943 .9945 .9946 .9948 .9949 .9951 .9952
2.6 .9953 .9955 .9956 .9957 .9959 .9960 .9961 .9962 .9963 .9964
2.7
2.8
.9965
.9974
.9966
.9975
.9967
.9976
.9968
.9977
.9969
.9977
.9970
.9978
.9971
.9979
.9972
.9979
.9973
.9980
.9974
.9981
17
 LECTURE 9 Continuous r.v.’s and pdf’s
• Readings: Sections 3.4-3.5 fX(x)
Sample Space

Outline
• PDF review Event {a < X < b }
a b x

• Multiple random variables

– conditioning
� b
– independence P(a ≤ X ≤ b) = fX (x) dx
a
• Examples
• P(x ≤ X ≤ x + δ) ≈ fX (x) · δ
Summary of concepts � ∞
• E[g(X)] = g(x)fX (x) dx
−∞
pX (x) fX (x)
FX (x) �
�
xpX (x) E[X] xfX (x) dx
x
var(X)
pX,Y (x, y) fX,Y (x, y)
pX|A(x) fX |A(x)
pX |Y (x | y ) fX |Y (x | y)

Joint PDF fX,Y (x, y) Buffon’s needle

• Parallel lines at distance d
Needle of length � (assume � < d)
� � • Find P(needle intersects one of the lines)
P((X, Y ) ∈ S) = fX,Y (x, y) dx dy
S q
x
l
d
• Interpretation:
P(x ≤ X ≤ x+δ, y ≤ Y ≤ y +δ ) ≈ fX,Y (x, y )·δ 2
• X ∈ [0, d/2]: distance of needle midpoint
to nearest line
• Expectations: • Model: X, Θ uniform, independent
� ∞ � ∞
E[g (X, Y )] = g (x, y)fX,Y (x, y) dx dy fX,Θ(x, θ ) = 0 ≤ x ≤ d/2, 0 ≤ θ ≤ π/2
−∞ −∞

�
• From the joint to the marginal: • Intersect if X ≤ sin Θ
2
fX (x) · δ ≈ P(x ≤ X ≤ x + δ) =
� � � �
�
P X≤ sin Θ = fX (x)fΘ(θ) dx dθ
2 x≤ 2� sin θ
� �
4 π/2 (�/2) sin θ
= dx dθ
πd 0 0
• X and Y are called independent if �
4 π/2 � 2�
fX,Y (x, y) = fX (x)fY (y), for all x, y = sin θ dθ =
πd 0 2 πd

18
 Conditioning

• Recall

P(x ≤ X ≤ x + δ) ≈ fX (x) · δ Joint, Marginal and Conditional Densities

• By analogy, would like:

P(x ≤ X ≤ x + δ | Y ≈ y) ≈ fX |Y (x | y) · δ

• This leads us to the definition:

fX,Y (x, y) Area of slice = Height of marginal
fX |Y (x | y) = if fY (y) > 0 density at x
fY (y)
Renormalizing slices for
fixed x gives conditional
• For given y, conditional PDF is a densities for Y given X = x
Slice through
(normalized) “section” of the joint PDF density surface
for fixed x

• If independent, fX,Y = fX fY , we obtain

Image by MIT OpenCourseWare, adapted from
fX |Y (x|y) = fX (x) Probability, by J. Pittman, 1999.

Stick-breaking example
• Break a stick of length � twice: 1
fX,Y (x, y) = , 0≤y≤x≤�
break at X: uniform in [0, 1]; �x
break again at Y , uniform in [0, X]
y
f X(x) f Y |X (y | x)
L

L x y
L x

fX,Y (x, y) = fX (x)fY |X (y | x) =

on the set:
y �
fY (y) = fX,Y (x, y) dx
L � �
1
= dx
y �x
1 �
= log , 0≤y≤�
� y
L x � � � �
1 � �
E[Y ] = yfY (y) dy = y log dy =
� 0 0 � y 4
E[Y | X = x] = yfY |X (y | X = x) dy =

19
 LECTURE 10 The Bayes variations

Continuous Bayes rule; pX,Y (x, y) pX (x)pY |X (y | x)

Derived distributions pX|Y (x | y) = =
pY (y) pY (y)
�
• Readings: pY (y) = pX (x)pY |X (y | x)
x
Section 3.6; start Section 4.1
Example:
Review • X = 1, 0: airplane present/not present
• Y = 1, 0: something did/did not register
pX (x) fX (x) on radar
pX,Y (x, y) fX,Y (x, y)
pX,Y (x, y) fX,Y (x, y) Continuous counterpart
pX |Y (x | y) = fX|Y (x | y) =
pY (y) fY (y)
� � ∞
fX,Y (x, y) fX (x)fY |X (y | x)
pX (x) = pX,Y (x, y) fX (x) = fX,Y (x, y) dy fX|Y (x | y) = =
y −∞ fY (y) fY (y)
�
fY (y) = fX (x)fY |X (y | x) dx
x
FX (x) = P(X ≤ x)
Example: X: some signal; “prior” fX (x)
E[X], var(X) Y : noisy version of X
fY |X (y | x): model of the noise

Discrete X, Continuous Y What is a derived distribution

pX (x)fY |X (y | x) • It is a PMF or PDF of a function of one

pX |Y (x | y) = or more random variables with known
fY (y)
probability law. E.g.:
� y
fY (y ) = pX (x)fY |X (y | x) f X ,Y(y,x)=1
x
1
Example:
• X: a discrete signal; “prior” pX (x)
• Y : noisy version of X
• fY |X (y | x): continuous noise model
1 x

Continuous X, Discrete Y
– Obtaining the PDF for
fX (x)pY |X (y | x)
fX |Y (x | y) = g(X, Y ) = Y /X
pY (y)
� involves deriving a distribution.
pY (y) = fX (x)pY |X (y | x) dx Note: g(X, Y ) is a random variable
x
Example:
When not to find them
• X: a continuous signal; “prior” fX (x)
(e.g., intensity of light beam); • Don’t need PDF for g(X, Y ) if only want
• Y : discrete r.v. affected by X to compute expected value:
� �
(e.g., photon count)
E[g(X, Y )] = g(x, y)fX,Y (x, y) dx dy
• pY |X (y | x): model of the discrete r.v.

20
 How to find them The continuous case

• Discrete case • Two-step procedure:

– Obtain probability mass for each – Get CDF of Y : FY (y) = P(Y ≤ y)

possible value of Y = g(X)

pY (y) = P(g(X) = y) – Differentiate to get

�
= pX (x) dFY
fY (y) = (y)
x: g(x)=y dy

x y
g(x)
Example
. . • X: uniform on [0,2]
. . • Find PDF of Y = X 3
. .
. . • Solution:

FY (y) = P(Y ≤ y) = P(X 3 ≤ y)

. . 1
= P(X ≤ y 1/3) = y 1/3
. . 2

. . fY (y) =
dFY
dy
(y) =
1
6y 2/3

Example The pdf of Y=aX+b

• Joan is driving from Boston to New York.

Her speed is uniformly distributed be- Y = 2X + 5:
tween 30 and 60 mph. What is the dis-
tribution of the duration of the trip? fX
faX faX+b
200
• Let T (V ) = .
V
• Find fT (t) -2 -1 2 3 4 9

f v(v0 )

1/30 � �
1 y−b
fY (y) = fX
|a| a

30 60 v0 • Use this to check that if X is normal,

then Y = aX + b is also normal.

21
 LECTURE 11 A general formula

Derived distributions; convolution; • Let Y = g(X)

covariance and correlation g strictly monotonic.
dg
slope (x)
• Readings: y dx

Finish Section 4.1;

Section 4.2
g(x)
[y, y+?]
Example
y x
f X ,Y(y,x)=1
[x, x+d]

• Event x ≤ X ≤ x + δ is the same as

g(x) ≤ Y ≤ g(x + δ)
or (approximately)
1 x
g(x) ≤ Y ≤ g(x) + δ|(dg/dx)(x)|

• Hence,
Find the PDF of Z = g(X, Y ) = Y /X ! !
! dg !
δfX (x) = δfY (y) !! (x)!!
FZ (z ) = z≤1 dx
where y = g(x)

FZ (z) = z≥1

The distribution of X + Y The continuous case

• W = X + Y ; X, Y independent • W = X + Y ; X, Y independent

. y
y
. (0,3)

. (1,2) w

.(2,1)

.
(3,0)
x
. w x
pW (w) = P(X + Y = w)
" x +y=w
= P(X = x)P(Y = w − x)
x
"
= pX (x)pY (w − x) • fW |X (w | x) = fY (w − x)
x
• fW,X (w, x) = fX (x)fW |X (w | x)
• Mechanics:
= fX (x)fY (w − x)
– Put the pmf’s on top of each other
# ∞
– Flip the pmf of Y • fW (w) = fX (x)fY (w − x) dx
−∞
– Shift the flipped pmf by w
(to the right if w > 0)
– Cross-multiply and add

22
 Two independent normal r.v.s The sum of independent normal r.v.’s

• X ∼ N (µx, σx2), Y ∼ N (µy , σy2), • X ∼ N (0, σx2), Y ∼ N (0, σy2),

independent independent

fX,Y (x, y) = fX (x)fY (y) • Let W = X + Y

$ %
1 (x − µx)2 (y − µy )2 # ∞
= exp − − fW (w) = fX (x)fY (w − x) dx
2πσxσy 2σx2 2σy2 −∞
#
1 ∞ 2 2 2 2
= e−x /2σx e−(w−x) /2σy dx
• PDF is constant on the ellipse where 2πσxσy −∞
2
(x − µx)2 (y − µ y ) 2 (algebra) = ce−γw
+
2σx2 2σy2
is constant • Conclusion: W is normal
– mean=0, variance=σx2 + σy2
• Ellipse is a circle when σx = σy
– same argument for nonzero mean case

Covariance Correlation coefficient

& '
• cov(X, Y ) = E (X − E[X]) · (Y − E[Y ]) • Dimensionless version of covariance:
, -
(X − E[X]) (Y − E[Y ])
• Zero-mean case: cov(X, Y ) = E [XY ] ρ = E ·
σX σY
cov(X, Y )
x x =
. . .... . .. .

. σX σY
.
.

.
. .. ... . .... ... . .. .. . ... . .

. . .... . .. .
. ... ...... ... .. . . .

. . ... ...... ... .. . . . • −1 ≤ ρ ≤ 1

. .. ... . ... ... . .. .. . ... . .
. . .. .. . . ... .. . .

. . . .. .. .
. . . .. .

. . . .. . . ... .. . . y y
• |ρ| = 1 ⇔ (X − E[X]) = c(Y − E[Y ])
. .
. . (linearly related)
. . .
.

• Independent ⇒ ρ = 0
• cov(X, Y ) = E[XY ] − E[X]E[Y ] (converse is not true)
 
n
" n
" "
• var  Xi = var(Xi) + 2 cov(Xi, Xj )
i=1 i=1 (i,j ):i=j
&

• independent ⇒ cov(X, Y ) = 0
(converse is not true)

23
 LECTURE 12 Conditional expectations

• Readings: Section 4.3; • Given the value y of a r.v. Y :

parts of Section 4.5 !
E[X | Y = y] = xpX|Y (x | y)
(mean and variance only; no transforms) x
(integral in continuous case)

Lecture outline • Stick example: stick of length !

break at uniformly chosen point Y
• Conditional expectation break again at uniformly chosen point X
– Law of iterated expectations y
• E[X | Y = y] = (number)
2
– Law of total variance

• Sum of a random number Y

of independent r.v.’s E[X | Y ] = (r.v.)
2
– mean, variance

• Law of iterated expectations:

!
E[E[X | Y ]] = E[X | Y = y]pY (y)= E[X]
y

• In stick example:
E[X] = E[E[X | Y ]] = E[Y /2] = !/4

var(X | Y ) and its expectation Section means and variances

" # Two sections:
• var(X | Y = y ) = E (X − E[X | Y = y])2 | Y = y
y = 1 (10 students); y = 2 (20 students)
• var(X | Y ): a r.v. 10 30
1 ! 1 !
with value var(X | Y = y) when Y = y y=1: xi = 90 y=2: xi = 60
10 i=1 20 i=11
• Law of total variance:

var(X) = E[var(X | Y )] + var(E[X | Y ]) 30

1 ! 90 · 10 + 60 · 20
E[X] = xi = = 70
30 i=1 30

Proof:
E[X | Y = 1] = 90, E[X | Y = 2] = 60

(a) Recall: var(X) = E[X 2] − (E[X])2 

90, w.p. 1/3
E[X | Y ] =
60, w.p. 2/3
(b) var(X | Y ) = E[X 2 | Y ] − (E[X | Y ])2
E[E[X | Y ]] = 1 · 90 + 2 · 60 = 70 = E[X]
3 3
(c) E[var(X | Y )] = E[X 2] − E[ (E[X | Y ])2 ]

1 2
(d) var(E[X | Y ]) = E[ (E[X | Y ])2 ]−(E[X])2 var(E[X | Y ]) = (90 − 70)2 + (60 − 70)2
3 3
600
Sum of right-hand sides of (c), (d): = = 200
3
E[X 2] − (E[X ])2 = var(X)

24
 Section means and variances (ctd.) Example

10 30
1 ! 1 ! var(X) = E[var(X | Y )] + var(E[X | Y ])
(xi−90)2 = 10 (xi−60)2 = 20
10 i=1 20 i=11
f X(x)

2/3

var(X | Y = 1) = 10 var(X | Y = 2) = 20
1/3

1 2
 Y=1 Y=2 x
10, w.p. 1/3
var(X | Y ) =
20, w.p. 2/3
E[X | Y = 1] = E[X | Y = 2] =
E[var(X | Y )] = 1 · 10 + 2 · 20 = 50
3 3 3

var(X | Y = 1) = var(X | Y = 2) =

var(X) = E[var(X | Y )] + var(E[X | Y ])

E[X] =
50
= + 200
3
= (average variability within sections) var(E[X | Y ]) =
+ (variability between sections)

Sum of a random number of Variance of sum of a random number

independent r.v.’s of independent r.v.’s

• N : number of stores visited • var(Y ) = E[var(Y | N )] + var(E[Y | N ])

(N is a nonnegative integer r.v.)
• E[Y | N ] = N E[X]
• Xi: money spent in store i var(E[Y | N ]) = (E[X])2 var(N )
– Xi assumed i.i.d.
• var(Y | N = n) = n var(X)
– independent of N
var(Y | N ) = N var(X)
• Let Y = X1 + · · · + XN E[var(Y | N )] = E[N ] var(X)

E[Y | N = n] = E[X1 + X2 + · · · + Xn | N = n]
= E[X1 + X2 + · · · + Xn]
= E[X1] + E[X2] + · · · + E[Xn] var(Y ) = E[var(Y | N )] + var(E[Y | N ])
= n E[X] = E[N ] var(X ) + (E[X])2 var(N )

• E[Y | N ] = N E[X]

E[Y ] = E[E[Y | N ]]
= E[N E[X]]
= E[N ] E[X]

25
 LECTURE 13 The Bernoulli process

The Bernoulli process • A sequence of independent

Bernoulli trials
• Readings: Section 6.1
• At each trial, i:
– P(success) = P(Xi = 1) = p
Lecture outline
– P(failure) = P(Xi = 0) = 1 − p
• Definition of Bernoulli process

• Random processes • Examples:

– Sequence of lottery wins/losses
• Basic properties of Bernoulli process
– Sequence of ups and downs of the Dow
• Distribution of interarrival times Jones
– Arrivals (each second) to a bank
• The time of the kth success
– Arrivals (at each time slot) to server
• Merging and splitting

Random processes Number of successes S in n time slots

• First view: • P(S = k) =

sequence of random variables X1, X2, . . .
• E[Xt] = • E[S] =
• Var(Xt) =
• Var(S) =
• Second view:
what is the right sample space?

• P(Xt = 1 for all t) =

• Random processes we will study:

– Bernoulli process
(memoryless, discrete time)
– Poisson process
(memoryless, continuous time)
– Markov chains
(with memory/dependence across time)

26
 Interarrival times Time of the kth arrival

• T1: number of trials until first success • Given that first arrival was at time t
i.e., T1 = t:
– P(T1 = t) = additional time, T2, until next arrival

– Memoryless property – has the same (geometric) distribution

– independent of T1
– E[T1] =

– Var(T1) = • Yk : number of trials to kth success

– E[Yk ] =
• If you buy a lottery ticket every day, what
is the distribution of the length of the
first string of losing days? – Var(Yk ) =

– P(Yk = t) =

Sec. 6.1 The Bernoulli Process 305

Splitting and Merging of Bernoulli Processes

Starting with a Bernoulli process in which there is a probability p of an arrival

at each time, consider splitting it as follows. Whenever there is an arrival, we
choose to either keep it (with probability q), or to discard it (with probability
1−q); see Fig. 6.3. Assume that the decisions to keep or discard are independent
for different arrivals. If we focus on the process of arrivals that are kept, we see
Splitting of a Bernoulli Process
that it is a Bernoulli process: in each time slot, there is a probability pq of a Merging of Indep. Bernoulli Processes
kept arrival, independent of what happens in other slots. For the same reason, 306 The Bernoulli and Poisson Processes Chap. 6

(using independent coin flips)

the process of discarded arrivals is also a Bernoulli process, with a probability
of a discarded arrival at each time slot equal to p(1 − q). Bernoulli (p)
time

time Merged process:

q Bernoulli (p + q − pq) time
Original
process time
Bernoulli (q)
1−q time

time
Figure 6.4: Merging of independent Bernoulli processes.

Figure 6.3: Splitting of a Bernoulli process. yields a Bernoulli process

yields Bernoulli processes
In a reverse situation, we start with two independent Bernoulli processes
(collisions
concentrate on the are
special counted as but
case where n is large one arrival)
p is small, so that the mean
np has a moderate value. A situation of this type arises when one passes from
(with parameters p and q, respectively) and merge them into a single process, discrete to continuous time, a theme to be picked up in the next section. For
as follows. An arrival is recorded in the merged process if and only if there some examples, think of the number of airplane accidents on any given day:
is an arrival in at least one of the two original processes. This happens with there is a large number n of trials (airplane flights), but each one has a very
probability p + q − pq [one minus the probability (1 − p)(1 − q) of no arrival in small probability p of being involved in an accident. Or think of counting the
either process]. Since different time slots in either of the original processes are number of typos in a book: there is a large number of words, but a very small
independent, different slots in the merged process are also independent. Thus, probability of misspelling any single one.
the merged process is Bernoulli, with success probability p + q − pq at each time Mathematically, we can address situations of this kind, by letting n grow
step; see Fig. 6.4. while simultaneously decreasing p, in a manner that keeps the product np at a
Splitting and merging of Bernoulli (or other) arrival processes arises in constant value λ. In the limit, it turns out that the formula for the binomial PMF
many contexts. For example, a two-machine work center may see a stream of simplifies to the Poisson PMF. A precise statement is provided next, together
arriving parts to be processed and split them by sending each part to a randomly with a reminder of some of the properties of the Poisson PMF that were derived
chosen machine. Conversely, a machine may be faced with arrivals of different in Chapter 2.
types that can be merged into a single arrival stream.

The Poisson Approximation to the Binomial

Poisson Approximation to the Binomial
The number of successes in n independent Bernoulli trials is a binomial random
variable with parameters n and p, and its mean is np. In this subsection, we • A Poisson random variable Z with parameter λ takes nonnegative
integer values and is described by the PMF

λk
pZ (k) = e−λ , k = 0, 1, 2, . . . .
k!
Its mean and variance are given by

E[Z] = λ, var(Z) = λ.

27
 LECTURE 14 Bernoulli review

The Poisson process • Discrete time; success probability p

Start Section • Number of arrivals in n time slots:

• Readings: LECTURE 166.2. BernoulliLECTURE
review 16
binomial pmf
• Discrete time; success probability
The Poisson p
process • D
The Poisson process • Interarrival times: geometric pmf
Lecture outline
• Number of arrivals in
• Readings: n time
Start slots:5.2.
Section • N
• Readings: Start Section 5.2. • Time to k arrivals: Pascal pmf
• Review of Bernoulli process binomial pmf b

• Definition of Poisson process • Memorylessness

• Interarrival time pmf: geometric pmf • I
Lecture outline Lecture outline

•• Distribution of number of arrivals • Time•toReview

k arrivals: Pascal pmf
of Bernoulli process • T
Review of Bernoulli process

•• Distribution • Memorylessness • M
Definition ofof interarrival
Poisson times
process • Definition of Poisson process

•• Other properties
Distribution of the of
of number Poisson
arrivalsprocess • Distribution of number of arrivals

• Distribution of interarrival times • Distribution of interarrival times

• Other properties of the Poisson process • Other properties of the Poisson process

Definition of the Poisson process PMF of NumberPoisson

of Arrivals N
Definition of the Poisson process PMF Definition
of Numberofofthe
Arrivals N process

t1 t2 t3 ! t1 t2 t3 !

x x x x x x xx x x x x xk −λτ
x x x x xx x x x
0 0 (λτ ) e
Time P (k, τ ) = , k = 0, 1, . . . Time
k!

•• Time • E[N ] •= λτ • E
P (k, τ )homogeneity:
= Prob. of k arrivals in interval P (k, τ ) = Prob. of k arrivals in interval
• Finely discretize [0, t]: approximately Bernoulli
Pof(k,duration
τ ) = Prob.
τ of k arrivals in interval 2 = λτof duration τ
• σN • σ
of duration τ
• Assumptions: • Assumptions:
λt(esdiscrete
• =
• MN (s) Nte(of −1) approximation): binomial • M
• Numbers of arrivals in disjoint time – Numbers of arrivals in disjoint time in-
– Numbers of arrivals in disjoint time in-
intervals
tervals are independent
are independent tervals are independent
• Taking δ → 0 (or n → ∞) gives:
For VERY
• –Small interval small
probabilities:
δ: – For VERY small δ:
(λτ )k −λτ
eaccording
 Example: YouP (k, get
τ ) =email  to a Exa
For VERY small   − λδ if k = 0
δ:1  1 −,λδ ifk k==0,01, . . .
Poisson process at a rate k!
of λ = 0.4 mes- Poi
P (k, δ) ≈ P (k, δ) ≈ λδ if k = 1


 − λδ, ifif kk=
1 λδ =1
0; sages per hour. You check your

 sag
0 if k > 1 0 email ifevery
k>1
P (k, δ ) ≈ λδ, if k = 1; • E[Nt] = λt, var(Nt) = λt thir

 thirty minutes.

– λ = “arrival rate” – λ = “arrival rate”
0, if k > 1.
– Prob(no new messages)= –
– λ: “arrival rate” – Prob(one new message)= –

28
 Example Interarrival Times
Interarrival Times
• You get email according to a Poisson • Yk time of kth arrival
process at a rate of λ = 5 messages per • Yk time of kth arrival !
hour. You check your email every thirty • Erlang distribution:
minutes.
• Erlang distribution:
k k−1 0
λ y e−λy
fYk (y) = k k−1 −λy , y≥0
λ y(k −e1)!
fYk (y) = , y≥0
• Prob(no new messages) = (k − 1)!
flr (l)
fY (y)
k
r=1
• Prob(one new message) = r=2
k=1 r=3
k=2

k=3

l
0 y

Image by MIT OpenCourseWare.

• First-order interarrival times (k = 1):
exponential
• Time of first arrival (k = 1):
(y) = λe−λy , f y
fY1exponential: ≥0
(y) = λe−λy , y ≥ 0
Y1
– –Memoryless
Memorylessproperty:
property: The
The time
time to the
to the
next arrival is independent of the past
next arrival is independent of the past

Bernoulli/Poisson Relation Adding Poisson

Merging PoissonProcesses
Processes
Bernoulli/Poisson Relation
• •Sum
Sumof of
independent
independentPoisson
Poissonrandom
randomvari-
! ! ! ! ! ! ! !
n = t /!
ables is Poisson
variables is Poisson
x x x np =" t
0 p ="!
Arrivals Time • •Sum of independent
Merging Poisson
of independent processes
Poisson processes
0 is is Poisson
Poisson
POISSON BERNOULLI Red bulb flashes
(Poisson)
POISSON BERNOULLI
Times of Arrival Continuous Discrete
All flashes
"1
Times Arrival
of Arrival
Rate Continuous
λ/unit time Discrete
p/per trial (Poisson)

PMF of Rate
Arrival # of Arrivals Poisson
λ/uni t time Binomial
p/per trial "2

Interarrival Time Distr. Exponential Geometric Green bulb flashes

PMF of # of Arrivals Poisson Binomial
1): (Poisson)
Time to k-th arrival Erlang Pascal
Interarrival Time Distr. Exponential Geometric
– What is the probability that the next
– What is the probability that the next
to the arrival comes from the first process?
past
Time to k-th arrival Erlang Pascal arrival comes from the first process?

m vari-

29
 Interarrival Times

• Yk time of kth arrival !

LECTURE 15 Review
• •Erlang distribution:
Defining characteristics 0
Poisson process — II
λk y k−1e−λyP (k, τ )
– Time homogeneity:
f (y) =
Yk
– Independence
, y≥0
• Readings: Finish Section 6.2. (k − 1)!
– Small interval probabilities (small δ):
fY (y) 
k
1 − λδ, if k = 0,
• Review of Poisson process 


k=1
P (k, δ) ≈ λδ, if k = 1,


• Merging and splitting 
k=2 0, if k > 1.

• Examples • Nτ is a Poisson r.v., with parameter λτ :

k=3
(λτ )k e−λτ
• Random incidence P (k, τ ) = , k = 0, 1, . . .
k! y

• E [Nτ ] = var(N
First-order τ ) = λτ
interarrival times (k = 1):
exponential
• Interarrival times (k = 1): exponential:
fY1 (y) = λe−λy , y≥0
fT1 (t) = λe−λt, t ≥ 0, E[T1] = 1/λ
– Memoryless property: The time to the
next arrival is independent of the past
• Time Yk to kth arrival: Erlang(k):
λk y k−1e−λy
fYk (y) = , y≥0
(k − 1)!

Adding Poisson Processes

• Sum of independent Poisson random vari-

ables is Poisson
Poisson fishing Merging Poisson Processes (again)

• Assume: Poisson, λ = 0.6/hour. • •Sum of independent

Merging Poisson
of independent processes
Poisson processes
is is
Poisson
Poisson
– Fish for two hours.
Red bulb flashes
– if no catch, continue until first catch. (Poisson)

All flashes
"1
a) P(fish for more than two hours)= (Poisson)

"2
b) P(fish for more than two and less than
Green bulb flashes
five hours)=
(Poisson)

c) P(catch at least two fish)=

– –What
Whatis isthe
theprobability
probability that
that the next
the next
arrival comes from the first process?
arrival comes from the first process?

d) E[number of fish]=

e) E[future fishing time | fished for four hours]=

f) E[total fishing time]=

30
 Light bulb example Splitting of
Splitting of Poisson
Poisson processes
processes

• Each light bulb has independent, Assume

• • Each that email
message traffic through
is routed a server
along the first
exponential(λ) lifetime is a Poisson process.
stream with probability p
Destinations of different messages are
• Install three light bulbs. Routings of different messages are in-
– independent.
Find expected time until last light bulb dependent
dies out.

USA
Email Traffic
leaving MIT p!
MIT
Server
! (1 - p) !

Foreign

• Each output stream is Poisson

• Each output stream is Poisson.

Renewal processes
Random incidence for Poisson Random incidence in
• Series of“renewal processes”
successive arrivals
• Poisson process that has been running
forever – i.i.d. interarrival
• Series times
of successive arrivals
(but not necessarily exponential)
– i.i.d. interarrival times
• Show up at some “random time”
(but not necessarily exponential)
(really means “arbitrary time”)
• Example:
• Bus interarrival times are equally likely to
Example:
be
Bus or
5 10 minutes
interarrival times are equally likely to
x x x x x
Time be 5 or 10 minutes
• If you arrive at a “random time”:
Chosen
time instant • If you arrive at a “random time”:
– what is the probability that you selected
– awhat is the probability
5 minute interarrivalthat you selected
interval?
a 5 minute interarrival interval?
– what is the expected time to next ar-
what is the expected time
– rival?
• What is the distribution of the length of to next arrival?
the chosen interarrival interval?

31
 LECTURE 16 Checkout counter model

Markov Processes – I • Discrete time n = 0, 1, . . .

• Readings: Sections 7.1–7.2 • Customer arrivals: Bernoulli(p)

– geometric interarrival times

• Customer service times: geometric(q)

Lecture outline

• Checkout counter example • “State” Xn: number of customers at

time n
• Markov process definition

• n-step transition probabilities

• Classification of states

0 1 2 3 .. . 9 10

Finite state Markov chains n-step transition probabilities

• State occupancy probabilities,
• Xn: state after n transitions
given initial state i:
– belongs to a finite set, e.g., {1, . . . , m}
– X0 is either given or random rij (n) = P(Xn = j | X0 = i)

• Markov property/assumption: Time 0 Time n-1 Time n

(given current state, the past does not
1
matter) r i1(n-1) p 1j
...

pij = P(Xn+1 = j | Xn = i)
i k
= P(Xn+1 = j | Xn = i, Xn−1, . . . , X0) r ik(n-1)
p kj j
...

r im(n-1) p mj
• Model specification: m

– identify the possible states – Key recursion:

m
!
– identify the possible transitions
rij (n) = rik (n − 1)pkj
– identify the transition probabilities k=1

– With random initial state:

m
!
P(Xn = j) = P(X0 = i)rij (n)
i=1

32
 Example Generic convergence questions:
• Does rij (n) converge to something?
0.5 0.8

0.5 0.5 0.5

1 2 3
1 2 1 1

0.2 n odd: r2 2(n)= n even: r2 2(n)=

n=0 n=1 n=2 n = 100 n = 101 • Does the limit depend on initial state?
r11(n)

r12(n) 0.4

r21(n) 1 2 3 4
0.3 0.3

r22(n)
r1 1(n)=

r3 1(n)=

r2 1(n)=

Recurrent and transient states

• State i is recurrent if:

starting from i,
and from wherever you can go,
there is a way of returning to i

• If not recurrent, called transient

3 4

6 7
5

1 2
8

– i transient:
P(Xn = i) → 0,
i visited finite number of times

• Recurrent class:
collection of recurrent states that
“communicate” with each other
and with no other state

33
 LECTURE 17 Review

Markov Processes – II
• Discrete state, discrete time, time-homogeneous
• Readings: Section 7.3 – Transition probabilities pij
– Markov property

Lecture outline
• rij (n) = P(Xn = j | X0 = i)
• Review
• Key recursion:
• Steady-State behavior !
rij (n) = rik (n − 1)pkj
k
– Steady-state convergence theorem

– Balance equations

• Birth-death processes

Warmup Periodic states

9 3 4 • The states in a recurrent class are

6 7
periodic if they can be grouped into
d > 1 groups so that all transitions from
5 1 2
8
one group lead to the next group

P(X1 = 2, X2 = 6, X3 = 7 | X0 = 1) =

P(X4 = 7 | X0 = 2) =

Recurrent and transient states

• State i is recurrent if: 9

starting from i, 5
4
and from wherever you can go,
there is a way of returning to i 6 8

3
• If not recurrent, called transient

• Recurrent class: 2 1 7
collection of recurrent states that
“communicate” to each other
and to no other state

34
 Expected Frequency of a Particular Transition
Consider n transitions of a Markov chain with a single class which is aperi-
odic, starting from a given initial state. Let qjk (n) be the expected number
of such transitions that take the state from j to k. Then, regardless of the
Steady-State Probabilities Visit frequency interpretation
initial state, we have
qjk (n)
• Do the rij (n) converge to some πj ? lim = πj pjk .
(independent of the initial state i)
n→∞ n!
πj = πk pkj
• Yes, if: k
Given the frequency interpretation of πj and πk pkj , the balance equation
– recurrent states are all in a single class,
m
! of being in j: πj
and • (Long run) frequency
πj = πk pkj
– single recurrent class is not periodic k=1
• Frequency
has an intuitive meaning. of transitions
It expresses the factk →
that
j: the
πk pexpected
kj frequency π
of visits to j is equal to the sum of the expected frequencies πk pkj of transition
• Assuming “yes,” start from key recursion
that lead to j; see• Fig. 7.13.
Frequency of transitions into j:
!
π p
! k kj
rij (n) = rik (n − 1)pkj k
k
1 π j pjj
– take the limit as n → ∞ π 1 p1j
!
πj = πk pkj , for all j
k 2
π 2p2j j
– Additional equation:

. . .
. . .
!
πj = 1
j π m pmj
m

Figure 7.13: Interpretation of the balance equations in terms of frequencies. In

a very large number of transitions, we expect a fraction πk pkj that bring the state
from k to j. (This also applies to transitions from j to itself, which occur with
frequency πj pjj .) The sum of the expected frequencies of such transitions is the
expected frequency πj of being at state j.

Example Birth-death processes

† In fact, some stronger statements are also true, such as the following. Wheneve
we carry out a probabilistic
1- p0 1experiment
- p1- q1 and generate a trajectory
1 - q m of the Markov chai

0.5 0.8 over an infinite time horizon, the observed long-term frequency with which state j
p p

0.5 from j to k will be exactly equal to πj p2 jk . Even

0 1 3 .. .
visited will be exactly equal0 to πj , 1and the observed long-term frequency of transition
m
though the trajectory is random, thes
equalities hold with essentialq 1 certainty,
q2 that is, with probability
q m 1.
1 2 pi

0.2 i i+1 πipi = πi+1qi+1

q i+1

• Special case: pi = p and qi = q for all i

ρ = p/q =load factor
p
πi+1 = πi = πiρ
q

π i = π 0 ρi , i = 0, 1, . . . , m

• Assume p < q and m ≈ ∞

π0 = 1 − ρ
ρ
E[Xn] = (in steady-state)
1−ρ

35
 LECTURE 18 Review
LECTURE 20 Review

Markov Processes – III • Assume a single class of recurrent states,

Markov Processes – III • Assume a single class of recurrent states,
aperiodic;
aperiodic. Then,
plus transient states. Then,

lim (n)
lim rrijij(n )==ππj j
Readings: Section 7.4 n→∞
n→∞
Readings: Section 6.4 where
where ππjj does not depend
does not dependon
onthe
theinitial
initial
conditions
conditions:
Lecture outline lim P(X = =
lim P(X =0)i)=
j | jX|0X
n n =ππjj
n→∞
n→∞
Lecture outline
• Review of steady-state behavior
• Review of steady-state behavior •• ππ11,,......,, ππm can
m can be
be found
found as
as the
theunique
unique
• Probability of blocked phone calls
solution
solution of the to the balance equations
balance equations
• Probability of blocked phone calls !
• Calculating absorption probabilities πj = πk pkj , ! j = 1, . . . , m,
πj = πk pkj
k
Calculating
• • Calculating absorption
expected timeprobabilities
to absorption k
together with
together with
!
• Calculating expected time to absorption !πj = 1
j πj = 1
j

Example The phone company problem

Example The phone company problem
0.5 0.8
0.5 0.8 • Calls originate as a Poisson process,
• Calls originate as a Poisson process,
rate
rate λλ
0.5
0.5
1 2 –– Each
Each call duration isis exponentially
call duration exponentially
1 2 distributed (parameter
distributed (parameter µ) µ)
0.2
0.2 B lines
–– B lines available
available

π1 = 2/7, π2 = 5/7
π1 = 2/7, π2 = 5/7 Discrete time intervals
•• Discrete intervals
• Assume process starts at state 1. of (small)
of (small) length
length δδ
• Assume process starts at state 1.

"#
• P(X1 = 1, and X100 = 1)=
• P(X1 = 1, and X100 = 1)= 0 1 i!1 i B-1 B

iµ#

• • PP(X100
(X = 1 and X101 = 2)
100 = 1 and X101 = 2) •• Balance
Balance equations:
equations: λπ i−1==iµπ
λπi−1 iµπi i

B
λi !B λ i
i
• πi = π0 λii π0 = 1/ ! λ
• πi = π0 µi i! π0 = 1/i=0 µ i!i
i
µ i! i=0 µ i!

36
 Calculating absorption probabilities Expected time to absorption

• What is the probability ai that:

process eventually settles in state 4,
1
given that the initial state is i? 3 0.5

0.5 44
1 0.4
54
0.6 0.2
0.2 1 2
1
3 0.3 0.8

0.5 44
0.4
• Find expected number of transitions µi,
0.6 0.2
until reaching the absorbing state,
1 2
given that the initial state is i?
0.8

For i = 4, ai =
For i = 5, ai = µi = 0 for i =

!
! For all other i: µi = 1 + pij µj
ai = pij aj , for all other i
j
j
– unique solution
– unique solution

Mean first passage and recurrence

times

• Chain with one recurrent class;

fix s recurrent

• Mean first passage time from i to s:

ti = E[min{n ≥ 0 such that Xn = s} | X0 = i]

• t1, t2, . . . , tm are the unique solution to

ts = 0,
!
ti = 1 + pij tj , for all i =
% s
j

• Mean recurrence time of s:

t∗s = E[min{n ≥ 1 such that Xn = s} | X0 = s]

"
• t∗s = 1 + j psj tj

37
 LECTURE 19 Chebyshev’s inequality
Limit theorems – I
• Random variable X
• Readings: Sections 5.1-5.3; (with finite mean µ and variance σ 2)
start Section 5.4 !
σ2 = (x − µ)2fX (x) dx
! −c ! ∞
• X1, . . . , Xn i.i.d. ≥ (x − µ)2fX (x) dx + (x − µ)2fX (x) dx
−∞ c
X1 + · · · + Xn
Mn =
n ≥ c2 · P(|X − µ| ≥ c)
What happens as n → ∞?

• Why bother? σ2
P(|X − µ| ≥ c) ≤
c2
• A tool: Chebyshev’s inequality

• Convergence “in probability”

1
P(|X − µ| ≥ kσ ) ≤
• Convergence of Mn k2
(weak law of large numbers)

Deterministic limits Convergence “in probability”

• Sequence an • Sequence of random variables Yn

Number a
• converges in probability to a number a:
“(almost all) of the PMF/PDF of Yn ,
• an converges to a eventually gets concentrated
lim a = a (arbitrarily) close to a”
n→∞ n
“an eventually gets and stays
(arbitrarily) close to a” • For every " > 0,

lim P(|Yn − a| ≥ ") = 0

n→∞
• For every " > 0,
there exists n0,
such that for every n ≥ n0,
we have |an − a| ≤ ". 1 - 1 /n pmf of Yn

1 /n

0 n
Does Yn converge?

38
 Convergence of the sample mean The pollster’s problem
(Weak law of large numbers)
• f : fraction of population that “. . . ”
• X1, X2, . . . i.i.d.
• ith (randomly selected) person polled:
finite mean µ and variance σ 2 
1, if yes,
X + · · · + Xn
Mn = 1 Xi =
0, if no.
n

• Mn = (X1 + · · · + Xn)/n
• E[Mn] = fraction of “yes” in our sample

• Goal: 95% confidence of ≤1% error

• Var(Mn) = P(|Mn − f | ≥ .01) ≤ .05

• Use Chebyshev’s inequality:

Var(Mn) σ2 2
σM
P(|Mn − µ| ≥ ") ≤ = P(|Mn − f | ≥ .01) ≤ n
"2 n"2 (0.01)2
σx2 1
• Mn converges in probability to µ = ≤
n(0.01)2 4n(0.01)2

• If n = 50, 000,
then P(|Mn − f | ≥ .01) ≤ .05
(conservative)

Different scalings of Mn The central limit theorem

• X1, . . . , Xn i.i.d. • “Standardized” Sn = X1 + · · · + Xn:

finite variance σ 2 Sn − E[Sn] Sn − nE[X]
Zn = = √
σSn nσ
• Look at three variants of their sum:
– zero mean
• Sn = X1 + · · · + Xn variance nσ 2
– unit variance

Sn • Let Z be a standard normal r.v.

• Mn = variance σ 2/n
n (zero mean, unit variance)
converges “in probability” to E[X] (WLLN)

Sn • Theorem: For every c:

• √ constant variance σ 2
n
P(Zn ≤ c) → P(Z ≤ c)
– Asymptotic shape?

• P(Z ≤ c) is the standard normal CDF,

Φ(c), available from the normal tables

39
 LECTURE 20 Usefulness
THE CENTRAL LIMIT THEOREM • universal; only means, variances matter
• accurate computational shortcut
• Readings: Section 5.4
• justification of normal models
• X1, . . . , Xn i.i.d., finite variance σ 2

• “Standardized” Sn = X1 + · · · + Xn:
What exactly does it say?
Sn − E[Sn] Sn − nE[X]
Zn = = √ • CDF of Zn converges to normal CDF
σ Sn nσ
– not a statement about convergence of
– E[Zn] = 0, var(Zn) = 1 PDFs or PMFs

• Let Z be a standard normal r.v.

(zero mean, unit variance) Normal approximation

• Theorem: For every c: • Treat Zn as if normal

– also treat Sn as if normal
P(Zn ≤ c) → P(Z ≤ c)

• P(Z ≤ c) is the standard normal CDF,

Φ(c), available from the normal tables
Can we use it when n is “moderate”?
• Yes, but no nice theorems to this effect
• Symmetry helps a lot

0.14 0.1
n =2 n =4
0.12
0.08
0.1

0.08 0.06

0.06
0.04
The pollster’s problem using the CLT
0.04

0.02
0.02
• f : fraction of population that “ . . .��
0 0
0 5 10 15 20 0 5 10 15 20 25 30 35

• ith (randomly selected) person polled:

0.25 0.035
n =32

0.2
0.03 1, if yes,
0.025 Xi =
0.15
0.02
0, if no.
0.1 0.015

0.01
0.05
0.005
• Mn = (X1 + · · · + Xn)/n
0 0

• Suppose we want:
0 2 4 6 8 100 120 140 160 180 200

P(|Mn − f | ≥ .01) ≤ .05

0.12 0.08

n = 16
0.07
n=8

• Event of interest: |Mn − f | ≥ .01

0.1

0.06

0.08
0.05

� �
� X1 + · · · + Xn − nf �
0.06 0.04

� �
� ≥ .01
0.03
0.04
�
0.02
0.02
n
0.01

0 0
0 5 10 15 20 25 30 35 40 0 10 20 30 40 50 60 70
� � √
1 0.06 � X + · · · + X − nf � .01 n
� 1 n �
0.9 n = 32
� √ � ≥
� �
0.05
0.8

0.7
nσ σ
0.04
0.6

0.5 0.03

0.4

0.3
0.02
√
0.2
0.01 P(|Mn − f | ≥ .01) ≈ P(|Z| ≥ .01 n/σ)
0.1
√
0
0 1 2 3 4 5 6 7
0
30 40 50 60 70 80 90 100 ≤ P(|Z| ≥ .02 n)

40
 Apply to binomial The 1/2 correction for binomial
approximation
• Fix p, where 0 < p < 1
• P(Sn ≤ 21) = P(Sn < 22),
• Xi: Bernoulli(p) because Sn is integer

• Sn = X1 + · · · + Xn: Binomial(n, p) • Compromise: consider P(Sn ≤ 21.5)

– mean np, variance np(1 − p)

Sn − np
• CDF of � −→ standard normal
np(1 − p)

Example
18 19 20 21 22
• n = 36, p = 0.5; find P(Sn ≤ 21)

• Exact answer:
21 � � �
� 36� 1 36
= 0.8785
k=0
k 2

De Moivre–Laplace CLT (for binomial) Poisson vs. normal approximations of

• When the 1/2 correction is used, CLT the binomial
can also approximate the binomial p.m.f.
(not just the binomial CDF) • Poisson arrivals during unit interval equals:
sum of n (independent) Poisson arrivals
P(Sn = 19) = P(18.5 ≤ Sn ≤ 19.5) during n intervals of length 1/n
– Let n → ∞, apply CLT (??)
18.5 ≤ Sn ≤ 19.5 ⇐⇒
– Poisson=normal (????)
18.5 − 18 Sn − 18 19.5 − 18
≤ ≤ ⇐⇒
3 3 3
0.17 ≤ Zn ≤ 0.5 • Binomial(n, p)
– p fixed, n → ∞: normal
P(Sn = 19) ≈ P(0.17 ≤ Z ≤ 0.5)
– np fixed, n → ∞, p → 0: Poisson

= P(Z ≤ 0.5) − P(Z ≤ 0.17) • p = 1/100, n = 100: Poisson

= 0.6915 − 0.5675 • p = 1/10, n = 500: normal

= 0.124

• Exact answer:
�36� � 1 �36
= 0.1251
19 2

41
 Θ

Θ Θ
pΘ(θ)
Θ pΘ(θ)
Θ pΘ(θ)
N
Θ Θ
pΘ(θ) N
Θ pΘ(θ) Θ Θ N
LECTURE 21 Types of Inference models/approaches
pX|Θ(x | θ)
Sample Applications pΘ(θ)pΘ(θ)
• Model building
pΘ(θ) versus
p
N
(x inferring
| θ) unkno
pX|Θwn
(x | θ)
• Readings: Sections• 8.1-8.2 pΘ(θ) N pΘ(θ) X|Θ X
Polling N N variables. E.g., assume X = aS + W
p (x | θ)
– Design of experiments/sampling – ModelNbuilding:
pX|Θ(x | θ) methodologies N X X|Θ X
“It is the mark of truly educated people N
| θ)(x | know
Θ̂
Lancet study on Iraq death toll pX|Θ(x “signal” S, observe X, infer a
to be deeply moved by–statistics.”
pX|Θ θ)
ΘΘ X Θ̂
X – Estimationp pX|Θ
(x |in
(x
θ) | the
θ)Θ̂ presence
Θ of
Estimator noise:
(Oscar Wilde) pX|Θ(x | θ) pX (x;Xθ) X|Θ
• Medical/pharmaceutical trials pknow a, observe X, Θ̂ estimate S. Θ
X
Θp(θ)
Θ (θ) Estimator Estimator
Θ̂ X pΘ(θ)
• Data mining X Θ̂
• Hypothesis testing: unknown
X takes one of
Reality Model Θ̂
N N possible values;
few Θ
W ∼ aim at small
Estimator pΘ(θ)
(e.g., customer arrivals)
–(e.g.,Netflix
Poisson)
competition Estimator Θ̂ Θ̂ fW (w)
N Θ ∈ {0, 1} X =Θ+W
Θ̂ Estimator probability
Estimator of incorrect decision
Θ pX|Θ (x(x| θ) pΘ(θ) N
Data • Finance pX|Θ
W ∼ fW (w) • ΘEstimation:
∈ {0,
| θ)
Θ X = aim
Estimator
1} Estimator Θ+W at a small
pX|Θ(xestimation
| θ) error
Estimator W ∼ fW W (w)
∼ fW (w)Θ ∈ {0,
Θ 1}
∈ {0, 1}X = Θ
X+=WΘ+W
• Design &Θ̂interpretation of experiments pΘ(θ) XX pΘ(θ)
N pX|Θ(x | θ)
• Classical statistics: X
– polling,
Matrix Completion Θ̂
pmedical/pharmaceutical
Y |X (y | x)
trials.
W ∼ .f.W (w) Θ ∈ {0, 1} =Θ̂Θ + W
X Θ̂ pX|Θ(x |Θθ) X
Θ̂ N N Θ̂
• Netflix competition
Y |X (·to
! Partially observed matrix:pgoal predict the
| ·) • Finance Θ
X̂ Estimator pΘ(θ)
unobserved entriesΘ̂ pY |X (y | x) fΘ(θ) 1/6 4 10 θ
Estimator X Θ̂
pX|Θ(x | θ) (x; (x
pX|Θ
pX θ) | θ) Θ Estimator
objects X̂ θ N pΘ(θ)
2 1 pY 4|X (y |5x) X̂ Θ̂ N
5 4 ? 1 3 Estimator
4
3 5
?
2
5 3 ?
θ measurement
pY (y ; θ)
Y Θ̂ X θ: unknownXparameter pΘ(not
(θ) a r.v.) N
Graph of S&P 500 index removed Θ Θ
Estimator
4 1X̂3 5 θ pX|Θ(x | θ)
2 1? 4 Θ̂
due to copyright restrictions. ◦ E.g., θ =
Θ̂
mass of electron
Estimator
sensors

1 5 5 4
2 ?5 ? 4 Θ̂ N pX|Θ(x | θ)
3 3 θ1 5 2 1 pY (y ; θ) pΘ(θ)
Θ p (θ)
3
4
1
51
2
3
3
Θ̂ N
pY |X (· | ·) • Bayesian: Use priors &XBayes rule
3 3? 5 Estimator
2? 1 p1Y (y ; θ) pX (x) Estimator pX|Θ(x | θ) X
1
5
3
2 ? X4 4
1 5 4 5pY |X (y | x)
Y X̂ N N Θ̂
Estimator
1 2 4 5?
pX (x) X ∈ {0, 1} ΘΘ X Θ̂
W ∼ fW (w) pΘ p∈(x
{0,
|(x1}| θ)
θ) X = Estimator
Θ+W
X̂ X|ΘX|Θ
• Signal processing
W ∼ fW (w) pΘp(θ)
Θ(θ) Θ̂ Estimator
θ
– Tracking, detection, speaker identification,. . . fΘ(θ) 1/6 X X4 10 θ
Y =X +W
NN Estimator
pY (y ; θ)
Θ̂ Θ̂
pX|Θ
pX|Θ(x(x
| θ)
| θ)
Estimator
Estimator
XX

Θ̂Θ̂

Estimator
Estimator
Bayesian inference: Use Bayes rule Estimation with discrete data
• Hypothesis testing
– discrete data fΘ(θ) pX |Θ(x | θ)
fΘ|X (θ | x) =
pΘ(θ) pX |Θ(x | θ) pX (x)
pΘ|X (θ | x) = !
pX (x)
pX (x) = fΘ(θ)pX |Θ(x | θ) dθ

– continuous data
pΘ(θ) fX |Θ(x | θ)
pΘ|X (θ | x) = • Example:
fX (x)
– Coin with unknown parameter θ
– Observe X heads in n tosses
• Estimation; continuous data
• What is the Bayesian approach?
fΘ(θ) fX |Θ(x | θ) – Want to find fΘ|X (θ | x)
fΘ|X (θ | x) =
fX (x) – Assume a prior on Θ (e.g., uniform)
Zt = Θ0 + tΘ1 + t2Θ2
Xt = Zt + Wt, t = 1, 2, . . . , n
Bayes rule gives:

fΘ0,Θ1,Θ2|X1,...,Xn (θ0, θ1, θ2 | x1, . . . , xn)

42
 pX|Θ(x | W θ)
Θp(θ
NpΘp(θ)Θ)(θ) pX (x) = f ( ) pf W (w)
X | (x | ) d
p
pX (x) X|Θ (x | θ) W pW ( w )
pΘ(θ)
pN N (x
N | θ) X Y = X + W
+
X|Θ
EXxaX 0 , p1Y} ( y ; ) Y = X + W
m{ ple:
N
X pXp|(x
pX|Θ Θ
X|Θ (x (x
| θ)| θ|) θ) Θ̂ object at unknown location X Θ
W Θ̂p W ( w )
Output of Bayesian Inference sensors
Least Mean Squares Estimation
pX|Θ(x | θ)
Θ̂
XX X Estimator pΘ(θ)
Y = X + W
• Posterior distribution: Estimator in the absence of information
• Estimation
X N Θ+W
Θ̂ Θ̂ Θ̂
Estimator W ∼ fW (w) Θ ∈ {0, 1} X=
– pmf pΘ|X (· | x) or pdf fΘ|X (· | x)
W ∼ fW (w) Θ ∈ {0, 1} X =Θ+W
Θ̂ pX|Θ(x | θ)
W E sEstimator
ft W
Estimator
∼ i m(w)
a t or Θ| (θ)
p X | (f1Θ = 1}1/6 X =4Θ + W10
∈) {0, θ
f=ΘP(θ)
(se nsor i1/6 “ se nses ” t4 h e o b j e c10
t | = ) θX
Estimator
Θ (θ)
fW W∼W ∼fW∼ f(w) (w
W (w)
Wf1/6 ) = hΘ( disΘ∈4t aΘ
{0, {e1}
∈n c∈ 1 } 1}
0,o{0,
f10 fr o X =
m X
se XΘ==
θnsor Θ+ i )Θ
W++ WW
Θ̂
W fW (w) Θ {0, 1} X =Θ+W
• If interested in a single answer: Θf(θ
fΘf(θ)Θ)(θ) 1/6
1 /1/6
6 4 4 4 101 010 θ θ θ Estimator
– Maximum a posteriori probability (MAP): fΘ(θ) 1/6 4 10 θ
• find estimate c, to:
◦ pΘ|X (θ∗ | x) = maxθ pΘ|X (θ | x) " #
minimizes probability of error; minimize E (Θ − c)2
often used in hypothesis testing
• Optimal estimate: c = E[Θ]
◦ fΘ|X (θ∗ | x) = maxθ fΘ|X (θ | x)
• Optimal mean squared error:
– Conditional expectation:
! " #
E (Θ − E[Θ])2 = Var(Θ)
E[Θ | X = y] = θfΘ|X (θ | x) dθ

– Single answers can be misleading!

LMS Estimation of Θ based on X LMS Estimation w. several measurements

• Two r.v.’s Θ, X • Unknown r.v. Θ

• we observe that X = x
• Observe values of r.v.’s X1, . . . , Xn
– new universe: condition on X = x
" #
• Best estimator: E[Θ | X1, . . . , Xn]
• E (Θ − c)2 | X = x is minimized by
• Can be hard to compute/implement
c=
" # – involves multi-dimensional integrals, etc.
• E (Θ − E[Θ | X = x])2 | X = x

≤ E[(Θ − g(x))2 | X = x]

" # " #
◦ E (Θ − E[Θ | X])2 | X ≤ E (Θ − g(X))2 | X

" # " #
◦ E (Θ − E[Θ | X ])2 ≤ E (Θ − g(X))2

" #
E[Θ | X] minimizes E (Θ − g (X))2
over all estimators g(·)

43
 N
Xp pXp|(x (x (x
| θ)| θ|) θ) Θ̂ object at unknown location X
W ∼ fW (w) Θ ∈X|Θ{0, ΘX|Θ
1} WXΘ̂ =p WΘ( w +) W Θ
pΘ(θ) sensors
pX|Θ(x | θ)
Θ̂
XX X Estimator pΘ(θ)
fΘ(θ) fΘ(θ) 1/6 4 10 Y = Xθ + W
fΘ(θ) Estimator
x
X 10
N Θ̂ Θ̂ Θ̂
Estimator W ∼ fW (w) Θ ∈ {0, 1} X= N Θ+W
4 fΘ(θ) f f(θ) (x | θ) x W ∼ fW (w) Θ ∈ {0, 1} X =Θ+W
fX|Θ(x | θ) Θ X|Θ pX|Θ(x | θ)
Θ̂
LECTURE
Θ 22 W E sEstimator
ft W
Estimator
∼ i m(w)
a t or Θ| (θ)
p X | (f1Θ = 1}1/6 X =4Θ + W10
∈) {0, θ
fX|Θ(xf | g( ) | θ)
θ)·(x x pX|Θ(x | θ) fΘ(θ) f=ΘP(θ) 1/6
fΘ(θ)
4 10 θ
g( · ) f (θ)3 x (se nsor i “ se nses ” t h e o b j e c t | = )X
X|Θ9 11 Estimator
Θ 5 y
• Readings: pp. 225-226; pSections
Θ (θ) 8.3-8.4 Θ (θ)
fW W∼W ∼fW∼ f(w) (w
Wf1/6 )4= hΘ
(w) Θ∈
( dis4t aΘ
{0, {ef1}
∈n c∈ 0, f10
o{0, } 1}
1(θ) frfo X =
fmX|Θ
se
X X ==
Θ
θnsor
(x +
|Θi )Θ
θ) W++WxW
W
fX|Θ(x | θ) Θ Θ (θ) Θ̂
g( · ) g( Θ̂
· ) = g(X) X
Θ̂ = fg(X)(x | θ) W fW (w) Θ {0, 1} X = Θ +ΘW
Θ Θf(θ
Θ)(θ) g( (x 0|10 )θ| θ
·1)/1/6
6 4 4 f4X|Θ 1fX|Θ
X|Θ fΘf(θ) 1/6 10 θ)·(x
g( x θ
θ) x
N Estimator
fΘ(θ)3 5 9 11 y
1/2 g( · ) Topics
Θ̂ = g(X)
Θ̂ 1/2
= g(X) fΘ(θ) 1/6 4 10· ) pΘ(θ)
θ = g(X)
pΘ(θ) p
X|Θ (x | θ)
Θ̂ fΘ(θ) Θ̂ = fg(X)(x | θ)
g( Θ̂
g( · )
Θ
• (Bayesian) Least1/2 means
θ − 1 squares (LMS)
X |Θ
N
θ − 1Θ̂ = g(X) 1/2 Θ̂ = g(X)
N fX|Θ(x | θ) Θ̂ 1/2
= g(X)
estimation X 1/2 g( · )
pΘ(θ) p fΘ(θ)
fΘ(θ) Estimator X|Θ (x | θ)
θ−1 − +1
θ1 1/2 θ−1
• θ + 11/2
(Bayesian) Linear θLMS estimation
pX|Θ(x |
Θ̂ θ) g( · ) θ − 1Θ̂ = g(X) N
1/2 fX|Θ(x | θ)
fΘ(θ) X
fX|Θ(x | θ) fΘ(θ)
θ+1 θ+1 W ∼ fW (w) Θ ∈ {0, 1} X = Θθ + W
+ 11/2
θ − 1 θ + 1
θ −p 1 (x | θ) g( · )
Θθ −1 X Estimator Θ̂ = g(X) X|Θ Θ̂
fX |Θ(x | θ) fΘ(θ)
g( · ) fX|Θ(x | θ)
θ+1 θ+1
fΘ(θ) Θθ −1 X Θ̂ = g(X)
pΘθ(θ) Θ̂ fΘ(θ) 1/6 4 10 θ Estimator
+1 fX|Θ(x | θ)
g( · )
g( · ) x fΘ(θ)
Θ̂ = g(X) 10 pΘθ(θ) Θ̂
fX|Θ
N (x | θ) Estimator +1
g( · )
• MAP estimate: Θ̂ = g(X)
Θ̂ = g(X)θ̂MAP maximizes fΘ|X (θ | x) fX|Θ
N (x | θ) Estimator
g(p· ) (x | θ) Θ̂ = g(X)
• LMSX|Θ
estimation: g(p· )
X|Θ (x | θ)
� � fΘ(θ)
Θ̂X
– Θ̂ = E[Θ | X] minimizes E (Θ − g(X ))2
=g(X) fΘ(θ) Θ̂X
= g(X)
over all estimators g(·)
Θ̂ 4 Θ̂ fΘ(θ) f f(θ) (x | θ) x
fX|Θ(x | θ) Θ X|Θ
– for any x, θ̂ �= E[Θ | X = x] �
Estimator Θ
minimizes E (Θ − θˆ)2 | X = x Estimator
fX|Θ(xf | g( ) | θ)
θ)·(x x x
over all estimates θ̂ g( · ) f X|Θ9
Θ θ3 5 11 y
pΘ(θ)
g( · ) g( Θ̂
· ) = g(X)
Θ̂ = fg(X) x | θ
X|Θ Θ N
Θ̂ = g(X)
Θ̂ 1/2
= g(X)
1/2 g · fΘ(θ)
pΘ(θ) p
X|Θ (x | θ)
x | θ) 1/2 θ−1
1/2
θ − 1Θ g X N fX|Θ(x | θ)
X
fΘ(θ)
θ−1 θ− θ1+1
θ+1 / pX|Θ(xΘ̂| θ) g( · )
Conditional mean squared error fΘ(θ)
Some properties of LMS estimation
fX|Θ(x | θ)
θ+1 θ+1
mator • E[(Θ − E[Θ | X ])2 | X = x] Θ θ − Θ̂ = E[Θ | X]
– Estimator:
X Estimator Θ̂ = g(X)
fX!Θ(x ! θ)
g( · )
– same as Var(Θ | X = x): variance of the fΘ(θ)
– Estimation error: Θ̃ = Θ̂Θ̂− Θ
fW (w) Θ ∈ {0, 1} X =Θ+W pΘθ(θ)
conditional distribution of Θ g( " )
Θ̂ = g(X)
1/6 4 10 fX|Θ (x | θ)
10
θ
x • E[Θ̃]N= 0 E[Θ̃ | XEstimator
= x] = 0
Θ̂ = g(X)
g(p· ) (x | θ)
X|Θ
• E[Θ̃h(X)] = 0, for any function h
Θ̂X= g(X)
• cov(Θ̃, Θ̂) = 0
4
fΘ(θ)
• SinceΘ̂Θ = Θ̂ − Θ̃:
Conditional mean (x | θ) error
squared
fX|Θ x
3 5 9 11 y var(Θ) = var(Θ̂) + var(Θ̃)
• E[(Θ − E[Θ | X])2 | X = x] Estimator
x g( · )
10 as Var(Θ | X = x): variance of the
– same
conditional distribution of Θ
Θ̂ = g(X)

1/2

4
−(θ)
θfΘ 1

+ 1(x | θ)
θfX|Θ x
3 5 9 11
Predicting X based on Yy
• Two r.v.’s X, Y g( · )
• we observe that Y = y
Θ̂ =
– new universe: condition ong(X)
Y =y
� �
• E (X − c)2 | Y = y is minimized
1/2 by
c= 44
 Linear LMS Linear LMS properties

• Consider estimators of Θ, Cov(X, Θ)

ˆ = aX + b
of the form Θ Θ̂L = E[Θ] + (X − E[X])
var(X)
� �
• Minimize E (Θ − aX − b)2 E[(Θ̂L − Θ)2] = (1 − ρ2)σΘ
2

• Best choice of a,b; best linear estimator:

Linear LMS with multiple data
∈ {0, 1} X =Θ̂Θ +W Cov(X, Θ)
L = E[Θ] + (X − E[X])
var(X) • Consider estimators of the form:

4 10 θ Θ̂ = a1X1 + · · · + anXn + b
x
10
• Find best choices of a1, . . . , an, b
• Minimize:

E[(a1X1 + · · · + anXn + b − Θ)2]

fΘ(θ)
fΘ(θ)
• Set derivatives to zero
4 fΘ(θ) f f(θ) (x | θ) x linear system in b and the ai
fX|Θ(x | θ) Θ X|Θ
• Only means, variances, covariances matter
Θ
fX|Θ(xf | g( ) | θ)
θ)·(x x x
g( · ) f X|Θ9
Θ θ3 5 11 y
pΘ(θ)
g( · ) g( Θ̂
· ) = g(X)
Θ̂ = fg(X) x | θ
X|Θ Θ N
Θ̂ = g(X)
Θ̂ 1/2
= g(X)
1/2 g · fΘ(θ)
pΘ(θ) p
X|Θ (x | θ)
1/2 θ−1
1/2
θ − 1Θ g X N fX|Θ(x | θ)
X
fΘ(θ)
The cleanest linear LMS example Big picture
θ−1 θ− θ1 +1
θ+1 / pX|Θ(xΘ̂| θ) g( · )
• Standard examples:
Xi = Θ + Wf i ,Θ (θ)Θ, W1 , . . . , Wn independent
fX|Θ(x | θ)
Θ ∼ µ, σ0 2 Wiθ +
∼ 10, θσ+ 2
i X1
Θθ − Estimator Θ̂ = g(X)– Xi uniform on [0, θ];
fX!Θ(x ! θ) �n
uniform prior on θ
µ/σ02 + 2
Xi/σi g( · )
fΘ(θ)
pΘθ(θ) Θ̂L = i=1Θ̂
n
� – Xi Bernoulli(p);
g( " ) 1/σi2 Θ̂ = g(X) uniform (or Beta) prior on p
N (x | θ)
fX|Θ i=0 Estimator
(weighted average of µ, X1, . . . , Xn)
Θ̂ = g(X) – Xi normal with mean θ, known variance σ 2;
· ) normal, normal prior on θ;
• g(
If pall (x | θ)Θ̂L = E[Θ | X1, . . . , Xn]
X|Θ X i = Θ + Wi

Θ̂X
= g(X)
• Estimation methods:
Choosing Xi in linear LMS – MAP
Θ̂
• E[Θ | X] is the same as E[Θ | X 3] – MSE
Estimator
• Linear LMS is different: – Linear MSE
◦ Θ̂ = aX + b versus Θ̂ = aX 3 + b
◦ Also consider Θ̂ = a1X + a2X 2 + a3X 3 + b

45
 X
pΘp(θ)
(θ) Θ̂ Θ
Estimator
Θ Estimator
Θ̂ X X pΘ(θ)
X Θ̂
Θ̂
NN ΘEstimator pΘ(θ)
Estimator Θ̂ Θ̂ W ∼ fW (w)
N Θ ∈ {0, 1} X =Θ+W
Θ̂ Estimator
Estimator
Θ p (x | θ)
pX|Θ(x | θ)
X|Θ pΘ(θ) N
W ∼ fW (w) Θ ∈ {0, ΘEstimator
1} Estimator
X =Θ+W pX|Θ(x | θ)
Estimator W ∼ fWW (w)
∼ f W (w)Θ ∈ {0,
Θ 1}
∈ {0, X
1} = Θ
X +
= WΘ + W
LECTURE 23 pΘ(θ) XX Classical
pΘ(θ)
N
statistics pX|Θ(x | θ)
X
W ∼ fW (w) Θ ∈ {0, 1} =Θ̂Θ + W
X Θ̂ pX|Θ(x | θ) X
N N
• Readings: Section 9.1 Θ̂
(not responsible for t-based confidence
fΘ(θ) 1/6 4 10 Estimator
θ
Estimator X Θ̂
pX|Θ(x | θ) (x; (x
pX|Θ
pX θ) | θ) Estimator
intervals, in pp. 471-473)
Θ̂ Estimator
X
X • also for vectors X and θ:
• Outline Estimator
pX1,...,Xn (x
Θ̂1 , . . . , xn ; θ1 , . . . , θm )
– Classical statistics Θ̂

– Maximum likelihood (ML) estimation • These areEstimator

NOT conditional probabilities;
Estimator
θ is NOT random
– Estimating a sample mean
W ∼– mathematically:
fW (w) Θ ∈ {0, 1} many
X=Θ models,
+W
– Confidence intervals (CIs)
one for each possible value of θ
– CIs using an estimated variance fΘ(θ) 1/6 4 10 θ

• Problem types:
– Hypothesis testing:
H0 : θ = 1/2 versus H1 : θ = 3/4
– Composite hypotheses:
H0 : θ = 1/2 versus H1 : θ =
� 1/2
– Estimation: design an estimator Θ̂,
to keep estimation error Θ̂ − θ small

Maximum Likelihood Estimation Desirable properties of estimators

(should hold FOR ALL θ !!!)
• Model, with unknown parameter(s):
X ∼ pX (x; θ)
• Unbiased: E[Θ̂n] = θ
• Pick θ that “makes data most likely”
– exponential example, with n = 1:
θ̂ML = arg max pX (x; θ) E[1/X1] = ∞ �= θ
θ
(biased)
• Compare to Bayesian MAP estimation:
• Consistent: Θ̂n → θ (in probability)
θ̂MAP = arg max pΘ|X (θ | x)
θ – exponential example:
pX |Θ(x|θ)pΘ(θ) (X1 + · · · + Xn)/n → E[X] = 1/θ
θ̂MAP = arg max
θ pX (x) – can use this to show that:
Θ̂n = n/(X1 + · · · + Xn) → 1/E[X] = θ
• Example: X1, . . . , Xn: i.i.d., exponential(θ)
n • “Small” mean squared error (MSE)
�
max θe−θxi
θ i=1
E[(Θ̂ − θ)2] = var(Θ̂ − θ) + (E[Θ̂ − θ])2
� �
= var(Θ̂) + (bias)2
n
�
max n log θ − θ xi
θ
i=1

n n
θ̂ML = Θ̂n =
x1 + · · · + xn X1 + · · · + X n

46
 Estimate a mean Confidence intervals (CIs)

• X1, . . . , Xn: i.i.d., mean θ, variance σ 2 • An estimate Θ̂n may not be informative
enough
Xi = θ + W i
• An 1 − α confidence interval
Wi: i.i.d., mean, 0, variance σ 2 is a (random) interval [Θ̂− +
n , Θ̂n ],
X1 + · · · + X n +
Θ̂n = sample mean = Mn = s.t. P(Θ̂−
n ≤ θ ≤ Θ̂n ) ≥ 1 − α, ∀ θ
n
– often α = 0.05, or 0.25, or 0.01
Properties: – interpretation is subtle

• E[Θ̂n] = θ (unbiased) • CI in estimation of the mean

Θ̂n = (X1 + · · · + Xn)/n
• WLLN: Θ̂n → θ (consistency)
– normal tables: Φ(1.96) = 1 − 0.05/2
• MSE: σ 2/n �
|Θ̂n − θ|
�
P √ ≤ 1.96 ≈ 0.95 (CLT)
σ/ n
• Sample mean often turns out to also be � �
1.96 σ 1.96 σ
the ML estimate. P Θ̂n− √ ≤ θ ≤ Θ̂n+ √ ≈ 0.95
n n
E.g., if Xi ∼ N (θ, σ 2), i.i.d.
More generally: let z be s.t. Φ(z) = 1 − α/2
� �
zσ zσ
P Θ̂n − √ ≤ θ ≤ Θ̂n + √ ≈1−α
n n

The case of unknown σ

• Option 1: use upper bound on σ

– if Xi Bernoulli: σ ≤ 1/2

• Option 2: use ad hoc estimate of σ

�
– if Xi Bernoulli(θ): σ̂ = Θ̂(1 − Θ̂)

• Option 3: Use generic estimate

of the variance
– Start from σ 2 = E[(Xi − θ)2]
n
2= 1 �
σ̂n (Xi − θ)2 → σ 2
n i=1
(but do not know θ)
n
1 �
Ŝn2 = ˆ n )2 → σ 2
(Xi − Θ
n − 1 i=1
(unbiased: E[Ŝn2] = σ 2)

47
 LECTURE 24 Review
• Maximum likeliho o d estimation
• Reference: Section 9.3
– Have model with unknown parameters:
• Course Evaluations (until 12/16) X ∼ pX (x; θ)
http://web.mit.edu/subjectevaluation – Pick θ that “makes data most likely”
max pX (x; θ)
476 Classical Statistical Inference Chap. 9 θ

Outline – Compare to Bayesian MAP estimation:

in the context of various probabilistic frameworks, which provide perspective and
a mechanism for quantitative analysis. pX |Θ(x|θ)pΘ(θ)
We first Reviewthe case of only two variables, and then generalize. We
• consider max pΘ|X (θ | x) or max
wish to model–theMaximum
relation between two variables of interest, x and y (e.g., years
θ θ pY (y)
likelihood estimation
of education and income), based on a collection of data pairs (xi , yi ), i = 1, . . . , n.
For example, – Confidence
xi could intervals
be the years of education and yi the annual income of the • Sample mean estimate of θ = E[X]
ith person in the sample. Often a two-dimensional plot of these samples indicates
Θ̂n = (X1 + · · · + Xn)/n
a systematic,• approximately linear relation between xi and yi . Then, it is natural
Linear regression
to attempt to build a linear model of the form
• 1 − α confidence interval
• Binary hypothesis
y ≈ θ0 testing
+ θ1 x, +
P(Θ̂−
n ≤ θ ≤ Θ̂n ) ≥ 1 − α, ∀ θ
where θ0 and –θ1 are unknown
Types parameters to be estimated.
of error
In particular, given some estimates θ̂0 and θ̂ 1 of the resulting parameters,
– Likelihood
the value yi corresponding to xiratio test (LRT)
, as predicted by the model, is • confidence interval for sample mean

ŷi = θ̂0 + θ̂1 xi .

– let z be s.t. Φ(z) = 1 − α/2
Generally, ŷi will be different from the given value yi , and the corresponding � �
zσ zσ
difference P Θ̂n − √ ≤ θ ≤ Θ̂n + √ ≈1−α
ỹi = yi − ŷi , n n
is called the ith residual. A choice of estimates that results in small residuals
is considered to provide a good fit to the data. With this motivation, the linear
regression approach chooses the parameter estimates θ̂ 0 and θ̂1 that minimize
the sum of the squared residuals,
n
� n
�
(yi − ŷi )2 = (yi − θ0 − θ1 xi )2 ,
i=1 i=1

over all θ1 and θ2 ; see Fig. 9.5 for an illustration.

Regression Linear regression
y
Residual × • Model y ≈ θ0 + θ1x
(xi , yi ) x yi − θˆ0 − θˆ1 xi × n
�
( y i − θ 0 − θ 1 xi )2
×
min
θ0 ,θ1 i=1
x y = θ̂0 + θ̂1 x
× × • Solution (set derivatives to zero):
× x + · · · + xn y + · · · + yn
x= 1 , y= 1
=0 yx n n
�n
• Data: (x1, y1), (x2, y2), . . . , (xn, yn)
θ̂1 = i=1(xi − x)(yi − y)
�n 2
i=1(xi − x)
Figure 9.5: Illustration of a set of data pairs (xi , yi ), and a linear model y =
θ̂ 0 + θ̂ 1 x, Model:
•obtained + θθ01, θx1 the sum of the squares of the residuals
y ≈ θ0over
by minimizing
yi − θ 0 − θ1 x i .
n
�
min ( yi − θ0 − θ 1 x i ) 2 (∗) θ̂0 = y − θˆ1x
θ0 ,θ1
i=1
• Interpretation of the form of the solution
• One interpretation: – Assume a model Y = θ0 + θ1X + W
Yi = θ0 + θ1xi + Wi, Wi ∼ N (0, σ 2), i.i.d. W independent of X, with zero mean
– Likelihood function fX,Y |θ (x, y; θ) is: – Check that
� � � �
n
1 � cov(X, Y ) E (X − E[X])(Y − E[Y ])
c · exp − (yi − θ0 − θ1xi)2 θ1 = = � �
2σ 2 i=1 var(X) E (X − E[X])2
– Take logs, same as (*)
– Solution formula for θˆ1 uses natural
– Least sq. ↔ pretend Wi i.i.d. normal estimates of the variance and covariance

48
 The world of linear regression The world of regression (ctd.)

• Multiple linear regression: • In practice, one also reports

– data: (xi, x�i, xi��, yi), i = 1, . . . , n – Confidence intervals for the θi
– model: y ≈ θ0 + θx + θ �x� + θ ��x�� – “Standard error” (estimate of σ)
– formulation: – R2, a measure of “explanatory power”
n
�
min (yi − θ0 − θxi − θ �x�i − θ ��x��i )2
θ,θ� ,θ �� i=1 • Some common concerns
– Heteroskedasticity
• Choosing the right variables
– Multicollinearity
– model y ≈ θ0 + θ1h(x)
e.g., y ≈ θ0 + θ1x2 – Sometimes misused to conclude causal
relations
– work with data points (yi, h(x))
– etc.
– formulation:
n
�
min (yi − θ0 − θ1h1(xi))2
θ i=1

Binary hypothesis testing Likelihood ratio test (LRT)

• Binary θ; new terminology: • Bayesian case (MAP rule): choose H1 if:

– null hypothesis H0: P(H1 | X = x) > P(H0 | X = x)
X ∼ pX (x; H0) [or fX (x; H0)] or
P(X = x | H1)P(H1) P(X = x | H0)P(H0)
– alternative hypothesis H1: >
P(X = x) P(X = x)
X ∼ pX (x; H1) [or fX (x; H1)]
or
P(X = x | H1) P(H0)
>
• Partition the space of possible data vectors P(X = x | H0) P(H1)
Rejection region R: (likelihood ratio test)
reject H0 iff data ∈ R
• Nonbayesian version: choose H1 if
• Types of errors:
P(X = x; H1)
– Type I (false rejection, false alarm): > ξ (discrete case)
P(X = x; H0)
H0 true, but rejected
fX (x; H1)
α(R) = P(X ∈ R ; H0) >ξ (continuous case)
fX (x; H0)

– Type II (false acceptance, • threshold ξ trades off the two types of error

missed detection):
– choose ξ so that P(reject H0; H0) = α
H0 false, but accepted
(e.g., α = 0.05)
β(R) = P(X �∈ R ; H1)

49
 LECTURE 25 Simple binary hypothesis testing
Outline
– null hypothesis H0:
• Reference: Section 9.4 X ∼ pX (x; H0) [or fX (x; H0)]
– alternative hypothesis H1:
• Course Evaluations (until 12/16)
X ∼ pX (x; H1) [or fX (x; H1)]
http://web.mit.edu/subjectevaluation
– Choose a rejection region R;
reject H0 iff data ∈ R
• Review of simple binary hypothesis tests
• Likelihood ratio test: reject H0 if
– examples
pX (x; H1) fX (x; H1)
• Testing composite hypotheses >ξ or >ξ
pX (x; H0) fX (x; H0)
– is my coin fair? – fix false rejection probability α
– is my die fair? (e.g., α = 0.05)

– goodness of fit tests – choose ξ so that P(reject H0; H0) = α

Example (test on normal mean) Example (test on normal variance)

• n data points, i.i.d. • n data points, i.i.d.

H0: Xi ∼ N (0, 1) H0: Xi ∼ N (0, 1)
H1: Xi ∼ N (1, 1) H1: Xi ∼ N (0, 4)

• Likelihood ratio test; rejection region: • Likelihood ratio test; rejection region:
√ � √ �
(1/ 2π)n exp{− i(Xi − 1)2/2} (1/2 2π)n exp{− i Xi2/(2 · 4)}
√ � >ξ √ � >ξ
(1/ 2π)n exp{− i Xi2/2} (1/ 2π)n exp{− i Xi2/2}
� �
– algebra: reject H0 if: Xi > ξ � – algebra: reject H0 if Xi2 > ξ �
i i
• Find ξ � such that • Find ξ � such that
� �
n � � �
n �
P Xi > ξ � ; H0 = α P Xi2 > ξ �; H0 = α
i=1 i=1
�
– use normal tables – the distribution of i Xi2 is known
(derived distribution problem)
– “chi-square” distribution;
tables are available

50
 Composite hypotheses Is my die fair?

• Got S = 472 heads in n = 1000 tosses; • Hypothesis H0:

is the coin fair? P(X = i) = pi = 1/6, i = 1, . . . , 6
– H0 : p = 1/2 versus H1 : p �= 1/2
• Observed occurrences of i: Ni
• Pick a “statistic” (e.g., S)
• Choose form of rejection region;
chi-square test:
• Pick shape of rejection region
(e.g., |S − n/2| > ξ) � (Ni − npi)2
reject H0 if T = >ξ
i npi
• Choose significance level (e.g., α = 0.05)
• Choose ξ so that:
• Pick critical value ξ so that:
P(reject H0; H0) = 0.05
P(reject H0; H0) = α
Using the CLT: P(T > ξ; H0) = 0.05

P(|S − 500| ≤ 31; H0) ≈ 0.95; ξ = 31

• Need the distribution of T :
(CLT + derived distribution problem)
• In our example: |S − 500| = 28 < ξ
H0 not rejected (at the 5% level) – for large n, T has approximately
a chi-square distribution
– available in tables

Do I have the correct pdf? What else is there?

• Partition the range into bins • Systematic methods for coming up with
– npi: expected incidence of bin i shape of rejection regions
(from the pdf)
– Ni: observed incidence of bin i • Methods to estimate an unknown PDF
(e.g., form a histogram and “smooth” it
– Use chi-square test (as in die problem)
out)
• Kolmogorov-Smirnov test:
form empirical CDF, F̂X , from data • Efficient and recursive signal processing

• Methods to select between less or more

complex models
– (e.g., identify relevant “explanatory
variables” in regression models)

• Methods tailored to high-dimensional

unknown parameter vectors and huge
number of data points (data mining)

(http://www.itl.nist.gov/div898/handbook/) • etc. etc.. . .

• Dn = maxx |FX (x) − F̂X (x)|
√
• P( nDn ≥ 1.36) ≈ 0.05

51
MIT OpenCourseWare
http://ocw.mit.edu

6.041 / 6.431 Probabilistic Systems Analysis and Applied Probability

Fall 2010

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.