Scribd
Upload
73 views

IE 325: Stochastic Models Homework Assignment 2: Fall 2015 Due: October 12, 17:00

This document contains 6 questions describing homework problems involving Markov chains. Question 1 defines a Markov chain with transition probabilities dependent on random variables ξ and asks to determine if it is a Markov chain, its states, and its transition probability matrix. Question 2 gives a transition probability matrix and asks to calculate various probabilities related to the Markov chain. Question 3 models speaking a word as a Markov chain and asks for expected time to finish and a probability after 3 time units. Question 4 models a teacher wearing a coat as a Markov chain and asks to model it and calculate a probability. Question 5 gives a transition probability matrix and asks to derive equations to calculate a probability related to the Markov chain. Question 6 asks to construct a transition matrix and calculate it

Uploaded by

emineb
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
73 views

IE 325: Stochastic Models Homework Assignment 2: Fall 2015 Due: October 12, 17:00

This document contains 6 questions describing homework problems involving Markov chains. Question 1 defines a Markov chain with transition probabilities dependent on random variables ξ and asks to determine if it is a Markov chain, its states, and its transition probability matrix. Question 2 gives a transition probability matrix and asks to calculate various probabilities related to the Markov chain. Question 3 models speaking a word as a Markov chain and asks for expected time to finish and a probability after 3 time units. Question 4 models a teacher wearing a coat as a Markov chain and asks to model it and calculate a probability. Question 5 gives a transition probability matrix and asks to derive equations to calculate a probability related to the Markov chain. Question 6 asks to construct a transition matrix and calculate it

Uploaded by

emineb
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

IE 325: Stochastic Models

Homework Assignment 2
Fall 2015
Due: October 12, 17:00
Question 1. Consider the Markov chain defined recursively by:
(
Xn + n+1 ,
if Xn + n+1 2
Xn+1 =
Xn + n+1 3, if Xn + n+1 3.
Where 1 , 2 , . . . are independent identically distributed random variables and,
P {i = 0} = p0 , P {i = 1} = p1 , P {i = 2} = p2 ,
p0 > 0, p1 > 0, p2 > 0, p0 + p1 + p2 = 1 and X0 = 0.
(a) Is X = (Xn )n0 a Markov chain? Explain briefly.
(b) What are the states of the Markov chain?
(c) Determine the one step transition probability matrix P.
Question 2. Consider the Markov chain with one-step transition probability matrix:
0
1

0 1/2 1/2
P =
1 1/4 3/4


and initial probabilities p0 = p1 = 12 .


(a) Compute the two-step transition probability matrix.
(b) Find P {X0 = 1, X2 = 0}.
(c) Find P {X2 = 0}.
(d) Find P {X0 = 1, X1 = 0|X2 = 0}.
(e) Find E[X2 |X0 = 1] and Var[X2 |X0 = 1].

Question 3. Markov chains are used to model speech. Consider saying the word beer.
Different people might make the phonemes b, ee, r longer or shorter, depending on
the speaker and where the emphasis is put. To model this, speaking the word beer is
considered to be a Markov chain on the state space E = {b, ee, r, .} with the
following transition matrix:

0.9

0
P =
0
0

0.1
0
0
0.99 0.01
0
.
0
0.95 0.05
0
0
1

(a) What is the expected amount of time it takes to reach the end of the word .?
(b) What is the probability that after three units of time, you have not yet finished enunciating the first phoneme b?
Question 4. A teacher has two coats. He does not want to get sick while going to school
and coming back to home during cold winter days. If the weather is cold and a coat is
available, he puts on it. If the weather is nice, he does not wear coat. The probability that
the weather is cold at any time is p, independent from everything else.
(a) Model the process as a Markov chain and calculate one-step transition probability
matrix P .
(b) If the teacher has one coat in his home this morning, what is the probability that,
tomorrow evening, he will have one coat in his office before leaving?
Question 5. A Markov Chain (Xn )n0 has the transition probability matrix:
0
1
2

0 0.2 0.3 0.5


P = 1 0.1 0.5 0.4
2 0
0
1
Let X0 = 0. Derive the equations necessary to calculate the probability that the time the
process ends up in state 2 is divisible by three. Do not solve the equations.
Question 6. Construct an m m one-step transition probability matrix P , where m 3
(it should not be identity matrix). Calculate P (n) for n = 5, 10, 20, 40. Based on your
calculations, how does P (n) change as n increases? Comment on your results.

You might also like