Scribd
Upload
36 views

Markov Process and Markov Chain - Solved Examples

Uploaded by

vawat75828
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
36 views

Markov Process and Markov Chain - Solved Examples

Uploaded by

vawat75828
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 13

Stochastic Model

Markov process and Markov


chain
Supervisor: Ms. Shaista Rais
Department of Computer Science
University of Karachi

Supervisor: Ms.Shaista Rais (DCS) University Of Karachi


Markov Process
Question: Consider a survey that is made on two brands of cotton fabric,
let’s say Gul Ahmed and Alkaram. Every time a customer (he/she) buys the
same brand or switch to another brand. The transition probabilities are
given below:
Transition probability matrix Future

From To Gul Ahmed Alkaram


P
Gul Ahmed 0.7 0.3
r
e Alkaram 0.6 0.4
s
e
Consider first row , we can say if a customer purchased
n
gulahmed brand in the last visit , the probability of switching
t
from gul ahmed to Alkaram brand is 30% in his next visit.
Supervisor: Ms.Shaista Rais (DCS) University Of Karachi
Cola example (COKE VS PEPSI)
• Given that a person last cola purchase was coke, there is a 90%
chance that is next cola purchase will also be coke. If a person last
cola purchase was Pepsi, there is a 80% chance that is next cola
purchase will also be Pepsi. Construct the TPM.

• Note: since there are two states coke and pepsi , so the TPM is
gonna be of 2*2 matrix
coke Pepsi

0.9 0.1
Coke
Pepsi 0.2 0.8
Supervisor: Ms.Shaista Rais (DCS) University Of Karachi
Question: Three players Kamran, Ali and Salman, are throwing a plastic ring to each
other. Kamran always throws the ring to Ali and Ali always throws the ring to Salman,
but Salman is as likely to throw the ball to Ali and Kamran. Find the transition matrix.

Since there are three states: so the transition matrix will be of 3*3 matrix
Kamran Always throws to Ali
From To Kamran Ali Salman
From To Kamran Ali Salman
Kamran
Kamran 0 1 0
Ali
Ali
Salman
Salman

Ali Always throws to Salman Salman is as likely to throw the ring to Kamran and Ali
From To Kamran Ali Salman From To Kamran Ali Salman
Kamran 0 1 0 Kamran 0 1 0
Ali 0 0 1 Ali 0 0 1
Salman Salman 0.5 0.5 0
Supervisor: Ms.Shaista Rais (DCS) University Of Karachi
Question: Brand Khadi has 40% market share in the local markets for its clothing product,
while the other two branded companies, Sapphire and J. have equal share each on January
1st, 2022. A study by the market research company has disclosed the following data for every
year.

• Brand khadi retains 70% of its customers and gains 5% from Saphhire
brand and 10% from J. brand.
• Sapphire retains 90% of its customers and gains 14% from Khadi and 5%
from J.
• J. retains 85% of its customers and gains 16% from Khadi and 5% from
Sapphire.
• Construct the TPM. From To Khadi Sapphire J.
Khadi 0.70 0.14 0.16
Sapphire 0.05 0.90 0.05
J. 0.10 0.05 0.85

Supervisor: Ms.Shaista Rais (DCS) University Of Karachi


Make a transition diagram from the TPM

0.14 0.90
Sapphire
0.70

0.05 0.05
Khadi 0.05

0.16

0.10 J.
0.85
Supervisor: Ms.Shaista Rais (DCS) University Of Karachi
MARKOV CHAIN
• How to calculate the probability of the states.

Example: 𝑃 𝑋3 = 4 ; 𝑃 𝑋2 = 5 𝑒𝑡𝑐
• How to calculate the probability after n-steps.
Example: 𝑝𝑖𝑗 (2) , 𝑝23 (3) , 𝑃 𝑋3 = 4 𝑋1 = 2) 𝑒𝑡𝑐

• How to find the probability of the chain.


Example: 𝑃 𝑋3 = 4, 𝑋1 = 2 ,

𝑃(𝑋3 = 2, 𝑋2 = 3, 𝑋1 = 1, 𝑋0 = 4)
Supervisor: Ms.Shaista Rais (DCS) University Of Karachi
• 1- step transition probability:

𝑝𝑖𝑗 = 𝑃 𝑋𝑛+1 = 𝑗 𝑋𝑛 = 𝑖)

• n-step transition probability:

𝑝𝑖𝑗 (𝑛) = 𝑃(𝑋𝑛+1 = 𝑗|𝑋1 = 𝑖)


Remember (n) = time , i = from, j = to
Supervisor: Ms.Shaista Rais (DCS) University Of Karachi
Supervisor: Ms.Shaista Rais (DCS) University Of Karachi
Supervisor: Ms.Shaista Rais (DCS) University Of Karachi
Calculating probability of the states
𝑋𝑛 = 𝑖
Where n = time n i= state

𝑃 𝑋𝑛 = 𝑎 = 𝑞𝑛 (𝑎)

We have formula for calculating probability of states:


𝑞𝑛 = 𝑞𝑜 𝑃𝑛
OR
𝑞𝑛+1 = 𝑞𝑛 𝑃
Supervisor: Ms.Shaista Rais (DCS) University Of Karachi
Ali uses his pointer or , ball point or marker to write notes well each day. The tpm of
the markov chain with these three states 1(pointer) , 2(ball point) , 3(marker) is

•P= P B M
P 0.1 0.5 0.4
B 0.6 0.2 0.2
M 0.3 0.4 0.3

And the initial probability is q0 = (0.7,0.2,0.1). Calculate P(X2 =3)

P(X2=3) = 𝑞2 = 3 , 𝑖𝑡 𝑚𝑒𝑎𝑛𝑠 𝑚 𝑡𝑟𝑦𝑖𝑛𝑔 𝑓𝑖𝑛𝑑 𝑡ℎ𝑒 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑎𝑓𝑡𝑒𝑟 2 𝑡𝑖𝑚𝑒 𝑝𝑒𝑟𝑖𝑜𝑑,


𝑞2 = 𝑞0 𝑃2

Supervisor: Ms.Shaista Rais (DCS) University Of Karachi


0.1 0.5 0.4 0.1 0.5 0.4
0.6 0.2 0.2 * 0.6 0.2 0.2
0.3 0.4 0.3 0.3 0.4 0.3

P^2= 0.43 0.31 0.26


0.24 0.42 0.34
0.3 0.35 0.29

(0.7,0.2,0.1).*P^2 = [0.385 0.336 0.279]

P(𝑋2 = 3) = 0.279 (ans)

Supervisor: Ms.Shaista Rais (DCS) University Of Karachi

You might also like