_ i Not Introduction Random process - | ’ random process (also called a stochastic process) is a collection of random variables indexed by time (or some other parameter). It describes how a system evolves randomly over time, Formally, a random process is a family of random variables: (X(1),t€T } where: + X(0) isa random variable for each t +t belongs to an index set T (often time). ‘A random process X(t) becomes a random variable when observed at a fixed time. That is, X(to)is just a random variable describing an outcome at a single point in time, rather than across time. Def: Strict Sense Stationary process . A random process is called a strongly stationary process (SSS) or strict sense stationary if all its statistical properties are invariant to a shift of time origin. This means that X (t) and X (t+1) have the same statistics for any Example: Bernoulli process is a SSS process and any t @ E(X(O)} = Constant (ii) Var{X(t)} = Constant Def: Wide sense stationary process E(X(t)} = J", X(©f (049 = 0 = a constant R(ty, ta) = E[X(t1)X(t2)] = a function of (ty - t2) - (X(O}isWss Define Markov processes. A stochastic process {X(t)} is a Markov process if, for all times t and states X (t): PUM Xb Xo) = pl X41 XO) Tats raeans ‘that Hu pribabtlity fransitfonting b be nore Jate XH depends fn the Curent State Xlt) and noon tte Seguene & states that preceded ré | Write down the classification of Markov Process. (a) Discrete-State Markov Process (b) Continuous-State Markov Process (c) Discrete-Time Markov Process (DTMP) (d) Continuous-Time Markov Process (CTMP) Write_down the relation_satisfied_by the steady-state distribution and the transition probability matrix of a regular Markov chain. Ina regular Markov chain, the steady-state distribution (also i na regular M ; the stea called the statio distribution) is the probability distribution that remains unchanged as time ae It satisfies a fundamental relation with the transition probability matrix. : CeSteady-State Equation Let: « Pbe the transition probability matrix of a Markov chain. 2 n= (rc, n2, ... Tin) be the steady-state probability distribution (a row vector). The steady-state equation is: xP=x or equivalently, mj Py=ni_ Vi | where Pij’s the probability of transitioning from state j to state i. Def: Poisson process QQ Th xld) represents the number af occurrences f° — a fr (ort) then the diets rondo procss Pacey B Called toe porsson PPPCRS / ort n = At) PlxH=nJ= ( State any two properties of Poisson process. i) Poisson process is a Markov process. ii) Sum of two independent Poisson process is a Poisson process. Example 1: The number of telephone calls arriving at a certain switch board within a time interval of length (measured in minutes) is a Poisson process X(t) with mean rate of 2 per minute. Find the probability of no telephone calls arriving at this switch board during a 5- minute period. Solution: ) 54 ye) ho the number of eatls ante at & boa rel in Hee tome tntawet e mea £25 permin ) N=0 ftir ak Fad ee ne —AE( nt)" X(E) Ha prisson prowssy PI xl = a PL Wo catls goig board PL xts) =o] = é 35" during 5 nin per fod) = yoSh KIO?State Chapman-Kolmogoroy theorem, For a Markov process (X(1)) with a transition probability P,j(€)(probability of transitioning from state iii to state j in time t, the theorem states: Py(t + SKY Pia(OPaj(S)(or all states i, j and intermediate state k | The process {X(¢)} whose probability distribution under certain conditions is given by on Grant = a Trae’ | Show that it is “not stationary”, | Solution: , aabsbutton 1,2, oo 0 -P(X(O =n) = g xtt) a | (DPato 2 = EDP + a) ELxee) = 27a OP DPI 2, ak +3 @b+--: aaa +P) (3 (t+) (at) | a [vt @ at +0) Tea | bx4+3% ae- Liat - ate iis? Tpat —— aa (ivad| j4at 2 = 1 x (reat) = 1 (ita) (él Fue on Poa! Sa} 4 Efyte)) -2n 2D a ~ 2 filme) Pn 2 n(nt) Pn - Ape n2o0 150) 2° 2 ntnt)Pa-! nzo0 = [foxixPo+! [xa xP) $2x3HP2) + 2 [= OFha- +a ae “ia” (Ci =_l(#) (teat) “ taal” Cael a e) (reat)? ~ Cygat)® =acieat) peat] 6) del*t8) aapoat =lt 2 iaat al oa I tat Li t(D Gn) yerl xe) = Ef eet]- elie) = |428at -(p = |4eat —1 =[age] Pac teu eleiey= = Conse bub varlece) of Constant a (tag ° at +34 ANd 2.Show that the random process X(t)=cos(t+6), | distributed in (0, 22) is () first order stationary (ii) stationary in the wide sense (ii) Ergodic Solution: 2 ven pyveers ie tonary where 0 is a random variable uniformly B not a DCLsAs .Guiven + xlt = Cos(tho — were 0’ iv uniformly Pn Pn ao | » fle) — tb |=4 ,0z20 Zam ano an Fist ovolur etatfona Yy ‘To ve } xl) Bb etxity) = = Constant fe To prove : prek! cryuey = J reese) de | Efeosterey = = J Fee( tt) 3 4° 2, f asiteorde , | ot Id 9 =a constant iC Jos (werner de = 9, nw an frleger, n#o} thre ) Yi) wa frst order Stalfonary.Pi To prove: xe) if wes To prove * a) Efytt)) = constant 5) Rit sta) ‘= 0 fanelton e lty-b2) Pref? a) EL xits) = Constant byt) 1) Rita) < ef ¥levd. xlt2)) = E[ es tt4@) cos (Lot 8)) 0) + - ef p Lees (a sy) +, egsa cos = [ies nes) Heestos) cE is tes ltitt2tea a+ EL Y% costti-t2)] = —t = LE [tos ceuttat 2) +h, Coste! 2) a) 2y° se Tore E[eosl tretoro@) = feaslesess9 5h de <0 | 0) S Rie) > toy tpios/41- | = os (42) | Sa fanttn Bt)Ge \ WIG rr nle) mean ~ @ dic. tm y, -elxt)] ecrey—o by 2 bem ¥q Tae T dt aes lt) = lim oa ; Ta” vee 1 fe rare 738 at , t pa sents 08) elem at iS ut oe st = bm, 3. ef — sno Sion >0 87 o 4 oe [ts id olen 3 al ~ ‘ +e Cos if an eve? a fen el SPaE ” c _L cosoSht a lim, op pari. = fin ree” — ba lem SoT = cosa (0) TS 0 TT =0LHS 2RHS- 2% The prow Kit) We mean — ergodPe Py Bayes theo rem *, aD Bp Ba) >~ 8 events — soe? and A 1g another Be ) Than : / .) Lh °) pian) — aD RATE Z plBt) xPLYBL) 1 folyey-o [Fl hea St of eu haus lPre Wit a yandom experhment even. arsockated we3) Show that tha random fdr Sense. Gerttana Contant and @ . ml in yandom YyarPable, é ioe ly dihsbuted se &bven X= A coswt ©) mn a @ a vagfomn adghied nf - hy) Fle) = tee fn azaeb ft)— gay OO + XID WSs ee arene = Conon ‘proof: cree) fe o4s >Elamtotia =f "A cos(wte) % a da eet " tosteoted da “2 2 ettetsed (Pf) RE tlvta) = a-furction q-é -Blestacesose T 4. [m Gen :7 Coane rotets ALE) Altos (wot) A and y) axe 1 p ERO) 2 towstont 2 (Pi) Reet) = 2 efrtue (%) A wale tite) % el vos (wtat®)) =f efetaevdaet) wing 018 “Kes oe Ces( voles )) e pr efes toys “2 4 a2 AZ ss uollrt ) as gute Is ° + Axtrs (10 Ltt )) =L(0)+ a BX tps volt) stRUL bs) = Pips (tit) >! ubtch afuntton 4 Wes So By @ 18 Zea Lt) 2 Wess: re4) Show that the pros xlé)= Acesrt+ BsPnrt ela Serie. GatPonary 9 tf EDAD = £(6) <0 ', €l4')-e(8], CAB) =0> AAG ‘ave random Varables - Gul Kren eva) 0 /ET8)=0 2D | gpabl= ecajecs) =9 > > (3) ECA) = ereijek Guben < vie) —Acosneé +Bsirt, guien xle) id Wss prree4s To prve! ‘ 3 lant Je prove: Ci) Exley = — afenta) <4 functor G lbr-ta) (8) Geren’ xle) = prerb-rBsore | efaley= ECAJ cesrt + ELA] Sr = oto =o C4Y? Y \(@ Ritsta) = EL ater Céay] = ELA cosrt pcos) | +B Sire Sinrte FA Bl esrtl ghortatsinrt ot) nety Star be | SE [Ai] wos Ati ceosata + ECB] Sn | LE LAL] |GosAtistrn tz +S fa br Coste) =k [cosrty cost tah nt sind &) by =k Cos (\(H-€2)) [s wpsla-8) acesh cosB 48PnA si) Rttstg) _fo-e = ey mins) =pfieT43) (itt) PL between | min i 2 vd fe pt) Fiche A [ee 7 = fre =) J 3 -£ =e =presented in a computer data base are following a Poisson process of rate 1 = 6 per minute. An experiment consists of monitoring the data base for m minutes and recording N(m) the number of queries presented (i) What is the probability that no queries arrive in one minute interval? | Gi) Whatis the probability that exactly 6 queries arriving in one-minute period? (iii) What is the probability that less than 3 queries arriving in one-minute period? Solution: |Jot N(m) Xlt) be Hu number Gf quences presented |fm m (or) ¢ mfnulés le) Gin Mean rete 7 d=bpemin t 20 | No anivals yn oPsson process) | Ohwed= roe" —p@ randcarel) b=) permin PY No quent ark in pee : . | Dplxl =o) = ote 2 & ae — it) pat bal> n= 8 Bo /mén wile nbs ardin pLExactly 69 Pee sob = Pew =6) = tml AE voy. amin) t2l ype bss Than 3 () neo pe , fn J mn interval) Pla than Syuents 67” oe we 3) ~plnud=o}rpl nied = Epo" 5 Muga reand! + Pls)= FJ) Cs eef ‘A machine goes out of order, whenever a component fails. The failure of this part allows a Poisson process with a mean rate of | per week. Find the probability that 2 | weeks have elapsed since last failure. If there are 5 spare parts of this component in an | inventory and that the next supply is not due in 10 weeks, find the probability that the | machine will not be out of order in the next 10 weeks. Solution: Mean vat. A=} y Trme inlrval ) t= 2 (9) diven * n2d h “ptt = EMD!" 5 © nl slere. in the & =P[X2) =0) = oR (xa)? _ go 91857 of pntanel, oe xs Se ast faites) plab fe “ =10 (ff) bvbven é Moan vote » A= 1) Th nels than oregual by S . ; (to) SS} = i xlto=o) + pl xtio) =! + me ; ff x(10) =2) 4 plxtio) a3) t plxlio) = 4) t plxtio) =5 = LO eM peta) & 0), 5) of 1 al 3f 7 =>4EUo> sl ' 0 4 (0002 +/90000 = et0f [ot LOH * 82% lees 720 _ 610 [380 +6000 F 20000 - $0000 #/0/0000 20 + = 01067 8.Three boys A, B and C are throwing a ball to each other. A always throws the ball to B and B always throws the ball to C but C is just as likely to throw the ball to B as to A. Show that the process is Markovian. Find the transition probability matrix and classify the states, simon: yp BE [ha {pm v ral of} ! ee dae Deer peace rors (#54) Para (Spb Yo Yo YX, y (Aa he herp = [ fa Yule Ye My, cpp al Yy & Pep He Yo My Ye Ve le,Stab A: pyer>e ) Py? >0, Perod =cp (315---)£) ale State A id aperPodéc - Slab B* pO toAB* pe" Soy pro) Pak? S01 perrod =Gicd (2)314-JA! 0, late Bid apertodi. [stake € RiP>o> Blo, Pevfod =EcD ([2)3)--Je) of Stated; Ls apervodie | 3 boys 0 ) pal? >0 Ast” 0 Pr,(?>0) Bf72>0) B30 p,t>e ; PxoV>07 Pal? 0 a frotle and Lhd J non-neuk Persistent ro Ihe choles are Figod | Pu| Sales are que 9.Suppose, the probability of a dry day following a rainy day is 1/3 and that are ‘| probability of a rainy day following a dry day is 1/2. Given that May | is a dry day, find the probability that (a) May 3 is also a dry day and (b) May 5 is also a dry day. Solution: a tpm of the Mower cha a relat) Ls hie PLROD = 3 pLD2R) = = Ya ae ra stochastted Siren Hf the tre J” ments of mab than tte sum any von Gd eguol & IU sly dbtbub “the Fyitgh sete pribellty } B (t °) 7 brtuen (Mog uo cry doy)py plr) [w P( May 3 Boa dry pli) = pep = i i lS) = pid.p = (3 + 42) i el | Solution: pie) = 43) (3) (& pop OLR zi pe) - pe). P= (2 ane be tt a(S 2) #3) May ya dry day) © I sates -s of the Markov chain with the transition probability matrix. P= [es 0 0 as where X, = {0,1,2} e(h h) day) = 2 2.) [’ Yo A wf, 13 sl | 1713 Aa & |€xam ple _ O Inc ths nature % Ha Shla of Me Marka? | Chan wet he fpm ie ° i Hi Oo \ / [* D os] 2 o 10 Solution >!) , Given Xn = foray a frile | CD thle 0 cad tate | 08 Lom mnt (7 with path olhar a ° | @) the ! ond Sale ae lommunhtalt wih aorh olfas - Glas 2 ond Hate eyez wple each olber Hrroweg by shte | > the markov thufn oy faredurl ble ab) ) fom C1) anol & we ae all th ghta a pone ae pescvitent > (3) d) Ove Can qe back toa Ga o in p>l2? ¢ ests? cht | jaoel C&S s) oats TO j222 ! ( steps) Gate 2 PN Sion = c astps) Rol? — on ae apeod le seh [star sete or re with au and (4) ne gee ta sates OF