Scribd
Upload
146 views3 pages

Stokastik

This document discusses continuous-time Markov chains, specifically pure birth processes. It defines a pure birth process as a Markov process where (1) the probability of transitioning from state k to state k+1 in a short time h is proportional to k*h + o(h) and (2) the probability of transitioning to any other state in time h is o(h). It also presents the postulates that define a Poisson process, a prototypical pure birth process, as a special case where the transition probabilities are independent of the current state. Finally, it introduces the concept of using a system of differential equations to model the transition probabilities P(t) over time t for a general pure birth process.

Uploaded by

Anonymous SIYSWsDtl
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
146 views3 pages

Stokastik

This document discusses continuous-time Markov chains, specifically pure birth processes. It defines a pure birth process as a Markov process where (1) the probability of transitioning from state k to state k+1 in a short time h is proportional to k*h + o(h) and (2) the probability of transitioning to any other state in time h is o(h). It also presents the postulates that define a Poisson process, a prototypical pure birth process, as a special case where the transition probabilities are independent of the current state. Finally, it introduces the concept of using a system of differential equations to model the transition probabilities P(t) over time t for a general pure birth process.

Uploaded by

Anonymous SIYSWsDtl
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

Chapter VI

Continuous Time Markov Chains

1. Pure Birth Processes

In this chapter we present several important examples of continuous time,

discrete state, Markov processes. Specifically, we deal here with a family

of random variables {X(t); 0 < t < co} where the possible values of X(t)

are the nonnegative integers. We shall restrict attention to the case where

(X(t)) is a Markov process with stationary transition probabilities. Thus,

the transition probability function for t > 0,

P;(t)=Pr{X(t+u)=jjX(u)=i}, i,j=0,1,2,...,

is independent of u ? 0.

It is usually more natural in investigating particular stochastic models

based on physical phenomena to prescribe the so-called infinitesimal

probabilities relating to the process and then derive from them an explicit

expression for the transition probability function. For the case at hand, we

will postulate the form of P;(h) for h small, and, using the Markov prop-

erty, we will derive a system of differential equations satisfied by P;(t) for

all t > 0. The solution of these equations under suitable boundary condi-

tions gives P;(t).

By way of introduction to the general pure birth process, we review

briefly the axioms characterizing the Poisson process.


1.1. Postulates for the Poisson Process

The Poisson process is the prototypical pure birth process. Let us point out

the relevant properties. The Poisson process is a Markov process on the

nonnegative integers for which

The precise interpretation of (i) is the relationship

The o(h) symbol represents a negligible remainder term in the sense that

if we divide the term by h, then the resulting value tends to zero as h tends

to zero. Notice that the right side of (i) is independent of x.

These properties are easily verified by direct computation, since the ex-

plicit formulas for all the relevant properties are available. Problem 1.13

calls for showing that these properties, in fact, define the Poisson process.

1.2. Pure Birth Process

A natural generalization of the Poisson process is to permit the chance of

an event occurring at a given instant of time to depend upon the number

of events that have already occurred. An example of this phenomenon is


the reproduction of living organisms (and hence the name of the process),

in which under certain conditions-sufficient food, no mortality, no mi-

gration, for example-the infinitesimal probability of a birth at a given in-

stant is proportional (directly) to the population size at that time. This ex-

ample is known as the Yule process and will be considered in detail later.

Consider a sequence of positive numbers, { AA } . We define a pure birth

process as a Markov process satisfying the following postulates:

As a matter of convenience we often add the postulate

With this postulate X(t) does not denote the population size but, rather, the

number of births in the time interval (0, t}.

Note that the left sides of Postulates (1) and (2) are just P.k+,(h) and

PA,x(h), respectively (owing to stationarity), so that o,,k(h) and o,.,(h) do not

depend upon t.

We define P(t) = Pr{X(t) = n}, assuming X(0) = 0.

By analyzing the possibilities at time t just prior to time t + h (h small),

we will derive a system of differential equations satisfied by P(t) for

t ? 0, namely

You might also like