Marshall University

Marshall Digital Scholar

Theses, Dissertations and Capstones

2020

Markov Chain Epidemic Models and Parameter Estimation

Oluwatobiloba Ige
[email protected]

Follow this and additional works at: https://mds.marshall.edu/etd

Part of the Biostatistics Commons, and the Mathematics Commons

Recommended Citation
Ige, Oluwatobiloba, "Markov Chain Epidemic Models and Parameter Estimation" (2020). Theses,
Dissertations and Capstones. 1307.
https://mds.marshall.edu/etd/1307

This Thesis is brought to you for free and open access by Marshall Digital Scholar. It has been accepted for
inclusion in Theses, Dissertations and Capstones by an authorized administrator of Marshall Digital Scholar. For
more information, please contact [email protected], [email protected].
MARKOV CHAIN EPIDEMIC MODELS AND PARAMETER ESTIMATION

A thesis submitted to
the Graduate College of
Marshall University
In partial fulfillment of
the requirements for the degree of
Master of Arts
in
Mathematics
by
Oluwatobiloba Ige
Approved by
Dr. Anna Mummert, Committee Chairperson
Dr. Avishek Mallick
Dr. Carl Mummert

Marshall University
May 2020
 ACKNOWLEDGEMENTS

First and foremost, I wish to express my deepest gratitude to my supervisor, Dr. Anna

Mummert, who in every way was supportive with her time, knowledge and resources. This thesis

would not have been possible without you. Thank you very much.

I am also extremely grateful to my thesis committee members, Dr. Carl Mummert and

Dr. Avishek Mallick. Your immerse contribution to my academic growth cannot be

overemphasized. Thank you for your support.

My sincere gratitude also goes to Dr. Akinsete. Your support during my program at

Marshall is invaluable. Thank you very much.

Finally, I would like to thank my family for the love and support I got from them. For

everyone who has contributed to my success in anyway, I say thank you.

iii
 TABLE OF CONTENTS

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Mathematical Modeling of Infectious Diseases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Compartments in Disease Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3 Introduction to Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.4 Parameter Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.5 Aim and Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Chapter 2 Epidemic Models using Discrete Time Markov Chains . . . . . . . . . . . . . . . . . . . . . . 11

2.1 Discrete Time Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2 SIS Epidemic Models using Discrete Time Markov Chains . . . . . . . . . . . . . . . . . . 13

2.2.1 Transition Probabilities of a DTMC SIS Epidemic Model . . . . . . . . . . . . . . 14

2.2.2 Pseudocode for DTMC SIS Models and Simulated Example . . . . . . . . . . . 17

2.3 SIR Epidemic Models using Discrete Time Markov Chains . . . . . . . . . . . . . . . . . . 19

2.3.1 Transition Probabilities of a DTMC SIR Epidemic Model . . . . . . . . . . . . . 21

2.3.2 Pseudocode for DTMC SIR Models and Simulated Example . . . . . . . . . . . 22

Chapter 3 Epidemic Models using Continuous Time Markov Chains . . . . . . . . . . . . . . . . . . . 25

3.1 Continuous Time Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.1.1 Generator Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.2 SIS Epidemic Models using Continuous Time Markov Chains . . . . . . . . . . . . . . . 30

3.2.1 Transition Probabilities of a CTMC SIS Epidemic Model . . . . . . . . . . . . . . 31

3.2.2 Pseudocode for CTMC SIS Models and Simulated Example . . . . . . . . . . . 33

3.3 SIR Epidemic Models using Continuous Time Markov Chains . . . . . . . . . . . . . . . 35

3.3.1 Transition Probabilities of a CTMC SIR Epidemic Model . . . . . . . . . . . . . 37

3.3.2 Pseudocode for CTMC SIR Models and Simulated Example . . . . . . . . . . . 38

iv
Chapter 4 Method of Maximum Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

4.1 Parameter Estimation for DTMC SIS Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4.1.1 Estimation Algorithm and Results for DTMC SIS Models . . . . . . . . . . . . . 43

4.2 Parameter Estimation for DTMC SIR Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.2.1 Estimation Algorithm and Results for DTMC SIR Models . . . . . . . . . . . . . 48

4.3 Parameter Estimation for CTMC SIS Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

4.3.1 Estimation Algorithm and Results for CTMC SIS Models . . . . . . . . . . . . . 56

4.4 Parameter Estimation for CTMC SIR Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

4.4.1 Estimation Algorithm and Results for CTMC SIR Models . . . . . . . . . . . . . 63

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Appendix A Approval Letter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

v
 LIST OF FIGURES

Figure 1 SIS Epidemic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Figure 2 SIR Epidemic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Figure 3 Discrete Time Stochastic Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Figure 4 Continuous Time Stochastic Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Figure 5 Trajectory of a Discrete Time SIS Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Figure 6 A DTMC SIS Epidemic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Figure 7 Trajectory of a Discrete Time SIR Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Figure 8 A DTMC SIR Epidemic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Figure 9 Embedded Discrete Time Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Figure 10 Trajectory of a Continuous Time SIS Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Figure 11 A CTMC SIS Epidemic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Figure 12 Trajectory of a Continuous Time SIR Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Figure 13 A CTMC SIR Epidemic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Figure 14 Histogram of β̂ and γ̂ for a DTMC SIS Model with β = 1.5 and γ = 0.5 . . . . 45

Figure 15 Histogram of β̂ and γ̂ for a DTMC SIS Model with β = 0.9 and γ = 0.3 . . . . 45

Figure 16 Histogram of β̂ and γ̂ for a DTMC SIR Model with β = 1.5 and γ = 0.5 . . . 50

Figure 17 Histogram of β̂ and γ̂ for a DTMC SIR Model with β = 0.9 and γ = 0.3 . . . 50

Figure 18 Estimating Maximum Likelihood for Continuous Time SIS Model . . . . . . . . . 52

Figure 19 Histogram of β̂ for a CTMC SIS Model with β = 1.5 and γ = 0.5 . . . . . . . . . 59

Figure 20 Histogram of γ̂ for a CTMC SIS Model with β = 1.5 and γ = 0.5 . . . . . . . . . 59

Figure 21 Histogram of β̂ for a CTMC SIS Model with β = 0.9 and γ = 0.3 . . . . . . . . . 60

Figure 22 Histogram of γ̂ for a CTMC SIS Model with β = 0.9 and γ = 0.3 . . . . . . . . . 60

Figure 23 Histogram of β̂ for a CTMC SIR Model with β = 1.5 and γ = 0.5 . . . . . . . . . 66

Figure 24 Histogram of γ̂ for a CTMC SIR Model with β = 1.5 and γ = 0.5 . . . . . . . . . 66

Figure 25 Histogram of β̂ for a CTMC SIR Model with β = 0.9 and γ = 0.3 . . . . . . . . . 67

Figure 26 Histogram of γ̂ for a CTMC SIR Model with β = 0.9 and γ = 0.3 . . . . . . . . . 67

vi
 LIST OF TABLES

Table 1 Simulation Parameter Estimates for a DTMC SIS Model . . . . . . . . . . . . . . . . . 46

Table 2 Computational Results for a DTMC SIS Model . . . . . . . . . . . . . . . . . . . . . . . . . 46

Table 3 Simulation Parameter Estimates for a DTMC SIR Model . . . . . . . . . . . . . . . . 51

Table 4 Computational Results for a DTMC SIR Model . . . . . . . . . . . . . . . . . . . . . . . . . 51

Table 5 Simulation Parameter Estimates for a CTMC SIS Model . . . . . . . . . . . . . . . . . 61

Table 6 Computational Results for a CTMC SIS Model with β = 1.5 and γ = 0.5 . . 61

Table 7 Computational Results for a CTMC SIS Model with β = 0.9 and γ = 0.3 . . 62

Table 8 Simulation Parameter Estimates for a CTMC SIR Model . . . . . . . . . . . . . . . . 68

Table 9 Computational Results for a CTMC SIR Model with β = 1.5 and γ = 0.5 . . 68

Table 10 Computational Results for a CTMC SIR Model with β = 0.9 and γ = 0.3 . . 69

vii
 ABSTRACT

Over the years, various parts of the world have experienced disease outbreaks. Mathematical

models are used to describe these outbreaks. We study the transmission of disease in simple cases

of disease outbreaks by using compartmental models with Markov chains. First, we explore the

formulation of compartmental SIS (Susceptible-Infectious-Susceptible) and SIR

(Susceptible-Infectious-Recovered) disease models. These models are the basic building blocks of

other compartmental disease models. Second, we build SIS and SIR disease models using both

discrete and continuous time Markov chains. In discrete time models, transmission occurs at fixed

time steps, and in continuous time models, transmission may occur at any time. Third, we

simulate examples of SIS and SIR disease models in discrete time and in continuous time to see

how the number of infected individuals changes over time. Fourth, we estimate the transmission

and recovery rates from simulated data using the method of maximum likelihood. The parameter

estimates in discrete time are obtained using computer algorithms and those in continuous time

are obtained using both computer algorithms and theoretical formulas. Finally, we compute the

bias and mean squared error of the estimators.

viii
 CHAPTER 1

INTRODUCTION

Over the years, various parts of the world have experienced infectious disease outbreaks.

These outbreaks have lead to the loss of lives and have also caused severe adverse economic effects

on the areas affected [27]. Infectious diseases are caused by pathogens such as bacteria and

viruses. Transmission of infectious disease can happen through physical contact with an infectious

person, use of contaminated objects, ingestion of contaminated food and water, or bites from

insects or animals. The spread of infectious diseases is a major concern in densely populated

developing countries. A major reason is that infectious diseases can spread rapidly in a very short

period of time, especially when they can be transmitted from person to person through physical

contact [1]. In developed countries, infectious disease can spread rapidly over a large area due to

the advancement in transportation networks in these countries [28].

Epidemics are infectious disease outbreaks that are confined to a certain area (e.g., a

country). The cholera epidemic in Nigeria in 2010 had a total of 41,787 cases, including 1,716

deaths [10]. Pandemics are infectious disease outbreaks that spread across a much larger area

(e.g., continents or the world). The “Asian” influenza pandemic between 1957 and 1958 affected

China, Singapore, Hong Kong and the United States, killing about two million people [19]. An

infectious disease is considered endemic to a location if the disease persists in the location.

Chickenpox is endemic in the United Kingdom. By age 9, over 60 percent of children in the U.K.

will acquire the infection [13].

Past and current disease outbreaks have had great adverse effects. The Spanish influenza

pandemic between 1918 and 1920, which was tagged the “mother” of all pandemics, infected over

500 million people, killing about 50 million of them [29]. The number of deaths caused by the

Spanish influenza was more than the number of deaths recorded in the First World War [29]. The

Severe Acute Respiratory Syndrome (SARS) epidemic of 2003 was truly a global concern [7, 24].

SARS caused high adverse economic effects on Asian countries due to the sharp decline in the

travel rate and tourism during the outbreak. The epidemic infected over 8,000 people, killing 774

people [7, 24]. Malaria is one of the leading causes of death in Africa and continues to claim a

1
large number of lives each year, with children under age 5 making up approximately 80% of the

deaths [21]. Recently, over 700 cases of measles were reported in the United States between

January and April of 2019. This is the largest number of measles cases reported in one year since

1994 [22]. The report raised some concerns because measles was said to be eradicated from the

United States in 2000 [8]. With advancements in the transportation system, disease outbreaks

may spread across continents in a matter of days infecting a large number of people even before

being detected [28]. The most recent pandemic is the coronavirus disease (COVID-19).

Originating from Wuhan, China, in December 2019, the disease has infected over 3,000,000 people

in over 200 countries, killing about 200,000 of them as of April 2020. The adverse effects of

disease outbreaks have prompted scientists to devise several means of making important

predictions about these disease outbreaks [6].

1.1 MATHEMATICAL MODELING OF INFECTIOUS DISEASES

Epidemiology is the study of the cause, spread and control of disease. Mathematical

modeling is a vital part of understanding disease outbreaks and making predictions in

epidemiology [14]. Models are representations of objects, ideas and scenarios on a small scale.

They are used in understanding the behavior of real life problems and making predictions about

future occurrence of events. Mathematical modeling is the representation of models using

mathematical equations.

Modeling in epidemiology is important because it can be used to predict how a disease

outbreak can affect a population. The incidence, prevalence, morbidity and mortality rates of a

disease outbreak can be predicted by using models. The incidence rate is the number of newly

infected persons over a specific period of time, the prevalence rate is the total number of infected

persons over a period of time, the morbidity rate is the rate of infection in a population, and the

mortality rate is the number of people who die in a given year and area divided by the population

of that area.

Models can be used to make predictions to understand if an outbreak may lead to an

epidemic or a pandemic. These predictions help in understanding how fast an infection can

spread, the effects of certain measures to contain the disease outbreak and even how long the

2
infection will persist in a given population. By analysing and making accurate predictions about

the outcome of a disease outbreak, the right measures to contain an outbreak can be taken.

When modeling a disease outbreak, various factors may be considered in the formulation

of the mathematical model. The type of infection, contact rate, latent period, age, sex,

surrounding temperature and other factors may be considered. Infections can be categorized into

two types: acute and chronic. Acute infections generally have a shorter life span than chronic

infections in humans. Acute infections can be present in humans from a few days to few weeks,

while chronic infections can remain in humans for a longer period of time up to an entire lifetime.

Examples of acute infections are influenza, measles, SARS and Ebola. Examples of chronic

infections are hepatitis C and the Human Immunodeficiency Virus (HIV).

Acute infections may be modeled with fewer equations and variables, since the infection

lasts just over a very short period of time in a person. A model for an acute infection may assume

a constant population over the period of the outbreak since the infection will last for a short

period of time. The population growth of the area of infection can be negligible. Chronic

infections are usually more complicated to model since there may be an increase or decrease in

population size during the course of the outbreak. It takes a longer time to study and collect

data. The assumed population size may be small, as in a household, a little larger, as in students

in a school, or even up to an entire country or the globe. Disease modeling has been successfully

used in real time to make predictions and understand the dynamics and analysis of the bovine

spongiform encephalopathy epidemic in cattle in Great Britain amongst other epidemics [20].

The number of equations and variables considered affect the behavior of the model, and

thus describe the disease outbreak characteristics. Different numbers of equations and variables

need to be used to accurately describe the behavior of different models. This means that some

models are simpler while some are more complex. The complexity of a model is usually

determined by the number of variables. Thus, the more equations and variables a model has, the

more complex it is to analyze.

Since models are representations of ideas and real life scenarios, it is impossible to

consider every single variable involved, so several assumptions need to be made. The more

assumptions made, the simpler the model and the easier it is to understand and modify, but this

3
comes at a cost. We generally try to find a balance between the assumptions and variables to be

considered in a model. If a model has too many assumptions, then it may not be helpful in

making predictions. Also, if a model has too many variables, it becomes complex and can be very

difficult to solve or modify. When making models we try to always use only the most important

factors as variables to get a simpler and more meaningful model. Recent models may contain

more variables since technological advancements in computing can help solve and analyse the

system of equations arriving from such a model.

1.2 COMPARTMENTS IN DISEASE MODELING

When modeling disease transmission, the population is split into different compartments,

each of which determines the group an individual in the population belongs to. No person is

allowed to be in more than one compartment at a particular point in time. The classical paper by

Kermack and McKendrick [17] laid the foundation of compartmental modeling in understanding

the spread of disease. These compartments vary depending on the type of infection that is to be

modeled. In general cases of acute infections, a member of the population falls into one of three

compartments: susceptible, infectious, or recovered.

The susceptible population consists of individuals who are free of infections. They are not

infected and are not carriers of the disease but can become infected if all transmission conditions

are satisfied. The infectious class consists of people who are carriers of the disease and are

infectious. They are able to transmit the disease to the susceptible class. The recovered class

consists of people who have been infected and then have recovered from the infection. Depending

on the disease, different compartments may be required. It is known that with certain infections,

like chickenpox, after recovery from the infection, a person is unlikely to be infected again and

thus has permanent immunity. The recovered compartment is required for the case of chickenpox.

Infections like influenza may be contacted again and there is no permanent immunity. So

influenza can be modeled with no recovered compartment.

The SIS (Susceptible-Infectious-Susceptible) epidemic model describes the transmission of

infectious disease when recovered individuals may become infectious again immediately (no

permanent immunity). When a disease is introduced into the population, transmission of infection

4
 β
S I
γ

Figure 1. SIS Epidemic Model

SIS Epidemic Model showing the transmission of infection in a fixed population. Here β and γ are

the transmission rate and recovery rate, respectively.

moves members of the susceptible compartment to the infectious compartment. Then later, the

infected individual recovers but does not gain immunity thereby becoming susceptible again. The

SIS model can be used to model the transmission of acute infections like influenza. Since the

infection generally lasts for a short period of time, we assume that the population size is constant

over the course of the outbreak, that is, the number of births and deaths are equal during this

time or there are no births or deaths during the outbreak period. It is also assumed that infections

can not be transmitted vertically, that is, infection cannot be passed from a mother to her unborn

child and so no child is born infected. Figure 1 shows a simple example of an SIS epidemic model.

The SIR (Susceptible-Infectious-Recovered) epidemic model describes the transmission of

infectious disease where the recovered individuals may no longer become infected again. A

susceptible person gets infected with disease and then later recovers from it. There is permanent

immunity in this model and the recovered individuals remain permanently in the recovered

compartment. Recovery can mean immunity to the disease or even death. Assumptions similar to

the SIS model can be used to simplify the model. We assume that the population size is constant

at every point in time of the outbreak and that the infection can only be transmitted horizontally,

that is, we consider transmission to be from person to person and not parent to child. The SIR

model can be used to model the transmission of chickenpox since immunity is gained after

recovery. Figure 2 shows a simple example of an SIR Epidemic Model.

The Kermack-McKendrick [17] model of 1927 is one of the simplest SIR models and is still

in use to date. It was used to explain the fluctuation in the number of infected people in the

Great Plague of London between 1665 and 1666 [5]. A complex SIR model can be obtained from

the Kermack-McKendrick model by making fewer assumptions and taking more parameters into

5
 β γ
S I R

Figure 2. SIR Epidemic Model

SIR Epidemic Model showing the transmission of infection in a fixed population. Here β and γ are

the transmission rate and recovery rate, respectively.

consideration.

1.3 INTRODUCTION TO STOCHASTIC PROCESSES

SIS and SIR epidemic models can be interpreted using stochastic processes. A stochastic

process is a collection of random variables {X(t) : t ∈ T } where each state X is a function of

time t, that is, a number X(t) is observed at each time t. The set T is the set of times when the

system can be observed. A stochastic process models how a random variable changes with time.

When the set T is countable and evenly spaced, the stochastic process is a discrete time process.

A simple example of a discrete time stochastic process is shown in Figure 3. When the set T

equals [0, ∞), the stochastic process is a continuous time process. A simple example of a

continuous time stochastic process is shown in Figure 4. The state space is defined depending on

what values the process can take. The state space tells us the states in which a stochastic process

can be observed. In disease modeling, the state spaces are usually countable. In the case of an

SIS model, the state space is the number of individuals in each of the two classes, the susceptible

class and the infectious class. For SIR models, the state space is the number of individuals in each

of the susceptible, infectious and recovered class.

There are basically two types of models: stochastic and deterministic. Stochastic models

can handle randomness and thus give different outputs for every input specified. Deterministic

models cannot handle randomness and thus give the same output for every input given [25]. Both

types of models are useful in disease modeling. Although deterministic models are usually simpler

than stochastic models, stochastic models are preferred since the transmission of disease in real

life is highly prone to randomness.

Stochastic processes in disease transmission are best modeled using Markov chains. In

6
 X(t0 ) X(t1 ) X(t2 ) X(tn ) X(t)

t0 t1 t2 tn t

Figure 3. Discrete Time Stochastic Process

A simple figure of a discrete time process. At each time tn , the observed value of the state is the

random variable X(tn ). The time difference between any two consecutive observations is constant.

X(t0 ) X(t1 ) X(t2 ) X(tn ) X(t)

t0 t1 t2 tn t

Figure 4. Continuous Time Stochastic Process

A simple figure of a continuous time process. At each time t, the observed value of the state is the

random variable X(t). The times of consecutive observations (t0 , t1 , . . .) are not evenly spaced.

most naturally occurring processes, the previous outcome may influence the next outcome of an

event. A stochastic process describing such an event is a Markov chain. The state space of a

Markov chain is usually discrete. The chance of transiting to any state depends on the current

state and time. Such events can be seen in disease modeling, where the probability of the number

of infectious persons at the nearest future time depends on the number of infectious persons in

the present time. That is, we need only information about the current state to be able to predict

the future state, and any additional information about the past state does not matter. This fact

is called the Markov property or memoryless property. The memoryless property makes Markov

chains an important tool in modeling naturally occurring events. For example, we may assume a

small population of fifty people with one infectious person. The probability of a new infection is

dependent only on the transmission of the infection by the one infectious person at that point in

time. It does not matter if the disease may have caused an epidemic in the past. Information

about the past has no effect on the future state transitions. The spread of the disease only

depends on the current state.

7
1.4 PARAMETER ESTIMATION

Statistical inference allows us to draw conclusions about a population system based on

observed data and parameters [12]. Parameters are numerical quantities that determine the

behaviour of a population system. We may take a sample of a population and make general

conclusions about the entire population and its parameters based on the observed samples. The

transmission rate β, recovery rate γ, and the basic reproduction number R0 are some parameters

of interest when modeling the spread of infectious diseases. The basic reproduction number R0 is

the number of secondary infections that are caused by one infectious person in a completely

susceptible population [30]. For the SIR and SIS disease models, the basic reproduction number is

β
R0 = ,
γ
1
where is the duration of infection [15]. For the SIS model, when R0 is greater than 1, the
γ
population will reach an endemic equilibrium, but when when R0 is less than 1, the infection

quickly dies down. Stochastic SIS models with R0 greater than 1 may have infection die out

because of the stochastic nature of the model. For the SIR model, when R0 is greater than 1, the

number of infectious individuals increases and then starts to decrease till the infection dies out.

When R0 is less than 1, the number of infectious individuals decreases until the infection dies

out [11].

In disease modeling, it can be difficult to collect all data relating to an outbreak, and so

inferential statistics are used to make generalizations about disease outbreaks in a population

based on the available observed data. Parameter estimation is one type of inferential statistics

and is important in describing the behaviour of a population system. It is the process of using

population data to estimate the parameters of a population system. Several methods have been

developed over the years. Point estimation is used to estimate a parameter as a single value from

a random sample.

Maximum likelihood estimation is a type of point estimation. It is one of the general

methods for parameter estimation and has been used in epidemiology [9]. Given a set of data, the

method of maximum likelihood estimates a parameter by finding the parameter value that

maximizes the likelihood of getting the observed data. The likelihood function is used in

8
determining the maximum likelihood estimate (MLE) of the parameters. Generally, if X(0) = x0 ,

X(1) = x1 , . . . , X(n) = xn are independent random variables and have a joint density function of

p[X(0), X(1), . . . , X(n) | θ], the likelihood function is a function of θ, where θ is now an unknown

parameter. The likelihood function is

n
Y
L(θ) = L(θ | x0 , x1 , . . . , xn ) = p(x0 , x1 , . . . , xn | θ) = p(xi | θ).
i=0

For a Markov process, the transition probabilities are not independent and are also regarded as

parameters. For some observed data, x0n = (x0 , x1 , ..., xn ), the likelihood function of the Markov

process is the product of the transition conditional probabilities and is given by

n
Y
L(θ) = L(θ | x0n ) = p(xn | xn−1 )p(xn−1 | xn−2 ) . . . p(x1 | x0 )p(x0 ) = p(x0 ) p(xi | xi−1 ).
i=1

The transition probability from state xi−1 to xi is denoted by pxi ←xi−1 , giving

p(xi | xi−1 ) = pxi ←xi−1 .

The likelihood function can be rewritten as

n n Y
n
n
Y Y
L(θ) = L(θ|x0n ) = p(x0 ) p(xi |xi−1 ) = p(x0 ) ji
pj←i , (1.1)
i=1 i=0 j=0

where nji is the count of transitions from i to j. It is possible for a process, after leaving state i

to j, to return back to state i, and then transition again to state j. For a Markov process, we can

estimate the parameter θ by maximizing the logarithm of the likelihood function (1.1) with the

result being the maximum likelihood estimator of θ. We may also choose to minimize the negative

log likelihood function to estimate the parameter θ. In estimating parameters in a discrete time

stochastic process, the times between consecutive events are equal and thus uniformly distributed.

In the case of a continuous-time stochastic process, the times between consecutive events are

random, and are exponentially distributed. The distribution of the times between consecutive

events can affect the likelihood function. More details on parameter estimation using the method

of maximum likelihood are given in Chapter 4.

9
 After obtaining the point estimates of the parameters using MLE, it is important to

compare our estimates with the actual values of the parameter. A point estimate can overestimate

or underestimate a parameter. Bias is the difference between the expected value of our estimates

and the actual value of our estimate. If θ̂ is an estimate of a parameter θ, the bias of θ̂ is

B(θ̂) = E(θ̂) − θ,

where E(θ̂) is the expected value of θ̂ [31]. The bias may be positive or negative. If B(θ̂) > 0, θ̂ is

an overestimate of θ and is a biased estimate. If B(θ̂) < 0, θ̂ is an underestimate of θ and is also a

biased estimate. If B(θ̂) = 0, θ̂ is an unbiased estimate. The mean squared error (MSE) is simply

the mean of the squares of the errors. The MSE measures variation in estimation and is given as

MSE(θ̂) = E(θ̂ − θ)2 .

The MSE is always non-negative. The smaller the MSE, the better the precision of the estimates.

1.5 AIMS AND OBJECTIVES

The aim of this thesis is to explain the formulations of the simple SIR and SIS epidemic

models for the spread of infectious diseases and also to estimate transmission and recovery rates

of the models. In Chapter 2, we will see how to build simple SIS and SIR models using discrete

time Markov chains. In Chapter 3, we build the same models using continuous time Markov

chains. Finally, in Chapter 4, we will use the method of maximum likelihood to estimate

transmission and recovery rates using data obtained from simulated outbreaks. For each estimate

of the transmission rate and recovery rate, we compute the bias and mean squared error of the

estimators.

10
 CHAPTER 2

EPIDEMIC MODELS USING DISCRETE TIME MARKOV CHAINS

In this chapter, we use discrete time Markov chains to formulate SIS and SIR epidemic

models.

2.1 DISCRETE TIME MARKOV CHAINS

Let {X(t) : t ∈ T } be random variables denoting the state of a system at time

t ∈ T = {0, ∆t, 2∆t, ...} with a discrete state space

{i0 , i∆t , i2∆t , ..., it−∆t , i, j} = S ⊂ {0, 1, 2, ..., N }.

A stochastic process is a Discrete Time Markov Chain (DTMC) if it satisfies the following

equation

P [X(t + ∆t) = j | X(t) = i, ..., X(∆t) = i∆t , X(0) = i0 ]

= P [X(t + ∆t) = j | X(t) = i] = pj←i (∆t).

A discrete time process satisfies the Markov property, that is, the process at any time

t + ∆t depends only on the state of the immediate past process in time t. As time goes on, the

process does not require the information from further back for future transitions. The probability

that the process will transition from state i at time t to state j at time t + ∆t is pj←i (∆t). We

assume that the process is time homogeneous, that is, the transition probability does not change

with time. The process is independent of t and

P [X(t + ∆t) = j | X(t) = i] = P [X(∆t) = j | X(0) = i] = pj←i (∆t).

The one-step transition probability is the probability of transiting from state i to state j

in one step, that is, in a period of ∆t. We denote the one-step transition probability as pj←i (∆t).

It is sometimes denoted by pji (∆t). The n-step transition probability is the probability of moving

11
from state i to state j in n steps, that is, in a period of n∆t and is given as

(n)
P [X(t + n∆t) = j | X(t) = i] = pj←i (n∆t).

The probability pj←i (∆t) may sometimes be zero.

A transition matrix P (∆t) is used to describe all possible one-step transitions in a discrete

time Markov chain. Since the states can take values from 0 to N , P (∆t) is an (N + 1) × (N + 1)

matrix. The matrix P (∆t) is given as

P (∆t) = (pj ←
−i )j,i∈S ,

that is,  
 p0←0 p0←1 ... p0←i ... p0←N 
 
p
 1←0 p1←1 ... p1←i ... p1←N  
 . .. .. .. 
 . .. ..
 . . . . . . 

P (∆t) = 

.

 pj←0 pj←1 ... pj←i ... pj←N 
 
 . .. .. .. .. .. 
 .. . . . . . 
 
 
pN ←0 pN ←1 . . . pN ←i . . . pN ←N

Each cell in the matrix is the probability of transiting from the state of the column to the

state of the row. For example, the element (j, i) corresponds to pj←i and is the probability of

transiting from state i to state j. The columns of the transition matrix P (∆t) form a probability

distribution and sum up to 1. We have

N
X
pj←i (∆t) = 1,
j=0

and the transition matrix is called a left stochastic matrix. If P (∆t) is transposed then the rows

of the new transition matrix sum to 1. The resulting matrix is called a right stochastic matrix.

We will regard P (∆t) as a left stochastic matrix.

States of a Markov chain are classified according to their properties. A Markov chain can

generally have two types of states: transient and recurrent. A state i of a Markov chain is a

12
transient state if, after the process leaves to another state j, the probability of returning to i is

less than 1. A state j of a Markov chain is a recurrent state if, after the process leaves to another

state k, the probability of returning to j is 1. This means that transition back to state j is almost

surely guaranteed. For both transient and recurrent states, there is no guarantee of a transition

back to the states. An absorbing state is a special kind of recurrent state. A state k is an

absorbing state if, after the process reaches k, the process cannot leave state k. No other state is

reachable from an absorbing state. A finite Markov chain must have at least one recurrent state.

2.2 SIS EPIDEMIC MODELS USING DISCRETE TIME MARKOV CHAINS

In formulating SIS epidemic models using discrete time Markov chains, we follow Allen’s

approach [2]. Let S(t) and I(t) denote discrete random variables counting the number susceptible

and infectious persons at each time t ∈ T = {0, ∆t, 2∆t, ...} with S(t), I(t) ∈ {0, 1, 2, ..., N }. These

two random variables count the number of individuals in each of the two compartments needed

for an SIS epidemic model, the susceptible and the infectious compartments. We make the

assumption that the population is constant throughout the observed time of the disease outbreak.

That is, at any time t, N = S(t) + I(t), thus

dN dS(t) dI(t)
= + = 0. (2.1)
dt dt dt

The discrete time derivative for the SIS model is given as Equation 2.1. Since there are only two

compartments in this model, it is sufficient to use the value of the infectious class to estimate the

value of susceptible class, reducing the dimension of the system.

By choosing the time step small enough, we may assume that at most one transition

happens during each time step. That is, for I(t) = i and a time step of ∆t, only one of the

following can occur:

∆t ∆t ∆t
i −→ i + 1, i −→ i − 1 or i −→ i.

For every change in time, one new person may get infected, recover from an infection and become

susceptible again, or there may be no change in the number of infectious persons in the

population. With a new infection in the population, the number of susceptible persons decreases

13
 i0 i∆t i2∆t i j

0 ∆t 2∆t t t + ∆t
∆t ∆t ∆t

Figure 5. Trajectory of a Discrete Time SIS Model

A trajectory of a DTMC SIS model showing transition states and uniform holding times between

consecutive state transitions.

by one and the infectious class goes up by one. A recovery in the population indicates that the

infectious class decreases by one and the susceptible class goes up by one. For the SIS model, we

define a trajectory as a sequence U of states with holding times as

U = (i0 , ∆t, i∆t , ∆t, . . . , i, ∆t, j).

This implies that the system started at state i0 . Then, after a period of ∆t units of time, the

system transitioned to state i∆t . The system stayed at state i∆t for another period of ∆t units of

time and then transitioned to the next state, and so on [23]. Figure 5 shows the trajectory of a

discrete time Markov chain SIS model.

2.2.1 TRANSITION PROBABILITIES OF A DTMC SIS EPIDEMIC MODEL

Several factors affect the transmission of infectious disease in a population such as the

climate, the population contact structure between the susceptible and the infectious populations,

and the probability of transmission [16]. For a simple SIS epidemic model, we consider only the

contact structure between the susceptible and infectious populations for disease transmission. We

define β > 0 and γ > 0 as the transmission and recovery rates of the population, respectively. We

make the assumption that there is homogeneous mixing of the susceptible and infectious

populations and the contact structure is frequency dependent. The contact structure being

frequency dependent implies that the transmission rate β does not change with population size.

This means that the transmission rate remains constant even as the number of infectious people

14
goes up. The number of susceptible individuals newly infected at time step t is therefore given

by [16]
βS(t)I(t)
.
N

The number of infectious individuals becoming susceptible is dependent on the number of

infectious individuals in a population. In this case, it is solely determined by the recovery rate.

The rate of infectious individuals becoming susceptible at any time t is therefore given by γI(t).

We have that the change in the susceptible class is

dS(t) βS(t)I(t)
=− + γI(t).
dt N

The probability of transiting from i to i + 1 is

βi(N − i)
pi+1←
−i (∆t) = ∆t,
N

where s = N − i because of the fixed population size. The number of individuals recovering at

any time t is given by γI(t) and, for every recovery, the change in the infectious class is given by

dI(t) βS(t)I(t)
= − γI(t).
dt N

The probability of transiting from i to i − 1 is

pi−1←
−i (∆t) = γi∆t.

The sum of the probabilities of all possible transitions must add up to one. So, the probability

that the number of infectious remains unchanged after a time step is

 
βi(N − i)
−i (∆t) = 1 −
pi← + γi ∆t.
N

15
The transition probabilities of the DTMC SIS epidemic model are then given by

βi(N − i)
∆t, j =i+1


N






γi∆t, j =i−1


pj ←
−i (∆t) =  
βi(N − i)
1 − + γi ∆t, j = i




 N



0,

otherwise.

For simplification and easy readability of the SIS epidemic model, we denote the transition

probabilities of the birth of an infection and recovery or death of an infection as b(i)∆t and

d(i)∆t, respectively. Therefore, the SIS epidemic model can be represented as [2]


b(i)∆t, j =i+1








d(i)∆t,
 j =i−1
pj ←
−i (∆t) =
1 − [b(i) + d(i)] ∆t, j = i








0,

otherwise.

To always have a valid transition probability between 0 and 1, we choose the time step ∆t small

enough so that [2]

max {[b(i) + d(i)]∆t} ≤ 1.

i∈{1,2,...,N }

We form the tridiagonal transition matrix P (∆t). Using the transition probabilities, P (∆t) is

given as

16
 
1 d(1)∆t 0 ... 0 0 
 
0 1 − [b(1) + d(1)]∆t d(2)∆t ... 0 0 
 
 
1 − [b(2) + d(2)]∆t
 
0 b(1)∆t ... 0 0 
 
 
0 0 b(2)∆t ... 0 0 
.
 
. .. .. .. .. ..
. .
. . . . .


 
 
0
 0 0 ... d(N − 1)∆t 0 

 
0
 0 0 ... 1 − [b(N − 1) + d(N − 1)]∆t d(N )∆t 
 
0 0 0 ... b(N − 1)∆t 1 − d(N )∆t

The state i = 0 is an absorbing state of the matrix since it is impossible to transition from

i = 0 to another state with i > 0. That is, it is not possible for a population with no infection to

leave the disease free state. If there can be a transition from i to j and from j to i, i and j are

said to be in the same communicating class. The state i = 0 forms a communicating class and the

states with i > 0 forms another communicating class. In the class with states i > 0, the

probability of transitioning between any two states in the class is positive, but also the probability

of transitioning from any one of the states out of the class to i = 0 is also positive. Thus the states

in the class with i > 0 are transient as it is possible to return after some time but not guaranteed.

2.2.2 PSEUDOCODE FOR DTMC SIS MODELS AND SIMULATED EXAMPLE

Provided all parameters and initial conditions are given, we can plot a graph of the SIS

epidemic model comparing the changes in the number of infectious individuals against time.

Using Allen’s approach [2], the algorithm of the code is as follows:

1. Initialize the values of β, γ, population size N , time step ∆t, and time duration of the

outbreak tend .

2. Create arrays S(t) and I(t). Set t = 0. Let I(0) be the initial number of infectious

individuals and S(0) = N − I(0).

3. For t from 0 to tend in an increment of ∆t, while t is less than tend :

17
  
βS(t)I(t)
(a) Compute the probability of a new infection (p1 ) using ∆t.
N
(b) Compute the probability of a recovery from infection (p2 ) using γI(t)∆t.

(c) Compute the probability of no change in the number of infectious using 1 − (p1 + p2 ).

(d) Select a random number u from a uniform distribution of (0,1).

(e) Compare u with p1 , p2 and 1 − (p1 + p2 ).

(f) If 0 < u ≤ p1 , decrease the number of susceptible individuals by 1 and increase the

number of infectious individuals by 1. Hence S(t + ∆t) = S(t) − 1 and

I(t + ∆t) = I(t) + 1.

(g) If p1 < u ≤ p1 + p2 , increase the number of susceptible individuals by 1 and decrease

the number of infectious individuals by 1. Hence S(t + ∆t) = S(t) + 1 and

I(t + ∆t) = I(t) − 1.

(h) Otherwise, p1 + p2 < u < 1. In this case, the number of susceptible and infectious

remains the same. Hence S(t + ∆t) = S(t) and I(t + ∆t) = I(t).

4. Plot the graph of the times t against the number of infectious at each time, I(t).

In the algorithm, we simulate a discrete random variable u to determine which of the

events occurred. In general, for n events, we partition the interval (0, 1) into n parts, according to

the probability of each of the individual events p1 , p2 , . . ., pn . The pieces of the partition are

(0, p1 ], (p1 , p1 + p2 ], . . . , (p1 + p2 + . . . + pn−1 , 1), where ni=1 pi = 1 is the sum of the column of
P

the transition matrix [4]. The ith event occurs when u falls inside the ith interval. The random

variable u has the same distribution as pi . We use the default random number generator

(Mersenne Twister) in Matlab with a period of 219937 − 1. The period is large enough to avoid

internal correlations for the computations in this work. We use these ideas in all our models.

We simulate an example of a DTMC SIS epidemic model. Suppose that there is an

influenza outbreak in a community of 100 people with 5 people initially infected. The

transmission rate β is 1, the recovery rate γ is 0.5, and the fraction of time between consecutive

events is 0.01. For 3 simulations, the graph of time against the number of infectious individuals is

18
Figure 6. A DTMC SIS Epidemic Model

Simulations of a DTMC SIS Epidemic Model with three stochastic paths using parameters N = 100,

I(0) = 5, β = 1, γ = 0.5 and ∆t = 0.01.

presented in Figure 6. The graph was plotted by following the pseudocode in this section, with

the aid of the epidemic model code used by Allen [2].

For this example, the basic reproduction number R0 is 2. For two of the three outbreaks

(red and green paths) in Figure 6, the population reaches an endemic equilibrium with about 50

infected people. The blue path in Figure 6 shows the infection quickly dies down even though

R0 > 1. This is due to the stochastic nature of the model.

2.3 SIR EPIDEMIC MODELS USING DISCRETE TIME MARKOV CHAINS

For SIR epidemic models, there are three compartments, and the random variables S(t),

I(t) and R(t) are required. As in SIS epidemic models, S(t) and I(t) count the number of

infectious and susceptible individuals in the population at time t ∈ T = {0, ∆t, 2∆t, ...} with

S(t), I(t) ∈ {0, 1, 2, ..., N }. The additional random variable R(t) counts the number of recovered

19
individuals in the population at time t ∈ T , also with R(t) ∈ {0, 1, 2, ..., N }. Recovered

individuals are no longer susceptible and are immune to the infection. The assumption of a

constant population still holds for the outbreak and, so, at any time t,

N = S(t) + I(t) + R(t).

Thus
dN dS(t) dI(t) dR(t)
= + + = 0.
dt dt dt dt

We use the susceptible and infectious classes to compute the recovered class to reduce the

dimension of the system. The process is bivariate, with the recovered class depending on the

susceptible and infectious class, and thus

R(t) = N − S(t) − I(t).

By choosing the time step small enough, we may assume that at most one transition happens

during each step. For a time step of ∆t, only one of the following can occur:

∆t ∆t ∆t
(s, i) −→ (s − 1, i + 1), (s, i) −→ (s, i − 1) or (s, i) −→ (s, i).

For every change in time, only one individual in the population may get infected, recover

completely from an infection with no way of becoming susceptible again, or there may be no

change in the number of infectious persons in the population. With a new infection in the

population, the number of susceptible persons decreases by one and the infectious class goes up

by one. A recovery indicates that the infectious class decrease by one and the recovered class goes

up by 1. For the SIR model, we define a trajectory as a sequence U of states with holding times

U = ((s0 , i0 ), ∆t, (s∆t , i∆t ), ∆t, . . . , (s, i), ∆t, (k, j)).

This implies that the system started at state (s0 , i0 ). Then, after a period of ∆t units of time, the

system transitioned to state (s∆t , i∆t ). The system stayed at state (s∆t , i∆t ) for another ∆t units

of time and then transitioned to the next state, and so on. Figure 7 shows the trajectory of a

20
 (s0 , i0 ) (s∆t , i∆t ) (s2∆t , i2∆t ) (s, i) (k, j)

0 ∆t 2∆t t t + ∆t
∆t ∆t ∆t

Figure 7. Trajectory of a Discrete Time SIR Model

A trajectory of a DTMC SIR model showing transition states and uniform holding times between

consecutive state transitions.

discrete time Markov chain SIR model.

2.3.1 TRANSITION PROBABILITIES OF A DTMC SIR EPIDEMIC MODEL

For a DTMC SIR epidemic model, the change in the susceptible class at time t is

dS(t) βS(t)I(t)
=− .
dt N

This is because individuals are transitioning from the susceptible compartment to the infectious

compartment, with no person returning back to the susceptible compartment. The number of

susceptibles will continue to decrease with time. This decrease implies a new infection in the

population and a transition from (s, i) to (s − 1, i + 1). The probability of new infection is

βsi
p(s−1,i+1)←
−(s,i) (∆t) = ∆t.
N

Individuals in the infectious compartment are also constantly transiting to the recovered

compartment so that the change in the infectious class at time t is

dI(t) βS(t)I(t)
= − γI(t).
dt N

Recovery means a transition from (s, i) to (s, i − 1). The probability of recovery is

p(s,i−1)←
−(s,i) (∆t) = γi∆t.

21
Since a constant population is assumed, the probability that the number of infectious remains

unchanged after a time step is

 
βsi
−(s,i) (∆t) = 1 − N + γi ∆t.
p(s,i)←

Therefore, we have transition probabilities of the SIR epidemic model as


βsi
∆t, (k, j) = (−1, 1)


N






γi∆t, (k, j) = (0, −1)


p(s+k,i+j)←
−(s,i) (∆t) =   
βsi

1− + γi ∆t, (k, j) = (0, 0)


 N



0,

otherwise.

For simplicity, as in the SIS epidemic model, the SIR epidemic model can be represented as

b(i)∆t, (k, j) = (−1, 1)








d(i)∆t,
 (k, j) = (0, −1)
p(s+k,i+j)←
−(s,i) (∆t) =
1 − [b(i) + d(i)] ∆t, (k, j) = (0, 0)








0,

otherwise.

2.3.2 PSEUDOCODE FOR DTMC SIR MODELS AND SIMULATED EXAMPLE

Using Allen’s approach [2], the algorithm of a DTMC SIR model code is as follows:

1. Initialize the values of β, γ, population size N , time step ∆t, and time duration of the

outbreak tend .

2. Create arrays S(t), I(t) and R(t). Set t = 0 and R(0) = 0. Let I(0) be the initial number of

infectious individuals and S(0) = N − I(0).

3. For t from 0 to tend in an increment of ∆t, while t is less than tend :

 
βS(t)I(t)
(a) Compute the probability of a new infection (p1 ) using ∆t.
N

22
 (b) Compute the probability of a recovery from infection (p2 ) using γI(t)∆t.

(c) Compute the probability of no change in number of infectious using 1 − (p1 + p2 ).

(d) Select a random number u from a uniform distribution of (0,1).

(e) Compare u with p1 , p2 and 1 − (p1 + p2 ).

(f) If 0 < u ≤ p1 , decrease the number of susceptible individuals by 1 and increase the

number of infectious individuals by 1. Hence we have

S(t + ∆t) = S(t) − 1, I(t + ∆t) = I(t) + 1, and R(t + ∆t) = R(t).

(g) If p1 < u ≤ p1 + p2 , decrease the number of infectious individuals by 1 and increase the

number of recovered individuals by 1. Hence we have

S(t + ∆t) = S(t), I(t + ∆t) = I(t) − 1, and R(t + ∆t) = R(t) + 1.

(h) Otherwise, p1 + p2 < u < 1. In this case, the number of susceptible, infectious and

recovered individuals remains the same. Hence

S(t + ∆t) = S(t), I(t + ∆t) = I(t), and R(t + ∆t) = R(t).

4. Plot the graph of the times t against the number of infectious at each time, I(t).

We simulate an example of a DTMC SIR epidemic model using the algorithm just stated.

Suppose that there is a measles outbreak in a community of 100 people with 5 people initially

infected. For this example, the transmission rate β is 1, the recovery rate γ is 0.5, and the time

between consecutive events is 0.01. For 3 simulations, the graph of time against the number of

infectious is given in Figure 8.

The path of the three simulations represented in Figure 8 shows that after the number of

infections peak, individuals gradually recover and the infection dies down eventually. Due to the

stochastic nature of the model, there may be no outbreak and the infection may die out quickly.

23
Figure 8. A DTMC SIR Epidemic Model

Simulations of a DTMC SIR Epidemic Model with three stochastic paths using parameters N = 100,

I(0) = 5, β = 1, γ = 0.5 and ∆t = 0.01.

24
 CHAPTER 3

EPIDEMIC MODELS USING CONTINUOUS TIME MARKOV CHAINS

In this chapter, we use continuous time Markov chains to study SIS and SIR epidemic

models.

3.1 CONTINUOUS TIME MARKOV CHAINS

Continuous time Markov chains are used to model systems where the process remains in a

state for a random amount of time before transitioning to the next state. Let {X(t) : t ∈ T } be

random variables denoting the state of a system at time t ∈ T = [0, ∞) with a discrete state space

S ⊂ {0, 1, 2, ..., N }. The random variables {X(t) : t ∈ T } form a Continuous Time Markov Chain

(CTMC) if for any sequence of times, t0 , t1 , . . . , tn−1 , tn , s, t with t0 < t1 < ... < tn−1 < tn < s < t,

the random variables {X(t) : t ∈ T } satisfy the following equation

P [X(t) = j | X(s) = i, X(tn ) = in , ..., X(t0 ) = i0 ] = P [X(t) = j | X(s) = i] = pj←i (t − s). (3.1)

The continuous time process satisfies the Markov property [2], that is, the process at time

t depends only on the state of the immediate past process in time s, as in Equation 3.1. Unlike a

discrete time Markov process, transitions in a continuous time Markov process do not happen at a

fixed unit of time but can occur at any time. Similarly to the DTMC, we denote the probability

that the process will transit from state i at time s, to state j at time t, to be pj←i (t − s). We also

make the assumption that pj←i (t − s) is independent of t at any given time, hence, the process is

time homogeneous and

P [X(t) = j | X(s) = i] = P [X(t − s) = j | X(0) = i] = pj←i (t − s).

The continuous time random process X(s) will remain in its state for ∆t = t − s units of time,

−i (t − s) = pj ←
before transiting to X(t) with a transition probability pj ← −i (∆t). We call the time
it takes for a process to transit from a state i to a state j the holding time or the interevent time.

In general, for any two consecutive states in and in+1 , a continuous time process in state in at

25
time tn will remain at in for time tn+1 − tn , before transiting to in+1 . We denote the interevent

time between in and in+1 as Tn . For a CTMC, the memoryless property of the stochastic process

along with time homogeneity forces the interevent times to also be memoryless. To show this, for

s, t ∈ T , consider the conditional probability that Tn > s + t given that Tn > s,

Prob(Tn > s + t | Tn > s).

For Tn > s, the process will remain in state in at any time tn + u while 0 ≤ u ≤ s, so

X(tn + u) = in for 0 ≤ u ≤ s. Also, for Tn > s + t, the process will remain in state in at any time

tn + u while 0 ≤ u ≤ s + t, so X(tn + u) = in for 0 ≤ u ≤ s + t. We want to show that

Prob(Tn > s + t|Tn > s) = Prob(Tn > t).

For Tn > t, the process will remain in state in at any time tn + u while 0 ≤ u ≤ t, so

X(tn + u) = in for 0 ≤ u ≤ t. The conditional probability that Tn > s + t given that Tn > s is

therefore

Prob(Tn > s + t | Tn > s)

= Prob(X(tn + u) = in : 0 ≤ u ≤ s + t | X(tn + u) = in : 0 ≤ u ≤ s).

Let A be the event that X(tn + u) = in for 0 ≤ u ≤ s and B be the event that

X(tn + u) = in for s ≤ u ≤ s + t. The occurrence of both A and B, that is, A ∩ B, is the event

that X(tn + u) = in for 0 ≤ u ≤ s + t. From the definition of conditional probability, we can show

the property

Prob(A ∩ B | A) = Prob(B | A).

Therefore, we have

Prob(Tn > s + t | Tn > s)

= Prob(X(tn + u) = in : s ≤ u ≤ s + t | X(tn + u) = in : 0 ≤ u ≤ s).

26
 Using the Markov property, the conditional probability simplifies to

Prob(Tn > s + t | Tn > s) = Prob(X(tn + u) = in : s ≤ u ≤ s + t | X(tn + s) = in ).

Since the random process is time homogeneous

Prob(Tn > s + t | Tn > s) = Prob(X(tn + u) = in : 0 ≤ u ≤ t | X(tn ) = in ),

and so

Prob(Tn > s + t | Tn > s) = Prob(X(tn + u) = in : 0 ≤ u ≤ t)

= Prob(Tn > t).

This shows that Tn has a distribution with the memoryless property, which implies that Tn is

exponentially distributed [26]. Let Prob(Tn > t) = Hn (t), we have

Hn (t) = e−λn t ,

where λn > 0 is the rate parameter. The cumulative distribution function Fn (t) is given by

Fn (t) = 1 − e−λn t .

3.1.1 GENERATOR MATRICES

A continuous time Markov chain can be exactly described by the probabilities of

transitioning from one state to another and the mean times spent in each state. The probabilities

of transitioning from one state to another form an embedded discrete time Markov chain as

shown in Figure 9. The mean time spent in each state gives the exponential distribution for the

interevent times. Next we show how to use the infinitesimal transition probabilities pj←i (∆t) to

form the embedded DTMC and find the mean interevent times.

To begin, we form the generator matrix Q of transition rates qj←i as the one-sided

derivatives of the infinitesimal transition probabilities at ∆t = 0. Using Allen’s approach [3] to

27
 i0 i1 i2 i j

t0 t1 t2 ∆t s t
T0 T1 Ti

Figure 9. Embedded Discrete Time Process

An illustration of an embedded discrete time process in a continuous time process.

obtain the transition rates, we assume the probabilities pj←i (∆t) are continuous and differentiable

at ∆t ≥ 0. There are no transitions in a time period of ∆t = 0 and so

pj←i (0) = 0, i 6= j,

and

pj←i (0) = 1, i = j.

To find the rates qj←i when i 6= j, we compute

pj←i (∆t) − pj←i (0) pj←i (∆t)

qj←i = lim = lim .
∆t→0+ ∆t ∆t→0 + ∆t

PN
In this case, we have j=0 pj←i (∆t) = 1, and it follows that

N
X
1 − pi←i (∆t) = pj←i (∆t).
j=0,
j6=i

Therefore, we have

P N
pi←i (∆t) − 1 − j6=i pj←i (∆t)
X
qi←i = lim = lim =− qj←i .
∆t→0+ ∆t ∆t→0+ ∆t
j=0,
j6=i

The relation between the infinitesimal transition probabilities pj←i and the transition

28
rates qj←i is often expressed as

pj←i (∆t) = qj←i · ∆t + o(∆t), i 6= j

and

pi←i (∆t) − 1 = qi←i · ∆t + o(∆t),

where a quantity e(∆t) is o(∆t) if it satisfies [18]

e(∆t)
lim = 0.
∆t→0+ ∆t

The rates qj←i are used to form the generator matrix

 
 q0←0 q0←1 ... q0←i ... q0←N 
 
q
 1←0 q1←1 ... q1←i ... q1←N  
 . .. .. .. 
 . .. ..
 . . . . . . 

Q=
 .

 qj←0 qj←1 ... qj←i ... qj←N 
 
 . .. .. .. .. .. 
 .. . . . . . 
 
 
qN ←0 qN ←1 . . . qN ←i . . . qN ←N

Using the generator matrix, the probability of transitioning from state i to state j for the

embedded DTMC is calculated as

qj←i
P . (3.2)
i6=j qj←i

In the embedded DTMC, the probability of transitioning from i to i is 0. The mean of the

exponential interevent time in state i is

−1
,
qi←i

meaning −qi←i = λi is the rate of the exponential distribution of the time for state i.

29
 i0 i1 i2 i j

t0 t1 t2 s t
T0 T1 Ti

Figure 10. Trajectory of a Continuous Time SIS Model

A trajectory of a CTMC SIS model showing transition states and non-uniform holding times be-

tween consecutive state transitions.

3.2 SIS EPIDEMIC MODELS USING CONTINUOUS TIME MARKOV CHAINS

In formulating SIS epidemic models using continuous time Markov chains, we follow

Allen’s approach [2]. The model uses the two compartments: susceptible and infectious. Let S(t)

and I(t) denote discrete random variables counting the numbers of susceptible and infectious

persons at any instant of time t ∈ [0, ∞) with S(t), I(t) ∈ {0, 1, 2, ..., N }. We make the

assumption that the population is constant through the observed time and so N = S(t) + I(t).

The process is univariate, and at any time t we have S(t) = N − I(t). For the continuous time SIS

epidemic model, we define a trajectory as a sequence U of states with the holding times

U = (i0 , T0 , i1 , T1 , . . . , i, Ti , j).

This implies that the system started at a state i0 . Then, after a period of T0 units of time, the

system transitioned to i1 , stayed at i1 for a period of T1 units of time, and then transitioned to

the next state i2 , and so on. Figure 10 shows the trajectory of a continuous time Markov chain

SIS model.

The infinitesimal transition probabilities of the embedded discrete time Markov chain are

valid for sufficiently small ∆t. If I(t) = i only one of the following transitions can occur:

∆t ∆t ∆t
i −→ i + 1, i −→ i − 1 or i −→ i.

Since a constant population is assumed, for every change in time the number of infectious

30
increases by one, reduces by one or there is no change in the number of infectious. If the number

of infectious increases by one, the susceptible compartment loses one person and the infectious

compartment gains the loss. If the number of infectious reduces by one, the susceptible

compartment gains one person. A recovery can mean a decrease in the number of infectious

individuals.

3.2.1 TRANSITION PROBABILITIES OF A CTMC SIS EPIDEMIC MODEL

For a simple CTMC SIS epidemic model, we assume a similar contact structure as in the

DTMC SIS epidemic model. The contact structure is frequency dependent. We also assume there

is homogeneous mixing of the population, and given that the transmission rate β is greater than

0, the infinitesimal probability of transiting from i to i + 1 is

βi(N − i)
pi+1←
−i (∆t) = ∆t + o(∆t).
N

If γ > 0 is the recovery rate, the number of individuals recovering at any time t is given by γI(t),

and therefore the infinitesimal probability of transition from i to i − 1 is

pi−1←
−i (∆t) = γi∆t + o(∆t).

The sum of the probabilities of all possible transitions must add up to one. So, the infinitesimal

probability that there is no change in the number of infected is

 
βi(N − i)
−i (∆t) = 1 −
p i← + γi ∆t + o(∆t).
N

31
Combining the probabilities of the three different possible transitions with time, the transition

probability of the embedded discrete time SIS epidemic model is then given by

βi(N − i)
∆t + o(∆t), j =i+1


N






γi∆t + o(∆t), j =i−1


pj ←
−i (∆t) =   
βi(N − i)

1− + γi ∆t + o(∆t), j = i


 N



o(∆t),

otherwise.

This can be represented as [2]


b(i)∆t + o(∆t),

 j =i+1





d(i)∆t + o(∆t),
 j =i−1
pj ←
−i (∆t) =
1 − [b(i) + d(i)] ∆t + o(∆t), j = i








o(∆t),

otherwise.

We use these probabilities to compute the transition rates and then form the infinitesimal

generator matrix
 
0 d(1) 0 ... 0 0 
 
0 −[b(1) + d(1)] d(2) ... 0 0 
 
 
−[b(2) + d(2)]
 
0 b(1) ... 0 0 
 
 
0 0 b(2) ... 0 0 
Q = . .
 
. .
.. .. .. .. .. 
. . . . . 

 
 
0
 0 0 ... d(N − 1) 0  
 
0
 0 0 . . . −[b(N − 1) + d(N − 1)] d(N ) 

 
0 0 0 ... b(N − 1) −d(N )

32
We use Equation 3.2 to find the transition probabilities of the embedded DTMC SIS model. From

the generator matrix Q, the probability of transiting from i to i + 1 (susceptible to infectious) is

b(i)
,
b(i) + d(i)

βsi
where b(i) = and d(i) = γi. Therefore the probability of a new infection is
N
 
βsi
N
 .
βsi
+ γi
N

Similarly, the probability of transiting from i to i − 1 (infectious to susceptible) is

d(i)
.
b(i) + d(i)

Therefore the probability of a recovery is

γi
 .
βsi
+ γi
N

The rate for the exponential interevent time for state i is

βsi
b(i) + d(i) = + γi.
N

3.2.2 PSEUDOCODE FOR CTMC SIS MODELS AND SIMULATED EXAMPLE

Given all parameters and initial conditions, we can plot a graph of the SIS epidemic model

in continuous time, comparing the changes in the number of infectious individuals against time.

Using Allen’s approach [2], the algorithm of the code is given as follows:

1. Initialize the values of β, γ, the population size N , and the time duration of the

outbreak tend .

2. Create arrays S(j), I(j), and t(j). Set j = 1 and t(1) = 0. Let I(1) be the initial number of

33
 infectious individuals and S(1) = N − I(1).

3. While I(j) > 0 and t(j) < tend ,

βS(j)I(j)
(a) Let a = and b = γI(j).
N
a
(b) Compute the probability of a new infection (p1 ) using .
a+b
b
(c) Compute the probability of a recovery from infection (p2 ) using .
a+b
(d) Select two random numbers u1 and u2 from a uniform distribution of (0,1).

(e) Compare u1 with p1 and p2 .

(f) If 0 < u1 ≤ p1 , decrease the number of susceptible individuals by 1, and increase the

number of infectious individuals by 1. Hence S(j + 1) = S(j) − 1 and

I(j + 1) = I(j) + 1.

(g) Otherwise, p1 < u1 ≤ 1. In this case, increase the number of susceptible individuals

by 1, and decrease the number of infectious individuals by 1. Hence

S(j + 1) = S(j) + 1 and I(j + 1) = I(j) − 1.

ln(u2 )
(h) Select the time of the next event using u2 as t(j + 1) = t(j) − .
a+b
(i) Update the vector index j to j + 1 and repeat.

4. Plot the graph of the times t(j) against the number of infectious at each time, I(j).

In the algorithm, u1 is used to determined which event occurred and u2 is used to

determine the time of occurrence. In the computation of the holding time, we can convert a

uniform random variable, u ∈ (0, 1), to an exponentially distributed random number using a

probability integral transform. The cumulative distribution function of the holding time is

Fi (t) = 1 − e−λi t .

Since Fi (Fi−1 (u)) = u, then

−1
1 − e−λi Fi (u)
= u,

−1
e−λi Fi (u)
= 1 − u,

34
Figure 11. A CTMC SIS Epidemic Model

Simulations of a CTMC SIS Epidemic Model with three different stochastic paths using parameters

N = 100, I(1) = 5, β = 1 and γ = 0.5.

ln (1 − u)
Fi−1 (u) = − ,
λi
ln (u)
Fi−1 (u) = − .
λi

Therefore, for any u from a uniform random variable on (0, 1),

ln (u)
Ti = Fi−1 (u) = − .
λi

We simulate an example of a CTMC SIS epidemic model. We use a population of 100

people with 5 people initially infected. We use the transmission rate β = 1 and the recovery rate

γ = 0.5. The graph of the time against the number of infectious individuals is presented in

Figure 11.

3.3 SIR EPIDEMIC MODELS USING CONTINUOUS TIME MARKOV CHAINS

In the formulation of a CTMC SIR epidemic model, we define a new random variable R(t)

counting the number of recovered persons to the compartments of a CTMC SIS epidemic model.

35
 (s0 , i0 ) (s1 , i1 ) (s2 , i2 ) (s, i) (k, j)

t0 t1 t2 ti ti+1
T0 T1 Ti

Figure 12. Trajectory of a Continuous Time SIR Model

A trajectory of a CTMC SIR model showing transition states and non-uniform holding times

between consecutive states.

There are three compartments in this model: susceptible, infectious and recovered. The

assumption of a constant population still holds for the observation and so, at any time t,

N = S(t) + I(t) + R(t).

The recovered class depends on the susceptible and the infectious class. The process is bivariate

with

R(t) = N − S(t) − I(t).

We can define a trajectory U for the continuous time SIR model as a sequence of states with

holding times

U = ((s0 , i0 ), T0 , (s1 , i1 ), T1 , . . . , (s, i), Ti , (k, j)).

This implies that the system started at a state (s0 , i0 ). Then, after T0 units of time, the system

transitioned to (s1 , i1 ). The system stayed at state (s1 , i1 ) for T1 units of time and then

transitioned to the next state (s2 , i2 ), and so on.

The infinitesimal transition probabilities are valid for sufficiently small ∆t, so for the

bivariate process with random variables (S(t), I(t)) only one of the following transitions can occur:

∆t ∆t ∆t
(s, i) −→ (s − 1, i + 1), (s, i) −→ (s, i − 1) or (s, i) −→ (s, i).

Since a constant population is assumed, for every change in time the number of infectious

increases by one, the number of recovered increases by one or no change happens. If the number

of infectious increases by one, the susceptible compartment loses one person. If the number of

36
recovered increases by one, the infectious compartment loses one person. Figure 12 shows the

trajectory of a CTMC SIR model.

3.3.1 TRANSITION PROBABILITIES OF A CTMC SIR EPIDEMIC MODEL

The same contact structure and homogeneous mixing of the population as in the SIR

DTMC model is assumed. The probability of a new infection, that is, transiting from (s, i) to

(s − 1, i + 1), is
βsi
p(s−1,i+1)←
−(s,i) (∆t) = ∆t + o(∆t).
N

The probability of a recovery, that is, transiting from (s, i) to (s, i − 1), is

p(s,i−1)←
−(s,i) (∆t) = γi∆t + o(∆t).

The probability that the number of infectious and recovered remains unchanged is

 
βsi
−(s,i) (∆t) = 1 − N + γi ∆t + o(∆t).
p(s,i)←

The transition probabilities of the embedded discrete time SIR epidemic model are

βsi
∆t + o(∆t), (k, j) = (−1, 1)


N






γi∆t + o(∆t), (k, j) = (0, −1)


p(s+k,i+j)←
−(s,i) (∆t) =  
βsi
1 − + γi ∆t + o(∆t), (k, j) = (0, 0)




 N



o(∆t),

otherwise.

For simplification, this is denoted as


b(i)∆t + o(∆t), (k, j) = (−1, 1)







d(i)∆t + o(∆t),
 (k, j) = (0, −1)
p(s+k,i+j)←
−(s,i) (∆t) =
1 − [b(i) + d(i)] ∆t + o(∆t), (k, j) = (0, 0)








o(∆t),

otherwise.

37
 We use Equation 3.2 to find the transition probabilities of the embedded DTMC SIR

model. From the generator matrix Q, the probability of transiting from (s, i) to (s − 1, i + 1)

(susceptible to infectious) is  
βsi
N
 .
βsi
+ γi
N

Also, the probability of transiting from (s, i) to (s, i − 1) (infectious to recovered) is

γi
 .
βsi
+ γi
N

The rate for the exponential interevent time for state i is

βsi
b(i) + d(i) = + γi.
N

3.3.2 PSEUDOCODE FOR CTMC SIR MODELS AND SIMULATED EXAMPLE

Provided all parameters and initial conditions are given, we can plot a graph of the SIR

epidemic model comparing the changes in the number of infectious individuals against time. The

algorithm of the code is as follows:

1. Initialize the values of β, γ, population size N , and the time duration of the outbreak tend .

2. Create arrays S(j), I(j), R(j) and t(j). Set j = 1, R(1) = 0 and t(1) = 0. Let I(1) be the

initial number of infectious individuals and S(1) = N − I(1).

3. While I(j) > 0 and t(j) < tend

βS(j)I(j)
(a) Let a = and b = γI(j).
N
a
(b) Compute the probability of a new infection (p1 ) using .
a+b
b
(c) Compute the probability of a recovery from infection (p2 ) using .
a+b
(d) Select two random numbers u1 and u2 from a uniform distribution of (0, 1).

(e) Compare u1 with p1 and p2 .

38
 (f) If 0 < u1 ≤ p1 , decrease the number of susceptible individuals by 1, and increase the

number of infectious individuals by 1. Hence we have

S(j + 1) = S(j) − 1, I(j + 1) = I(j) + 1, and R(j + 1) = R(t).

(g) Otherwise, p1 < u1 < 1. In this case, decrease the number of infectious individuals by

1, and increase the number of recovered individuals by 1. Hence we have

S(j + 1) = S(j), I(j + 1) = I(j) − 1, and R(j + 1) = R(t) + 1.

ln(u2 )
(h) Select the time of the next event using u2 as t(j + 1) = t(j) − .
a+b
(i) Update the vector index j to j + 1 and repeat.

4. Plot the graph of the times t(j) against the number of infectious at each time, I(j).

We simulate an example of a CTMC SIR epidemic model. We use a population of 100

people with 5 people initially infected. We use the transmission rate β = 1 and the recovery rate

γ = 0.5. The graph of the time against the number of infectious individuals is presented in

Figure 13.

39
Figure 13. A CTMC SIR Epidemic Model

Simulations of a CTMC SIR epidemic model with three stochastic paths using parameters N = 100,

I(1) = 5, β = 1 and γ = 0.5.

40
 CHAPTER 4

METHOD OF MAXIMUM LIKELIHOOD

For independent random variables X(0) = x0 , X(1) = x1 , . . . , X(n) = xn with joint

density function p(X(0), X(1), ...X(n) | θ) with parameter θ, the likelihood function is

n
Y
L(θ) = L(θ | x0 , x1 , ..., xn ) = p(x0 , x1 , ..., xn | θ) = p(xi | θ).
i=0

For a Markov process, the transition probabilities are not independent. The transition probability

to a new state depends on the current state. Assuming the transition probabilities depend on a

parameter θ, the likelihood function of a Markov process is the product of the transition

conditional probabilities,

n
Y
L(θ) = p(xn | xn−1 , θ)p(xn−1 | xn−2 , θ)...p(x1 | x0 , θ)p(x0 ) = p(x0 ) p(xi | xi−1 , θ). (4.1)
i=1

The maximum likelihood estimate (MLE) θ̂ is the value of θ that maximizes L(θ). For

computational convenience, we maximize the logarithm of the likelihood function by finding the

value of θ that satisfies the equation

∂ log L(θ)
= 0.
∂θ

Given some observed data, the MLE may be difficult to obtain analytically from the likelihood

function, and so a suitable optimization algorithm may be used to approximate the MLE. Many

optimization algorithms are designed to minimize the output of a function. A general

optimization technique is to minimize the negative logarithm of the likelihood function. This

process is equivalent to maximizing the logarithm of the likelihood function. We use the

fminsearch algorithm in Matlab to approximate the MLEs of β and γ.

41
4.1 PARAMETER ESTIMATION FOR DTMC SIS MODELS

Consider a discrete time SIS model with transition probabilities


 βi(N − i) ∆t,

j =i+1
N






γi∆t, j =i−1


pj ←
−i (∆t) =  
βi(N − i)
1− + γi ∆t, j = i





 N



0,

otherwise .

The parameters β and γ can be estimated for sufficiently observed data. The likelihood function is

N
n
Y ji
L(β, γ) = P (X(t = 0) = i0 ) pj←i ,
i,j=0

where X(t = 0) is the random variable representing the initial number of infectious persons, N is

the population size and nji is count of transitions from i to j. The product runs through all

possible transition probabilities given by the transition matrix. We set pj←i = 0 when a transition

did not occur from state i to state j, that is, nji = 0, and let 00 = 1. The initial number of

infectious persons is known and so P ((X(t = 0) = i0 ) = 1. The likelihood function from

Equation 4.1 is
N
Y
L(β, γ) = [(pi+1←i )ni+1,i (pi−1←i )ni−1,i (pi←i )ni,i ] .
i=1

The logarithm of a function is strictly increasing, so the logarithm of the likelihood function will

be maximized exactly when the likelihood function is maximized. Taking the logarithm of the

likelihood function converts the products to sums for easier computation. We have

N      
X βi βi
log L(β, γ) = ni+1,i log (N − i) + ni−1,i log(γi) + ni,i log 1 − γi + (N − i) .
N N
i=1

Taking the partial derivative of the logarithm of the likelihood function with respect to β, we have

N    
∂ log L(β, γ) X 1 (ni,i )i(N − i)
= (ni+1,i ) − .
∂β β (N − N γi − βi(N − i))
i=1

42
Also, taking the partial derivative of the logarithm of the likelihood function with respect to γ, we

have
N    
∂ log L(β, γ) X 1 (ni,i )i
= (ni−1,i ) − .
∂γ γ (N − N γi − βi(N − i))
i=1

The MLE of β is β̂, the value β that satisfies the equation

∂ log L(β, γ)
= 0.
∂β

The MLE of γ is γ̂, the value γ that satisfies the equation

∂ log L(β, γ)
= 0.
∂γ

4.1.1 ESTIMATION ALGORITHM AND RESULTS FOR DTMC SIS MODELS

Given some observed data, the likelihood function can be computed using Matlab. For

this algorithm, we need not count the number of transitions between any two consecutive states.

We compute the probabilities for the likelihood function at any point in time using data from the

simulated outbreak. We progress through the data in time steps to compute the likelihood

function. The Matlab algorithm to compute the MLE of β and γ is:

1. Simulate an outbreak with known β and γ while discarding simulations with no infections at

t = 10. Let k be the smaller of the first index when the I vector is 0 (I(k) = 0) or the length

of the infectious vector.

2. The following steps are used to compute a function l(β̃, γ̃), where β̃ and γ̃ are parameters.

The algorithm uses the vector I from step (1) and the population size N .

(a) Initialize a vector p with length k − 1 for the transition probabilities.

(b) For j from 2 to k in an increment of 1:

i. If I(j) = I(j − 1) + 1, the number of susceptible individuals decreased by 1, and

the number of infectious individuals increased by 1. Hence we have

β̃[N − I(j − 1)]I(j − 1)

p(j − 1) = ∆t.
N

43
 ii. If I(j) = I(j − 1) − 1, the number of susceptible individuals increased by 1, and

the number of infectious individuals decreased by 1. Hence we have

p(j − 1) = γ̃I(j − 1)∆t.

iii. Otherwise, we have I(j) = I(j − 1). In this case, the number of susceptible and

infectious remains the same. Hence

!
β̃[N − I(j − 1)]I(j − 1)
p(j − 1) = 1 − + γ̃I(j − 1) ∆t.
N

Pk−1
(c) Let l(β̃, γ̃) = − j=1 (log p(j)). Hence l represents the negative of the logarithm of the

likelihood function.

3. Use fminsearch on the function l(β̃, γ̃) to identify values of β̃ and γ̃ that minimize l

starting with initial search values of β̃ = 0.1 and γ̃ = 0.01. This process produces

approximations to the MLE values β̂ and γ̂, respectively.

4. Repeat steps (1) to (3) 10,000 times, and store the approximations to β̂ and γ̂ found in

step (3) each time.

5. Plot the distribution of data from step (4).

We use the algorithm just described to computationally produce a distribution of the MLE

values β̂ and γ̂. We use a discrete time SIS model with parameters N = 100, I(0) = 5, ∆t = 0.01

and t = 50 for β = 1.5 and γ = 0.5 and for β = 0.9 and γ = 0.3. For 10,000 simulations, the

histograms of β̂ and γ̂ are given in Figure 14 and in Figure 15, respectively, for the different

values of β and γ. Simulations with infections dying out before t = 10 are discarded and the

simulations are repeated. The discarded simulations did not reach endemic equilibrium, and

therefore did not last long enough for the fminsearch algorithm to accurately determine β̂ and γ̂.

The number of repeated simulations is given in Table 2. From the histograms in Figure 14

and Figure 15, we see that the distributions of β̂ and γ̂ are normal and symmetric about the

known values of β and γ.

44
Figure 14. Histogram of β̂ and γ̂ for a DTMC SIS Model with β = 1.5 and γ = 0.5

Parameter estimation distribution of β̂ and γ̂ for a DTMC SIS model with β = 1.5 and γ = 0.5.

Each distribution has 10,000 simulations with N = 100, I(0) = 5, ∆t = 0.01 and t = 50. Left

panel: Parameter distribution of β̂. Right panel: Parameter distribution of γ̂.

Figure 15. Histogram of β̂ and γ̂ for a DTMC SIS Model with β = 0.9 and γ = 0.3

Parameter estimation distribution of β̂ and γ̂ for a DTMC SIS model with β = 0.9 and γ = 0.3.

Each distribution has 10,000 simulations with N = 100, I(0) = 5, ∆t = 0.01 and t = 50. Left

panel: Parameter distribution of β̂. Right panel: Parameter distribution of γ̂.

45
 β = 1.5 γ = 0.5 β = 0.9 γ = 0.3 β = 1.5 γ = 0.5 β = 0.9 γ = 0.3
β̂ γ̂ β̂ γ̂ β̂ γ̂ β̂ γ̂
1.4962 0.5177 0.8669 0.3023 1.4900 0.5007 0.8875 0.3080
1.4980 0.4983 0.8852 0.3087 1.5303 0.5038 0.8785 0.3260
1.5413 0.4825 0.9029 0.3103 1.4910 0.4975 0.8922 0.2945
1.4911 0.5036 0.9389 0.2938 1.4963 0.4909 0.9309 0.3038
1.4947 0.5064 0.8786 0.3037 1.5217 0.4924 0.9158 0.2875
1.4622 0.5102 0.9112 0.2978 1.5587 0.4950 0.9202 0.2980
1.4841 0.5140 0.8869 0.2962 1.5018 0.4954 0.9312 0.3042
1.4627 0.5109 0.9075 0.3057 1.4914 0.5077 0.9365 0.3077
1.4851 0.4889 0.8914 0.2963 1.4709 0.5010 0.8791 0.3108
1.4984 0.4995 0.9087 0.3029 1.5203 0.5051 0.9206 0.2971

Table 1. Simulation Parameter Estimates for a DTMC SIS Model

Parameter estimations of β̂ and γ̂ for simulations of DTMC SIS model with β = 1.5 and γ = 0.5

(left) and with β = 0.9 and γ = 0.3 (right). All simulations are performed with N = 100, I(0) = 5,

∆t = 0.01 and t = 50.

For the two pairs of β and γ, 20 estimates of β̂ and γ̂ are presented in Table 1. The bias

and MSE are presented in Table 2. The bias and MSE are small and so the MLE gives good

estimates of β and γ.

Known values β = 1.5 γ = 0.5 β = 0.9 γ = 0.3

β̂ γ̂ β̂ γ̂
Bias 0.2485 × 10−3 0.1545 × 10−3 0.7072 × 10−3 0.1104 × 10−3
MSE 0.9019 × 10−3 0.1054 × 10−3 0.6850 × 10−3 0.0787 × 10−3
Number of
48 60
Repeated simulations

Table 2. Computational Results for a DTMC SIS Model

Computational results of β̂ and γ̂ for simulations of DTMC SIS model with β = 1.5 and γ = 0.5

(left) and β = 0.9 and γ = 0.3 (right). Computational results for 10,000 simulations of discrete

time SIS model with N = 100, I(0) = 5, ∆t = 0.01 and t = 50.

46
4.2 PARAMETER ESTIMATION FOR DTMC SIR MODELS

Consider a discrete time SIR model with transition probabilities


βsi
∆t, (k, j) = (s − 1, i + 1)


N






γi∆t, (k, j) = (s, i − 1)


p(k,j)←
−(s,i) (∆t) =  
βsi
1− + γi ∆t, (k, j) = (s, i)





 N



0,

otherwise.

As with the SIS model, we can estimate parameters β and γ. The likelihood function is

N
Y
L(β, γ) = P (X(t = 0) = (s0 , i0 )) (p(k,j)←(s,i) )n(k,j)(s,i) ,
i,j,s,k=0

where n(k,j)(s,i) is the transition count from state (s, i) to state (k, j). The likelihood function is

N Y
Y N
L (β, γ) = (p(s−1,i+1)←(s,i) )n(s−1,i+1),(s,i) (p(s,i−1)←(s,i) )n(s,i−1),(s,i) (p(s,i)←(s,i) )n(s,i),(s,i) .
i=1 s=0

Taking the logarithm of the likelihood function, we have

N X
N h  
X βsi
log L(β, γ) = n(s−1,i+1),(s,i) log + n(s,i−1),(s,i) log(γi)
N
i=1 s=0
   i
βsi
+ n(s,i),(s,i) log 1 − + γi .
N

Taking the partial derivative of the logarithm of the likelihood function with respect to β, we have

N N 
n(s,i),(s,i) si
  
∂ log L(β, γ) X X 1
= n(s−1,i+1),(s,i) − .
∂β β (N − N γi − βsi)
i=1 s=0

Also, taking the partial derivative of the logarithm of the likelihood function with respect to γ, we

have
N N    
∂ log L(β, γ) X X 1 (ni,i )i
= (ni−1,i ) − .
∂γ γ (N − N γi − βsi)
i=1 s=0

47
We denote the MLE of β as β̂, the value of β that satisfies the equation

∂ log L(β, γ)
= 0.
∂β

We denote the MLE of γ as γ̂, the value γ that satisfies the equation

∂ log L(β, γ)
= 0.
∂γ

4.2.1 ESTIMATION ALGORITHM AND RESULTS FOR DTMC SIR MODELS

Given some observed data, the likelihood function can also be obtained using Matlab. For

this algorithm, we need not count the number of transitions between any two consecutive states.

We progress through the data in time steps to get the likelihood function. The Matlab algorithm

of the MLE of β and γ is given as follows:

1. Simulate an outbreak with known β and γ while discarding simulations with no infections at

t = 7. Let k be the smaller of the first index when the I vector is 0 (I(k) = 0) or the length

of the infectious vector.

2. The following steps are used to compute a function l(β̃, γ̃), where β̃ and γ̃ are parameters.

The algorithm uses the vectors I and S from step (1) and the population size N .

(a) Initialize a vector p for the transition probabilities with length k − 1.

(b) For j from 2 to k in an increment of 1:

i. If I(j) = I(j − 1) + 1, the number of susceptible individuals decreased by 1, and

the number of infectious individuals increased by 1. Hence we have

β̃S(j − 1)I(j − 1)
p(j − 1) = ∆t.
N

ii. If I(j) = I(j − 1) − 1, the number of infectious individuals decreased 1, and the

number of recovered individuals increased by 1. Hence we have

p(j − 1) = γ̃I(j − 1)∆t.

48
 iii. Otherwise, I(j) = I(j − 1). In this case, the numbers of susceptible, infectious and

recovered remained the same. Hence we have

!
β̃S(j − 1)I(j − 1)
p(j − 1) = 1 − + γ̃I(j − 1) ∆t.
N

Pk−1
(c) Let l(β̃, γ̃) = − j=1 (log p(j)) where l is the negative of the logarithm of the likelihood

function.

3. Use fminsearch on the function l(β̃, γ̃) to identify values of β̃ and γ̃ that minimize l

starting with initial search values of β̃ = 0.1 and γ̃ = 0.01. This process produces

approximations to the MLE values β̂ and γ̂, respectively.

4. Repeat steps (1) to (3) 10,000 times, and store the approximations to β̂ and γ̂ found in

step (3) each time.

5. Plot the distribution of data from step (4).

We computationally produce a distribution for the MLE values of β̂ and γ̂, for a discrete

time SIR model. We use parameters N = 100, I(0) = 5, ∆t = 0.01 and t = 50 for β = 1.5 and

γ = 0.5 and for β = 0.9 and γ = 0.3. For 10,000 simulations, the histograms of β̂ and γ̂ are given

in Figure 16 and Figure 17. Simulations with infections dying out before t = 7 are discarded. The

discarded simulations are presumed to not last long enough for fminsearch to accurately

determine the values of β̂ and γ̂. The choice of which simulations to discard may be adjusted to

increase the accuracy of the histogram. The choice of t = 7 in this simulation improved the

accuracy of the histogram.

For the two pairs of β and γ, 20 estimates of β̂ and γ̂ are presented in Table 3. The bias,

MSE and the number of repeated simulations are presented in Table 4. The histogram of γ̂ for

γ = 0.5 and γ = 0.3 is right skewed due to some simulations having the number of infectious

dying out right after time t = 7. The bias and MSE are small, so the MLE gives a good estimate

of β and γ.

49
Figure 16. Histogram of β̂ and γ̂ for a DTMC SIR Model with β = 1.5 and γ = 0.5

Parameter estimation distribution of β̂ and γ̂ for a DTMC SIR model with β = 1.5 and γ = 0.5.

Each distribution has 10,000 simulations with N = 100, I(0) = 5, ∆t = 0.01 and t = 50. Left panel:

Parameter estimation distribution of β̂ using fminsearch. Right panel: Parameter estimation

distribution of γ̂ using fminsearch.

Figure 17. Histogram of β̂ and γ̂ for a DTMC SIR Model with β = 0.9 and γ = 0.3

Parameter estimation distribution of β̂ and γ̂ for a DTMC SIR model with β = 0.9 and γ = 0.3.

Each distribution has 10,000 simulations with N = 100, I(0) = 5, ∆t = 0.01 and t = 50. Left panel:

Parameter estimation distribution of β̂ using fminsearch. Right panel: Parameter estimation

distribution of γ̂ using fminsearch.

50
 β = 1.5 γ = 0.5 β = 0.9 γ = 0.3
β̂ γ̂ β̂ γ̂
1.4633 0.5591 0.7789 0.3253
1.2879 0.4374 0.8381 0.2830
1.5262 0.4609 0.9281 0.2838
1.4284 0.5318 0.8760 0.3374
1.5316 0.5614 0.9036 0.2666
1.6757 0.5359 0.8930 0.2949
1.6276 0.4807 0.8867 0.2697
1.6627 0.4845 1.1176 0.2861
1.6689 0.4842 0.9544 0.2661
1.5112 0.4741 0.8565 0.2907
1.3503 0.5216 0.9925 0.3355
1.4809 0.4800 0.8543 0.3179
1.3747 0.5022 0.8397 0.3273
1.4969 0.4755 0.9436 0.3502
1.5238 0.4881 0.8017 0.3159
0.7832 0.7964 0.7801 0.3250
1.2235 0.5122 1.0029 0.2808
1.5345 0.4659 0.7896 0.3064
1.7693 0.4678 0.8152 0.2559
1.5651 0.4744 0.8291 0.3225

Table 3. Simulation Parameter Estimates for a DTMC SIR Model

Parameter estimations of β̂ and γ̂ for simulations of DTMC SIR model with β = 1.5 and γ = 0.5

(left) and β = 0.9 and γ = 0.3 (right). All simulations are performed with N = 100, I(0) = 5,

∆t = 0.01 and t = 50.

Known values β = 1.5 γ = 0.5 β = 0.9 γ = 0.3

β̂ γ̂ β̂ γ̂
Bias 0.0083 0.0058 0.0060 0.0041
MSE 0.0217 0.0027 0.0087 0.0011
Number of
59 60
Repeated simulations

Table 4. Computational Results for a DTMC SIR Model

Computational results of β̂ and γ̂ for simulations of DTMC SIR model with β = 1.5 and γ = 0.5

(left) and β = 0.9 and γ = 0.3 (right). Computational results for 10,000 simulations with N = 100,

I(0) = 5, ∆t = 0.01 and t = 50.

51
 s0 s1 s2 sk−1 sk

t0 t1 t2 tk−1 tk t

T0 T1 Tk−1

Figure 18. Estimating Maximum Likelihood for Continuous Time SIS Model

Illustration of a continuous time stochastic process observed between t0 units of time and t units

of time.

4.3 PARAMETER ESTIMATION FOR CTMC SIS MODELS

We follow the approach of Perkins [23]. Consider a CTMC SIS model with trajectory

U = (s0 , T0 , s1 , T1 , ..., sk−1 , Tk−1 , sk ),

The system starts at state s0 at t0 units of time and enters the last state sk at tk units of time as

shown in Figure 18. There are a total of k transitions in this process. At any given time, all

observations occur within the time interval (t0 , t) where t ≥ tk , and there is no transition in the

interval (tk , t).

The likelihood function is

k−1
Y Pk−1
L(β, γ) = (λsi e−λsi Ti )(psi+1 ←si )(e−λsk (t− i=0 Ti )
),
i=0

where λsi e−λsi Ti is the probability of the holding time Ti in state si , psi+1 ←si is the probability of
Pk−1
transiting from state si to si+1 and e−λsk (t− Ti )
is the probability that no additional
i=0

transitions occur after time sk up to time t. We observe that k−1

P
i=0 Ti = tk . Let t − tk = Tk , so

that
Pk−1
e−λsk (t− i=0 Ti )
= e−λsk (Tk ) ,

52
and
k−1
Y
−λsk (Tk )
L(β, γ) = e (λsi e−λsi Ti )(psi+1 ←si ).
i=0

For a continuous time SIS model, only two types of transitions can occur. Either a susceptible

person becomes infected or an infected person becomes susceptible. For any transition from state

si to state si+1 , there is either a transition from i to i + 1 or from i to i − 1.

Let {tβ1 , tβ2 , ..., tβn } be the set of times when there is a transition from i to i + 1 and let

{tγ1 , tγ2 , ..., tγm } be the set of times when there is a transition from i to i − 1. This means that the

first transition from i to i + 1 in the system happened at time tβ1 , the first transition from i to

i − 1 in the system happened at time tγ1 and so on. There are n transitions from i to i + 1 and m

transitions from i to i − 1. There are a total of k transitions in the system from state s0 up to

state sk .

From the generator matrix Q, the probability of transiting from i to i + 1 (susceptible to

infectious) is
b(i)
pi+1←i = ,
b(i) + d(i)
βsi
where b(i) = and d(i) = γi. The probability of a new infection is
N

βsi
N
pi+1←i = .
βsi
+ γi
N

Similarly, the probability of transiting from i to i − 1 (infectious to susceptible) is

d(i)
pi−1←i = ,
b(i) + d(i)

and so the probability of transiting from infectious to susceptible is

γi
pi−1←i =  .
βsi
+ γi
N

We split the state transition probabilities, psi+1 ←si , into the two possible cases of transitions:

53
from susceptible to infectious and infectious to susceptible. The likelihood function becomes

k−1
Y
(λsi e−λsi Ti )(psi+1 ←si ) =
i=0
susceptible to infectious
z }| {
βS(a)I(a)
 
tβn      
 Y βS(a)I(a) βS(a)I(a) N
+ γI(a) exp − + γI(a) Ta 
 
N N βS(a)I(a)
 
a=tβ1 + γI(a)
  N 
tγm      
 Y βS(b)I(b) βS(b)I(b) γI(b)
× + γI(b) exp − + γI(b) Tb  ,
 
N N βS(b)I(b)
b=tγ1 + γI(b)
| {z N }
infectious to susceptible

where S(a) = N − I(a) and S(b) = N − I(b). The equation then simplifies to

k−1 tβn     
Y Y βS(a)I(a) βS(a)I(a)
(λsi e−λsi Ti )(psi+1 ←si ) = exp − + γI(a) Ta
a=t
N N
i=0 β1

tγm
Y   
βS(b)I(b)
 
× γI(b) exp − + γI(b) Tb .
N
b=tγ1

The likelihood function can be written as

    tY
γm     
βS(tk )I(tk ) βS(b)I(b)
L(β, γ) = exp − + γI(tk ) Tk γI(b) exp − + γI(b) Tb
N N
b=tγ1
tβn
Y  βS(a)I(a)  
βS(a)I(a)
 
× exp − + γI(a) Ta ,
a=t
N N
β1

By taking the logarithm of the likelihood function, we have

  tγm    
βS(tk )I(tk ) X βS(b)I(b)
log L(β, γ) = − + γI(tk ) Tk + log(γI(b)) − + γI(b) Tb
N N
b=tγ1
tβn      
X βS(a)I(a) βS(a)I(a)
+ log − + γI(a) Ta .
a=t
N N
β1

54
Taking the partial derivative of the logarithm of likelihood function with respect to β, we have

tβn      X tγm    
∂ log L(β, γ) X 1 S(a)I(a) S(b)I(b)
= − Ta + − Tb
∂β a=tβ1
β N N
b=tγ1
 
S(tk )I(tk )
− Tk .
N

Distributing the sums, we have

 
tβn    X tγm   
∂ log L(β, γ) X S(a)I(a) S(b)I(b)
=− Ta + Tb 
∂β a=t
N N
β1 b=tγ1
  tβn  
S(tk )I(tk ) X 1
− Tk + .
N a=t
β
β1

We combine the first two terms since together, they consist of all possible transitions from state

s0 up to state sk , and so

  tβn   k−1   
∂ log L(β, γ) S(tk )I(tk ) X 1 X S(ti )I(ti )
=− Tk + − Ti .
∂β N a=t
β N
β1 i=0

Adding the first term to the last term and since there is a total of n transitions from tβ1 to tβn

k   
∂ log L(β, γ) n X S(ti )I(ti )
= − Ti .
∂β β N
i=0

The maximum likelihood estimate β̂ is the value of β such that

∂ log L(β, γ)
= 0.
∂β

Therefore, we have that

n
β̂ = k  . (4.2)
X S(ti )I(ti )Ti
N
i=0

Similarly, to obtain the maximum likelihood estimate γ̂, we take the partial derivative of

55
the log likelihood function with respect to γ, obtaining

tβn tγm  
∂ log L(β, γ) X X 1
= −I(tk )Tk − [I(a)Ta ] + − I(b)Tb .
∂γ a=t
γ
β1 b=tγ1

Distributing the sum, we have

 
tγm   tβn tγm
∂ log L(β, γ) X 1 X X
= −I(tk )Tk + − [I(a)Ta ] + [I(b)Tb ] .
∂γ γ a=t
b=tγ1 β1 b=tγ1

We combine the last two terms since together, both terms sum all possible times of transitions

from state s0 up to state sk , and so

tγm   k−1
∂ log L(β, γ) X 1 X
= −I(tk )Tk + − [I(i)Ti ] .
∂γ γ
b=tγ1 i=0

Adding the first term to the last term, and since there is a total of m transitions from tγ1 to tγm

k
∂ log L(β, γ) m X
= − [I(i)Ti ] .
∂γ γ
i=0

The maximum likelihood estimate γ̂, is the value of γ such that

∂ log L(β, γ)
= 0,
∂γ

therefore, we have that

m
γ̂ = k
. (4.3)
X
[I(ti )Ti ]
i=0

4.3.1 ESTIMATION ALGORITHM AND RESULTS FOR CTMC SIS MODELS

The likelihood function for simulated data can be approximated using Matlab. We use

fminsearch and the theoretical formulas Equation 4.2 and Equation 4.3 to compute the

maximum likelihood estimate of β and γ. The Matlab algorithm is:

1. Simulate an outbreak with known β and γ while discarding simulations with no infectious

56
 individuals at t = 10. Let k be the smaller of the first index when the I vector is 0

(I(k) = 0) or the length of the infectious vector.

2. The following steps are used to compute a function l(β̃, γ̃), where β̃ and γ̃ are parameters.

The algorithm uses the vector I and t from step (1) and the population size N .

(a) Initialize counters n and m as n = 0 and as m = 0 for counting transitions from I(j) to

I(j) + 1 and from I(j) to I(j) − 1, respectively.

(b) Initialize a vector p of length k − 1 for the transition probabilities.

(c) For j from 2 to k in an increment of 1:

i. Calculate the interevent time between consecutive states using the time vector

from the simulated outbreak as T (j − 1) = t(j) − t(j − 1).

ii. If I(j) = I(j − 1) + 1, the number of susceptible individuals decreased by 1, and

the number of infectious individuals increased by 1. Hence we have

!
β̃[N − I(j − 1)]I(j − 1)
p(j − 1) = ×
N
!!
β̃[N − I(j − 1)]I(j − 1)
exp − + γ̃I(j − 1) T (j − 1),
N

and we increase the counter from n to n + 1.

iii. Otherwise, I(j) = I(j − 1) − 1, so the number of susceptible individuals increased

by 1, and the number of infectious individuals decreased by 1. Hence we have

!!
β̃[N − I(j − 1)]I(j − 1)
p(j − 1) = (γ̃I(j − 1)) × exp − + γ̃I(j − 1) T (j − 1),
N

and we increase the counter from m to m + 1.

(d) Let l(β̃, γ̃) be the negative log likelihood

 
k−1
!
X β̃[N − I(k)]I(k)
l(β̃, γ̃) = − log p(k) − + γ̃I(k) · (tend − t(k)),
N
j=1

where t(k) is the time of the last event and tend is the length of time of the outbreak.

57
 3. Use fminsearch to identify values of β̃ and γ̃ that minimize l starting with initial search

values of β̃ = 0.1 and γ̃ = 0.01. These values are our approximations of β̂ and γ̂.

4. Use Equation 4.2 and Equation 4.3 with m and n to find the theoretical MLE of β and γ.

5. Store the approximations of β̂ and γ̂ from step (3) and the theoretical values from step (4).

6. Repeat steps (1) through (5) 10,000 times.

7. Plot the distribution of the data from step (6).

We computationally produce a distribution for the MLE values of β̂ and γ̂, for a

continuous time SIS model. We use parameters N = 100, I(1) = 5 and t = 50 for β = 1.5 and

γ = 0.5 and for β = 0.9 and γ = 0.3. For 10,000 simulations, the histograms of β̂ are given in

Figure 19 and Figure 21. The histograms for γ̂ are given in Figure 20 and Figure 22. Simulations

with infections dying out before t = 10 are repeated.

For the two pairs of β and γ, 20 estimates of β̂ and γ̂ obtained using fminsearch and the

theoretical formulas are presented in Table 5. The bias, MSE and the number of repeated

simulations are presented in Table 6 and Table 7. From the histograms of the MLEs, we see that

the distributions of β̂ and γ̂ are normal and symmetric about the known values of β and γ. The

bias and MSE are small, so the MLE gives a good estimate of β and γ.

4.4 PARAMETER ESTIMATION FOR CTMC SIR MODELS

For a continuous time SIR model, the MLEs are derived in a similar manner to the

continuous time SIS model. The likelihood function is

k−1
Y
L(β, γ) = e−λsk (Tk ) (λsi e−λsi Ti )(psi+1 ←si ).
i=0

For a continuous time SIR model, only two types of transitions can occur. Either a susceptible

person becomes infected or an infected person recovers. For any transition from state si to state

si+1 , there is either a transition from (s, i) to (s − 1, i + 1) (susceptible to infectious) or from (s, i)

to (s, i − 1) (infectious to recovered).

58
Figure 19. Histogram of β̂ for a CTMC SIS Model with β = 1.5 and γ = 0.5

Parameter estimation distribution of β̂ for a CTMC SIS model with β = 1.5 and γ = 0.5. Each

distribution has 10,000 simulations with N = 100, I(1) = 5 and t = 50. Left panel: Parameter

estimation distribution of β̂ using fminsearch. Right panel: Parameter estimation distribution of

β̂ using theoretical estimates.

Figure 20. Histogram of γ̂ for a CTMC SIS Model with β = 1.5 and γ = 0.5

Parameter estimation distribution of γ̂ for a CTMC SIS model with β = 1.5 and γ = 0.5. Each

distribution has 10,000 simulations with N = 100, I(1) = 5 and t = 50. Left panel: Parameter

estimation distribution of γ̂ using fminsearch. Right panel: Parameter estimation distribution of

γ̂ using theoretical estimates.

59
Figure 21. Histogram of β̂ for a CTMC SIS Model with β = 0.9 and γ = 0.3

Parameter estimation distribution of β̂ for a CTMC SIS model with β = 0.9 and γ = 0.3. Each

distribution has 10,000 simulations with N = 100, I(1) = 5 and t = 50. Left panel: Parameter

estimation distribution of γ̂ using fminsearch. Right panel: Parameter estimation distribution of

γ̂ using theoretical estimates.

Figure 22. Histogram of γ̂ for a CTMC SIS Model with β = 0.9 and γ = 0.3

Parameter estimation distribution of γ̂ for a CTMC SIS model with β = 0.9 and γ = 0.3. Each

distribution has 10,000 simulations with N = 100, I(1) = 5 and t = 50. Left panel: Parameter

estimation distribution of γ̂ using fminsearch. Right panel: Parameter estimation distribution of

γ̂ using theoretical estimates.

60
 β = 1.5 γ = 0.5 β = 0.9 γ = 0.3
β̂fminsearch β̂theoretical γ̂fminsearch γ̂theoretical β̂fminsearch β̂theoretical γ̂fminsearch γ̂theoretical
1.4522 1.4524 0.5006 0.5002 0.8591 0.8598 0.2936 0.2935
1.5399 1.5404 0.4774 0.4764 0.8910 0.8895 0.3099 0.3095
1.5113 1.5121 0.4925 0.4921 0.9411 0.9407 0.3160 0.3160
1.5135 1.5135 0.5173 0.5173 0.8783 0.8793 0.2768 0.2767
1.5244 1.5252 0.4900 0.4894 0.8874 0.8883 0.2940 0.2938
1.5148 1.5147 0.5111 0.5112 0.8568 0.8565 0.2841 0.2841
1.4694 1.4703 0.4952 0.4949 0.9116 0.9111 0.3206 0.3206
1.4799 1.4805 0.5009 0.5006 0.8721 0.8717 0.2943 0.2943
1.4761 1.4766 0.4787 0.4783 0.8727 0.8733 0.2962 0.2956
1.4809 1.4812 0.5055 0.5049 0.9104 0.9099 0.2978 0.2975
1.4295 1.4300 0.5039 0.5033 0.9292 0.9292 0.3014 0.3010
1.5412 1.5418 0.4988 0.4983 0.9104 0.9098 0.3063 0.3054
1.4836 1.4826 0.4952 0.4950 0.8901 0.8900 0.3021 0.3023
1.4908 1.4905 0.5168 0.5170 0.9559 0.9553 0.3012 0.3013
1.5157 1.5156 0.5022 0.5021 0.8737 0.8726 0.2977 0.2976
1.5384 1.5375 0.5032 0.5032 0.9174 0.9165 0.3128 0.3127
1.4619 1.4626 0.5222 0.5217 0.9044 0.9039 0.2824 0.2825
1.4518 1.4514 0.4725 0.4726 0.8735 0.8734 0.3054 0.3039
1.4921 1.4930 0.5096 0.5094 0.8644 0.8649 0.3051 0.3048
1.5359 1.5359 0.5053 0.5053 0.9290 0.9297 0.2967 0.2965

Table 5. Simulation Parameter Estimates for a CTMC SIS Model

All simulations are performed with N = 100, I(1) = 5 and t = 50.

Known values β = 1.5 γ = 0.5

β̂fminsearch β̂theoretical γ̂fminsearch γ̂theoretical
Bias 0.6029 × 10−3 0.0361 0.5303 × 10−3 0.0169
MSE 0.0014 0.0027 0.0002 0.0004
Number of
50
Repeated simulations

Table 6. Computational Results for a CTMC SIS Model with β = 1.5 and γ = 0.5

Computational results for 10,000 simulations with N = 100, I(1) = 5 and t = 50.

61
 Known values β = 0.9 γ = 0.3
β̂fminsearch β̂theoretical γ̂fminsearch γ̂theoretical
Bias −0.2547 × 10−3 0.0187 0.5441 × 10−3 0.0046
MSE 0.8401 × 10−3 0.0012 0.1000 × 10−3 0.0001
Number of
45
Repeated Simulations

Table 7. Computational Results for a CTMC SIS Model with β = 0.9 and γ = 0.3

Computational results for 10,000 simulations with N = 100, I(1) = 5 and t = 50.

Let {tβ1 , tβ2 , ..., tβn } be the set of times when there is a transition from i to i + 1 and let

{tγ1 , tγ2 , ..., tγm } be the set of times when there is a transition from i to i − 1. This means that the

first transition from i to i + 1 in the system happened at time tβ1 , the first transition from i to

i − 1 in the system happened at time tγ1 and so on. There are n transitions from i to i + 1 and m

transitions from i to i − 1. There are a total of k transitions in the system from state s0 up to

state sk .

The probability of transiting from (s, i) to (s − 1, i + 1) (susceptible to infectious) is

 
βsi
N
p(s−1,i+1)←(s,i) = .
βsi
+ γi
N

Also, the probability of transiting from (s, i) to (s, i − 1) (infectious to recovered) is

γi
p(s,i−1)←(s,i) =  .
βsi
+ γi
N

We split the state transition probabilities, psi+1 ←si , into the two possible types of state transitions

in an SIR model. Transition can be from susceptible to infectious or from infectious to recovered.

62
The likelihood function becomes

k−1
Y
(λsi e−λsi Ti )(psi+1 ←si ) =
i=0
susceptible to infectious
z }| {
βS(a)I(a)

tβn      
 Y βS(a)I(a) βS(a)I(a) N
+ γI(a) exp − + γI(a) Ta 
 
N N βS(a)I(a)
 
a=tβ1 + γI(a)
  N 
tγm      
 Y βS(b)I(b) βS(b)I(b) γI(b)
× + γI(b) exp − + γI(b) Tb  ,
 
N N βS(b)I(b)
b=tγ1 + γI(b)
| {z N }
infectious to recovered

where S(a) = N − I(a) − R(a) and S(b) = N − I(b) − R(b). In finding the MLEs, we follow the

same steps as in the continuous time SIS, and we obtain β̂ as

n
β̂ = k  , (4.4)
X S(ti )I(ti )Ti
N
i=0

and γ̂ as
m
γ̂ = k
. (4.5)
X
[I(ti )Ti ]
i=0

4.4.1 ESTIMATION ALGORITHM AND RESULTS FOR CTMC SIR MODELS

For simulated data, the likelihood function can be obtained using Matlab. We use

fminsearch and the theoretical formulas Equation 4.4 and Equation 4.5 to find the maximum

likelihood estimate of β and γ. The Matlab algorithm is given as follows:

1. Simulate an outbreak with known β and γ while discarding simulations with no infectious at

t = 9. Let k be the smaller of the first index when the I vector is 0 (I(k) = 0) or the length

of the infectious vector.

2. The following steps are used to compute a function l(β̃, γ̃), where β̃ and γ̃ are parameters.

63
 The algorithm uses the vector I and t from step (1) and the population size N .

(a) Initialize counters n and m as n = 0 and m = 0 for counting transitions from I(j) to

I(j) + 1 and from I(j) to I(j) − 1, respectively.

(b) Initialize a vector p of length k − 1 for the transition probabilities.

(c) For j from 2 to k in an increment of 1:

i. Calculate the interevent time between consecutive states using the time array from

the simulated outbreak as T (j − 1) = t(j) − t(j − 1).

ii. If I(j) = I(j − 1) + 1, the number of susceptible individuals decreased by 1, and

the number of infectious individuals increased by 1. Hence we have

! !!
β̃S(j − 1)I(j − 1) β̃S(j − 1)I(j − 1)
p(j−1) = exp − + γI(j − 1) T (j−1)
N N

and we increase the counter from n to n + 1.

iii. Otherwise I(j) = I(j − 1) − 1, the number of infectious individuals decreased by 1,

and the number of recovered individuals increased by 1. Hence we have

!!
β̃[S(j − 1)]I(j − 1)
p(j − 1) = γI(j − 1) exp − + γ̃I(j − 1) T (j − 1),
N

and we increase the counter from m to m + 1.

(d) Let l(β̃, γ̃) be the negative log likelihood

 
k−1
!!
X β̃S(k)I(k)
l(β̃, γ̃) = − log p(j) + − + γ̃I(k) · (tend − t(k))
N
j=1

where t(k) is the last event and tend is the length of time of the outbreak.

3. Use fminsearch to identify values of β̃ and γ̃ that minimize l starting with initial search

values of β = 0.1 and γ = 0.01. The final values of β̃ and γ̃ will be our approximations to β̂

and γ̂, respectively.

4. Use Equation 4.5 and Equation 4.4 with m and n to find the theoretical MLE of β and γ.

64
 5. Store the approximations of β̂ and γ̂ from step (3) and the theoretical values from step (4).

6. Repeat steps (1) through (5) 10,000 times.

7. Plot the distribution of the data from step (6).

We find the MLEs β̂ and γ̂ for a continuous time SIR model using fminsearch and the

theoretical MLE formulas Equation 4.4 and Equation 4.5, respectively. We simulate a continuous

time SIR model with parameters N = 100, I(1) = 5 and t = 50 for β = 1.5 and γ = 0.5 and for

β = 0.9 and γ = 0.3. For 10,000 simulations, the histograms of β̂ are given in Figure 23 and

Figure 25 while the histograms of γ̂ are given in Figure 24 and Figure 26. Simulations with

infections dying out at t = 7 are repeated for simulations with β = 1.5 and γ = 0.5. Simulations

with infections dying out at t = 9 are repeated for simulations with β = 0.9 and γ = 0.3.

For the two pairs of β and γ, 20 estimates of β̂ and γ̂ obtained using fminsearch and the

theoretical formulas are presented in Table 8. The bias and MSE are presented in Table 9 and

Table 10. The histograms of γ̂ for γ = 0.5 and γ = 0.3 are right skewed due to some simulations

having the number of infectious dying out right after time t = 7 (or t = 9). The time constraint

may be increased to allow only simulations that last long enough to predict β and γ. The bias

and MSE are small, so the MLE gives a good estimate of β and γ.

65
Figure 23. Histogram of β̂ for a CTMC SIR Model with β = 1.5 and γ = 0.5

Parameter estimation distribution of β̂ for a CTMC SIR model with β = 1.5 and γ = 0.5. Each

distribution has 10,000 simulations with N = 100, I(1) = 5 and t = 50. Left panel: Parameter

estimation distribution of β̂ using fminsearch. Right panel: Parameter estimation distribution of

β̂ using theoretical estimates.

Figure 24. Histogram of γ̂ for a CTMC SIR Model with β = 1.5 and γ = 0.5

Parameter estimation distribution of γ̂ for a CTMC SIR model with β = 1.5 and γ = 0.5. Each

distribution has 10,000 simulations with N = 100, I(1) = 5 and t = 50. Left panel: Parameter

estimation distribution of γ̂ using fminsearch. Right panel: Parameter estimation distribution of

γ̂ using theoretical estimates.

66
Figure 25. Histogram of β̂ for a CTMC SIR Model with β = 0.9 and γ = 0.3

Parameter estimation distribution of β̂ for a CTMC SIR model with β = 0.9 and γ = 0.3. Each

distribution has 10,000 simulations with N = 100, I(1) = 5 and t = 50. Left panel: Parameter

estimation distribution of β̂ using fminsearch. Right panel: Parameter estimation distribution of

β̂ using theoretical estimates.

Figure 26. Histogram of γ̂ for a CTMC SIR Model with β = 0.9 and γ = 0.3

Parameter estimation distribution of γ̂ for a CTMC SIR model with β = 0.9 and γ = 0.3. Each

distribution has 10,000 simulations with N = 100, I(1) = 5 and t = 50. Left panel: Parameter

estimation distribution of γ̂ using fminsearch. Right panel: Parameter estimation distribution of

γ̂ using theoretical estimates.

67
 β = 1.5 γ = 0.5 β = 0.9 γ = 0.3
β̂fminsearch β̂theoretical γ̂fminsearch γ̂theoretical β̂fminsearch β̂theoretical γ̂fminsearch γ̂theoretical
1.7428 1.7550 0.5171 0.4354 0.8892 0.8998 0.3570 0.3224
1.4552 1.4885 0.5192 0.4375 0.8338 0.8497 0.3335 0.3046
1.4008 1.4411 0.6393 0.5085 0.8571 0.8595 0.2753 0.2588
1.3492 1.3850 0.4775 0.4095 0.9494 0.9601 0.3277 0.3010
1.3213 1.3805 0.6281 0.5163 0.8495 0.8687 0.3545 0.3200
1.5680 1.6086 0.6607 0.5244 0.7703 0.7768 0.2697 0.2476
1.7140 1.7373 0.4758 0.4054 0.8451 0.8705 0.3595 0.3268
1.7136 1.7485 0.5850 0.4837 0.7937 0.8104 0.33201 0.3046
1.4325 1.4941 0.6208 0.4942 0.7923 0.8076 0.3291 0.3009
1.4730 1.5110 0.5763 0.4874 0.7715 0.7910 0.3785 0.3415
1.2484 1.2705 0.5198 0.4433 0.9420 0.9527 0.3555 0.3295
1.7468 1.7579 0.5171 0.4309 0.8796 0.8895 0.2907 0.2741
1.7136 1.7377 0.6328 0.5137 1.0995 1.1070 0.3330 0.2982
1.5640 1.6160 0.5470 0.4542 0.8160 0.8361 0.3834 0.3441
1.6864 1.6974 0.5714 0.4718 0.8322 0.8431 0.3176 0.2956
1.5315 1.5705 0.5602 0.4528 0.7607 0.7753 0.3027 0.2769
1.7573 1.7836 0.5937 0.4768 0.8772 0.8787 0.3044 0.2869
1.3245 1.3807 0.6570 0.5254 0.9978 1.0160 0.3597 0.3228
1.5227 1.5504 0.6453 0.5235 0.8028 0.8145 0.3013 0.2787
1.4305 1.4797 0.5761 0.4676 0.7542 0.7748 0.3449 0.3111

Table 8. Simulation Parameter Estimates for a CTMC SIR Model

All simulations are performed with N = 100, I(1) = 5 and t = 50.

Known values β = 1.5 γ = 0.5

β̂fminsearch β̂theoretical γ̂fminsearch γ̂theoretical
Bias −0.0375 0.1877 0.1372 −0.0230
MSE 0.0318 0.0621 0.0281 0.0034
Number of
52
Repeated Simulations

Table 9. Computational Results for a CTMC SIR Model with β = 1.5 and γ = 0.5

Computational results for 10,000 simulations with N = 100, I(1) = 5 and t = 50.

68
 Known values β = 0.9 γ = 0.3
β̂fminsearch β̂theoretical γ̂fminsearch γ̂theoretical
Bias −0.0069 0.0816 0.0350 0.0117
MSE 0.0101 0.0162 0.0030 0.0012
Number of
84
Repeated Simulations

Table 10. Computational Results for a CTMC SIR Model with β = 0.9 and γ = 0.3

Computational results for 10,000 simulations with N = 100, I(1) = 5 and t = 50.

69
 REFERENCES

[1] E. Alirol, L. Getaz, B. Stoll, F. Chappuis, and L. Loutan, Urbanisation and infectious

diseases in a globalised world, Lancet Infect Dis. 11 (2011), no. 2, 131–141.

[2] L.J.S. Allen, An introduction to stochastic epidemic models, Mathematical epidemiology,

Lecture Notes in Math., vol. 1945, Springer, Berlin, 2008, pp. 81–130. MR 2428373

[3] , An introduction to stochastic processes with applications to biology, second ed., CRC

Press, 2010.

[4] , A primer on stochastic epidemic models: formulation, numerical simulation, and

analysis, Infectious Disease Modelling 2 (2017), 128–142.

[5] F. Brauer, Compartmental models in epidemiology, Mathematical Epidemiology (F. Brauer,

Pauline van den Driessche, and Jianhong Wu, eds.), Springer, 2008, pp. 19–79.

[6] D.L. Buckeridge, A. Okhmatovskaia, S. Tu, M. O’Connor, C. Nyulas, and M.A. Musen,

Predicting outbreak detection in public health surveillance: quantitative analysis to enable

evidence-based method selection, AMIA Annual Symposium Proceedings 2008 (2008), 76–80.

[7] M.D. Christian, S.M. Poutanen, M.R. Loutfy, M.P. Muller, and D.E. Low, Severe acute

respiratory syndrome, Clinical Infectious Diseases 38 (2004), no. 10, 1420–1427.

[8] N.S. Clemmons, G.S. Wallace, M. Patel, and P.A. Gastaaduy, Incidence of measles in the

United States, 2001-2015, JAMA 318 (2017), no. 13, 1279–1281.

[9] S.R. Cole, H. Chu, and S. Greenland, Maximum likelihood, profile likelihood, and penalized

likelihood: a primer, American Journal of Epidemiology 179 (2014), no. 2, 252–260.

[10] M.M. Dalhat, A.N. Isa, P. Nguku, S. Nasir, K. Urban, M. Abdulaziz, R.S. Dankoli,

P. Nsubuga, and Poggensee G., Descriptive characterization of the 2010 cholera outbreak in

Nigeria, BMC Public Health 14 (2014), no. 1, 1167.

70
[11] K. Dietz, The estimation of the basic reproduction number for infectious diseases, Statistical

Methods in Medical Research 2 (1993), no. 1, 23–41, PMID: 8261248.

[12] P. Driscoll, F. Lecky, and M. Crosby, An introduction to statistical inference — 3, Emergency

Medicine Journal 17 (2000), no. 5, 357–363.

[13] C.K. Fairley and E. Miller, Varicella-zoster virus epidemology — a changing scene, Journal

of Infectious Disease 174 (1996), Suppl 3:314–319.

[14] A. Huppert and G. Katriel, Mathematical modelling and prediction in infectious disease

epidemiology, Clinical Microbiology and Infection 19 (2013), no. 11, 999–1005.

[15] J.H. Jones, Notes on R0 , Stanford University, Stanford, 2007.

[16] M.J. Keeling and R. Pejman, Modeling infectious diseases in humans and animals, Princeton

University Press, 2008.

[17] W.O. Kermack and A.G. McKendrick, A contribution to the mathematical theory of

epidemics, Proceedings of the Royal Society of London. Series A, Containing Papers of a

Mathematical and Physical Character 115 (1927), no. 772, 700–721.

[18] L. Kleinrock, Queueing systems, volume 1, theory, Wiley-Interscience, New York, NY, USA,

1975.

[19] J.T. Macfarlane and W.S. Lim, Bird flu and pandemic flu, BMJ 331 (2005), no. 7523,

975–976.

[20] A. Mummert and O. Otunuga, Parameter identification for a stochastic SEIRS epidemic

model: case study influenza, Journal of Mathematical Biology 72 (2019), no. 2, 705–729.

[21] World Health Organization, World Malaria Report 2011, Geneva, Switzerland, 2011.

[22] M. Patel, A.D. Lee, S.B. Redd, N.S. Clemmons, R.J. McNall, A.C. Cohn, and P.A.

Gastanaduy, Increase in measles cases in the United States, MMWR 68 (2019), 402–404.

71
[23] T.J. Perkins, Maximum likelihood trajectories for continuous-time Markov chains,

Proceedings of the 22nd International Conference on Neural Information Processing Systems

(Red Hook, NY, USA), NIPS09, Curran Associates Inc., 2009, p. 14371445.

[24] W. Qiu, C. Chu, A. Mao, and J. Wu, The impacts on health, society, and economy of SARS

and H7N9 outbreaks in China: a case comparison study, Journal of Environmental and

Public Health 2018 (2018), 7 pages.

[25] P. Renard, A. Alcolea, and D. Ginsbourger, Stochastic versus deterministic approaches,

Environmental Modelling: Finding Simplicity in Complexity 2 (2013), 113–149.

[26] J.F. Shortle, J.M. Thompson, D. Gross, and C.M. Harris, Fundamentals of queueing theory,

John Wiley & Sons, Inc., 2018.

[27] K.M. Smith, C.C. Machalaba, R. Seifman, Y. Feferholtz, and W.B. Karesh, Infectious disease

and economics: the case for considering multi-sectoral impacts., One health (Amsterdam,

Netherlands) 7 (2019), no. 100080, doi:10.1016/j.onehlt.2018.100080.

[28] A.J. Tatem, D.J. Rodger, and S.I Hay, Global transport networks and infectious disease

spread, Advances in Parasitology 62 (2006), 293–343.

[29] J.K. Taubenberger and D.M. Morens, 1918 influenza: the mother of all pandemics, Emerging

Infectious Diseases 12 (2006), no. 1, 15–22.

[30] P. van den Driessche, Reproduction numbers of infectious disease models, Infectious Disease

Modelling 2 (2017), no. 3, 288–303.

[31] D. Wackerly, W. Mendenhall, and R.L. Scheaffer, Mathematical statistics with applications.,

Thomson Brooks/Cole, 2008.

72
 APPENDIX A

APPROVAL LETTER

73