ANUx Introduction to Actuarial Science

Lesson 3

Analysis of State Transitions

Introduction to Analysis of State Transitions
The cash flow calculations and models you have seen so far in the course have all assumed that the
cash flows are guaranteed to occur. For example, the annuity certain example you looked at in
Lesson 2 assumed that there was nothing that would stop the policy remaining in force over its
entire term. However, there are many factors that could cause the actual cash flows to differ from
those projected in the cash flow model. This is fundamentally what is of interest to actuaries, the
uncertainty in these cash flows, how to measure this uncertainty, and how to manage this
uncertainty.
So what are these factors that could cause the projected cash flows to differ? Firstly, if the premium
income were invested by the insurer in assets that did not provide a certain interest rate, this would
change the interest rate and the Actual Reserves held at future time periods. Perhaps the policy
might have a condition that allows it to be closed down before the end of the term, and a lump sum
claim made to the policyholder to cover the future claims forgone.
Apart from what was just mentioned, there are likely to be many other factors that might cause the
actual cash flows to differ from those projected in an insurance policy. You might like to share some
examples in the forum, where a thread has been created specifically for this topic.
It is important for actuaries to be able to model the occurrence of events that affect cash flows this
is what Analysis of State Transitions is all about. For the moment well consider non-financial
events only, such as sickness and death. Later on in the course we will consider the impact of
financial events such as interest rates. Modelling these uncertain events is fundamental to what
actuarial work is all about, no matter whether the actuary is working in insurance, pensions,
investment, or any other line of work.
Models of Analysis of State Transitions are typically represented in a way similar to the following
example:

Healthy (H)

Temporarily
Ill (TI)

Permanently
Ill (PI)

Dead (D)

Each of the boxes represents possible states for an individual, whilst lines with arrows represent
possible transitions between states. For example a Healthy individual can become Temporarily Ill,
Adam Butt
Version 1 (2015)

Page 1

ANUx Introduction to Actuarial Studies Lesson 3 Analysis of State Transitions

and a Temporarily Ill individual can also become Healthy. A Healthy individual or Temporarily Ill
individual can become Permanently Ill, but a Permanently Ill individual cannot become Healthy or
Temporarily Ill. Absorbing states are states that cannot be transitioned out of once they are entered
(i.e. they only have arrows leading to them and not away from them). Obviously in this example the
Dead state is absorbing; you clearly cannot exit this state once you enter it, but you can enter it
from all of the other states!
These models are very important to actuaries, as the cash flows that affect individuals and the
insurer will depend on the state the individual is in. For example, imagine an insurance product that
requires policyholders to pay a premium to the insurer when they are healthy and take up the
product, and to continue paying premiums each subsequent year that they are healthy. No claim is
paid to the policyholder while they are healthy. But the product provides a regular claim to a
policyholder who is temporarily ill, and a one-off claim to a policyholder who becomes permanently
ill or dies. The policy ceases after one of these one-off claims. Clearly the future movements of
policyholders between states will affect cash flows in (premiums) and out (claims) of the insurer, and
the insurer will need to know the probabilities of movements between these states in order to
ensure the Actual Reserves are sufficient to pay the claims to policyholders as they occur.
In the remainder of this weeks material will we will investigate how to model transitions between
the states. For simplicity, we will consider a basic two-state model only, with a single transition
between the Alive and the Dead state:

Dead (D)

Alive (A)

An example of a product for which this model might be useful is one that requires policyholders to
pay regular premiums when they are alive and then provides a one-off claim when the policyholder
dies. This is obviously a very simple product, but will provide a nice framework from which to view
an example of actuarial work. In practice, actuaries will typically need to consider much more
complicated models of state when working with insurance and other financial products.

Assessment Question 3.1

Which of the following statements best describes what an absorbing state is?
A)
B)
C)
D)

The state that an individual starts in

A state that an individual cannot leave after they have entered it
A state that has many potential paths of entry into
A state that can only be entered into from one other state

Adam Butt
Version 1 (2015)

Page 2

ANUx Introduction to Actuarial Studies Lesson 3 Analysis of State Transitions

The Two-State Model

Now we have our two-state model, we are ready to investigate how this model is used by actuaries
in practice. Well start from our example of a product that requires policyholders to pay regular
premiums when they are alive and then provides a one-off payment when the policyholder dies.

x
Dead (D)

Alive (A)

For this type of policy, once a policyholder dies and they receive their one-off claim from the insurer,
the insurer has no further obligations in respect to that policyholder. Hence, the insurer is primarily
interested in policyholders who are Alive and the probability that they might transition to the Dead
state. We are now going to introduce some notation that will help us model this.
We first introduce x , where is the transition intensity of mortality per annum (i.e. from state
Alive to Dead this is also known as the force of mortality) and x is the factor (or factors) that
affect . For the remainder of the course well assume x is age or time. We would obviously
expect mortality to be affected by age, although there could be other factors such as gender, socioeconomic status, etc. that affect but which we will ignore to keep things simple. Actuaries in
practice will have much more complicated models for than what we will consider here.

An Example Using Car Speed

We can think of

as somewhat equivalent to the instantaneous speed of a car. Lets just imagine

for a moment that s is the speed of a car at exact time s . If s was a constant (i.e. not
dependent on s ), then the distance travelled between time 0 and time t would be:

Distance

Speed Time

This of course assumes that speed is measured in equivalent units to time. For example, if a car
travelled at 60 km/hr for 3 hours it would travel 60 x 3 = 180 km.
Lets make things a little more complicated now. If s is dependent on s (i.e. if changes over
time), then our formula Distance = Speed x Time no longer works as simply as it does above. We
can, however, use integration techniques to calculate the distance travelled. We can see from our
original example that distance is the result of the integration of speed between time 0 and time t ,
or simply the area under a plot of speed against time:

Adam Butt
Version 1 (2015)

Page 3

ANUx Introduction to Actuarial Studies Lesson 3 Analysis of State Transitions

Speed

= 60 km/hr

t = 3 hours

Now let us assume that

Distance
0

s . Now the distance travelled between time 0 and time t would be:

t
s

Time

ds

s.ds
0

s2
2

t2
2

Expressing this is a plot we can see that the area under the line

Speed

s is equal to t 2 / 2 :

Adam Butt
Version 1 (2015)

Page 4

Time

ANUx Introduction to Actuarial Studies Lesson 3 Analysis of State Transitions

Back to the Two-State Model

What we have done with the car example is move from an instantaneous calculation

, to a

calculation that gives us a value over a period of time 0 to t . Typically actuaries are interested in
something similar they would like to know the probability that an individual who is currently alive
at age x will be dead at age x t . This is commonly defined by actuaries as t q x . Equivalently, the
probability that an individual who is currently alive at age x will be alive at age x t is defined as
t

px . Since these options are exhaustive (i.e. you can only be alive or dead) then t px

q .

t x

Intuitively, we might expect t q x to be calculated as follows:

t x

x s

ds

This approach works in the car distance example, as there are no probabilities attached to the
whether the car is driving or not. However, that is not the case here, as an individual could die at
age x j between age x and age x t ; they cannot die again between age x j and age x t
and therefore

x s

is irrelevant when s

Transition
Intensity

j for someone who has died at age x

j.

irrelevant
here

x
Dies x

Time (Age)

Another way of demonstrating the incorrectness of this approach is through a numerical example.
Lets assume that the transition intensity of mortality is constant at a rate of 10% per annum.
Were the above formula correct the probability of dying in the next 15 years, 15q x , would be equal to
10% x 15 = 1.5, a number which is outside the possible bounds of 0 to 1 for a probability to take!
Therefore we need a different approach to determining t q x .
Adam Butt
Version 1 (2015)

Page 5

ANUx Introduction to Actuarial Studies Lesson 3 Analysis of State Transitions

Assessment Question 3.2

Which of the following statements best describes what a transition intensity is?
A)
B)
C)
D)

The state that an individual moves into

The state that an individual moves out of
The probability that an individual moves from one state to another over a period of time
The instantaneous rate of movement from one state to another

Assessment Question 3.3

What is the meaning of the notation t px ?
A)
B)
C)
D)

The probability that an individual aged 0 will die before age x t .

The probability that an individual aged x will die before age x t .
The probability that an individual aged 0 will survive until age x t .
The probability that an individual aged x will survive until age x t .

Adam Butt
Version 1 (2015)

Page 6

ANUx Introduction to Actuarial Studies Lesson 3 Analysis of State Transitions

Calculating Probabilities from Transition Intensities

In determining t q x , it will be helpful to utilise the following conditional rule of probability for events
A and B :

P(A | B)

P(A B )
P(B )

A Dice-Based Example
For those unfamiliar with conditional probability, we give the following simple example using dice.
Imagine that two six-sided dice are rolled. A is the event where the sum of the two dice is 10 and
B is the event where the number on the first dice is 5. Therefore A | B (which means event A
given that event B has already occurred) is the event where the sum of the two dice is 10, given
that the number on the first dice is 5. We want to calculate the probability of A | B . Lets look at
this in terms of the probabilities on the right hand side of the above equation:

P(A

B) is the joint probability of A and B ; i.e. the probability that both A and B will

occur. There are 36 different combinations of numbers that can be rolled on two dice (6 on
the first dice multiplied by 6 on the second dice). The only combination of these that leads
to both A and B occurring is where the first dice is 5 and the second dice is also 5.
P(A B) 1 / 36

P(B) is the probability that the number on the first dice is 5.

P(B)

1/ 6

Thus the required probability is:

P(A | B )

1 / 36
1/ 6

1/ 6

This result makes sense intuitively. If we know the first dice is 5, then the only way the two dice can
add up to 10 is for the second dice to be 5, which has a probability of 1/6. Note that the probability
of the sum of two dice being 10 without knowing the results of one of the dice is 1/12. In this
instance, knowing the result of one of the dice has given us a higher probability of the sum of the
two dice being 10.

Adam Butt
Version 1 (2015)

Page 7

ANUx Introduction to Actuarial Studies Lesson 3 Analysis of State Transitions

Back to the Two-State Model

In the two-state model we will let A be the event of surviving from age 0 to age x t and B be the
event of surviving from age 0 to age x . Therefore A | B is simply the event of surviving from age x
to age x t . Event A B (which means event A and event B both occurring) is just event A ,
since surviving from age 0 to age x t must also entail surviving from age 0 to age x .
Using notation previously introduced, and the conditional probability formula, this is equivalent to
stating:

p0
x p0

x t

px

We will use this result throughout the calculations below.

Recall that the result we want to calculate is t q x . We determine t q x as follows:

t x

px

p0

x t

t x
x

q
t

t x

which we already know

p0

p0
x

p0

x t

p0

by using the probability rule above

p0

As t goes to zero, the left hand side of this equation become the instantaneous rate of mortality,
i.e. the transition intensity x . The right hand side of the equation can be rearranged so it becomes
a differentiation from first principles.

lim t
t

qx
t

1
lim
t 0
x p0

x t

p0

p0

d
x p0
dx
x p0
d
ln
dx

p0

d
since
ln f (x )
dx

d
f (x )
dx
f (x )

Note that ln is a log with base e, where e is a mathematical constant approximate to 2.71828
Adam Butt
Version 1 (2015)

Page 8

ANUx Introduction to Actuarial Studies Lesson 3 Analysis of State Transitions

Given this, we can also state:

d
ln
ds

x s

x s

p0

We will now integrate both sides of this equation:

t
x s

ds

ln

x s

p0

t
0

ln

x t

p0

ln

p0

ln

p0
x p0

x t

ln t px

t
t

px

exp

x s

ds

Note that exp[] is e[] this result holds because exp[ln{}] = {}

t x

px

t qx

exp

t
x s

ds

The following questions will all use the above result.

Practice Question 3.1

An individual has a constant transition intensity of mortality of

0.03 per annum. Calculate the

probability that they will die within the next 5 years.

Assessment Question 3.4

An individual has a constant transition intensity of mortality of
probability that they will die within the next 20 years.

0.01 per annum. Calculate the

Assessment Question 3.5

An individual has a transition intensity of mortality for the next 10 years of s 0.01s per annum
(where s is the number of years from today). Calculate the probability that they will survive for the
next 10 years.

Adam Butt
Version 1 (2015)

Page 9

ANUx Introduction to Actuarial Studies Lesson 3 Analysis of State Transitions

Assessment Question 3.6 (Hard)

The Gompertz-Makeham law of mortality states that the transition intensity of mortality for humans
of age x can be expressed in the form

e x . It has been observed in a population that the

probability of a 20 year old surviving until age 50 is equal to 0.96, and the probability of a 20 year old
surviving until age 85 is equal to 0.50. Assuming that the Gompertz-Makeham law of mortality holds
and that
0.085 , calculate and .
In order to solve this question, you will need to know the result:

e f (x )dx

e f (x )
; where f (x ) is a linear function of x .
d
f (x )
dx

Extension Question 3.1

An individual currently aged 30 experiences a transition intensity of mortality of
x

5 10

2 10 5e 0.085x , where x is equal to age. Calculate the probability that they will die

between age 75 and 80.

Note that this question is asking for a deferred probability of mortality, which is commonly defined by
actuaries as

q , where n is the deferral period that the individual must survive during. In this

n |t x

question, you are asked to calculate

q .

45|5 30

Extension Question 3.2

Show that an alternative formulation for t q x is:
t
t qx

px

x s

ds

Hint: Use the result P(A | B)

x

P(A B )
and let A be the event of dying between age x and age
P(B )

t , and B be the event of surviving from age 0 to age x , attempting to express P(A

B) in

terms of q only.

Given the discussion on the relationship between distance, speed and time earlier in this Lesson,
why does this result make intuitive sense?

Adam Butt
Version 1 (2015)

Page 10

ANUx Introduction to Actuarial Studies Lesson 3 Analysis of State Transitions

Extension Question 3.3

Using

e x , calculate q x in terms of x ,

and

After doing this, use a spreadsheet tool to calculate q x for x

results using the

and

. Note that q x is equivalent to 1q x .

30, 31, 32,..., 80 and graph the

from Extension Question 3.1 (i.e.

You might like to investigate the effect of changing the values of

Adam Butt
Version 1 (2015)

Page 11

5 10

and

2 10 5e 0.085x ).

on the q x values.

ANUx Introduction to Actuarial Studies Lesson 3 Analysis of State Transitions

How Actuaries use State-Based Models

So what does this mean to an actuary in practice? If an actuary is able to determine the transition
intensities then they can use this information to calculate probabilities of transition in their
modelling. Later on in the course, this will become particularly relevant when we want to model the
outcome of selling an insurance product. Actuaries typically estimate the transition intensities and
probabilities by reference to relevant past data, although for the purposes of this course we will
assume that transition intensities and probabilities are known and that data analysis is not required.
Although we have only looked at mathematical relationships between transition intensities and
probabilities of transition in a simple two-state model, in practice most situations that will be
considered by an actuary will have more than two states, which makes the mathematical modelling
far more complicated.
In Lesson 4 we will consider a specific example of the simple two-state model, the life table.

Adam Butt
Version 1 (2015)

Page 12

ANUx Introduction to Actuarial Studies Lesson 3 Analysis of State Transitions

Summary

Analysis of State Transitions involve investigating the states an individual could be in that
will affect the cash flows of a financial system, as well as the potential transitions between
states. Absorbing states are states that cannot be transitioned out of once they are entered.

The two-state model we will consider throughout this course looks like:
x
Dead (D)

Alive (A)

is the transition intensity of mortality per annum (i.e. from state Alive to Dead this is
also known as the force of mortality), where x is age or time.
x

q is the probability that an individual who is currently alive at age x will be dead before

t x

age x t , whilst t px is the probability that an individual who is currently alive at age x will
be alive at age x t . They are calculated as:
t
t px

exp

x s

ds

t x

px

t qx

exp

t
x s

ds

Generating the above formulae required the use of the conditional probability formula:
P(A | B)

Adam Butt
Version 1 (2015)

P(A B )
P(B )

Page 13