T MU MD 20003 GU

Guide

Quantified Safety Risk Assessment

Version 1.0
Issue date: 19 December 2019

© State of NSW through Transport for NSW 2019

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

Important message
This document is one of a set of standards developed solely and specifically for use on
Transport Assets (as defined in the Asset Standards Authority Charter). It is not suitable for any
other purpose.

The copyright and any other intellectual property in this document will at all times remain the
property of the State of New South Wales (Transport for NSW).

You must not use or adapt this document or rely upon it in any way unless you are providing
products or services to a NSW Government agency and that agency has expressly authorised
you in writing to do so. If this document forms part of a contract with, or is a condition of
approval by a NSW Government agency, use of the document is subject to the terms of the
contract or approval. To be clear, the content of this document is not licensed under any
Creative Commons Licence.

This document may contain third party material. The inclusion of third party material is for
illustrative purposes only and does not represent an endorsement by NSW Government of any
third party product or service.

If you use this document or rely upon it without authorisation under these terms, the State of
New South Wales (including Transport for NSW) and its personnel does not accept any liability
to you or any other person for any loss, damage, costs and expenses that you or anyone else
may suffer or incur from your use and reliance on the content contained in this document. Users
should exercise their own skill and care in the use of the document.

This document may not be current and is uncontrolled when printed or downloaded. Standards
may be accessed from the Transport for NSW website at www.transport.nsw.gov.au

For queries regarding this document, please email the ASA at

[email protected] or visit www.transport.nsw.gov.au

© State of NSW through Transport for NSW 2019

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

Standard governance
Owner: Manager Safety and Risk Assurance, Asset Standards Authority
Authoriser: Director Safety, Quality, Environment and Risk, Asset Standards Authority
Approver: Executive Director, Asset Standards Authority on behalf of the ASA Configuration Control
Board

Document history
Version Summary of changes
1.0 First issue. Incorporation of content from T MU MD 20000 GU v2.0 and TfNSW internal document
30-SD-492.

© State of NSW through Transport for NSW 2019 Page 3 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

Preface
The Asset Standards Authority (ASA) is a key strategic branch of Transport for NSW (TfNSW).
As the network design and standards authority for NSW Transport Assets, as specified in the
ASA Charter, the ASA identifies, selects, develops, publishes, maintains and controls a suite of
requirements documents on behalf of TfNSW, the asset owner.

The ASA deploys TfNSW requirements for asset and safety assurance by creating and
managing TfNSW's governance models, documents and processes. To achieve this, the ASA
focuses on four primary tasks:

• publishing and managing TfNSW's process and requirements documents including TfNSW
plans, standards, manuals and guides

• deploying TfNSW's Authorised Engineering Organisation (AEO) framework

• continuously improving TfNSW’s Asset Management Framework

• collaborating with the Transport cluster and industry through open engagement

The AEO framework authorises engineering organisations to supply and provide asset related
products and services to TfNSW. It works to assure the safety, quality and fitness for purpose of
those products and services over the asset's whole-of-life. AEOs are expected to demonstrate
how they have applied the requirements of ASA documents, including TfNSW plans, standards
and guides, when delivering assets and related services for TfNSW.

Compliance with ASA requirements by itself is not sufficient to ensure satisfactory outcomes for
NSW Transport Assets. The ASA expects that professional judgement be used by competent
personnel when using ASA requirements to produce those outcomes.

About this document

This document provides guidance for quantified safety risk assessments. This document also
provides guidance on the application of the assessment outputs to support a reasoned safety
argument.

The guidance in this document reflects the full scope of Transport assets operated and
managed by TfNSW.

This guide supports T MU MD 20001 ST System Safety Standard for New or Altered Assets.

This document is a new document that replaces T MU MD 20000 GU Risk Tolerability,

Quantified Risk Assessment and its Role in the Assurance of Change, version 2.0.

The changes in this document from its predecessor T MU MD 20000 GU include the following:

• document title change

• refinement of the scope and content

© State of NSW through Transport for NSW 2019 Page 4 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

• incorporation of content from TfNSW internal document 30-SD-492 Safety Risk Tolerability
and Targets

This document is a first issue.

© State of NSW through Transport for NSW 2019 Page 5 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

Table of contents
1. Introduction .............................................................................................................................................. 7
2. Purpose .................................................................................................................................................... 7
2.1. Scope ..................................................................................................................................................... 8
2.2. Application ............................................................................................................................................. 8
3. Reference documents ............................................................................................................................. 8
4. Terms and definitions ............................................................................................................................. 9
5. Key concepts ......................................................................................................................................... 10
5.1. So far as is reasonably practicable within TfNSW ............................................................................... 10
5.2. TfNSW safety risk tolerability requirements ......................................................................................... 11
5.3. Risk assessment types ........................................................................................................................ 11
6. Quantified safety risk assessment in TfNSW ..................................................................................... 12
6.1. Approach to quantified safety risk assessment ................................................................................... 13
6.2. Fatalities and weighted injuries............................................................................................................ 13
6.3. Collective risk versus individual risk .................................................................................................... 14
6.4. Data accuracy and assumptions ......................................................................................................... 15
7. Using a QSRA to support a safety argument ..................................................................................... 15
7.1. Cost-benefit analysis ........................................................................................................................... 16
7.2. Value of preventing a fatality ............................................................................................................... 16
7.3. Gross disproportion ............................................................................................................................. 17
7.4. Net present value and discounting ...................................................................................................... 18
7.5. Sensitivity analysis ............................................................................................................................... 18
7.6. Societal concern .................................................................................................................................. 19
8. Undertaking a QSRA ............................................................................................................................. 19
8.1. Key considerations .............................................................................................................................. 19
8.2. Seek assistance ................................................................................................................................... 20

© State of NSW through Transport for NSW 2019 Page 6 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

1. Introduction
TfNSW has an extensive portfolio of assets and operations and is undergoing a period of
sustained investment which will further expand the networks and services. TfNSW has a legal
duty to ensure that transport assets and transport operations are managed to a level which is
safe, so far as is reasonably practicable (SFAIRP).

For novel or complex systems or high-value investments, quantified safety risk assessment
(QSRA) is a tool that can support the decision making process to demonstrate that safety risks
are managed SFAIRP. QSRA is a technique for calculating the safety risk exposure to a defined
population as a result of the operation of an asset or system. This technique is widely used in
the transport industry to assess safety risks to customers, workers and members of the public in
proximity to the system.

QSRA has two primary purposes, the first is during the plan and acquire stages of a project
where it can be used to set safety targets and to inform investment decisions and in turn to
support the safety argument to enter operational service.

The second purpose is during the operate / maintain stage where a QSRA can be used to
monitor an organisation's risk profile, prioritise risk areas and support decision making to
support the ongoing safe operations.

The decision to employ QSRA methodology is one that requires specialist guidance and
consideration of the novelty, costs, potential for benefit and influence of the change that can be
attributed to the employment of quantified as opposed to qualitative risk assessment
methodology.

A QSRA should be seen as a key input into the safety decision making process but it should
always be used as part of a wider set of considerations to develop a reasoned safety argument.

2. Purpose
This document provides guidance in the use of QSRA and discusses issues and implications
associated with the assurance of change or the management of safe operations within the
Transport cluster.

The development of QSRA requires a degree of specialist knowledge and experience and
guidance needs to be sought where necessary.

This document provides guidance to develop a QSRA such that a common approach can be
implemented across the Transport cluster. This document also provides a set of criteria to be
used when developing a reasoned safety argument to support either an investment decision or
a SFAIRP determination.

© State of NSW through Transport for NSW 2019 Page 7 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

2.1. Scope
This document provides guidance on the application of QSRAs to support new or altered
transport assets and the management of transport operations with the Transport cluster.

This document supports the overall TfNSW asset acceptance process and T MU MD 20001 ST
System Safety Standard for New or Altered Assets; however it does not define that process.
This document covers QSRA for the development of SFAIRP arguments for significant changes
to support the TfNSW safety acceptance requirements.

This guide is also consistent with the requirements of the TfNSW safety management system
with respect to safety change management and safety risk management.

This document does not cover the process to conduct a QSRA, although it sets out key
considerations. The guidance in this document does not cover criteria that an organisation
needs to meet to be authorised as an Authorised Engineering Organisation (AEO); the guidance
should be followed by an AEO under contract or a TfNSW change manager.

2.2. Application
This document is applicable to all transport assets either owned by TfNSW or for which
operation or maintenance or both are contracted either directly or indirectly by TfNSW. It applies
to all changes to existing transport assets and the introduction of new transport assets on the
Transport Network. It also applies to operational changes that can impact the risk profile of the
transport assets or any group exposed to safety risk either directly or indirectly from the
operation or maintenance or both of the transport assets. This includes but is not limited to the
general public, transport services customers and transport workers.

This document applies to all stages of the asset life cycle, although the greatest benefits are
realised when undertaken in the plan stage of a project when a QSRA can be used to set safety
targets for the change which can form the basis of the safety assurance process throughout the
acquire stage.

QSRA provides valuable support to safety decision making throughout the operate / maintain
stage.

3. Reference documents
The following documents are cited in the text. For dated references, only the cited edition
applies. For undated references, the latest edition of the referenced document applies.

Transport for NSW standards

30-ST-164 TfNSW Enterprise Risk Management (TERM) Standard (restricted document – for
internal TfNSW use only)

T MU MD 20001 ST System Safety Standard for New or Altered Assets

© State of NSW through Transport for NSW 2019 Page 8 of 21
 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

T MU MD 20002 ST Risk Criteria for Use by Organisations Providing Engineering Services

Legislation

Maritime Safety Act 1998

Rail Safety National Law (NSW) 2012

Work Health and Safety Act 2011

Other reference documents

NSW Treasury 2017, NSW Government Guide to Cost-Benefit Analysis (TPP17-03)

Transport for NSW 2015, Asset Management Framework Overview (available on request from
[email protected])

Transport for NSW 2018, Principles and Guidelines for Economic Appraisal of Transport
Investment and Initiatives

Department of the Prime Minister and Cabinet, Office of Best Practice Regulation 2018, Best
Practice Regulation Guidance Note: Value of statistical life

Rail Safety and Standards Board (RSSB) 2008, The Weighting of Non-Fatal Injuries (T440)

Rail Safety and Standards Board (RSSB) 2019, Taking Safe Decisions, version 3.0

4. Terms and definitions

The following terms and definitions apply in this document:

AEO Authorised Engineering Organisation

ALARP as low as reasonably practicable

ASA Asset Standards Authority

BCR benefit-cost ratio

CBA cost-benefit analysis

collective risk the total risk generated by a system to all exposed groups, which might include
passengers, staff and members of the public

CPI consumer price index

ETA event tree analysis

FTA fault tree analysis

FWI fatalities and weighted injuries

gross disproportion a legal principle to be applied when making safety decisions

individual risk the risk to a theoretical individual using the systems

© State of NSW through Transport for NSW 2019 Page 9 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

NPV net present value

PCBU person conducting a business or undertaking

QSRA quantified safety risk assessment; a mathematical method for calculating safety risk.

SFAIRP so far as is reasonably practicable

SMS safety management system

TfNSW Transport for NSW

Transport Network the transport system (transport services and transport infrastructure)
owned and operated by TfNSW, its operating agencies or private entities upon which TfNSW
has power to exercise its functions as conferred by the Transport Administration Act or any
other Act

VoSL value of a statistical life; a financial value applied to a theoretical loss of life

VPF value of preventing a fatality; a similar concept to 'value of a statistical life'

5. Key concepts
Key concepts associated with the TfNSW approach to quantified safety risk assessments
(QSRA) are covered in Section 5.1 to Section 5.3.

5.1. So far as is reasonably practicable within TfNSW

Transport for NSW is responsible under legislation, regulations and NSW Government policy to
manage risks to safety SFAIRP. For certain modes, such as train services, the concept of
SFAIRP is a defined term in the Rail Safety National Law (NSW) (RSNL). For other modes such
as the oversight of the road network, a safe systems approach is used, which is an
internationally acknowledged policy approach to improving road safety.

The Work Health and Safety Act (WHS Act) requires a person conducting a business or
undertaking (PCBU) to ensure safety SFAIRP. Therefore, while the definitions and applications
may vary, TfNSW applies the SFAIRP principles for each mode where it is involved in delivering
or establishing the regulatory regime.

In practice, the application of SFAIRP requires a judgement to be made while following a risk
management approach of the safety risks of an activity against the resources needed to
eliminate or reduce those safety risks in terms of time, cost, or effort. No hard and fast legal
definition exists of what is required to reduce risks SFAIRP and so the test of 'reasonably
practicable' is applied.

The SFAIRP approach puts the onus on the duty holder to determine whether the costs or
business impacts of additional measures to control the risk (over and above those risk controls

© State of NSW through Transport for NSW 2019 Page 10 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

already in place) would be grossly disproportionate to the risk reduction benefit that they would
achieve.

In some organisations or jurisdictions this approach is called as low as reasonably practicable

(ALARP). However, TfNSW uses the SFAIRP approach on all activities, as required under the
WHS Act and RSNL.

5.2. TfNSW safety risk tolerability requirements

Due to the multi-modal nature of TfNSW, safety risks are not measured consistently for the
travelling public as a whole. As such TfNSW has taken the decision to move away from setting
tolerability levels for individual transport modes. The overarching requirement is for operations
to be able to demonstrate that risks are managed SFAIRP and that no individual risks are
intolerable.

When developing a new project or mode or setting a safety policy, a QSRA can be used to set
quantified safety targets specific to that change or system; however the quantified safety targets
should not be confused with tolerability levels. Certain modes and operators may have
developed baseline tolerability and target requirements and all applicable requirements can be
checked as part of any QSRA activity.

5.3. Risk assessment types

When choosing the type of risk assessment, a project needs to consider if the risk assessment
technique is suitable to the type, complexity and nature of hazards and safety risks. Risk
assessments assist in understanding controls and assess the effect of risk reduction measures.
As part of this, risk assessment is grouped into two methods - qualitative risk assessment and
quantitative risk assessment.

5.3.1. Qualitative risk assessment

Qualitative risk assessment is commonly used across industry and relies upon knowledge and
understanding of hazards, likelihoods and controls. Qualitative risk assessment methodologies
are generally the primary risk assessment activity due to the relative ease and comprehension
of the process, accessibility and low financial impacts.

These methods generally rely upon the application of relative or descriptive scales to assess the
likelihood and the consequences of a risk and then generally use a matrix to categorise them
and determine their acceptability.

Qualitative methods do not employ mathematical analysis to identify the likelihood of a

consequence. Rather, they use evidence, stakeholder inputs and judgement to analyse the
likelihood and impact. Project risk workshops are an example of an activity supporting the
qualitative approach.

© State of NSW through Transport for NSW 2019 Page 11 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

In the majority of cases, qualitative risk assessment provides an adequate means of supporting
safety decision making and the integration of safety into the design, construction and system
integration into the transport network.

T MU MD 20002 ST Risk Criteria for Use by Organisations Providing Engineering Services

provides the consequence and likelihood criteria AEOs use when undertaking qualitative risk
assessments. On TfNSW internal projects, 30-ST-164 TfNSW Enterprise Risk Management
(TERM) Standard should be referenced.

5.3.2. Semi-quantitative risk assessment

The term 'semi-quantitative risk assessment' is sometimes used when numerical methods or
calculations are used to determine the categorisation of likelihood or consequence. This method
of assessment is a variant of the qualitative technique and should not be confused with a full
QSRA.

5.3.3. Quantitative risk assessment

QSRA methods are based on a mathematical analysis of potential failure modes and likely
effects to calculate risks using specific criteria. QSRAs are complex, requiring specialist
expertise and are costly when compared with qualitative methods. Generally, quantified
methods are considered appropriate for system level assessment or for risks that are identified
as high consequence but low likelihood by an initial qualitative assessment.

Decisions that entail one or more of the following characteristics, should, among other
appropriate techniques, consider applying a QSRA method:

• relatively high number of people potentially affected

• system level assessments considering a range of hazardous events

• decision commits TfNSW to a long-term outcome or arrangement

• novel outcomes with no relevant standards or guidance

• complex and wide range of variables affect the decision-making

This requires that most major procurements of fixed infrastructure and new services
demonstrate that a QSRA was undertaken or at a minimum provide an assured argument as to
why it was considered not relevant. This is particularly the case for major public transport
projects. However, major road projects, whether focused on efficiency, safety, or both, are
similarly subject to these principles.

6. Quantified safety risk assessment in TfNSW

Guidance on the application of QSRA within TfNSW is covered in Section 6.1 to Section 6.4,
including key concepts and applications of QSRA within the TfNSW environment and portfolio.

© State of NSW through Transport for NSW 2019 Page 12 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

The employment of a QSRA can be a costly, time consuming and specialist exercise. When
considering to employ a QSRA, a project needs to consider the cost-benefit ratio of a QSRA in
supporting a SFAIRP argument. As such, TfNSW supports the scalable QSRA model of a
QSRA and Semi-QSRA. The concept of a scalable QSRA approach is employed successfully in
other organisations within the transport industry.

Section 6.1 to Section 6.4 also cover key QSRA concepts in support of defining QSRA for
TfNSW applications.

6.1. Approach to quantified safety risk assessment

QSRA is an established technique for estimating the risk associated with a particular hazard or
hazardous event. It usually involves a progressive process of risk management from
identification of safety risks (hazards) through modelling of the system or scenario that is the
source of risk and then numerically analysing the risk to provide a quantified estimate of the
risk. It requires a quantified analysis of frequency and consequence.

QSRA is generally conducted through the use of fault tree analysis (FTA) or event tree analysis
(ETA) or a combination of both. It also requires a consequence analysis. Though other means
of conducting QSRA, such as in spreadsheets, are available, FTA and ETA are established and
proven techniques that have been in use across high reliability and safety critical industries for
many years.

When conducting FTA or ETA activities, practitioners should recognise the limitations of these
tools. Specifically, around the suitability for modelling situations where there are deliberate
violations (terrorism, road rule violations, pedestrian violations on tram ways, trespass and so
on). Additional limitations exist when there is limited failure rate data available.

QSRA relies upon the estimation of probability or frequency of failures and events that
contribute to a hazard or hazardous event as well as assessment of the consequences of each
event. As such it is reliant on a source of good and applicable data on which to base the
analysis.

6.2. Fatalities and weighted injuries

As with any modelling technique, a common metric for comparing the outputs is required. With
QSRA the baseline unit of measurement is a death resulting from a fatal injury. Not all accidents
result in a fatality; however, that does not mean that those risks can be ignored.

A number of methods have been developed which compare non-fatal injuries to fatalities and
provide a relative weighting. In the rail environment, TfNSW has adopted the fatalities and
weighted injuries (FWI) definitions used by the Rail Safety and Standards Board (RSSB) in the
UK research brief The Weighting of Non-Fatal Injuries (T440) of May 2008.

© State of NSW through Transport for NSW 2019 Page 13 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

The FWI approach states that it would be worth the same amount of effort to prevent 10 major
injuries as it would be to prevent one fatality. Therefore, a major injury is weighted at one tenth
of a fatality or 0.1 FWI. This approach is used for minor reportable and first aid injuries. The FWI
ratios are given in Table 1.

Table 1 - Fatality and weighted injury ratios

Accident severity Fatality equivalent FWI

Fatality 1 1.0
Major injury 10 0.1
Minor injury 200 0.005
First aid injury 1000 0.001

The FWI definition provides a standard approach for assessing the consequences of an event,
and the risk is considered by applying the risk over a year. The metric for quantified risk using
this approach can be expressed as fatalities and weighted injuries per year or FWI/yr.

QSRA undertaken on behalf of TfNSW uses the FWI criteria as described in Table 1 and
present risk outputs as FWI/yr.

Some international benchmarking criteria base their approach on fatalities only and do not
consider injuries. When using this approach in a benchmark assessment for TfNSW these
fatalities are presented in addition to the FWI/yr and described as fatalities per year or Fat/yr.

6.3. Collective risk versus individual risk

To support the principles of QSRA, collective risk and individual risk are two key categories to
be considered in measuring the population exposure levels.

6.3.1. Collective risk

Collective risk considers the total risk associated with the system under consideration.
Collective risk is the sum of all the risks and applies to anyone affected by those risks including
customers, workers and members of the public.

Changes to the collective risk are used as the basis for cost-benefit analysis.

6.3.2. Individual risk

Individual risk is an approach that considers the level of risk to an individual hypothetical
person. In a transport network context this is usually considered to be a typical commuter
because such a commuter is generally the predominant user of the network.

A typical commuter is assumed to make on average 500 journeys per year (2 journeys per day,
5 days per week, for 50 weeks per year).

© State of NSW through Transport for NSW 2019 Page 14 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

Individual risk is calculated by taking the collective risk, dividing that by the total number of
journeys per year, then multiplying by 500 to represent the typical commuter.

Given the complexity and variance of a typical commuter journey, it is not practicable to
consider beyond a single transport mode. As such, the outcome will not be the risk to a typical
commuter for their entire journey, but just the exposure on a single mode or on the given
system under consideration.

Given that growth inevitably happens on transport systems throughout their life, individual risk is
generally used to set safety targets in a project so that those targets are not impacted as much
by changes in passenger numbers.

6.3.3. Intolerable individual risk levels

Individual risk can also be used to determine if a particular risk exposure is too high and
deemed intolerable. An individual risk of 1 in 10,000 fatalities per year for a customer or
member of the public is generally considered intolerable. Similarly, an individual risk of 1 in
1000 fatalities per year for a worker is considered intolerable. Where an individual risk is
assessed as greater than these levels, action has to be taken to reduce those risks, regardless
of costs and SFAIRP considerations.

6.4. Data accuracy and assumptions

With the complex calculations involved in developing a QSRA, large amounts of data is required
as input to the models. In order for the models to present a realistic analysis of the risk the data
should be as accurate as possible. However, it can be difficult to source data for low likelihood
events and therefore expert judgment is often used.

Where expert judgement is used it should be from an experienced or qualified individual, or

preferably consensus of a group of experts and the source should be documented.

The person developing the QSRA model should always be conscious of conservatism, as small
margins of safety can compound throughout the calculation resulting in significant over-
estimations of the level of risk. For example, a natural tendency to round numbers up or down
or be conservative to include a degree of safety can exist.

Properly managed, these adjustments build in a natural margin of error as it means that the
calculated risk is usually overestimated, but when the outputs are used as part of a cost-benefit
analysis to support an investment decision they may need to be re-visited to improve the level
of accuracy.

7. Using a QSRA to support a safety argument

A QSRA is a tool for assessing the level of safety risk presented by a system or operation. The
tool can also be used to determine the level of risk reduction that could be achieved if additional

© State of NSW through Transport for NSW 2019 Page 15 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

safety measures were introduced. This can then be used to support a safety argument by
assessing whether the time, cost, or effort of introducing those safety measures would be
reasonably practicable given the risk reduction benefit that they would achieve.

The assessment of time and effort as part of a safety argument are subjective and should be
discussed as part of a reasoned safety argument. Section 7.1 to Section 7.6 describe the
techniques for assessing the cost aspects by applying a financial consideration to the safety risk
benefits.

7.1. Cost-benefit analysis

One of the most important factors when determining whether a safety measure may be
reasonably practicable to implement is assessing the cost of implementing the measure against
the potential safety benefit. To do this a cost-benefit analysis (CBA) is used.

By using the QSRA model, safety risk benefits can be assessed using various safety measures
such as the FWI/yr approach. Using the techniques described in Section 7.2 to Section 7.6, it is
possible to directly apply a financial value to those safety benefits. These safety benefits can be
used to inform a decision as to whether those measures would be considered reasonably
practicable to implement (noting that this would be from a cost perspective only).

When undertaking a CBA for a given control, there needs to be a clear understanding on the
time period that those costs and benefits will be assessed. For example, if a control has an
expected design life of 20 years, then that would be a reasonable timeframe over which to
assess the costs and benefits. Where no defined period exists then a good practice is to
undertake the base case assessment over 30 years and apply a sensitivity analysis as part of
the reasoned safety argument.

CBA is a guidance tool only as it provides an objective view of the costs and benefits of a safety
measure that need to be considered as part of a reasoned safety argument.

7.2. Value of preventing a fatality

When undertaking a CBA based on the findings of a QSRA, the safety risk benefits should be
translated to financial benefits such that they can be objectively assessed against the costs.

Both the federal government and the NSW Government have undertaken assessments to
determine the financial value to be placed on preventing a theoretical fatality. This is known as
the value of preventing a fatality (VPF) and is applied to FWI calculations, not just fatal injuries.
These assessment are carried out on a 'willingness to pay' basis and are developed using
various techniques to determine how much society would be willing to pay to prevent someone
from dying.

The Commonwealth Department of Prime Minister and Cabinet, Office of Best Practice
Regulation, provided a figure of $4.9m in October 2018 for the value of a statistical life (VoSL) in

© State of NSW through Transport for NSW 2019 Page 16 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

Best Practice Regulation Guidance Note: Value of statistical life. VoSL is a similar concept to
VPF.

The NSW Government published a figure of $6.42m calculated in December 2011 for the VoSL
in accordance with Principles and Guidelines for Economic Appraisal of Transport Investment
and Initiatives (Transport Economic Appraisal Guidelines), in June 2018.

These two figures are notably different; this is likely due to the questions that were used when
undertaking the willingness to pay analysis. The NSW assessment used an example scenario
close to the interviewees’ home location and as such there would be a natural bias that more
would be spent to fix a problem than if it had been in another part of the state.

In order to provide an impartial assessment, the Commonwealth figure is used as the base
case, with the option to consider the NSW figure as part of the sensitivity analysis if it is
undertaken.

VPF figures should be updated to reflect their current value by applying the compounded
Australian consumer price index (CPI) increases from the date the figure was published to the
time that the assessment is made.

To avoid confusion, TfNSW requires the term value of preventing a fatality (VPF) to be used
consistently in any assessment.

7.3. Gross disproportion

When any safety decision is made to support a SFAIRP position, the decision needs to be
balanced in favour of safety. At the core of SFAIRP is the concept of 'reasonable practicability',
which means that a safety measure or benefit can only be ruled out as not 'reasonably
practicable' where the sacrifice in money, time, trouble or other cost of the measure or benefit is
'grossly disproportionate' to that benefit being the reduction in risk.

No guidance has come from the law courts or otherwise on what represents a level of gross
disproportion. The person undertaking the assessment is entirely responsible for determining
whether the cost is grossly disproportionate or not. Hence CBA alone cannot justify a safety
related decision and should form part of a reasoned safety argument.

When QSRA and CBA techniques were in their infancy, factors based on good practice were
used to determine what constituted gross disproportion. With the development of sophisticated
modelling tools and a much more developed understanding of CBA techniques, these factors
are no longer the default position and current good practice is to allow the duty holder to
determine their own approach to determining gross disproportion.

It is normal that there is a level of uncertainty in the results of a QSRA. A pragmatic approach
can be taken to consider the amount of uncertainty and if the assessor can demonstrate that
there is a high level of uncertainty in the data and assumptions have been rounded up, or down,

© State of NSW through Transport for NSW 2019 Page 17 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

to err on the side of safety, then the assessor could reasonably claim that the test of gross
disproportion has been met.

If doubt or uncertainty exists regarding the level of safety built into the QSRA, then the assessor
can undertake sensitivity analysis by increasing the safety benefit by a factor of two or three and
testing the effect that this has on the benefit-cost ratio (BCR) and using this to support the
reasoned safety argument.

Sensitivity factors beyond 1:3 should not be used. No requirement exists to increase the factors
relative to the scale of the risk.

The legal obligation to comply with the SFAIRP test falls on duty holders because the duty
holders are best placed to make such judgements. Duty holders should be satisfied on a case-
by-case basis that the SFAIRP test is being met.

7.4. Net present value and discounting

When conducting CBA, a more accurate result will be obtained if future costs and benefits are
converted to present values, usually termed 'net present value' (NPV). The value of money
changes over time so costs or benefits that will be incurred or obtained later in the asset life will
be at the value of money at that time. The use of a discount rate within the analysis allows the
costs and benefits in future years to be properly accounted for.

Unlike with a typical capital investment, a CBA for a safety benefit is based on the value of
preventing a fatality figure. The VPF figure is a willingness to pay estimate and is expected to
rise broadly in line with inflation at a similar rate as the value of money. These variables in effect
cancel each other out and the application of NPV to safety CBA can generally be disregarded.

For high value investments when the BCR is close to parity (that is, around 1:1), then applying a
NPV sensitivity analysis to the CBA may be beneficial. Guidance on how to apply this can be
found in the NSW Government Guide to Cost-Benefit Analysis (TPP17-03) published by NSW
Treasury.

7.5. Sensitivity analysis

When undertaking a CBA, it is common to find a very large variance between the cost value and
the benefit value such that it is evident that a decision can be made without the need to
undertake any sensitivity analysis. Where this is the case, a statement is made to that effect in
the reasoned safety argument.

When the costs and benefits are of a similar order of magnitude, then it is prudent to undertake
a sensitivity analysis to inform the reasoned safety argument. The level of sensitivity analysis
will be proportionate to the complexity of the assessment and the variance between the two
values. The analysis should test each of the considerations mentioned in Section 7.1 to
Section 7.4 to determine their effect on the outcome.

© State of NSW through Transport for NSW 2019 Page 18 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

7.6. Societal concern

The term 'societal concern' refers to a political decision to implement a safety measure where
the cost of that measure is greater than the safety benefit that would be realised by
implementing it.

Decisions based on societal concern are made at the government or Transport cluster level and
do not need to be considered by practitioners applying QSRA and CBA in accordance with this
document.

8. Undertaking a QSRA
There are many tools and techniques that can be used when developing a QSRA and each
approach has benefits and drawbacks. When a QSRA is developed, the scope should be
defined and an understanding of the purpose and outcomes of a QSRA are required.

For example, a small change to an existing well-defined system may be done using some
simple documented calculations in a short period of time and at relatively low cost. At the other
extreme, to develop a multiple systems wide QSRA from scratch would be a very extensive and
costly exercise.

The decision to maintain a detailed QSRA through the operating life cycle will require significant
ongoing resource commitment.

8.1. Key considerations

The items in Section 8.1.1 to Section 8.1.6 should be considered before embarking on a QSRA.

8.1.1. Define the scope and the outcomes

The first step is to determine the scope of the QSRA and what the business is trying to achieve
from the assessment. Determine if the QSRA is to be used to look at a single issue, support a
safety argument for a change involving multiple issues or be used as an ongoing risk
management tool.

8.1.2. Identify if a benchmark model can be used

Determine whether an existing model can be modified or used as a benchmark or whether a
new model will have to be developed.

8.1.3. Commit the resources to undertake the QSRA

Specialist external resources as well as a significant commitment from Transport cluster
specialist resources will likely be needed to source the data and validate the model. The

© State of NSW through Transport for NSW 2019 Page 19 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

Transport cluster specialist resources will be from all stages of the asset life cycle including
planners, engineers, operators, maintainers and other specialists.

An estimate of the costs and business impacts should be approved at the executive level. The
cost of developing the QSRA may well be more than the cost of implementing the safety
measures so it may make more business sense just to implement them. Alternatively, the QSRA
may provide the opportunity to make significant savings.

8.1.4. Decide on the approach and any specialist software packages

Where a benchmark model is available it should be used. The decision is then taken whether to
use it in its original form or to tailor the model.

If a model is to be developed then a suitable technique should be selected. This would generally
be a variation on a FTA and ETA. Specialist input will likely be needed to support this.

For complex models, specialist software packages can be used as they reduce the potential for
human errors to occur in the calculations.

8.1.5. Document data sources and assumptions

A QSRA provides the argument and justification supporting the development of safety critical
decisions. As such, all data sources used should be documented. Where data is based on
expert judgment, the source of the information should be recorded. Any assumptions should be
documented and justified.

All records should be kept and maintained in compliance with NSW Government and TfNSW
records management requirements.

8.1.6. Reporting requirements

The outputs of a QSRA need to have a summary report detailing how and when the
assessment was made, key findings, and so on. A QSRA report needs to be framed and
contain appropriate level of details to support decision makers to make assured and justified
arguments for what controls should or should not be implemented.

A QSRA report captures and describes the inputs and assumptions of the model, methodology
and outputs. This includes written arguments supporting numerical decisions and outputs.

8.2. Seek assistance

A key factor when considering or developing a QSRA is to ensure that adequate specialist
support is available to ensure that it is done properly.

Detailed QSRA is a highly specialised field that can require considerable skill and experience to
implement effectively. Someone without that skill and experience may not conduct a good
QSRA, which can result in poor investment decisions or negative safety outcomes.

© State of NSW through Transport for NSW 2019 Page 20 of 21

 T MU MD 20003 GU
Quantified Safety Risk Assessment
Version 1.0
Issue date: 19 December 2019

Assistance should be sought from an expert who can guide you in the development of a QSRA
and the analysis of the outcomes. Initial support can be found within the Transport cluster by
engaging with Systems Safety Assurance resources.

© State of NSW through Transport for NSW 2019 Page 21 of 21