68r-11 Escalation Estimating Using Indices and Monte Carlo Simulation
68r-11 Escalation Estimating Using Indices and Monte Carlo Simulation
11
ESCALATI
ONESTI
MATI
NGUSING
INDI
CESANDMONTECARLO
SIMULATI
ON
AACE International Recommended Practice No. 68R-11
Any terms found in AACE Recommended Practice 10S-90, Cost Engineering Terminology, supersede terms defined in
other AACE work products, including but not limited to, other recommended practices, the Total Cost Management
Framework, and Skills & Knowledge of Cost Engineering.
Contributors:
Disclaimer: The content provided by the contributors to this recommended practice is their own and does not necessarily
reflect that of their employers, unless otherwise stated.
This document is copyrighted by AACE International and may not be reproduced without permission. Organizations may obtain permission
to reproduce a limited number of copies by entering into a license agreement. For information please contact [email protected]
AACE® International Recommended Practice No. 68R-11
ESCALATION ESTIMATING USING INDICES AND
MONTE CARLO SIMULATION
TCM Framework: 7.6 – Risk Management
May 2, 2012
TABLE OF CONTENTS
INTRODUCTION
Scope
This recommended practice (RP) of AACE International defines basic principles and methodological building blocks
for estimating escalation using forecasted price or cost indices while also addressing uncertainty using Monte Carlo
simulation. The methods in this RP are an extension of the principles and methods in RP 58R-10, Escalation
Estimating Principles and Methods Using Indices, from a probabilistic and scenario/sensitivity viewpoint. This RP
will guide practitioners in developing or selecting appropriate methods for their definitions and situation. While
this RP discusses the relationships of escalation estimating to other risk cost and schedule accounts (namely
contingency), dealing with those cost types is not the focus of this RP. This RP assumes that practitioners are
already familiar with Monte Carlo simulation as typically applied in spreadsheet applications.
Escalation estimating is an element of both the cost estimating and risk management processes. Like other risks,
escalation is amenable to mitigation, control, etc. However, this RP is focused on escalation quantification, not on
treatment (i.e., how it is addressed through contracting, bidding, schedule acceleration, hedging, etc.) or control.
In terms of cost estimating, this RP covers practices applicable to all classes of estimates[2]. Escalation uncertainty is
partly driven by schedule risk; therefore this RP also references AACE’s RPs on integrated cost and schedule risk
analysis and contingency estimating[3,4]. The examples in this RP emphasize capital cost estimating and scheduling,
but the principles apply equally to operating, maintenance and other cost and time evaluations. While a model
such as that covered in this RP could be used for schedule optimization in consideration of escalation, optimization
is not covered here.
May 2, 2012
As with RP 58R-10, Escalation Estimating Principles and Methods Using Indices, this RP recommends segregating
escalation versus exchange rate impacts and their estimation for projects with resources priced in currencies other
than the base currency.
BACKGROUND
Background on the topic of escalation is covered in RP 58R-10, Escalation Estimating Principles and Methods Using
Indices, which should be studied prior to this one. That RP highlights that escalation is always uncertain and should
therefore be considered in the risk management process. However, escalation is usually quantified using different
methods than used for other risks (i.e., contingency). Being driven by conditions in the economy, which are
external to the project, it is less amenable to quantification techniques that use project system empirical data (e.g.,
parametric contingency estimating), or project team input (.e.g., methods dependent on brainstorming or Delphi
techniques). Given its economic nature, it is recommended that those with the most economics expertise (e.g.,
economists) be included in the process of developing escalation estimating methods and estimates, including
evaluations of uncertainty.
As with contingency estimates, it is the estimator’s responsibility to quantify the uncertainty of the escalation
estimate, usually by providing a distribution or range (e.g., 80% confidence or P10/P90 range), so that
management can make effective investment and project decisions and fund the escalation account as they see fit
in accordance with their decision and risk management policies.
The methods elaborated in this RP may not be applicable for small projects with short durations in fairly stable
economies. However, this RP addresses large projects with multi-year durations that typically dominate capital
budgets. Often, the economy is not stable over these durations, and escalation may be the largest single cost
account in these projects. In this situation, robust methodologies are needed.
Probabilistic methods in industry escalation estimating are not widely used or reported. A few references are
included in this RP[6,8,9]. However, the increasing number of extended duration mega-projects and increasing
volatility in the global economy has increased the need for improved practices and we hope this RP will help
trigger increased development and reporting in this area of practice.
General Principles
As discussed in RP 58R-10, Escalation Estimating Principles and Methods Using Indices, there is no preferred way to
quantify risks, including escalation. Each method has advantages and disadvantages and its advocates. However,
there is general agreement that any recommended practice or method for estimating or forecasting the cost of
uncertainty should address the principles identified in RP 40R-08, Contingency Estimating: General Principles. The
methods discussed in this RP are consistent with those principles.
In recognition of the differences between contingency and escalation estimating, the principles in RP 40R-08 are
clarified or expanded as follows:
• Differentiate between escalation, currency and contingency
• Leverage economist’s knowledge (based on macroeconomics)
• Use indices appropriate to each account including addressing differential price trends between accounts
• Use indices that address levels of detail for various estimate classes
May 2, 2012
This RP focuses on the use of probabilistic methods; specifically use of Monte Carlo simulation applied to an
escalation indices model. Other approaches are available to assess escalation uncertainty such as scenario and
sensitivity analysis, but these practices are not within the scope of this RP.
RP 58R-10, Escalation Estimating Principles and Methods Using Indices, includes extensive discussion of the
principles of the base method or model for estimating escalation using indices in a deterministic way. It starts with
that base model and adds Monte Carlo functionality to address uncertainty in each of the model’s input variables.
To summarize this RP’s approach, the uncertain input variables in the escalation estimate model are substituted
with probability distribution functions (PDFs) with appropriate consideration of correlation. The resulting Monte
Carlo simulation version of the model is then run to obtain an escalation cost outcome distribution from which
management can run business case sensitivities and select a value to budget based on their desired confidence of
underrun (usually 50%). Such a Monte Carlo model can be complex for several reasons. One example is the timing
of spending needs to be quantified in a model (i.e., a logistic function) that is practical and amenable to Monte
Carlo manipulation and reflects the schedule uncertainty resulting from contingency evaluations (e.g., replace a
variable in the logistic function with a distribution). Linking the method to a cost-loaded schedule risk tool to deal
with probabilistic timing is not simple because escalation cash flow must reflect the spending by the supplier. Also,
escalation cost items may not be aligned with the schedule activities. Another complication is that a price index in
one period may have some dependence on the index in the prior period.
These complications invite creativity in model application. This RP presents approaches that have proven
reasonably effective; however, they should be considered just a reliable starting point for the model developer’s
consideration in developing approaches to suit a specific situation.
Model Variables
The escalation estimating model described in RP 58R-10, Escalation Estimating Principles and Methods Using
Indices, includes the following major uncertain input variables:
• Cost estimate including cost contingency
• Schedule (start/finish timing of spending including schedule contingency)
• Cash flow pattern (spending pattern within the schedule timing)
• Indices (cost or price index forecasts)
The following sections discuss Monte Carlo application in consideration of each of these variables. In general, the
schedule start/finish and the indices are the two main variables driving escalation uncertainty. [6]
May 2, 2012
For escalation, PDFs must be assigned to the start/finish in respect to each cost account, not schedule activities. If
the schedule contingency is estimated using a CPM-model based method[3], then milestones could be included in
that CPM model, and PDFs captured for them, that approximate the start/end of activities that correspond to
start/finish of escalation model cost accounts. For example, if the escalation model cost account was piping
material, milestones for first piping order and last piping delivery could be established in the CPM model and PDFs
recorded for them for use in the escalation model.
If a non-CPM based schedule contingency method such as expected value[4] was used that results in a single project
completion variance PDF, then a reasonable approach is to keep the escalation account start dates fixed and
replace the finish dates with PDFs reflecting the project completion variance PDF. For example, if the overall
project PDF had a P10/P90 range of -2 months to +10 months, create a custom PDF for the escalation piping
material account finish date such that the P10 date is the original date less 2 months and so on for each
probability-level. The approximation of locking all starts and having variable finish is balanced approximation; the
important thing is to include schedule uncertainty in the model.
May 2, 2012
Variable: Indices
RP 58R-10, Escalation Estimating Principles and Methods Using Indices, discusses price index forecasting and
structuring these indices in alignment with the by-period approach and cost accounts selected. Price indices are
highly uncertain, driven by micro- and macro-economic conditions and markets, and they are best understood by
economists and supply-chain experts. The challenge is to derive PDFs to use in place of fixed index values (although
the fixed value would usually become the most likely value in a PDF). RP 66R-11, Selecting Probability Distribution
Functions for Use in Cost and Schedule Risk Simulation Models, discusses the selection of PDFs to consider[7].
Dependency issues will be discussed later.
Estimating the range of the index PDFs should consider the principles of leveraging expert input, and leveraging
historical knowledge. One recommended approach in accordance with these principles would be to examine the
index historical record and use the high/low range of by-period increase/decrease making sure the historical
record was long enough to include a broad range of economic conditions (note: Monte Carlo can be used to
develop PDFs of historical index data)[8]. For example, if the 30 year price index for a certain commodity including
boom and bust times had a maximum annual increase of +21% and maximum decrease of -17%, and the
economists’ forecast change for the coming year was a decrease of -9%, you could use a PDF with the forecast as
the most likely value and the historical range as the bounds. The bounds could then be tested against and adjusted
by the expert opinion using a reasonable heuristic. Experts tend not to incorporate drastic changes in the short
run; however, their view (and consideration of the PDF) should be tempered by the fact that history, as discussed
in RP 58R-10, is full of “black swan” (long tail) risk events that occur with dismaying frequency; history can repeat
and in reasonably short time durations.
Given that 58R-10, Escalation Estimating Principles and Methods Using Indices, is predicated on base index forecasts
provided by economists, another approach is to have them provide the bounds of uncertainty in their forecasts. Most
economists study economic scenarios (usually reflecting best and worst case assumptions) and can provide the ranges for their
indices based on these scenarios.
Treatment of Dependencies/Correlation
Monte Carlo simulation methods require that the dependency or correlation between the model variables be
established. There is some correlation between each of the escalation estimating variables; however, the following
guidelines focus on the most significant correlations. Model developers will have to balance their risk profile with
the practicalities of their model.
The escalation model will however have a separate start/finish for each cost account and each is a variable. As a
general guideline, the start/finish of each account is likely to be highly correlated.
May 2, 2012
The escalation model will however have a separate pattern for each cost account and each is a variable. As a
general guideline, the spending pattern of some of the accounts may be correlated; however, there is no general
rule. For example, if engineering spending is late, that may imply late design releases to vendors causing their
spending to also be late.
Variable: Indices
As a general guideline, this variable can be treated as independent of the others.
The escalation model will have separate indices for each cost account and for each period. As a general guideline,
the price trends between accounts will be significantly correlated (i.e., a hot economy tends to affect all cost
accounts to some extent).
The price index from year to year will also be correlated. For example, if an economist predicted a price increase
for a certain commodity of +18% next year, and the historical max is +21%, it is unlikely that the second year
change will also be near the max increase; it will more likely decrease. The challenge with this price wave behavior
is that a big change in one year will affect the succeeding years[6]. The correlation diminishes with each additional
year. Given that econometric models do not typically forecast drastic price swings, and considering practicality (we
are already correlating indices between accounts) and bottom line effects, the recommended approach would be
to ignore the year to year correlation of indices for a given account. However, model developers may consider
approaches (e.g. use three-year rolling averages) to address this after examining price wave behavior in the
historical record.
As discussed, the recommended practice for PDF selection for cost contingency and schedule start/finish in the
escalation model is to use the PDF outputs from cost and schedule contingency models. This integration offers an
additional benefit from the escalation method; it can produce the outcome distribution of escalation costs alone,
but also the outcome distribution of all cost including contingency and factoring in schedule risk. Escalation and
contingency costs can be established separately for control purposes, but for business case scenario and sensitivity
analysis, the total cost distribution can be used. A total cost distribution is important because when non-integrated
methods are used, only the mean cost outcomes of escalation and cost contingency are additive (using the P90 of
escalation and P90 of contingency would overstate the overall cost risk at P90).
As discussed in RP 58R-10, Escalation Estimating Principles and Methods Using Indices, the principles of escalation
estimating outlined above are fairly straight-forward. However, putting them into practice can be a significant
effort. Certainly, no one wants to or has the time to build unique escalation risk models for each project; the only
efficient method is to build a tool to automate the process. The steps to build probabilistic functionality in an
escalation estimating tool include the following:
• Review industry literature on escalation and contingency estimating
• Obtain buy-in on the approach from the stakeholders (based on principles)
• Determine which escalation estimating model variables to assign PDFs (main emphasis on schedule
start/finish and indices)
• Select appropriate PDFs (for indices, obtain input from economists and market experts)
• Establish dependencies/correlations between variables
May 2, 2012
• Develop output products to meet stakeholder needs (i.e., escalation for control and overall cost
distribution for business decisions)
• Test and validate the model against historical experience and review with stakeholders
• Document the model and methodology used in the basis of estimate documentation.
The time and cost for this effort is highly variable depending upon the skills and knowledge of the developers
and users regarding Monte Carlo methods and tools, the status and complexity of the base (non-probabilistic)
escalation tool in place, the degree of buy-in and training at the end, and so on. Depending on these variables,
the entire effort could take from several weeks to several months. The first step is obtaining buy-in from
management based on principles; they must first accept that significant uncertain costs must be treated
probabilistically. Given the scale of escalation risk, implementing a method such as this is generally justified for
companies with multiple large projects with multi-year durations. Once that is accepted, the resources
required and the following strengths and weaknesses may be considered:
Strengths:
• Consistent with contingency and escalation risk analysis and estimating principles
• Provides information to support making decisions in consideration of risk
• Leverages common base escalation methods (i.e., indices by period)
• Uses practical Monte Carlo simulation methods and tools
Weaknesses:
• Economic forecasting is highly uncertain compounded by uncertainties in other planning elements
(however, it is a strength to quantify this uncertainty for decision makers).
• Given the number of uncertain variables in escalation estimating and the element of time, a Monte Carlo
simulation version of an escalation tool becomes complex and takes significant time and resources to
develop.
REFERENCES
1. AACE International, Recommended Practice 58R-10, Escalation Estimating Principles and Methods Using
Indices, AACE International, Morgantown, WV (latest revision).
2. AACE International, Recommended Practice 17R-97, Cost Estimate Classification System, AACE
International, Morgantown, WV, (latest revision).
3. AACE International, Recommended Practice 57R-09, Integrated Cost and Schedule Risk Analysis Using Risk
Drivers and Monte Carlo Simulation of a CPM Model, AACE International, Morgantown, WV (latest
revision)
4. AACE International, Recommended Practice 65R-11, Integrated Cost and Schedule Risk Analysis and
Contingency Determination Using Expected Value, AACE International, Morgantown, WV (latest revision)
5. AACE International, Recommended Practice 40R-08, Contingency Estimating: General Principles, AACE
International, Morgantown, WV, (latest revision). [This RP also applies to estimating other risk funds such
as escalation.
6. Touran, Ali, and Ramon Lopez, “Modeling Cost Escalation in Large Infrastructure Projects”, Journal of
Construction Engineering and Management, August 2006.
7. AACE International, Recommended Practice 66R-11, Selecting Probability Distribution Functions for Use in
Cost and Schedule Risk Simulation Models, AACE International, Morgantown, WV (latest revision)
8. Butts, Glenn, “Escalation: How Much is Enough?”, 2007 AACE International Transactions (EST.08), AACE
International, Morgantown, WV, 2007.
9. Hollmann, John K, and Larry R. Dysert, “Escalation Estimating: Lessons Learned in Addressing Market
Demand”, 2008 AACE International Transactions (EST.08), AACE International, Morgantown, WV, 2008
May 2, 2012
CONTRIBUTORS
Disclaimer: The content provided by the contributors to this recommended practice is their own and does not
necessarily reflect that of their employers, unless otherwise stated.