Building

Descrip/ve Models:
Simula/on

MIT Center for

Transportation & Logistics ctl.mit.edu
1
Ways of Studying a System

System

Experiment with Experiment with

the actual system a model of the system

Physical Model Mathema/cal Model

Analy/cal Methods Simula/on

2
Selec/on of Different Methods
•  Op/miza/on (LP, IP, MILP, NLP)
n  Finds “best” solu/on or recommenda/on (Prescrip/ve)
n  Best suited for determinis/c problems with no uncertainty
•  Regression
n  Measures impact of independent variables on a dependent variable
(Predic/ve)
n  Best suited for establishing rela/onships between variables
•  Simula/on
n  Captures outcomes of different policies within an uncertain or
stochas/c environment (Descrip/ve)
n  Best suited for situa/ons where the exact rela/onship between
decisions and outcome are not known

3
Pros and Cons of Using Simula/on
•  Use simula/on models to . . .
n  capture complex system interac/ons.
n  model system uncertainty.
n  systema/cally experiment with new system designs or decision rules.
n  train or experiment without disrup/ng real opera/ons.
n  generate data to analyze, describe, and visualize interac/ons,
outcomes, and sensi/vi/es.

•  Do not use simula/on models if . . .

n  the problem can be solved analy/cally (simula/on is expensive!)
n  direct experimenta/on with the actual physical system is feasible and
cost effec/ve.
n  cost and/or /me requirements are prohibi/vely expensive.

4
Simula/on Models in Supply Chains
•  Manufacturing processes
n  How well do sequen/al processes interact with each
other?
•  Flow of goods in a warehouse
n  How do different picking rules perform?
•  Vehicle rou/ng from a distribu/on center
n  Which rou/ng heuris/cs work best?
•  Service opera/ons in a call center
n  How does the system operate with more staff?
•  Inventory replenishment policies
n  How well do different re-order points and order quan//es
perform?
5
Simula/on Example:
Suzie’s Sushi Shack

6
Suzie’s Sushi Shack
•  Suzie sells sushi in the Seychelles from her shack at the seashore . Each
day she buys certain fish from the local fishermen and creates a number
of her special rolls. If she has extra unsold rolls at the end of the day, she
has to throw them out (no one eats day old sushi). If she sells out what
she has during a day, she is done and cannot make any more rolls that
day. Each roll costs 25 Seychelles Rupees (SCR) to make and she sells
them for 80 SCR each.
•  The demand for her sushi rolls is highly variable. She has collected two
months of demand, to include the number of /mes customers requested
rolls when she was already sold out.
•  Her rule of thumb has been to make 1.5 /mes the average number of rolls
typically demanded each day. She is repor/ng an average profit of about
800 SCR a day.
•  Suzie wants to know if this policy makes sense, and if not, how many rolls
she should make each day to maximize her profit.

7
Simple Steps in a Simula/on Study
1.  Formulate & plan the study
2.  Collect data & define a model
3.  Construct model & validate
Go to Step 2 as many /mes as needed!
4.  Make experimental runs
5.  Analyze output

Step 1. Study Plan

•  Develop a simula/on model where daily demand varies.
•  A “produc/on policy” will be applied.
•  Based on the demand and the policy, we will calculate profitability.
•  We will assess profitability and other performance metrics of different policies

8
Collect Data and Define Model

9
Step 2. Collect Data and Define Model
a)  Determine random variables
X = number of sushi rolls demanded each day

b)  Determine rela/onships between various variables

s = sales price of each sushi roll (SCR/roll)
c = cost of each sushi roll (SCR/roll)
Q = quan/ty of sushi rolls made each day
R = daily revenue = s*MIN(Q,X)
C = daily cost = c*Q
c)  Determine performance metrics
Daily Profit = R – C
Daily Unfilled Demand = MAX(X-Q, 0)
Daily Extra Rolls = MAX(Q-X, 0)
d)  Collect data & es/mate probability distribu/ons

10
Determining Random Variable Distribu/on
Uniform Distribu/on
•  If sample data is not available . . .
n  Determine the “range” of the variable
w  Talk to stakeholders or experts worst best
case case
w  Get worst/best/most likely values
Triangular Distribu/on
n  Use known distribu/ons
w  Number of customers arriving ~ Poisson
w  No informa/on ~ Uniform or Triangular
worst most best
•  If sample data is available . . . case likely case

n  Examine histograms
n  Calculate summary sample sta/s/cs
n  Conduct Chi-Square tests to fit sample to “tradi/onal” distribu/ons,
(i.e., Normal, Poisson, Exponen/al etc. )
n  Use a “custom” empirical distribu/on (be careful of over-fiong!!)
w  Discrete Empirical – use % of observa/on as probabili/es
w  Con/nuous - Use histogram to compute probabili/es of each range and
then “uniform” within the range
11
Tes/ng the Fit for Sample Data

12
Chi-Square Test
•  Recall the general approach
n  Create buckets or categories, c, that make sense
n  Count the expected and observed (actual) values in each category
n  Calculate the Chi-square sta/s/c and find the p-value
n  If the p-value is greater than level of significance, reject the null hypothesis
that the data follows the proposed (expected ) distribu/on
⎛ (Observed − Expected )2 ⎞
χ2 = ∑⎜ ⎟ df = c − 1
⎜ Expected ⎟
⎝ ⎠

9
8
Min 7
7 25th Pct 17
Frequency

6
5
Mode 18
4 Median 20.00
3
2
Mean 19.77
1 75th Pct 24
0
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 Max 31
Number of Rolls Demanded per Day StDev.S 4.91
IQ Range 7
13
 Looking at the data, we could set c=1, but let’s use
Chi-Square Test c=2 in order to illustrate the process.

=COUNTIF(DemandArray,"<=”&ValueCell)
Day Demand (X)
1 26 X≤ CumObs Obs ExpCumProb Expected
2 24 6 0 0 0.00 0.15
3 25 8 1 1 0.01 0.34
4 31 10 2 1 0.02 0.90
5 18 12 5 3 0.06 2.01 H0: X~N(20,5)
6 26 14 7 2 0.12 3.79
7 17 16 14 7 0.22 6.08 H1: X not ~N(20,5)
8 24 18 26 12 0.36 8.28
9 15 20 35 9 0.52 9.57
22 42 7 0.68 9.39
alpha = 0.01
10 16
11 12 24 50 8 0.81 7.82
12 26 26 57 7 0.90 5.53
13 22 28 58 1 0.95 3.32
14 22 30 58 0 0.98 1.69 p = 0.83
15 18 32 60 2 0.99 0.73
16 15 34 60 0 1.00 0.27 Do NOT Reject H0
17 15 36 60 0 1.00 0.08
38 60 0 1.00 0.02
18 21
40 60 0 1.00 0.00
Assume X~N(20,5)
19 22
20 26
. .
=CumObsi – CumObsi-1
. . 0.83
. .
=NORM.DIST(ValueCell,Mean,StdDev,1)
60 16
=N*(ExpCumProbi – ExpCumProbi-1)

=CHISQ.TEST(ObsArray, ExpectedArray) 14
Define and Validate the Model

15
Genera/ng Random Variables
•  Underlying principle
n  Generate a random (or usually pseudo-random) number
n  Transform that RV to fit the desired distribu/on
•  Manual Techniques:
n  Rolling a die ~U(1, 2, 3, 4, 5, 6)
n  Turning a rouleue wheel ~U(38 op/ons) ala Monte Carlo
n  Flipping open a book and picking last digit
n  Random number tables
•  In Spreadsheets – there are other func/ons in add-ins:
n  RAND() = Returns a con/nuous variable between 0 and 1
n  Uniform Distribu/on ~U(a,b)
w  ~U(a, b) = a + (b – a) * RAND()
w  ~U(500, 900) =500 + 400 * RAND()
n  Normal Distribu/on ~N(μ, σ)
w  ~N(μ, σ) =NORMINV( RAND(), μ, σ)
w  ~N(650, 75) =NORMINV( RAND(), 650, 75)
n  Rand() refreshes upon hiong enter, F9, or Ctrl-R

16
The Sushi Simula/on Model Data:
Sales Price
Cost
80 SCR
25 SCR
=INT(NORM.INV(RAND(),μ,σ)) Mean μ 20 #/day
StdDev σ 5 #/day
=MAX(Demand-ProducConQuanCty,0) Produc/on Quan/ty 30 #/day

=MAX(ProducConQuanCty - Demand,0)
=MIN(Demand, ProducConQuanCty)*Price
=ProducConQuanCty*Cost
Day Demand #Short #Extra Revenue Cost Profit
=Revenue - Cost
1 17 0 13 1360 750 610
2 18 0 12 1440 750 690
3 26 0 4 2080 750 1330
4 33 3 0 2400 750 1650 Notes:
5 28 0 2 2240 750 1490 -  Ran 300 simulated days
6 25 0 5 2000 750 1250
7 18 0 12 1440 750 690
-  Refreshing the RAND()
8 18 0 12 1440 750 690 func/on changes these
9 15 0 15 1200 750 450 numbers!
. . . . . . . . . . . . . . . . . . . . .
Demand #Short #Extra Revenue Cost Profit
Min 1 0 0 80 750 -670 Min
25th Pct 17 0 7 1360 750 610 25th Pct Valida/on:
Mode 21 0 9 1680 750 930 Mode
Median 20 0 10 1600 750 850 Median
-  Does the “as-is”
Mean 19.45 0.02 10.57 1554 750 804 Mean simula/on match
75th Pct 23 0 13 1840 750 1090 75th Pct reality?
Max 32 2 29 2400 750 1650 Max
StDev.S 5.11 0.17 5.07 406 0 406 StDev.S
17
IQ Range 6 0 6 480 0 480 IQ Range
Valida/on of Model
8
7
6
5
•  Valida/on Runs 4

n  Made 40 runs of 300 days each 3

2
n  Sample mean ranged from 763 to 850 1

w  Mean of Sample Means ~N(806, 20.9) 0

790 795 800 805 810 815 820 825 830 835 840 845 850
w  99% Confidence Interval [797, 816]
n  Hypothesis Test:
w  H0: X| = 800
w  H1: X| ≠ 800
n  This is a two tailed test, ~N since 40 obs.
n  For alpha = 0.01
w  Cri/cal Value = NORM.S.INV(0.01/2) = +/- 2.57
n  Test sta/s/c (using the sample standard devia/on)
w  z =( X| - μX| )/σX| = =( X| - μX| )/(σx/√n) = (806 – 800)/(20.9/√40) = 1.82
n  So, we reject H0 if z < -2.57 or if z > +2.57
n  Since z = 1.82, we cannot reject the Null Hypothesis and assume the
model is valid in that it replicates the current situa/on.
18
Make Experimental Runs

19
Making Experimental Runs
•  Need to make mul/ple runs for each policy
•  Use Hypothesis Tests to evaluate results
•  Remember, simula/on will never recommend a solu/on –
only evaluate the quality of proposed solu/ons
ProducKon E[Profit] StdDev
QuanKty [Profit]
30 806 20.9
25 906 10.13
20 921 8.51
15 784 3.01

•  These few runs suggest that Suzie should consider making

fewer Sushi rolls each day. Too much is being thrown out.
•  In later courses we will show how to set the op/mal Q!
20
Final Notes on Simula/on

21
Simula/on Modeling Tips for Spreadsheets
•  Start simple. Add complexity as needed
n  Helps model valida/on
n  Keep track of model versions
•  Keep input parameters in separate sheets
•  Keep performance metrics in separate sheets
n  Macros will be useful to compile mul/ple run results
•  Random numbers change every /me a cell changes.
n  Consider crea/ng a separate table with all random numbers in a
separate tab.
n  Consider a different random stream for each random variable
n  consider “variance-reduc/on” when comparing mul/ple scenarios
•  Remember output results are random variables
n  Need mul/ple runs for each scenario
n  If dynamic system, may need a warm-up period and run it for a long
/me. Ini/al condi/ons mauer on dynamic systems.
n  Think carefully about number of “runs” needed for each “scenario”.
n  When comparing scenarios you need to use sta/s/cal hypothesis
tes/ng to give final conclusions
22
Types of Simula/on Models
•  Classes of Models
n  System Dynamics
w  Models the non-linear behavior of complex systems
n  Monte Carlo Simula/on
w  Uses random sampling to mimic specified distribu/on
n  Discrete Time Simula/on
w  Models the opera/on of a system as a discrete sequence of events in /me. Each
event occurs at a par/cular instant in /me and marks a change of state in the
system.
n  Agent Based Simula/ons
w  Models behavior and interac/on of individual agents within a system using defined
rules

•  Suzie’s Sushi Simula/on Model

n  Sta/c Monte Carlo Simula/on with Stochas/c Demand
n  Each day did not depend on the “state of the system” from the
previous day

23
Common pi•alls in simula/on studies
•  Not well defined objec/ves
•  Lack of communica/on with stakeholders
•  Inappropriate level of detail
•  Trea/ng study as a primarily mathema/cal or programming
exercise
•  Failure to take into account all sources of randomness in the
model
•  Making a single “run” to obtain conclusions
•  Using output sta/s/cs as “true” answer
•  Using wrong measures to evaluate the system

24
Ques/ons, Comments, Sugges/ons?
Use the Discussion Forum!

“Dexter – simula/ng his way to dinner /me”

Yankee Golden Retriever Rescued Dogs
(www.ygrr.org)
MIT Center for [email protected]
Transportation & Logistics ctl.mit.edu