2024/12/26

Introduction to Monte Carlo

Methods

Overview
 The aim is to introduce the fundamental concepts
behind Monte Carlo methods. The specific learning
objectives are:
a) To become familiar with the general scope of Monte
Carlo methods;
b) To understand the key components of the Monte Carlo
method;
c) To understand to role of “randomness” in Monte Carlo;
d) To understand the importance of careful choice of
random number generators;
e) To be able to write simple Monte Carlo programs.

1
 2024/12/26

What is Meant by Monte Carlo

Method?
 The term “Monte Carlo method” is used to embrace a
wide range of problem solving techniques which use
random numbers and the statistics of probability.

 Not surprisingly, the term was coined after the casino in

the principality of Monte Carlo. Every game in a casino is a
game of chance relying on random events such as ball
falling into a particular slot on a roulette wheel, being
dealt useful cards from a randomly shuffled deck, or the
dice falling the right way!

 In principle, any method that uses random numbers to

examine some problem is a Monte Carlo method.

First application of Monte

Carlo Method
 The needle experiment of Comte de Buffon,
1733 (french biologist, 1707-1788)
 What is the probability 𝑝, that a needle (length
𝐿), which randomly falls on a sheet, crosses
one of the lines (distance 𝐷)?
 N trials, n hits
𝑛 2𝐿 2𝐿𝑁
= →𝜋=
𝑁 𝜋𝐷 𝑛𝐷

2
 2024/12/26

First application of Monte

Carlo Method
 Let 𝑥 be the distance from the center of the needle to the
closest line, let 𝜃 be the acute angle between the needle
and the lines.
 The uniform probability density function of 𝑥 between 0 and
𝐷/2 is 2/𝐷; the uniform probability density function of 𝜃
between 0 and 𝜋/2 is 2/𝜋.
 The two random variables, 𝑥 and 𝜃, are independent, so the
joint probability density function is the product 4/𝐷𝜋.
 The needle crosses a line if 𝑥 ≤ 𝑠𝑖𝑛𝜃.
 The probability that the needle will cross a line
4 2𝐿
𝑝= 𝑑𝑥𝑑𝜃 =
𝐷𝜋 𝐷𝜋

First application of Monte

Carlo Method
 Convergence: about 𝑁 = 1000000 trials needed for
only 2 digits after the comma!! (poor convergence,
but foolproof)

 Typical: intuitively understandable even without any

mathematical reasoning. But: Error estimates?
Optimal choice of 𝐿, 𝐷?

3
 2024/12/26

A Classic Example – The

Calculation of 
 The value of  can be obtained by finding
the area of a circle of unit radius. The
diagram represents such a circle centered
at the origin (O) inside of a unit square.
 Trial “shots” (●) are generated in the
OABC square. At each trial, two
independent random numbers are
generated on a uniform distribution
between 0 and 1. These random numbers
represent the x and y coordinates of the
shot. If N is the number of shots fired and
H is the number of hits (within the circle):
4×𝐴 4𝐻
𝜋≈ =
𝐴 𝑁

The Calculation of  (contd)

A simple C function to calculate 

double calculate(int nTrials)

{
int hit = 0;
double x, y, distanceFromOrigin;
for (int i = 1; i <= nTrials; i++)
{
x = myRand(); /* random number between 0 and 1 */
y = myRand();
distanceFromOrigin = sqrt(x*x +y*y);
if (distanceFromOrigin <= 1.0) hit++;
}
return 4*(double) hit/nTrials;
}

4
 2024/12/26

Introduction
 The Monte Carlo method provides approximate solutions
to a variety of mathematical problems by performing
statistical sampling experiments on a computer.

 The method applies to problems with no probabilistic

content as well as to those with inherent probabilistic
structure.

Another Example – Radioactive

Decay
 The kinetics of radioactive decay are first-order,
which means that the decay constant () is
independent of the amount of material (𝑁). In
general:
∆𝑁(𝑡)
= −𝜆
𝑁(𝑡)∆𝑡

 The decay of radioactive material, although

constant with time is purely a random event. As
such it can be simulated using random numbers

5
 2024/12/26

Radioactive Decay (contd)

C function to simulate radioactive decay

void RadioactiveDecay(int maxTime, int maxAtom, double lambda)

{
int numLeft = maxAtom, time, counter;
counter = maxAtom;
for (time = 0; time <= maxTime; time++) /* time loop */
{
for (atom = 1; atom <= numLeft; atom++)
if (myRandom() < lambda) counter--;
numLeft = counter;
}
printf((%d\t%f\n”, time, (double) numLeft/maxAtom);
}

Monte Carlo Integration

 If we want to find the numerical value of the integral
𝑆= 𝑓 𝑥 𝑑𝑥

 We can divide the region [0,1] evenly into 𝑀 slices with 𝑥𝑜 =

0 and 𝑥𝑀 = 1 and the integral can be approximated as
𝑆= ∑ 𝑓 𝑥 + 𝑂(ℎ )
which is equivalent to sampling from the region [0,1] with an equal
weight 1.
 We can select 𝑥𝑛 from a uniform random number generator in
the region [0, 1] to accomplish the same goal.

6
 2024/12/26

Monte Carlo Integration

(contd)
 The general MC procedure for integrating a function 𝑓(𝑥) is to
randomly obtain values of 𝑥 within the upper and lower
bounds of the integrand, determine the value of the function,
accumulate a sum of these values and finally obtain an
average by dividing the sum by the number of trials.

 In general, to calculate an integral of two independent values

𝑓(𝑥, 𝑦) over an area in the (𝑥, 𝑦) −values:
(a) Choose random points in the area and determine the value
of the function at these points.
(b) Determine the average value of the function.
(c) Multiply the average value of the function by the area over
which the integration was performed.
This procedure can be easily extended to multiple integrals

Simple Monte Carlo

Integration (contd)
 MC integration is not usually a computationally viable
alternative to other methods of integration such as
Simpson’s rule. However, it can be of some value in
solving multiple integrals. Consider, the following multiple
integral
(𝑥 + 𝑦 + 𝑧 ) 𝑑𝑥𝑑𝑦𝑑𝑧

𝑥𝑦𝑧(𝑥 + 𝑦 + 𝑧 ) 𝑥 𝑦 𝑧
=
3 𝑥 𝑦 𝑧

 This integral has an analytical solution (RHS), but many

multiple integrals are not so easily evaluated in which
case MC integration is a useful alternative.

7
 2024/12/26

Simple Monte Carlo

Integration (contd)
The skeletal outline of simple C program to solve this integral is:
main()
{
int i;
double xL, xU, yU,yL, zU, zL, x, y, z, xRange, yRange, zRange,
function, diff, sum = 0.0, accur = 0.001, oldVal = 0.0;
printf(“Enter xL “);
scanf(“%f”, &xL);
/* add similar statements for xU, yL, yU, zL and zU here */
xRange = xU – xL;
/* add similar statements for yRange, and zRange here */
do
{

Simple Monte Carlo

Integration (contd)
i = i + 1;
x = myRand()*xRange + xL;
/* add similar statements fo y and z here */
function = x*x + y*y + z*z;
sum = sum + function;
integrand = (sum/i)*xRange*yRange*zRange;
diff = fabs(integrand – oldVal); /*absolute error*/
oldVal = integrand;
} while (diff > accur);
printf(“The integrand is %f”, integrand);
}

8
 2024/12/26

Simpson’s rule

𝑥−𝑚 𝑥−𝑏 (𝑥 − 𝑎)(𝑥 − 𝑏) (𝑥 − 𝑎)(𝑥 − 𝑚)

𝑃 𝑥 =𝑓 𝑎 +𝑓 𝑚 + 𝑓(𝑏)
𝑎−𝑚 𝑎−𝑏 (𝑚 − 𝑎)(𝑚 − 𝑏) (𝑏 − 𝑎)(𝑏 − 𝑚)
𝑏−𝑎 𝑎+𝑏
𝑓 𝑥 𝑑𝑥 ≈ 𝑓 𝑎 + 4𝑓 + 𝑓(𝑏)
6 2

A Simple Integral
 Consider the simple integral:
𝐼= 𝑓 𝑥 𝑑𝑥

 This can be evaluated in the same

way as the 𝜋 example. By randomly
tossing darts at a graph of the
function and tallying the ratio of
hits inside and outside the function.

9
 2024/12/26

A Simple Integral
(continued…)
 𝑅 =
{(𝑥, 𝑦): 𝑎 𝑥  𝑏, 0  𝑦  max 𝑓(𝑥)}
 Randomly tossing 100 or so
darts we could approximate the
integral…
 𝐼 = [fraction under 𝑓(𝑥)] *
(𝑎𝑟𝑒𝑎 𝑜𝑓 𝑅)

 This assumes that the dart

player is throwing the darts
randomly, but not so random as
to miss the square altogether.

A Simple Integral
(continued…)
 Generally, the more iterations of the game the better the
approximation will be. 1000 or more darts should yield a
more accurate approximation of the integral than 100 or
fewer.

 The results can quickly become skewed and completely

irrelevant if the games random numbers are not
sufficiently random.

 Say for each iteration of the game the “random” trial

number in the interval was exactly the same. This is
entirely non-random. Depending on whether or not the
trial number was inside or outside of the curve the
approximation of integral 𝐼 would be either 0 or . This is
the worst approximation possible

10
 2024/12/26

Numerical Recipes uses

this volume as an example
of Monte Carlo integration.

Key Components of a Monte

Carlo Algorithm
 Having examined some simple Monte Carlo
applications, it is timely to identify the key
components that typically compose a Monte Carlo
algorithm.
 They are:
• Random number generator
• Sampling rule
• Probability Distribution Functions (PDF)
• Error estimation

11