Monte Carlo Simulation

Fawaz hrahsheh
Dr. A. obeidat
Department
of physics
just
History
What is Monte Carlo (MC) method ?

The Monte Carlo method :is a numerical method
for statistical simulation which utilizes sequences
of random numbers to perform the simulation
What the meaning of MC simulation?
MC simulation is a versatile tool to analyse
and evaluate complex measurements
Constructing a model of a system.
Experimenting with the model to
draw inferences of the systems behavior
A simulation model
Decision and
uncontrollable
variables
Simulation
model
Measures of
performance or
behaviour of the
system
Inputs
outputs
A simulation model cont..
Model inputs capture the environment of the problem
The simulation model
Conceptual model: set of assumptions that define
the system
Computer code: the implementation of the
conceptual model
Outputs describe the aspects of system behaviour that
we are interested in

Random numbers

Uniform Random numbers or pseudo-random numbers
(PRN) are essentially independent random variables
uniformly Distributed over the unit interval (0,1).

The PRNs are good if they are uniformly distributed,
statistically independent and reproducible.

Linear congruential generator
Generating a random sequence of numbers {X1,X2,.,Xk} of
length M over the interval [0,M-1]
Xi=mod(AXi-1+C,M)
R=Xi/M
mod(b,M)=b-int(b/M)*M
Starting value X0 is called seed
M,A and C are nonnegative integers known
Modulus, multiplier and increment, respectively
M is must be prime number(2-1,2 -1,..)

program random_num
implicit none
integer,parameter::m=2**21,a=17,c=43
integer::i,n,s
real(8)::R,Xo
read*,n
!n is the number of random points
Xo=27.0d0
!------------------------
do i=1,n
s=A*Xo+c
Xo=mod(s,m)
R=Xo/m
write(*,*) R
end do
!------------------------
end program random_num
Fortran program
Classic Example

t
t
t t
4 4
2
2
= =


r
r
circle of area
square of area
Find the value of ?
Use the reject and accept method
Or hit and miss method
The area of square=(2r)
The area of circle = r
square of area
circle of area


= * 4 t
Cont....
dots of number total
circle inside dots of
square of area
circle of area
. . .
. . . #.
. .
. .
=
Hit and miss algorithm
Generate two sequences of N of PRN :: Ri,,Rj
Xi=-1+2R i
Yj=-1+2R j
Start from s=zero
If (X+Y<1) s=s+1
# of dots inside circle=s
total number of dots=N
N S / * 4 = t
PROGRAM Pi
implicit none
integer,parameter::mm=(2**17)-1,aa=(2**7)-1
REAL(8)::X, Y, Pi
Real(8),dimension(:),allocatable::R
INTEGER::I,N
Pi = 0.0d0
READ*, N
allocate(R(n))
DO I = 1,N
CALL RANDOM_NUMBER(R(n))
CALL RANDOM_NUMBER(R(n+1))
X =-1.0d0+2.0d0*R(n)
Y=-1.0d0+2.0d0*R(N+1)
IF (X*X+Y*Y<1.0d0) Pi=Pi+1.0d0
END DO
Pi=4*Pi/N
PRINT*,Pi
END program pi
Fortran program

Monte Carlo Integration
Hit and miss method
Sample mean method
importance sampled method
Hit and Miss method
Generate two sequence of N of PRN (Ri,R j)
i& j=1,2,.,N
}
=
b
a
dx x f I ) (
R b a e ,
0 f(x) Ymax ,for X (a,b)
Xi=a+Ri (b-a)
Yi=Ymax R j
start from s=0
if Yj<f(x) s=s+1
I=Ymax(b-a) S/N
program Hit_miss
implicit none
real(8),dimension(:),allocatable::R
real(8)::X,Y,m,a,b,integ
integer::i,N,s
read*,a !the lower value of the interval
read*,b !the upper value of the interval
M=f(b)
if (dabs(f(a))>dabs(f(b))) M=f(a)
read*,N !the number of random numbers
allocate(R(n))
s=0
do i=1,N
call random_number(R(n))
call random_number(R(n+1))
X=a+R(n)*(b-a)
Y=M*R(n+1)
if (y<f(x)) s=s+1
end do
INTEG=M*(b-a)*s/N
print*,integ
contains
real(8) function F(X)
real(8),intent(in)::X
F=2.0*X+1.0
end function F
end program Hit_miss
Cont.
If there is a part of the function under X-axis
) )( (
) (
.
. .
a b R M
dx x f
area Total
curve under area
N
s
b
a
+
}
= =
}
+ =
b
a
N
s
a b R M dx x f ) )( ( ) (
This must now be corrected for the fact that the line y = -R has
been used as the baseline for the integral instead of the line
y = 0. This is accomplished by subtracting the rectangular
area R(b-a).


}
+ =
b
a
a b R
N
s
a b R M dx x f ) ( ) )( ( ) (
then
Types of distribution
Gaussian or normal distribution Uniform distribution
Sample Mean method
Rewrite
}
=
b
a
dx x f I ) (
By
}
=
a
a
dx x x h I ) ( ) ( |
}
=
b
a
dx x 1 ) ( | 0 ) ( > x |
Where is p.d.f
|
If x1,x2,x3,..,xN are i,i,d uniformly distributed on [a,b],then
) ( ~
}
= f a b dx x f I
b
a
) ( ) (

= ) (
=
N
i
i
x f
N
f
1
) (
1
,
Theorem.
) ( ) ( ) ( x x f x h | =
From the theorem choose and
Cont
a b
x

=
1
) ( |
) ( ) ( ) ( x f a b x h =
Then an estimate of I is

=
=
N
i
i
x f
N
a b
I
1
) (
) (

) ( f
a
b
F(x)
You can calculate the value of error from the variance
)

var(I error =
2 2
) var( ) ( ) ( = f f f
) var(
) (
)

var(
2
f
N
a b
I

=
x
~
= =
N
i
i
N
i
i
x f
N
x f
N
1
2
1
2
)] (
1
[ ) (
1
Sample Mean MC algorithm
Generate sequence of N of PRN : Ri
Compute Xi=a+Ri (b-a)
compute f(Xi) , i=1,2,3,.,N

=
=
N
i
i
x f
N
a b
I
1
) (
) (

use
note:: if f(x) is not square integrable ,then the MC
Estimate will still converge to the true value, but
The error estimate becomes unreliable.
program Sampled_Mean
implicit none
real(8),dimension(:),allocatable::R
real(8)::a,b,sum,integ,X
integer::i,N
read*,a ! Lower value
read*,b ! upper value
read*,n ! number of random points
allocate(R(n))
sum=0.0d0
do i=1,n
call random_number(R(n))
call random_number(R(n+1))
X=a+R(n)*(b-a)
sum=sum+f(x)
int=((b-a)/N)*sum
end do
write(*,*) "integ=",integ
contains
real(8) function F(X)
real(8),intent(in)::X
F=2*X+1.0d0
end function F
end program Sampled_Mean
Generalization to multidimensional cases
The estimate of I is

=
= =
N
i
i
N
i
i
X f
N
V
x h
N
I
1 1
) ( ) (
1

Rewrite d-dimension integral

}
=
I
dx x f I ) (
by
}
=
I
dx x x h I ) ( ) ( |
Because the uniform distribution ,choose
0 ) ( > x |
}
=
I
1 ) ( dx x | and
V
a b
x
d
i
i i
1
) (
1
) (
1
=
[

=
=
|
[
= =
=
d
i
i i i i
x Vf x f a b x h
1
) ( ) ( )] ( [ ) (
Importance Sampled method
It is quite obvious that most of the integral comes from the
region of the peak. But if we generate points evenly in the
interval [a, b], most points wont be in the peak area ,and
their contribution to the total will be relatively small.

The idea behind importance sampling is to transform
f(x) into another, flatter function
which is then MC integrated of
course there has to be
a back-transformation
to give the original integral
we really want.


Important properties of continuous and discrete pdf s
Steps of Importance Sampled method
rewrite
}
=
b
a
dx x f I ) (
by
}
=
b
a
dx x g x h I ) ( ) (
where
) (
) (
) (
x g
x f
x h =
}
=
b
a
x dG x h I ) ( ) (
where
}
=
x
dx x g x G
0
) ( ) (
Now make a variable change
) (x G u =
and
) (x dG du =
) (
1
u G x

=
Then
}
=

) (
) (
1
1
)) ( (
)) ( (
b G
a G
du
u G g
u G f
I
Cont.
now the value of the integration equal to the average value

=
=

N
i
i
i
u G g
u G f
N
I
1
1
1
)) ( (
)) ( ( 1
note that, the function must be invertible
The error calculated by variance method
) var(I error =
2
2
2
) var( ) ( ) ( =
g
f
g
f
g
f
) (x G
] [
1
) var(
2
2
2
) ( ) ( =
g
f
g
f
N
I
) var(
1
) var(
g
f
N
I =
Example (Importance-Sampled method )
find
}
=

1
0
2
dx e I
x
In this region, the function decreases from 1 to 1/e. The
simple exponential function does the same, so lets
use that for g(x) . We first have to normalize g , so we
x
e

calculate

}

= +

1
0
1
1
1
e
e
e
dx e
x
Then our normalized weighting function is
1
) (

=

e
e e
x g
x
Cont....
} }

= =

x x
x
dx
e
e
dx x g x G
0 0
1
1
) ( ) (
1
) 1 (
) (

=

e
e e
x G
x
) (x G u =
let
)
1
1 ( log ) (
1
e
e
u u G x
e

= =

Then
)
1
1 ( log ) (
1
e
e
u u G
i e i

Cont
note:: from the last result we redistribute the PRN
according to the pdf (g(x)) , then the new values
(i.e., ) are uniform random numbers used
To find the value of




on the interval of integration ,where the average
value is the estimator value for the original
integral.


) (
1
i
u G

=
=

N
i
i
i
u G g
u G f
N
I
1
1
1
)) ( (
)) ( ( 1
program important_sample
implicit none
real(8)::sum,u,R,integ
integer::i,N
read*,N
sum=0.0
do i=1,N
call random_number(R)
u=G_inv(R)
sum=sum+f(u)/g(u)
end do
integ=sum/N
write(*,*) integ
contains
real(8) function F(x)
real(8),intent(in)::x
F=dexp(-x**2)
end function F
real(8) function g(x)
real(8),intent(in)::x
g=dexp(-x)
end function g
real(8) function G_inv(x)
real(8),intent(in)::x
G_inv=-dlog(1-x*1.718d0/2.718d0)
end function G_inv
end program important_sample
Why monte carlo ?
Method Conv.rate in one
dim
Conv.rate in d-dim
Basic MC
Trapezoidal rule
Simpsons rule
2
1

N
2
N
4
N
2
1

N
d
N
2
d
N
4
MC is from the best methods to find the partition function
numerically , then you can solve the stochastic processes.
Suppose we want to solve the following integral Using any
other numerical method ,(i.e. Trapezoidal or Simpson)
}
=
I i i
dx x f I ) (
Where i=1,2,3,,30
So we generate a grid in the domain of integration ,suppose
a grid with 10 nodes on each coordinate axis in 30 dimension cube
[ 0,1] be chosen .in this case, we have 10 abscissas.
Suppose a time of 10 second is necessary for calculating
One value of the function . Therefore , a time of
10 years will be necessary for evaluating the integral.

Thank you