+

Numerical Integration
Techniques
A Brief Introduction

+
Objectives
Start

Writing your OWN Programs

Make

Numerical Integration accurate

Make

Numerical Integration fast

CUDA acceleration

The same Objective

Lord, make me accurate and fast.
- Mel Gibson, Patriot

+
Schedule

+
Preliminaries
Basic

Calculus

Derivatives
Taylor series expansion

w dw

Basic

Programming Skills

Octave

+
Numerical Differentiation

Definition of Differentiation

df
f (x h) f (x)
f '(x) lim
h0
dx
h

Problem: We do not have an infinitesimal h

Solution: Use a small h as an approximation

+
Numerical Differentiation
Forward Difference

Approximation Formula

df
f (x h) f (x)
f '(x)approx
dx
h

Is it accurate?

+
Numerical Differentiation
Error Analysis

Taylor Series expansion uses an infinite sum of terms to

represent a function at some point accurately.

1 2
1 3
f(x h) f (x) hf '(x) h f ''(x) h f '''(x)
2
6

which implies

f (x h) f (x)
1
1 2
f '(x)approx
f '(x) hf ''(x) h f '''(x)
h
2
6

Truncation Error
1
1 2
et hf ''(x) h f '''(x) O(h)
2
6

+
Numerical Differentiation
Error Analysis

Roundoff Error

A computer can not store a real number in its

memory accurately.

Every number is stored with an inaccuracy

proportional to itself.

Denoted

Total Error

er

e et er

Usually we consider Truncation Error more.

10

+
Numerical Differentiation
Backward Difference

Definition

f (x) f (x h)
f '(x)approx
h

Truncation Error

1
1
et hf ''(x) h2 f '''(x) O(h)
2
6

No Improvement!

11

+
Numerical Differentiation
Central Difference

Definition

f (x h) f (x h)
f '(x)approx
2h

Truncation Error

1 2
1 4 (5)
et h f '''(x)
h f (x) O(h2 )
6
120

More accurate than Forward Difference and Backward

Difference

12

+
Numerical Differentiation

13

Example

Compute the derivative of function

f (x) ex

At point x=1.15

f '(1.15) f (1.15) 3.1582

14

Use Octave to compare these

+
methods
Blue Error of Forward Difference
Green Error of Backward Difference
Red Error of Central Difference

+
Numerical Differentiation
Multi-dimensional & High-Order
Multi-dimensional

Apply Central Difference for different parameters

d2 f d df f (x 2h) f (x 2h) 2 f(x)

2
dx dx dx
4h2
High-Order

Apply Central Difference several times for the same

parameter
2
f
f f (x h, y h) f(x h, y h) f (x h, y h) f (x h, y h)

xy y x
4h2

15

+
Euler Method

16

IVP
The

Initial Value Problem

Differential Equations
Initial Conditions

y'(t) f (t, y(t))

y(t0 ) y0
Problem

What is the value of y at time t?

+
Euler Method

17

Explicit Euler Method

Consider

Forward Difference

y(t t) y(t)
y'(t)
t
Which

implies

y(t t) y(t) ty'(t)

+
Euler Method

18

Explicit Euler Method

Split

time t into n slices of equal length t

t0 0

ti i t
t t
n
The

Explicit Euler Method Formula

y(ti1 ) y(ti ) ty'(ti )

+
Euler Method
Explicit Euler Method - Algorithm

19

+
Euler Method

20

Explicit Euler Method - Error

Using

Taylor series expansion, we can compute

the truncation error at each step

y(t t) y(t) ty'(t) O(t )

2
etstep O(t )
2

We

assume that the total truncation error is the

sum of truncation error of each step

t
et O(t2 ) O(t)
t

This

assumption does not always hold.

+
Euler Method

21

Implicit Euler Method

Consider

Backward Difference

y(t) y(t t)
y'(t)
t
Which

implies

y(t) y(t t) ty'(t)

+
Euler Method

22

Implicit Euler Method

Split

the time into slices of equal length

y(ti1 ) y(ti ) ty'(ti1 )

The

above differential equation should be

solved to get the value of y(ti+1)
Extra computation
Sometimes worth because implicit method is
more accurate

+
Euler Method

23

A Simple Example
Try

to solve IVP

y'(t) et t

y(0) 1
What
The

is the value of y when t=0.5?

analytical solution is

1 2
y e t 2
2
t

+
Euler Method

24

A Simple Example
Using

explicit Euler method

ti

yi1 yi dt(e ti )
We

choose
accuracy

different dts to compare the

dt1 0.1 t 0, 0.1, 0.2,..., 0.5

dt2 0.05 t 0, 0.05, 0.1,..., 0.5
dt3 0.01 t 0, 0.01, 0.02,..., 0.5

+
Euler Method

25

A Simple Example

0.1
0.2
0.3
0.4
0.5

exact

dt=0.0
5

error

dt=0.0
25

error

dt=0.0
125

error

1.10016 1.10030 0.00014 1.10022 0.00006 1.10019 0.00003

1.20126 1.20177 0.00050 1.20151 0.00024 1.20138 0.00011
1.30418 1.30525 0.00107 1.30470 0.00052 1.30444 0.00025
1.40968 1.41150 0.00182 1.41057 0.00089 1.41012 0.00044
1.51846 1.52121 0.00274 1.51982 0.00135 1.51914 0.00067

At some given time t, error is proportional to dt.

+
Euler Method

26

Instability
For

some equations called Stiff Equations,

Euler method requires an extremely small dt
to make the result accurate

y'(t) ky(t), k 0
The

Explicit Euler Method Formula

yi1 yi tkyi (1 tk)yi

The

choice of t matters!

+
Euler Method

27

Instability
Assume

k=5

y'(t) 5y(t)

y(0) 1
Analytical

Solution is
5t

y(t) e
Try

Explicit Euler Method with different dts

28

+ Choose dt=0.002, s.t.

Works!

1
0 1 tk 1 t
k

29

+ Choose dt=0.25, s.t.

1
2
1 1 tk 0 t
k
k

Oscillates, but works.

30

+ Choose dt=0.5, s.t.

Instability!

2
t 1 tk 1
k

+
Euler Method

31

Stiff Equation Explicit Euler Method

For large dt, explicit Euler Method does not guarantee

an accurate result

exact

dt=0.5

0.4

0.135335

0.8
1.2
1.6
2

error

dt=0.2
5

error

dt=0.0
02

error

6.389056 -0.25 2.847264 0.13398 0.010017

82.89722
0.018316
-1.5
5
0.015625 1.853096 0.017951 0.019933
906.7147
0.002479 2.25
85
0.000977 1.393973 0.002405 0.02975
10061.73
0.000335 -3.375
321
0.000061 1.181943 0.000322 0.039469
111507.9
0.000045 5.0625
831
0.000015 0.663903 0.000043 0.04909

+
Euler Method

32

Stiff Equation Implicit Euler Method

Implicit

Euler Method Formula

yi1 yi dt5yi1
Which

implies

yi
yi1
1 5dt

33

Choose dt=0.5,
Oscillation eliminated!
Not elegant, but works.

+
The Three-Variable Model
Single Cell
The

Differential Equations
tV (DV) (I fi I so I si ) I ext

tv (1 p)(1 v) / v pv/ v

w
(1
p)(1
w)
/

pw/

w
w
t
I fi vp(V Vc )(V Vm) / d
I so (V Vo )(1 p) / o p/ r
I si w(1 tanh[k(V Vcsi )]) / (2 si )
p

1 if V Vc
0 if V Vc

1 if V VV
0 if V VV

34

+
The Three-Variable Model
Single Cell
Simplify

the model

tV (I fi I so I si ) I ext

tv (1 p)(1 v) / [1000q19.2(1 q)] 0.3pv

w (1 p)(1 w) /11 pw/ 667
t
I fi 0.04vp(V 72)(15 V)
I so (V 85)(1 p) / 8.3 2 p
I si w(1 tanh[0.1V]) / 0.897
p

1 if V 72
0 if V 72

1 if V 79.5
0 if V 79.5

35

+
The Three-Variable Model
Single Cell
Using

explicit Euler method

Select simulation time T

Select time slice length dt
Number of time slices is nt=T/dt
Initialize arrays vv(0, , nt), v(0, , nt), w(0, , nt)
to store the values of V, v, w at each time step

36

+
The Three-Variable Model
Single Cell
At

each time step

Compute p,q from value of vv of last time step

Compute Ifi, Iso, Ifi from values of vv, v, w of
previous time step

I fi 0.04 v(i 1)p(vv(i 1) 72)(15 vv(i 1))

I so (vv(i 1) 85)(1 p) / 8.3 2 p
I si w(i 1)(1 tanh[0.1vv(i 1)]) / 0.897

37

+
The Three-Variable Model
Single Cell
At

each time step

Use explicit Euler method formula to compute

new vv, v, and w
vv(i) vv(i 1) dt((I fi I so I si ) I ext )
v(i) v(i 1) dt(1 p)(1 v(i 1)) / (1000q19.2(1 q)) 0.3pv(i 1)
w(i) w(i 1) dt(1 p)(1 w(i 1)) /11 pw(i 1) / 667

38

39

The Three-Variable Model

+
Heat Diffusion Equations
The

Model

The

first equation describes the heat conduction

u
u

t
u f
t0

Function u is temperature distribution function

Constant is called thermal diffusivity

The

second equation initial temperature

distribution

40

+
Heat Diffusion Equation
Laplace Operator
Laplace

Operator (Laplacian)

Definition: Divergence of the gradient of a function

f 2 f (f )
Divergence

measures the magnitude of a vector

fields source or sink at some point

div( f ) f
Gradient

is a vector point to the direction of

greatest rate of increase, the magnitude of the
gradient is the greatest rate

41

+
Heat Diffusion Equation
Laplace Operator
Meaning

of the Laplace Operator

u (u)
(thegradientof temperaturedistribution)
themagnitudeof temperaturechangeover space
Meaning

of Heat Diffusion Operator

At some point, the temperature change over time

equals the thermal diffusivity times the magnitude of
the greatest temperature change over space

42

+
Heat Diffusion Equation
Laplace Operator
Cartesian

coordinates

x1, x2 ,, xn
n

2 f
f 2
i1 xi
1D

space (a cable)

2 f
f 2
x

43

+
Heat Diffusion Equation
Laplace Operator
Compute
Similar

Laplacian Numerically (1d)

to Numerical Differentiation

2 f f (x dx) f (x dx) 2 f (x)

f 2
x
dx2

44

+
Heat Diffusion Equation
Laplace Operator
Boundaries
Assume

(1d), take x=0 for example

the derivative at a boundary is 0

f
0
x x0
f (x) f (x)

0
2x
f (x) f (x)

Laplacian

at boundaries
f

x0

f (x) f (x) 2 f (0) 2 f (x) 2 f (0)

x2
x2

45

+
Heat Diffusion Equation

Exercise

Write a program in
Octave to solve the
following heat diffusion
equation on 1d space:

u
t u

u(x, 0) 10sin(
0 x 10
0.9

x
)
10

46

+
Heat Diffusion Equation

Exercise

TIPS:

Write a program in
Octave to solve the
following heat diffusion
equation on 1d space:

Store the values of u in a

2d array, one dimension is
the time, the other is the
cable(space)

Use explicit Euler Method

Choose dt, dx carefully to

avoid instability

You can use mesh()

function to draw the 3d
graph

u
t u

u(x, 0) 10sin(
0 x 10
0.9

x
)
10

47

48

Store u in a two-dimensional array

u(i,j) stores value of u at point x=xi, time t=tj.
u(i,j) is computed from u(i-1,j-1), u(i,j-1), and u(i+1,j+1).

+
Heat Diffusion Equation

Explicit Euler Method Formula

u(x, t) u(x, t dt) dtu

Discrete Form

u(i 1, j 1) u(i 1, j 1) 2u(i, j 1)

u(i, j) u(i, j 1) dt
dx2
2u(1, j 1) 2u(0, j 1)
u(0, j) u(0, j 1) dt
dx2
2u(n1, j 1) 2u(n, j 1)
u(n, j) u(n, j 1) dt
dx2

49

+
Heat Diffusion Equation

Stability

2dt
0 1
1
2
dx
2dt dx2

dt must be small enough to avoid instability

50

+
The END
Thank You!

51