Outline

Introduction to Numerical • Numerical modeling

– Terminology
Modeling and Device Modeling
– Basic operations
– Examples of simple algorithms
Lecture 1 – Sources of error
• Modeling physical systems
Special Topics: – Main steps and considerations
Device Modeling

Introduction Introduction
• Most real-word applications lead to mathematical • The simplified analytical model (e.g., set of equations)
problems which cannot be solved with exact for problem is converted into a numerical model
formulas, or analytically – Can be solved in a finite number of basic arithmetic
• A common approach is to reduce a problem to operations
special cases and simplified situations, and study – Approximation of an analytical model (error)
those in detail • A numerical model is solved with numerical methods,
• The aim is to uncover generally applicable introducing additional errors
concepts and properties, which can guide us in • Machine representation: computers have a limit on
more difficult problems how small or large a number can be

Example: sine function approximated

Numerical modeling:
by Taylor series expansion
terminology
Sequences and series
• A sequence is a (possibly infinite) collection of
numbers lined up in some order
• A series is a (possibly infinite) sum
– Example: Taylor’s series Images from: http://www.sosmath.com/calculus/taypol/taypol05.html
n
1
Tn  x    k ! f  x  x  x 
k

(k )
o o The approximation of f(x)=sinx, around x0=0 by the Taylor
k 1
series becomes more accurate (better over a larger interval
1 1
f  x o   f  x o  x  x o   f  x o  x  x o   ...   x o  x  x o n around the center) with the number of terms increasing
2 (n)
f
2 n! from 1 to 13

1
 Numerical modeling: Examples: converging and
terminology diverging sequences
Convergence and divergence
• Sequence (aj) with j=[0,∞] is said to be e-close to • Unbounded sequences, i.e., sequences that contain
a number b if there exists a number N ≥ 0 (it can arbitrarily large numbers, always diverge
be very large), such that for all n ≥ N, |aj − b|≤ e • e−n → 0 as n →∞, and convergence is very fast
• A sequence (aj) with j=[0,∞] is said to converge to • n/(n + 2) → 1 as n →∞, and convergence is rather
b if it is e-close to b for all e> 0 (however small) slow
• Notation: aj → b, or limj→∞ aj = b • log(n) →∞ as n →∞, so the sequence diverges
• If a sequence does not converge, it diverges

Numerical modeling: Examples: converging and

terminology diverging series
Convergence and divergence N
• Geometric series converges for |x|<1

• Define the Nth partial sum S N  a 0  a1  ...  a n   aj
x
j
 1  x  x  ... 
2 1
j0
j0 1 x
• The series  j a j converges if the sequence of partial • Harmonic series diverges
sums SN converges to some number b as N →∞ 
1 1 1

  1   ...
• Notation:  aj  b j 1

j
j
2 3
j0

• If a series does not converge, it diverges • Series    1   0  1  0  1  ... also diverges

j0

Numerical modeling: Numerical modeling:

terminology terminology
• Discretization is the process of transferring • Iteration (from the Latin iterare, “to plow once
continuous functions, models, and equations into again”) is a repetition of a mathematical or
discrete counterparts Function
computational procedure applied to the result of a
previous application, typically as a means of
• Usually carried out as a first
obtaining successively closer approximations to
step toward making them
the solution of a problem
suitable for numerical http://www.cradle-cfd.com/tec/column01/012.html

evaluation and • Iterative methods are used to produce approximate

Surface
implementation as a computer mesh
numerical solutions to mathematical problems
program • When programmed, implemented through loops
http://mfem.org/examples/

2
 Converging iterations
Example: Solution of an
equation x=F(x) using the
Numerical modeling:
method of iterations
• Start with an initial
basic operations
approximation x0, and
compute the sequence
• Interpolation
x1=F(x0) x2=F(x1) x3=F(x2) … • Derivatives
Diverging iterations • Each computation of the
type xn+1=F(xn) is called a • Integration
fixed-point iteration: as n
grows we would like xn to • Root finding
be closer to the root
– Nonlinear equations
• If the sequence {xn}
converges to a limiting – Differential equations
value α, then x= α is the
root: α=F(α)

Numerical modeling: Numerical modeling:

interpolation interpolation
• Experiment is usually represented by a discrete set of • For a function defined and
datapoints [xi, f(xi)]; interpolation is required to find the continuous over an interval
value of f(xk) for any arbitrary point xk [a,b], for each e >0 there exists
– Often we want an analytical function describing the whole data set polynomial P(x) such that
• One of the most useful and well-known approaches to f ( x )  P ( x )  e for all x in [a,b]
functions mapping over a range of values is with the
algebraic polynomials • Another important reason for considering the class of
n 1
polynomials in the approximation of functions is that the
Pn ( x )  a n x  a n 1 x  ... a 1 x  a 0
n
derivative and indefinite integral of a polynomial are easy to
determine and are also polynomials
– Taylor's polynomials are good approximation at one point, x0

Numerical modeling: Numerical modeling:

differentiation differentiation
• The derivative of the function f at x0 is defined as • Start with Taylor series for f(x0+h) and express first derivative:
f ( x0  h )  f ( x0 ) f ( x0  h )  f ( x0 ) h h
2

f ( x )  lim f ( x 0 )    f ( x 0 )  f ( x 0 )  ... 
h 0 h h  2! 3! 
• General approximation would be to compute for small h:
f ( x0  h )  f ( x0 ) • Neglecting ( ) leads to error ~h
h • Can reduce the error by using both
• Two problems with this approach using numerically: forward- and backward differences
– Very small h – division by zero forward-difference for h>0,
f ( x0  h )  f ( x0  h ) backward-difference for h<0
f ( x 0 )   O (h )
2
– Subtract two numbers which only differ by a small amount
2h
remainder ~h2

3
 Numerical modeling: Numerical modeling:
integration integration
• Evaluation of definite integral of a function as • The [a,b] is divided into N intervals, Trapezoid rule

numerical quadrature: equally spaced by h = xi −xi−1

b
n • Trapezoid rule:
 f ( x ) dx   f ( xi )  xi b h
 f ( x0  a )  f ( x 0  h )  ...  f ( x N  b ) 
 f ( x ) dx 
a
i0
a 2
Simpson’s rule
• Divide interval into multiple slices, and find area • Simpson’s rule: approximate function
under the curve over interval [a,b] with parabola over each interval
• Simplest approximation with rectangles; slightly b h
more sophisticated: trapezoid rule, Simpson’s rule  a
f ( x ) dx 
3
[ f (a )  4 f (a  h)  2 f (a  2h) 
Use 3 points for each interval,
 4 f ( a  3 h )  ...  f ( b )] number of intervals should be even

Numerical modeling: Numerical modeling:

root finding nonlinear equations
• Finding a root, or solution, of an equation of the • Bisection method – the simplest and most robust; works for one
form f(x) = 0, for a given function f or more roots
– Also called a zero of the function f • Let f(x) be a continuous function on [a,b] with f(a) and f(b) of
opposite sign; we search for p such that f(p)=0
• Selection of a particular numerical method
• The method calls for a repeated
depends on the equation type: linear, non-linear,
halving (or bisecting) of subintervals
differential (ordinary or partial) of [a, b] and, at each step, locating the
• System of equations half containing p
• After n steps interval is reduced to
(b−a)/2n, continue until relative error
objective is reached

Numerical modeling: Numerical modeling:

nonlinear equations differential equations
• Newton’s (or the Newton-Raphson) method exploits • Principally, divided into two categories:
derivatives of f(x) to accelerate conversion to f(p)=0 – the ordinary differential equations (ODE) which
• Start with Taylor’s expansion of the f(x) about p0, which is contain functions of only one independent variable
close to the root p, and neglect terms of second order, since – the partial differential equations (PDE) having
p-p0 is small functions of several independent variables
f ( p0 )
p  p0   p1
f ( p 0 ) • ODEs can themselves be grouped into
• After n iterations: subcategories according to their order n and
f ( p n 1 ) whether they are linear
p n  p n 1  , for n  1
f ( p n 1 ) • Most general form of ODE: F(x,y,y’,y’’, …,y(n))=0
Intersection of
• Stop when rel. error objective is reached tangent with x gives
next point p1
Wolfram website: http://www.wolframalpha.com

4
 Numerical modeling: Numerical modeling:
differential equations differential equations
• In solving differential equation we make distinction • Euler method: first order ODE with initial value
between the general and a particular solution • Consider the problem of calculating the shape of a curve
– Example: equation y’=y has the general solution y=Cex, f(x): it starts at a given point x0 (initial value) and satisfies
where C is an arbitrary constant a given differential equation, so we can calculate the
derivative at x0 (find tangent)
• Every nth order ODE has n integrating constants, that x2 … xn
• Take a small step h along the tangent
can be either the initial values, or boundary conditions and repeat the procedure x1
(determined at the integration limits) leading to initial f(x)
y ( x )  f ( x , y ( x )), y ( x 0 )  y 0 x0
value or boundary condition problems
y n 1  y n  hf ( x n , y n ), yn  y ( xn ) - approximate
– Same for first-order differential equations solution

Numerical modeling:
Numerical modeling:
differential equations
differential equations
Euler method Runge-Kutta method
• Runge–Kutta method: 4th order method for ODE y'(x) = -2 x y(x)
y(0) = 5
• Offers much better accuracy than the Euler method (2 nd [0,3]
degree method); can further adapt using variable step size
k 1  hf ( x n , y n ) y'(x) = -2 x y(x)
1 1 y(0) = 5
k 2  hf ( x n  h, yn  k1 ) [0,3]
2 2
Sample solution:
1 1 http://www.wolframalpha.com
k 3  hf ( x n  h, yn  k2 )
2 2
k 4  hf ( x n  h , y n  k 3 )
1 1 1 1
y n 1  y n  k1  k2  k3  k 4  O (h )
5

6 3 3 6

Numerical modeling: Numerical modeling:

differential equations matrices
• Finite differences method - finite differences • Two principal computational problems are
approximate the derivatives associated with matrices
• Finite elements method subdivides a large problem – Systems of linear equations: Ax=b
into smaller, simpler, parts - finite elements (FE’s) – Eigenvalue problems: Ax=lx
• The simple equations that model these FE’s are • Calculation of determinants(detA, det|A-lI|)
then assembled into a larger system of equations • Typical problems: large round-off errors (iterative
that models the entire problem approaches such as Gaussian elimination), ill-
• FEM then uses variational methods to approximate conditioned matrices (detA close to 0, solution is
a solution by minimizing an associated error unstable)
function

5
 Numerical modeling:
Numerical modeling: errors
Monte Carlo simulations
• Multiple real-life processes are stochastic in • The accuracy is always lower than in analytical
nature: characterized by probability distribution solution
and expectation value • Sources of error:
• Monte Carlo simulations are designed to repeat – Errors in given input data, operator error, etc.
the same process very large number of times to – Simplification error
obtain the solution close to the expectation value – Rounding (or chopping) error during computation
• Rely heavily on random number generators – Truncation error (e.g., an infinite series is broken off
– Eventually the sequence of numbers from the generator after a finite number of terms)
will repeat itself – Discretization error

Modeling physical system:

Modeling physical system
Problem formulation
• Model is defined through • The general goal of modeling is to improve
– Objectives: problem formulation understanding of physical processes
– A set of governing equations • It is critical to define the specific goals of a
– Geometry problem formulation
– Boundary conditions (or initial values) • Example: drug delivery
– Material properties and other parameters through skin, using a patch
• Most models do not have simple analytical solution • The general problem is
– Example: calculation of the motion of a cannon ball in quite complex
two dimensions if include drag force

Modeling physical system: Modeling physical system:

Problem formulation Geometry
Goal: to study the drug transport Goal: to study the drug transport inside
primarily in the skin region the patch as well as in the skin • The computational domain is the chosen region of the
physical domain (actual geometry) where computations
will be performed
• The larger the computational domain, the more
computation is required
• Choice of 1D vs 2D vs 3D
• How can symmetry be used to reduce the domain?
• What regions need to be included

6
 Modeling physical system: Modeling physical system:
Geometry Governing equations
• We need as many equations to describe the model as there
are distinct “physics”
• Example: to model transport processes in a biomedical
system have to consider three most distinct “physics” -
fluid flow, heat transfer, and mass transfer. The following
equations should be included:
– Conservation equation for total mass (continuity equation)
– Momentum conservation equations (fluid flow equations)
– Energy conservation equation (heat transfer equation)
– Mass species conservation equation (mass transfer equation)
Example: arterial tissue surrounding blood

Modeling physical system: Modeling physical system:

Governing equations Boundary conditions
• Next simplification step: what terms should • Boundary conditions are statements describing
remain in the governing equations how the process relates to its surroundings
• Transient or steady-state (time dependence) – alternatively, initial conditions are specified
• Gradient terms and resultant transport vs uniform • How many boundary conditions are needed
distributions – all of the external boundaries of the computational
domain need boundary conditions for each of the
• Generation (depletion) or source (sink) terms primary variables
• If approximating functions – how many terms to • What they are specifically – opportunity for
retain? simlification

Modeling physical system: Modeling physical system:

Boundary conditions Material properties
• Are we likely to find the property data needed for
the exact material that we need?
Less
• Can we estimate the property using empirical
realistic predictive equations
• Possible substitutions, simplifications, and their
More realistic: effect on the accuracy of the solution
the patch will
lose drug while
• Uniform vs. non-uniform properties
tissue will gain • Best decided at the problem formulation stage

7
 Summary References
• Most real-word applications lead to mathematical
• A. Klein and A. Godunov, Introductory Computational
problems which cannot be solved with exact Physics, Cambridge University Press, 2006
formulas, or analytically • R. L. Burden, J. D. Faires, Numerical Analysis, 9 th edition,
• A common approach is to reduce a problem to Brooks/Cole, Cengage Learning, 2011
special cases and simplified situations, and study • G. Dahlquist, Å. Björck, Numerical methods in scientific
those in detail computing, Vol.1, SIAM, Philadelphia, 2008
• Errors accumulate through every simplification • A. Datta and V. Rakesh, An Introduction to Modeling of
Transport Processes, Cambridge University Press, 2010
step at the model level, and at every
approximation related to model implementation