ESOGÜ Electrical-Electronics

Engineering Department
2023-2024 Fall Semester
Numerical Methods
Week-15: Lecture Notes
•Ordinary Differential Equations
•Euler’s method
•Improvements of Euler’s method
•Runge-Kutta methods
•Systems of Equations
•Adaptive Runge-Kutta Methods

1
 Ordinary Differential Equations
Part- 7

• A differential equation is a mathematical

equation that relates some function with its
derivatives.
• In applications, the functions usually represent
physical quantities, the derivatives represent
their rates of change, and the equation defines
a relationship between the two.
dv c v- dependent variable
=g− v
dt m t- independent variable
3

• When a function involves one dependent variable, the

equation is called an ordinary differential equation
(or ODE). A partial differential equation (or PDE)
involves two or more independent variables.
• Differential equations are also classified as to their
order.
– A first order equation includes a first derivative as its
highest derivative.
– A second order equation includes a second derivative.
• Higher order equations can be reduced to a system of
first order equations, by redefining a variable.

2
Some ODE Examples:

• Newton’s 2nd law of motion:

𝑑𝑣
𝐹 = 𝑚𝑎 = 𝑚
𝑑𝑡

• Swinging pendulum:

• Numerical solutions of ODEs?

5

Runge-Kutta Methods
Chapter 25

• This chapter is devoted to solving ordinary differential

dy
equations of the form = f ( x, y ) .
dx

Example: Falling parachutist (from Chapter-1)

3
Recall that the method was of the general form:
New value = old value + (slope×step size)
or, in mathematical terms, yi+1 = yi + ϕh.
This formula can be
applied step by step to
compute out into the
future and, hence,
trace out the trajectory
of the solution.

Euler’s Method
• The first derivative provides a direct estimate of the
slope at xi
φ = f ( xi , yi )
where f(xi,yi) is the differential equation evaluated at
xi and yi. This estimate can be substituted into the
equation:
yi +1 = yi + f ( xi , yi )h
• A new value of y is predicted using the slope to
extrapolate linearly over the step size h.
• The Euler method is a first-order numerical procedure
for solving ODEs with a given initial value. It is the
simplest Runge–Kutta method. 8

4
Example:

10

5
Error Analysis for Euler’s Method/
• Numerical solutions of ODEs involves two types of
error:
– Truncation error
• Local truncation error

f ′( xi , yi ) 2
Ea = h
2!
Ea = O ( h 2 )
• Propagated truncation error
– The sum of the two is the total or global truncation error
– Round-off errors
11

• The Taylor series provides a means of quantifying the

error in Euler’s method. However;
– The Taylor series provides only an estimate of the local
truncation error-that is, the error created during a single
step of the method.
– In actual problems, the functions are more complicated
than simple polynomials. Consequently, the derivatives
needed to evaluate the Taylor series expansion would not
always be easy to obtain.
• In conclusion,
– the error can be reduced by reducing the step size
– If the solution to the differential equation is linear, the
method will provide error free predictions as for a straight
line the 2nd derivative would be zero.
12

6
 13

Improvements of Euler’s method

• A fundamental source of error in Euler’s method is that the
derivative at the beginning of the interval is assumed to apply
across the entire interval.
• Two simple modifications are available to circumvent this
shortcoming:
– Heun’s Method
– The Midpoint (or Improved Polygon) Method

14

7
Heun’s Method/
• One method to improve the estimate of the slope
involves the determination of two derivatives for the
interval:
– At the initial point
– At the end point
• The two derivatives are then averaged to obtain an
improved estimate of the slope for the entire interval.
Predictor : yi0+1 = yi + f ( xi , yi )h
f ( xi , yi ) + f ( xi +1 , yi0+1 )
Corrector : yi +1 = yi + h
2
15

16

8
Heun’s Method is a predictor-corrector method.
• Corrector step can be used more than once to get better estimates
for yi+1.
Predictor: y0i+1= yi+ f(xi, yi) h
Corrector: y1i+1= yi+ [ f(xi, yi) + f(xi+1, y0i+1)]/2 * h
Corrector: y2i+1= yi+ [ f(xi, yi) + f(xi+1, y1i+1)]/2 * h … continue
until the error falls below the tolerance
• If f=f(x) only, than the predictor step is not required. Corrector step
becomes
f ( xi ) + f ( xi +1 )
yi +1 = yi + h
2
Note the similarity between the above formula and the trapezoidal
Rule.
• Heun’s Method is 2nd order accurate. It can obtain exact results
when the solution y(x) is quadratic.
• It has a global error of O(h2) and local error of O(h3).
17

Example:

Heun Method with 1 corrector result: Heun method h = 1

90

80

70

60

50
y

40

Heun Method with 15 correctors result: 30

20

10

0
0 0.5 1 1.5 2 2.5 3 3.5 4
x

M-file: p7_ch25_ex2.m 18

9
Example: Repeat the previous example (Euler Meth.) by applying
Heun’s method. Compare the results.

(Heun method used with

1 corrector )

M-file: p7_ch25_ex3.m

19

The Midpoint (or Improved Polygon) Method/

• Uses Euler’s method to predict a value of y at the
midpoint of the interval:
yi +1 = yi + f ( xi +1/ 2 , yi +1/ 2 )h
• Similar to Heun’s Method this also tries to improve
the Euler’s Method by using a better slope.
• If f=f(x) only, than there is no need to perform the
first step.
• Midpoint Method is 2nd order accurate. Its global
error is O(h2).
20

10
yi +1 = yi + f ( xi +1/ 2 , yi +1/ 2 )h

21
Graphical depiction of the midpoint method.

Example: Repeat the same example by applying Midpoint

method. Compare the results.

M-file: p7_ch25_ex4.m
22

11
Example: Repeat the same example by applying Midpoint
method. Compare the results.

M-file: p7_ch25_ex5.m

23

Runge-Kutta (RK) Methods

• Runge-Kutta methods achieve the accuracy of a
Taylor series approach without requiring the
calculation of higher derivatives.
yi +1 = yi + ϕ ( xi , yi , h) h
ϕ = a1k1 + a2 k2 +  + an kn Increment function
a's = constants
k1 = f ( xi , yi )
k2 = f ( xi + p1h, yi + q11k1h) p’s and q’s are constants

k3 = f ( xi + p2 h, yi + q21k1h + q22 k2 h)

kn = f ( xi + pn −1h, yi + qn −1k1h + qn −1,2 k2 h +  + qn −1,n −1kn −1h)
24

12
• k’s are recurrence functions. Because each k is a functional
evaluation, this recurrence makes RK methods efficient for
computer calculations.
• Various types of RK methods can be devised by employing
different number of terms in the increment function as
specified by n.
• First order RK method with n=1 is in fact Euler’s method.
• Once n is chosen, values of a’s, p’s, and q’s are evaluated by
setting general equation equal to terms in a Taylor series
expansion.
yi +1 = yi + (a1k1 + a2 k 2 )h
k1 = f ( x i , yi )
k 2 = f ( xi + p1h, yi + q11k1h)
25

• Values of a1, a2, p1, and q11 are evaluated by

setting the second order equation to Taylor series
expansion to the second order term. Three
equations to evaluate four unknowns constants are
derived.
a1 + a2 = 1
1
a 2 p1 = A value is assumed for one of the
2 unknowns to solve for the other
1 three.
a2 q11 =
2

26

13
• Because we can choose an infinite number of values
for a2, there are an infinite number of second-order RK
methods.
• Every version would yield exactly the same results if
the solution to ODE were quadratic, linear, or a
constant.
• However, they yield different results if the solution is
more complicated (typically the case).
• Three of the most commonly used methods are:
– Heun Method with a Single Corrector (a2=1/2)
– The Midpoint Method (a2=1)
– Ralston’s Method (a2=2/3)

27

28

14
 Third Order Runge-Kutta Methods
• For n=3, a derivation similar to the one for the second-order method can be performed.
• It gives: six equations with eight unknowns. Therefore, values for 2 of the unknowns
must be specified a priori. One solution:

• Note that if the derivative is a function of x only, this third-order method reduces to
Simpson’s 1/3 rule.
• 3rd-order RK methods have local and global errors of O(h4) and O(h3), respectively.
29

Fourth Order Runge-Kutta Methods

• For n=4, similar derivations can be performed.
• The following is the most commonly used 4th order form:

30

15
 Fourth Order Runge-Kutta Methods

31

Higher Order Runge-Kutta Methods

Butcher’s (1964) fifth-order
RK method:

32

16
 Systems of Equations
• Many practical problems in engineering and science
require the solution of a system of simultaneous ordinary
differential equations rather than a single equation:
dy1
= f1 ( x, y1 , y2 , , yn )
dx
dy2
= f 2 ( x, y1 , y2 , , yn )
dx

dyn
= f n ( x, y1 , y2 , , yn )
dx

• Solution requires that n initial conditions be known at the

starting value of x. 33

Adaptive Runge-Kutta Methods

• For an ODE with an abrupt
changing solution, a constant step
size can represent a serious
limitation.

34

17
Step-Size Control/
• The strategy is to increase the step size if the error is too small
and decrease it if the error is too large. Press et al. (1992) have
suggested the following criterion to accomplish this:
α
Δ new
hnew = h present
Δ present

Δpresent= computed present accuracy

Δnew= desired accuracy
α= a constant power that is equal to 0.2 when step size
increased and 0.25 when step size is decreased

35

• Implementation of adaptive methods requires

an estimate of the local truncation error at each
step.
• The error estimate can then serve as a basis for
either lengthening or decreasing step size.

36

18
(a) A bell-shaped forcing function
that induces an abrupt change
in the solution of an ODE.

(b) The solution. The points

indicate the predictions of an
adaptive step-size routine.

37

38

19