EEP312

NUMERICAL METHODS
AND ANALYSIS
 CHAPTER 7
NUMERICAL SOLUTIONS OF
DIFFERENTIAL EQUATIONS
7.1 The Milne’s Method
7.2 The Runge-Kutta Method
 NUMERICAL SOLUTIONS OF
DIFFERENTIAL EQUATIONS
• The fundamental problem is to find the solution of the
first-order differential equation
𝑑𝑦
= 𝑓(𝑥, 𝑦)
𝑑𝑥
Which satisfies the initial condition y=y0 when x=x0. Our
objective is to tabulate the solution curve point by point,
beginning at (x0,y0) and continuing thereafter at selected
values of x, usually equally spaced, until the solution has
been extended over a required range.
 7.1 THE MILNE’S METHOD
• To develop the Milne’s method, we begin with the equation
2𝑟 − 1 3𝑟 2 − 6𝑟 + 2 4𝑟 3 − 18𝑟 2 + 22𝑟 − 6
ℎ𝑓 ′ 𝑥0 + 𝑟ℎ = ∆𝑓0 + 𝑓0 + ∆3 𝑓0 + ∆4 𝑓0 + ⋯
2 6 24
Written in terms of y rather than f, and we evaluate it for r=1,2,3, and 4, getting
′
1 1 2 1 3 1 4
𝑦1= ∆𝑦0 + ∆ 𝑦0 − ∆ 𝑦0 + ∆ 𝑦0 + ⋯
ℎ 2 6 12
′
1 3 2 1 3 1 4
𝑦2= ∆𝑦0 + ∆ 𝑦0 − ∆ 𝑦0 − ∆ 𝑦1 + ⋯
ℎ 2 3 12
′
1 5 2 11 3 1 4
𝑦3= ∆𝑦0 + ∆ 𝑦0 − ∆ 𝑦0 + ∆ 𝑦0 + ⋯
ℎ 2 6 4
′
1 7 2 13 3 25 4
𝑦4= ∆𝑦0 + ∆ 𝑦0 − ∆ 𝑦0 + ∆ 𝑦0 + ⋯
ℎ 2 3 12
 7.1 THE MILNE’S METHOD
• Neglecting differences beyond the fourth and replacing the
remaining differences by their equivalent expressions in terms
of the successive functional values:
′
1
𝑦1= −3∆𝑦0 − 10𝑦1 + 18𝑦2 − 6𝑦3 + 𝑦4
12ℎ
′
1
𝑦2= 𝑦0 − 8𝑦1 + 8𝑦3 − 𝑦4
12ℎ
′
1
𝑦3= −𝑦0 + 6𝑦1 − 18𝑦2 + 10𝑦3 + 3𝑦4
12ℎ
′
1
𝑦4= 3𝑦0 − 6𝑦1 + 36𝑦2 − 48𝑦3 + 25𝑦4
12ℎ
 7.1 THE MILNE’S METHOD
• Now, if we subtract the second equation in the last set from twice the sum
of the first and the third equations and solve the result for y4, we will obtain
4ℎ
𝑦4 = 𝑦0 + 2𝑦 ′1 − 𝑦 ′ 2 + 2𝑦 ′ 3
3
Or in more general terms,
4ℎ
𝑦𝑛+1 = 𝑦𝑛−3 + 2𝑦 ′ 𝑛−2 − 𝑦 ′ 𝑛−1 + 2𝑦 ′ 𝑛
3
If we know the values of y and y’ down to and including their values at xn, it
does enables us to reach out one step further and compute yn+1. With yn+1
𝑑𝑦
known, we can return to the differential equation 𝑑𝑥 = 𝑓(𝑥, 𝑦) and compute
y’n+1. Then using equation 𝑦𝑛+1 again, we can find yn+2, and so on, step by
step, until the solution has been extended over the desired range.
 7.2 THE RUNGE-KUTTA METHOD
• In the Runge-Kutta method, three or more estimates of Δy are computed,
and then the value of Δy which is finally used to determine y1 is taken to be
a linear combination of all of these. Specifically, in Kutta’s third-order
approximation we let
∆1𝑦 ≡ 𝑘1 = 𝑓(𝑥0 , 𝑦0)∆𝑥 ≡ 𝑓 𝑥0 , 𝑦0 ℎ
∆2 𝑦 ≡ 𝑘2 = 𝑓(𝑥0 + 𝑝∆𝑥, 𝑦0 + 𝑝∆1𝑦)∆𝑥 ≡ 𝑓 𝑥0 + 𝑝ℎ, 𝑦0 + 𝑝𝑘1 ℎ
∆3𝑦 ≡ 𝑘3 = 𝑓(𝑥0 + 𝑞∆𝑥, 𝑦0 + 𝑟∆2𝑦 + 𝑞 − 𝑟∆1𝑦)∆𝑥 ≡ 𝑓 𝑥0 + 𝑞ℎ, 𝑦0 + 𝑟𝑘2 + 𝑞 − 𝑟𝑘1 ℎ
And then put
∆𝑦 = 𝑎𝑘1 + 𝑏𝑘2 + 𝑐𝑘3
Where a, b, c, p, q, and r are constants to be determined to ensure the
highest possible accuracy in Δy.
 7.2 THE RUNGE-KUTTA METHOD
• We first expand 𝑘1 , 𝑘2 , 𝑘3 , and then ∆𝑦 in terms of powers of ∆𝑥 ≡ ℎ, using implicit
differentiation to compute the necessary derivatives. Using subscripts to indicate
the partial derivatives of f evaluated at h=0, that is, letting
𝜕(𝑥, 𝑦) 𝜕(𝑥, 𝑦) 𝜕 2 (𝑥, 𝑦)
𝑓0 = 𝑓 𝑥0 , 𝑦0 , 𝑓1 = ቤ ,𝑓 = ቤ ,𝑓 = ቤ ,…
𝜕𝑥 𝑥0 , 𝑦0 2 𝜕𝑥 𝑥0 , 𝑦0 11 𝜕𝑥 2 𝑥0 , 𝑦0
𝑑𝑘𝑖 𝑑 2 𝑘𝑖
And using and , we have
𝑑ℎ 𝑑ℎ2
𝑘1 = 𝑓0 ℎ
2
𝑑𝑘𝑖 2 2
𝑑𝑘𝑖 2
𝑑𝑘𝑖 ℎ2
𝑘2 = 𝑓0 + 𝑓1 𝑝 + 𝑓2 𝑝 ℎ + 𝑓11 𝑝 + 2𝑓12 𝑝 + 𝑓22 𝑝 +⋯ ℎ
𝑑ℎ 𝑑ℎ 𝑑ℎ 2
2 3
2 2 𝑝 ℎ
= 𝑓0 ℎ + 𝑓1 + 𝑓2 𝑓0 𝑝ℎ + 𝑓11 + 2𝑓12 𝑓0 + 𝑓22 𝑓0 +⋯
2
 7.2 THE RUNGE-KUTTA METHOD
𝑘3
𝑑𝑘2 𝑑𝑘1
= ൭𝑓0 + 𝑓1𝑞 + 𝑓2 𝑟 + (𝑞 − 𝑟) ℎ
𝑑ℎ 𝑑ℎ
2
2
𝑑𝑘2 𝑑𝑘1 𝑑𝑘2 𝑑𝑘1
+ ൝𝑓11𝑞 + 2𝑓12 𝑞 𝑟 + (𝑞 − 𝑟) + 𝑓22 𝑟 + (𝑞 − 𝑟)
𝑑ℎ 𝑑ℎ 𝑑ℎ 𝑑ℎ
𝑑 2𝑘2 𝑑 2𝑘1 ℎ2
+ 𝑓2 𝑟 2 + (𝑞 − 𝑟) 2
ൡ + ⋯൱ℎ
𝑑ℎ 𝑑ℎ 2
3
ℎ
= 𝑓0 ℎ + 𝑓1 + 𝑓2𝑓0 𝑞ℎ2 + 𝑓11 + 2𝑓12𝑓0 + 𝑓22 𝑓02 𝑞2 + 2𝑓2 𝑓1 + 𝑓2 𝑓0 𝑝𝑟 +⋯
2
 7.2 THE RUNGE-KUTTA METHOD
• Substituting,
∆𝑦 = 𝑎𝑘1 + 𝑏𝑘2 + 𝑐𝑘3
2 𝑝2 ℎ3
= 𝑎𝑓0 ℎ + 𝑏 𝑓0 ℎ + 𝑓1 + 𝑓2 𝑓0 𝑝ℎ2 + 𝑓11 + 2𝑓12 𝑓0 + 𝑓22 𝑓0 +⋯
2
ℎ 3
2
+ 𝑐 𝑓0 ℎ + 𝑓1 + 𝑓2 𝑓0 𝑞ℎ2 + 𝑓11 + 2𝑓12 𝑓0 + 𝑓22 𝑓0 𝑞 2 + 2𝑓2 𝑓1 + 𝑓2 𝑓0 𝑝𝑟 +⋯
2
𝑏𝑝 2 + 𝑐𝑞 2
= 𝑎 + 𝑏 + 𝑐 𝑓0 ℎ + 𝑏𝑝 + 𝑐𝑞 𝑓1 + 𝑓2 𝑓0 ℎ2 + 𝑓11 + 2𝑓12 𝑓0 + 𝑓22 𝑓0 2 + 𝑐𝑝𝑟𝑓2 𝑓1 + 𝑓2 𝑓0 ℎ3
2
+⋯
Now, since 𝑦 ′ = 𝑓(𝑥, 𝑦), by Maclaurin’s expansion,
ℎ 2 ℎ 3
∆𝑦 = 𝑦 − 𝑦0 = 𝑦 ′ 0 ℎ + 𝑦 ′′ 0 + 𝑦 ′′′ 0 + ⋯
2! 3!
ℎ 2 ℎ3
2
= 𝑓0 ℎ + 𝑓1 + 𝑓2 𝑓0 + 𝑓11 + 2𝑓12 𝑓0 + 𝑓22 𝑓0 + 𝑓2 𝑓1 + 𝑓2 𝑓0 +⋯
2! 3!
 7.2 THE RUNGE-KUTTA METHOD
• Hence ∆𝑦 will agree with Maclaurin expansion of ∆𝑦, through terms in ℎ3 =
(∆𝑥)3 , provided
1 𝑏𝑝2 + 𝑐𝑞2 1 1
𝑎 + 𝑏 + 𝑐 = 1, 𝑏𝑝 + 𝑐𝑞 = , = , 𝑐𝑝𝑟 =
2 2 6 6
We have four equations in the six unknown parameters a, b, c, p, q, r. The first
three equations are linear in a, b, c and can easily be solved to express a, b,
c in terms of p and q. Then the fourth equation can be used to express r in
terms of p and q also.
6𝑝𝑞 − 3 𝑝 + 𝑞 + 2 2 − 3𝑞 2 − 3𝑝 𝑞 𝑞−𝑝
𝑎= ,𝑏 = ,𝑐 = ,𝑟 =
6𝑝𝑞 6𝑝(𝑝 − 𝑞) 6𝑞(𝑞 − 𝑝) 𝑝(2 − 3𝑝)
 7.2 THE RUNGE-KUTTA METHOD
• Since p and q are arbitrary, we have a two-parameter family of formulas which can be
used for the step-by-step solution of the equation 𝑦 ′ = 𝑓(𝑥, 𝑦) with an error which is of the
order (∆𝑥)4 = ℎ4 .
Here are some cases.
Case 1.
1 3 1 2
𝑎 = , 𝑏 = 0, 𝑐 = , 𝑝 = , 𝑞 = 𝑟 =
4 4 3 3
1
∆𝑦 = 𝑘1 + 3𝑘3
4
Where
𝑘1 = 𝑓 𝑥0 , 𝑦0 ℎ
1 1
𝑘2 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘1 ℎ
3 3
2 2
𝑘3 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘2 ℎ
3 3
 7.2 THE RUNGE-KUTTA METHOD
Case 2.
1 3 2
𝑎 = ,𝑏 = 𝑐 = ,𝑝 = 𝑞 = 𝑟 =
4 8 3
1
∆𝑦 = 2𝑘1 + 3𝑘2 + 3𝑘3
8
Where
𝑘1 = 𝑓 𝑥0 , 𝑦0 ℎ
2 2
𝑘2 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘1 ℎ
3 3
2 2
𝑘3 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘2 ℎ
3 3
 7.2 THE RUNGE-KUTTA METHOD
Case 3.
1
∆𝑦 = 𝑘1 + 2𝑘2 + 2𝑘3 + 𝑘4
6
Where
𝑘1 = 𝑓 𝑥0 , 𝑦0 ℎ
1 1
𝑘2 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘1 ℎ
2 2
1 1
𝑘3 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘2 ℎ
2 2
𝑘4 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘3 ℎ
The solution process based on this case is often referred to specifically as the
Runge-Kutta method.
 7.2 THE RUNGE-KUTTA METHOD
Case 4.
1
∆𝑦 = 𝑘1 + 3𝑘2 + 3𝑘3 + 𝑘4
8
Where
𝑘1 = 𝑓 𝑥0 , 𝑦0 ℎ
1 1
𝑘2 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘1 ℎ
3 3
2 1
𝑘3 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘2 − 𝑘1 ℎ
3 3
𝑘4 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘3 − 𝑘2 − 𝑘1 ℎ
 7.2 THE RUNGE-KUTTA METHOD
Any of the Runge-Kutta formulas can be used to solve simultaneous and higher-order differential
equations. For instance, using Case 3, we can tabulate the solution of the system of equations
𝑑𝑦 𝑑𝑧
= 𝑓 𝑥, 𝑦, 𝑧 = 𝑔 𝑥, 𝑦, 𝑧
𝑑𝑥 𝑑𝑥
At intervals of ∆𝑥 ≡ ℎ by computing
∆1 𝑦 ≡ 𝑘1 = 𝑓 𝑥0 , 𝑦0 , 𝑧0 ℎ
∆1 𝑧 ≡ 𝑙1 = 𝑔 𝑥0 , 𝑦0 , 𝑧0 ℎ
1 1 1
∆2 𝑦 ≡ 𝑘2 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘1 , 𝑧0 + 𝑙1 ℎ
2 2 2
1 1 1
∆2 𝑧 ≡ 𝑙2 = 𝑔 𝑥0 + ℎ, 𝑦0 + 𝑘1 , 𝑧0 + 𝑙1 ℎ
2 2 2
1 1 1
∆3 𝑦 ≡ 𝑘3 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘2 , 𝑧0 + 𝑙2 ℎ
2 2 2
1 1 1
∆3 𝑧 ≡ 𝑙3 = 𝑔 𝑥0 + ℎ, 𝑦0 + 𝑘2 , 𝑧0 + 𝑙2 ℎ
2 2 2
 7.2 THE RUNGE-KUTTA METHOD
∆4 𝑦 ≡ 𝑘4 = 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘3 , 𝑧0 + 𝑙3 ℎ
∆4 𝑧 ≡ 𝑙4 = 𝑔 𝑥0 + ℎ, 𝑦0 + 𝑘3 , 𝑧0 + 𝑙3 ℎ
And then using the formulas
1
∆𝑦 = 𝑘1 + 2𝑘2 + 2𝑘3 + 𝑘4
6
1
∆𝑦 = 𝑙1 + 2𝑙2 + 2𝑙3 + 𝑙4
6
 7.2 THE RUNGE-KUTTA METHOD
• Example: Tabulate the solution of 𝑦 ′ = 𝑥 2 + 𝑦 at the interval h=0.1. If y=-1 when x=0.
• Solution: Using Runge-Kutta formulas in Case 3, for the first increment, we have
𝑘1 = −0.1000, 𝑘2 = −0.1048, 𝑘3 = −0.1050, 𝑘4 = −0.1095, 𝑎𝑛𝑑 ∆𝑦 = −0.1048
Hence, 𝑦1 = 𝑦0 + ∆𝑦 = −1.1048
For the second increment, we have
𝑘1 = −0.1095, 𝑘2 = −0.1137, 𝑘3 = −0.1139, 𝑘4 = −0.1179, 𝑎𝑛𝑑 ∆𝑦 = −0.1138
Hence, 𝑦2 = 𝑦1 + ∆𝑦 = −1.2186
For the third increment, we have
𝑘1 = −0.1179, 𝑘2 = −0.1215, 𝑘3 = −0.1217, 𝑘4 = −0.1250, 𝑎𝑛𝑑 ∆𝑦 = −0.1216
Hence, 𝑦3 = 𝑦2 + ∆𝑦 = −1.3402
This process can be continued as desired.
 7.2 THE RUNGE-KUTTA METHOD
• Example: Tabulate the solution of 𝑦 ′ = 𝑥 2 + 𝑦 at the interval h=0.1. If y=-1
when x=0.
• Solution:
We can calculate one more value of y, this time using Milne’s method, which
we can now apply since we have 𝑦0, 𝑦1, 𝑦2, and 𝑦3. To do this, we must first
compute 𝑦′0, 𝑦′1, 𝑦′2, and 𝑦′3 from the differential equation, getting
𝑦′0 = −1.0000, 𝑦 ′1=-1.0948, 𝑦 ′ 2 = −1.1786,𝑦 ′ 3 = −1.2502
4ℎ
With this value, we can use 𝑦𝑛+1 = 𝑦𝑛−3 + 2𝑦 ′ 𝑛−2 − 𝑦 ′ 𝑛−1 + 2𝑦 ′ 𝑛
3
𝑦4 = −1.4682
Using the value, we find from the differential equation that
𝑦′4 = −1.3082
 7.2 THE RUNGE-KUTTA METHOD
• Example: Tabulate the solution of 𝑦 ′ = 𝑥 2 + 𝑦 at the interval h=0.1. If y=-1
when x=0.
• Solution:
4ℎ
We can use 𝑦𝑛+1 = 𝑦𝑛−3 + 2𝑦 ′
𝑛−2
− 𝑦 ′
𝑛−1
+ 2𝑦 ′
𝑛 to find 𝑦5 .
3
The differential equation 𝑦 ′ = 𝑥 2 + 𝑦 can be solved exactly without
recourse
to numerical methods. The required solution of the differential equation is
𝑦 = 𝑒 𝑥 − 𝑥 2 − 2𝑥 − 2
For x=1,2,3, and 4, this gives us the correct values:
𝑦1 = −1.1048, 𝑦2 = −1.2186, 𝑦3 = −1.3401, 𝑦4 = −1.4682