A tutorial on the Aitken convergence accelerator

By Cedrick Collomb

Abstract. This tutorial introduces the concept and usefulness of convergence accelerators
hopefully in a simple manner intelligible to any reader with minimal mathematical and
engineering skills. The Aitken convergence accelerator is derived as an example with both
intuitive explanations (every step is purposely made unnecessarily detailed for ease of
understanding) and a simple demonstration. Source code is provided as a way to skip the
potential formula obfuscation in order to help the reader implement and use the Aitken
convergence accelerator immediately after reading this document.

1. Introduction
There exists in Mathematics and Computer Science a large number of iterative algorithms
whose goal is usually to reach a solution to a problem within a certain tolerance within a
given number of iterations. Iterating means going over a pattern of steps and procedures
that can sometimes be complex and sometimes take a substantial amount of time even for
fast modern computers. Examples of common iterative algorithms are root solvers,
matrix inversion, linear equation solvers, and integrators. Each of these have deep
implications in domains such as Engineering, in general; Computer Graphics; Video
Games where runtime performance is critical; and also surprisingly in Internet search
engines for features such as page rank [4]. What if there existed methods to reach the
same result faster with less iteration? This is the purpose of convergence accelerators.
The rest of the tutorial is organized as follow: Section 2 introduces the Aitken formula
graphically. Section 3 introduces the Aitken formulas analytically. Section 4 gives an
example use of the Aitken formulas. Section 5 demonstrates that the Aitken formula
accelerates convergence under given conditions. Section 6 introduces iterative Aitken
formulas and Meyers iterated Aitken formulas. Section 7 extends Meyers formulas.
Section 8 concludes this tutorial.
2. A graphical explanation of the Aitken 2 accelerator2 formula
Imagine you have a convergent sequence ( xn )n iteratively defined by xn +1 = f ( xn ) ,
and that this sequence converges to x with f ( x ) = x .

Given an initial start point x0 , you can construct the sequence ( xn )n graphically. Figure
1 shows how to proceed. Place x0 on the x axis, find the point of the curve y = f ( x )

[email protected] / http://ccollomb.free.fr/
2
The Aitken accelerator being called process is due to the fact that in (3), the numerator is a difference

squared i.e. ( ) and the denominator is a difference of a difference i.e. as in ( ) .

2

that is vertically aligned with x0 , the y value of this point is f ( x0 ) . Since by definition

x1 = f ( x0 ) we now have found the value of x1 . In order to place x1 graphically on the x

axis, find the point of the curve y = x that is horizontally aligned with f ( x0 ) and project
this point vertically on the x axis. Repeating the same procedure will give you x2 , then
x3 , and you can continue to get as many terms of the sequence ( xn )n as needed.

Figure 1

y=x

y = f ( x0 ) + ( x x0 )
f ( x1 )

f ( x0 )

f ( x1 ) f ( x0 )
x1 x0

y = f ( x)

P1
P0

x0

x1

x2

Now lets focus on the first two points P0

x0
f ( x0 )

those two points as shown in Figure 1.

The equation of this line is y = f ( x0 ) + ( x x0 )
and f ( x1 ) by x2 gives
y = x1 + ( x x0 )

and P1

x1
f ( x1 )

, and the line going through

f ( x1 ) f ( x0 )
x1 x0
x2 x1
x1 x0

. Replacing f ( x0 ) by x1

(1)

The idea behind Aitken is to approximate x by the intersection of the line going through
P0 and P1 , with the line y = x . Finding the intersection is relatively simple and just
x x
require solving x = x1 + ( x x0 ) 2 1 , where the only unknown is x .
x1 x0

Regrouping the terms in x on the left side of the equal sign gives
x x
x x
x 1 2 1 = x1 x0 2 1 and after multiplying both sides of the equation by x1 x0
x1 x0
x1 x0

we find x ( ( x1 x0 ) ( x2 x1 ) ) = x1 ( x1 x0 ) x0 ( x2 x1 ) therefore we have

x=

x1 ( x1 x0 ) x0 ( x2 x1 )
( ( x1 x0 ) ( x2 x1 ) )

(2)

Subtracting and adding x0 ( x1 x0 ) in the middle of the numerator of (2) gives

x=

x1 ( x1 x0 ) x0 ( x1 x0 ) + x0 ( x1 x0 ) x0 ( x2 x1 )
which first two terms of the
( ( x1 x0 ) ( x2 x1 ) )

numerator can be factored by ( x1 x0 ) and which last two terms of the numerator can be

( x1 x0 ) + x0 ( ( x1 x0 ) ( x2 x1 ) )
factored by x0 which gives x =
.
( ( x1 x0 ) ( x2 x1 ) )
2
x1 x0 )
(
+ x . Therefore
Simplifying by ( ( x1 x0 ) ( x2 x1 ) ) gives x =
( ( x1 x0 ) ( x2 x1 ) ) 0
2

( x1 x0 )
x = x0 +
( ( x1 x0 ) ( x2 x1 ) )
2

(3)

This is the Aitken formula to find an approximate x of x given three consecutive terms
of a sequence.
3. An analytical explanation of the Aitken formulas
As for the previous section you have a convergent sequence ( xn )n iteratively defined

by xn +1 = f ( xn ) , and that this sequence converges to x with f ( x ) = x . Now lets admit

that f ( x ) has a Taylor development at x of order 2.

f ( x + h ) = f ( x ) + hf ( x ) + O ( h 2 )

(4)

We define x = x + h , therefore h = x x . Let = f ( x ) . Replacing x by x + h , h by

x x , f ( x ) by x , and f ( x ) by in (4), as well as neglecting the order two error

O (x x)

) gives

f ( x) = x + ( x x )

(5)

Note that we do not know x and also generally do not know in (5). Applying the
previous formula to two continuous elements of the sequence xi and xi +1 , gives two
equations with two unknowns x and .

xi +1 = f ( xi ) = x + ( xi x )
xi +1 = (1 ) x + xi

(6)

xi + 2 = f ( xi +1 ) = x + ( xi +1 x )
xi + 2 = (1 ) x + xi +1

(7)

Subtracting (6) from (7) gives xi + 2 xi +1 = ( xi +1 xi ) . Therefore gives us .

xi + 2 xi +1
xi +1 xi

(8)

and
that will come handy very soon.
1
1
x x
( x x ) ( xi + 2 xi +1 )
1 = 1 i + 2 i +1 = i +1 i
xi +1 xi
xi +1 xi
xi +1 xi
1
=
and
1 ( xi +1 xi ) ( xi + 2 xi +1 )
xi + 2 xi +1

Combining (8) and (9) gives

=
1 ( xi +1 xi ) ( xi + 2 xi +1 )

We use (8) to calculate

(9)
(10)

We use (6) and (9) to find a first formula for x .

Using (6) and solving for x gives x =

xi +1 xi
1

(11)

xi +1 xi + xi xi
1
x x + x (1 )
Factoring the right side of the numerator gives x = i +1 i i
1
x x
Simplifying gives x = i +1 i + xi
1
x x
Therefore x = xi + i +1 i and using (9) gives
1
Subtracting and adding xi gives x =

( xi +1 xi )
( xi +1 xi ) ( xi + 2 xi +1 )
2

x = xi +

(12)

Using (11) in a different way leads to the second expression for x .

xi +1 xi +1 + xi +1 xi
1
1

x
+

x
(
) i +1 ( i +1 xi )

Subtracting and adding xi +1 in (11) gives x =

Factoring the numerator gives x =
Simplifying gives x = xi +1 +

x = xi +1 +

( xi +1 xi )
, and using (10) gives
1

( xi + 2 xi +1 )( xi +1 xi )
( xi +1 xi ) ( xi +2 xi +1 )

(13)

The third expression for x comes from (7).

xi + 2 xi +1
1
x xi + 2 + xi + 2 xi +1
Subtracting and adding xi + 2 gives x = i + 2
1
(1 ) xi + 2 + ( xi + 2 xi +1 )
Factoring the numerator gives x =
1
( xi + 2 xi +1 )
Simplifying gives x = xi + 2 +
1
Using (7) and solving for x gives x =

Using (10) gives the most common formulation of the Aitken accelerator.

( xi + 2 xi +1 )
x = xi + 2 +
( xi +1 xi ) ( xi + 2 xi +1 )
2

(14)

To resume there are three equivalent ways to express x , equations (13) and (14) could
have been directly expressed from (12) but it was worth deriving from the original
equations for pedagogic reasons. The high level item that is very important to understand
is that the Aitken accelerator gives an approximate of the solution of your problem ( x )
based on three consecutive terms of a convergent sequence.

( xi +1 xi )
(12) x = xi +
( xi +1 xi ) ( xi + 2 xi +1 )
( x x )( x x )
(13) x = xi +1 + i + 2 i +1 i +1 i
( xi +1 xi ) ( xi +2 xi +1 )
2
( xi + 2 xi +1 )
(14) x = xi + 2 +
( xi +1 xi ) ( xi + 2 xi +1 )
2

4. Example of use of the Aitken accelerator

The Newton method is a well known method to find roots of the equation g ( x ) = 0 . The
iterative procedure is given by f ( x ) = x

g ( x)
. An extension of the Newton method
g ( x)

described in [8] is the Steffensen method that uses for every three iterations two iterations
of the Newton method f and the Aitken formula (14) on the last three elements of the
sequence for the last iteration.

5. A simple demonstration
Definition: (Tn ) n is an accelerating sequence3 of ( Sn )n if the following properties are
true: (Tn ) n converges to the same limit S as ( Sn )n , (Tn ) n converges faster than

( Sn )n

with the following criterion lim

Tn S
=0.
Sn S

Theorem: Let ( Sn )n be a converging sequence to S , verifying lim

n

 1;1 . The sequence (Tn )n defined by

accelerating sequence of ( Sn )n .

S n +1 S
= with
Sn S

( Sn +1 Sn )
,
Tn = S n +
( Sn +1 Sn ) ( Sn +2 Sn+1 )
2

is an

Demonstration:
a. Lets demonstrate that lim Tn = S
n

Adding S + S in all parenthesis and dividing numerator and denominator by S n +1 S

S Sn
1 +
( Sn +1 S n )
S n +1 S

gives Tn = S n +
simplifying and taking the limit when

S Sn Sn+ 2 S
1
1 +

S n +1 S S n +1 S

1
1
n gives lim Tn = S + lim ( Sn +1 Sn ) since 1 , the denominator is not
n
n
1
1 +1

zero therefore (Tn ) n

converges and lim Tn = S .

n

Using the same notations and definitions as in [2].

b. Lets demonstrate that (Tn ) n converges faster than ( Sn )n .

( Sn +1 Sn )
( Sn +1 S n )
Sn S +
( Sn+1 Sn ) ( Sn+ 2 Sn +1 ) = 1 + ( Sn +1 Sn ) ( Sn+ 2 Sn+1 )
=
2

Tn S
Sn S

Sn S

Sn S

Adding S + S in all parenthesis and dividing numerator and denominator by S n +1 S

S S n
S n +1 S
1 +
1 +

S n +1 S
Sn S
T S
gives n
= 1+
therefore
Sn S

S Sn
Sn+ 2 S
1 +
1 +

S n +1 S
S n +1 S
1
2
1 ( 1 + )
1)
Tn S
(

lim
= 1+
= 1 2
=0
n S S
1

+
2
1
n
2

Therefore (Tn ) n converges faster than ( Sn )n .

6. Iterative Aitken Accelerator

One might wonder why should we stop there? The Aitken accelerator gives an estimate
of the limit of a convergent sequence given three consecutive terms, what if we created a
sequence of the Aitken estimations and applied the Aitken formula on that sequence?
What if we created a convergent sequence that used the Aitken accelerator to converge
faster to the limit? Those are great ideas and it is indeed possible to use the Aitken
iterator iteratively to accelerate the convergence further.
a. Simple iterative method.
Lets define the sequence ( x

i
n n

result of creating ( xni )

i 1
n+ 2

by x = x
i
n

(x

i 1
n +1

(x

i 1
n+ 2

xni +11 )

xni 1 ) ( xni +12 xni +11 )

which is the

by applying equation (14) to three consecutive terms of

( x ) . Following the assumption that ( x ) converges, it is reasonable to believe

that ( x )
will also converge and will converge faster. This is reasonable but although
i 1
n
n

i 1
n
n

i
n n

it works in some instances it is very important to realize it is not always true.

So this is a very simple way to iteratively apply the Aitken accelerator. The advantage of
this method is that you can get closer to the limit with few more arithmetic operations.
The other advantage is that you can apply this process without the need of using f nor a

sequence ( xn )n iteratively defined by xn +1 = f ( xn ) . The disadvantage is that you need

to store a certain number of values of ( xni 1 )

(x )

i
n n

that will be used to define the sequence

Table 1 shows the result of the iterative Aitken accelerator based on 5 steps of the
9x ex
sequences defined by f ( x ) =
which limit is LambertW (1) 4 . Reaching the
10
same accuracy without the iterated Aitken accelerator would require calculating x33 .

n
0
1
2
3
4
Error

(x )

Table 1

(x )

1
n n

n n

0.00000
-0.10000
-0.18048
-0.24592
-0.29953
0.26762

(x )

2
n n

-0.51239 -0.56476
-0.53054
-0.54239

0.02475

0.00238

Table 2 shows the result of the iterative Aitken accelerator based on 5 values of the serie
n

defined by xn =

( 1)

that converges to

. Reaching the same accuracy without the

(1 + 2 j )
4
iterated Aitken accelerator would require calculating x1951 .
j =0

n
0
1
2
3
4
Error

(x )

Table 2

n n

1.00000
0.66666
0.86666
0.72380
0.83492
0.04952

(x )

1
n n

(x )

2
n n

0.79166 0.78552
0.78333
0.78630

0.00091 0.00001

b. Meyers iterative method

First let us rewrite (13) by replacing xi + 2 by f ( xi +1 ) , and by replacing xi +1 by f ( xi ) . It
shows the Aitken formula in another form.
4

See [9] for more details on the Lambert W function.

x = xi +1 +

( f (x ) x )
( f (x ) x )
( f (x ) x )( f (x ) x )
i

i +1

i +1

(15)

i +1

i +1

Let us now construct a sequence ( yn ) n .

y0 is the iteration guess start value, chosen sometimes randomly or with a rough
estimation depending the problem you are trying to solve.
y1 is the first iteration and defined for the original sequence by y1 = f ( y0 ) . Lets define
a0 = 1 then y1 = f ( y0 ) = y0 + f ( y0 ) y0 = y0 + a0 ( f ( y0 ) y0 ) .

Lets apply (15) to y0 and y1 to determine y2 .

( f (y ) y )
( f (y ) y )
( f (y ) y )( f (y ) y )
( f (y ) y )
If we define a =
and use the fact that a
( f (y ) y )( f (y ) y )
y = y +a a ( f (y ) y )
y y = a a ( f (y ) y )
y2 = y1 +

1 0

1 0

= 1 then

(16)

We want to determine y3 by using (13) to y1 , y2 and another value y2 . An interesting

value for y2 is given by

y2 = y2 + a1a0 ( f ( y2 ) y2 )
y2 y2 = a1a0 ( f ( y2 ) y2 )

(17)

The idea is to interpolate or extrapolate in the direction f ( y ) y by the same distance

a1a0 used for y2 .
Using (13) to y1 , y2 and y2 gives y3 = y2 +

( y y )( y y )
(y y )(y y )
2

Using (16) and (17) gives y3 = y2 +

( a a ( f ( y ) y )) ( a a ( f ( y ) y ))
( a a ( f ( y ) y )) ( a a ( f ( y ) y ))
1 0

1 0

1 0

1 0

Simplifying numerator and denominator by a1a0 gives

( f ( y1 ) y1 )
a a ( f ( y2 ) y2 )
( f ( y1 ) y1 ) ( f ( y2 ) y2 ) 1 0
( f ( y1 ) y1 )
If we define a2 =
then y3 = y2 + a2 a1a0 ( f ( y2 ) y2 )
( f ( y1 ) y1 ) ( f ( y2 ) y2 )
y3 = y2 +

 i 1
Repeating the exact same steps to create yi = yi + ai ( f ( yi ) yi ) and applying (13)
i =0
to yi 1 , yi and yi gives the Meyers iterative Aitken formulas.
i

yi +1 = yi + ai ( f ( yi ) yi )
i =0
( f ( yi1 ) yi 1 )
With ai =
for 1 i and a0 = 1 .
( f ( yi1 ) yi 1 ) ( f ( yi ) yi )

(18)

It is very important to realize here that it has not been demonstrated that ( yn ) n
is convergent, nor that it respects the necessary conditions to be accelerated by
the Aitken formula. Although in practice the Meyers formula works really well
in many cases and its convergence is in many case very impressive, it might be
necessary to verify that it applies to the problem at hand. Nevertheless the
advantage of this method is that it does not require storing all previous values of
the sequence and it does not require more use of the function f .

Table 3 compares the results of the normal sequence, the iterative Aitken sequence and
the Meyers method, based on 5 values of the sequence defined by f ( x ) = cos ( x ) that
converges to approximately 0.73908513. Reaching the same accuracy without Meyers
method, requires computing x43 with basic iterations and x16 with the iterative Aitken.

N
0
1
2
3
4
Error

(x )

n n

Table 3

(x )

2
n n

(y )

n n

0.36235775 0.73727953 0.36235775

0.93506365
0.70250260
0.59376721
0.74284646
0.82883886
0.73905422
0.67573215
0.73908511
0.06335298 0.00180561 0.25620330 107

c. Other iterative ideas that do not work as well

There are several creative ways you can try to iterate with the Aitken accelerator.
Following are some examples of ideas that do not work really well in practice.
Reusing the method presented in 6.a by using (15) instead of (14). It accelerates the
convergence however in most case the method presented in 6.a converges faster.
10

Moreover this method requires calling the function g during the iterations, and it will
require n iterations to reach the final value.
Another method is based on using (15) on two preceding values of the sequence. This
method works well in practice and has the same advantages as the Meyers method, no
storage is required and no additional calls to f are required. However in most cases the
Meyers method converges much faster.
A last method that might sound reasonable at first but is flawed is to use (14) on three
preceding values of the sequence. Although this sounds good at first, it does not use the
function g and therefore is not likely to converge on the right value. Imagine that two
sequences have the same initial three values, with this method they would end-up having
the same calculated wrong limit.

7. Using Meyers method to solve g ( x ) = 0

Creating a function f that has for fixed point x = f ( x ) the root of g is relatively easy
if we define f by the following f ( x ) = x + w ( x ) g ( x ) , w ( x ) needs to be picked so that

the sequence converges around x .

The Meyers formulas (18) applied to f following this definition give:

yi +1 = yi + ai ( w ( yi ) g ( yi ) )
i=0
With ai =

(w( y

i 1

( w ( y ) g ( y ))
) g ( y )) ( w ( y ) g ( y ))
i 1

i 1

i 1

(19)

for 1 i and a0 = 1 .

Most of the time w ( x ) = 1 will be good enough to solve g .

8. Conclusion
A well known result in the domain of data compression is that there is no single
compression algorithm that can compress all data. In other words for a given compression
algorithm there exist some data that can not be compressed by the algorithm. Delahaye
and Germain-Bonne [5] have shown a very similar result in the domain of convergence
acceleration by proving that several families of sequences have no algorithm accelerating
the convergence of every sequence of the family.
Therefore, as with the multitude of compression methods there are also several other
convergence accelerations methods such as Eulers transformation, Wynns algorithm,
Brezinskis algorithm, Levins transforms, and extrapolation methods.

11

The positive side of this incomplete solution is that there is always an interest to discover
new acceleration formulas for sequences, and the reader of this tutorial can take part in
this research effort.

9. References
1. C. Brezinski, Algorithmes d'Acclration de la Convergence, tude Numrique,
Editions Technip (1978)
2. C. Brezinski, M.R. Zaglia, Extrapolation Methods, Studies in Computational
Mathematics, North Holland (1990)
3. A. Meyers, Accelerating Aitkens convergence accelerator, World Wide Web site at
the address http://www.tm.bi.ruhr-uni-bochum.de/profil/mitarbeiter/meyers/aitken.pdf
4. S.D. Kamvar, et al., Extrapolation Methods for Accelerating PageRank
Computations, World Wide Web site at the address
http://www.stanford.edu/~sdkamvar/papers/extrapolation.pdf, (2003)
5. J. P. Delahaye and B. Germain-Bonne, Resultats negatifs en acceleration de la
convergence, Numer. Math, Vol. 35 (1980), pp. 443-457,
6. X. Gourdon and P. Sebah, Convergence acceleration of series, World Wide Web site at
the address
http://numbers.computation.free.fr/Constants/Miscellaneous/seriesacceleration.html,
(2002)
7. David A. Smith; William F. Ford, Numerical Comparisons of Nonlinear Convergence
Accelerators, Mathematics of Computation, Vol. 38, No. 158. (Apr., 1982), pp. 481-499
8. J. Mathews, K. Fink, Numerical Methods Using MATLAB 3rd edition, Prentice Hall,
(1999)
9. Robert M. Corless, et al., On the Lambert W Function, Advances in Computational
Mathematics, volume 5 (1996), pp. 329-359.

10. Appendix. Non optimized C++ code.

// Returns an estimate of the limit given three consecutive terms of a sequence
float AitkenExtrapolation( const float x0, const float x1, const float x2 )
{
float d1 = x1 - x0;
float d2 = x2 - x1;
return x0 + d1 * d1 / ( d1 - d2 );
}
// Produce another sequence of n-2 terms using the Aitken extrapolation
void IteratedAitken( float* const dst, const float* const src, const long n )
{
for ( long i = 0; i < n - 2; i++ )
{
dst[ i ] = AitkenExtrapolation( src[ i ], src[ i + 1 ], src[ i + 2 ] );
}
}
// Typedef used for Meyers functions
typedef float (*pSimpleFunction)( float x );

12

// Fixed point iteration using Meyers method. To accelerate Newton method use f=x-u(x)/u'(x)
float MeyersFixedPoint( pSimpleFunction f, const float startPoint, const long maxIteration, const
float epsilon )
{
long iter = maxIteration;
float res = f( startPoint ); // First iteration
float last = res - startPoint;
float prod = 1.0f;
float step = FLT_MAX;
while ( ( 1 < iter ) && ( epsilon <= fabsf( step ) ) ) // One iteration has already been done
{
float delta = f( res ) - res;
float diff = last - delta;
prod *= last / diff;
step = prod * delta;
res += step;
last = delta;
iter--;
}
return res;
}
// Root finding using Meyers method. To accelerate Newton method use g=u(x)/u'(x)
float MeyersRoot( pSimpleFunction g, const float startPoint, const long maxIteration, const float
epsilon )
{
long iter = maxIteration;
float last = g( startPoint );
float res = startPoint + last; // First iteration
float prod = 1.0f;
float step = FLT_MAX;
while ( ( 1 < iter ) && ( epsilon <= fabsf( step ) ) ) // One iteration has already been done
{
float delta = g( res );
float diff = last - delta;
prod *= last / diff;
step = prod * delta;
res += step;
last = delta;
iter--;
}
return res;
}

13