Sie sind auf Seite 1von 5

# Convergence Rates on Root Finding

Com S 477/577
Oct 5, 2004

A sequence xi R converges to if for each > 0, there exists an integer N () such that
|xl | > for all l N (). The Cauchy convergence criterion states that a sequence xi R
is convergent if and only if for each > 0 there exists an N () such that |xl xm | < for all
l, m N ().
Let a sequence xi R be generated by an iteration function , that is,
xi+1 = (xi ),

i = 0, 1, 2, . . .

Let be a fixed point of , that is, = (). Suppose that the sequence {xi } is generated in the
neighborhood of . The corresponding iteration method is said to be of at least pth order if there
exists a neighborhood N () of such that for all x0 N () the generated sequence xi+1 = (xi ),
i = 0, 1, . . ., satisfies
|xi+1 | C|xi |p ,
where C < 1 if p = 1. In the case of first order convergence, for instance, we have
|xi | C|xi1 | C 2 |xi2 | C i |x0 |.
Since C < 1, it follows that
lim |xi | = lim C i |x0 | = 0,

## namely, the sequence {xi } will converge to .

Now suppose is sufficiently often differentiable in N (). If xi N () and if (k) () = 0 for
k = 1, 2, . . . , p 1 but (p)() 6= 0, that is, is a zero of order p, then it follows from the Taylor
expansion that
xi+1 = (xi )
= () +



(p) ()
(xi )p + O (xi )p+1 .
p!

Because () = , we obtain
xi+1
(p) ()
=
.
i (xi )p
p!
lim

## For p = 2, 3, . . ., the method is of (precisely) pth order.

The method is of first order if p = 1 and | ()| < 1. When 0 < () < 1, the sequence {xi }
will converge monotonically to as shown in the left figure below. When 1 < () < 0, the
sequence will alternate about during convergence as shown in the right figure.
1

( x)

( x)

xi 0

x i+1

x i +2

xi

x i +2

x i+1

In the below we study the convergence rates of several root finding methods introduced before.

## Newtons method has the iteration function

f (x)
(x) = x
f (x)

with

f () = 0.

Suppose f is sufficiently continuously differentiable in some neighborhood N (). In the nondegenerate case, f () 6= 0. So we have
() = ,

2

() = 1


2

f (x)
x=
= 0,

since f () = 0,


2


2

## f (x) f (x)f (x)

2f (x)f (x) f (x)f (x) f (x)f (x) f (x) 2f (x)f (x)

() =


4

f (x)
x=

3

2

2
f () f () f () f () f () + 2f ()f () f ()
=

4
f ()

3
f () f ()
,
since f () = 0
=

4
f ()
f ()
f ()
6
=
0.

## So Newtons method is quadratically convergent.

In the degenerate case, is an m-fold zero of f , for some m > 1, that is, f (i) () = 0, for
i = 0, 1, . . . , m 1. We will leave to the students to determine the order of convergence in this case.
2

## Linear Convergence of Regula Falsi

For clarity of analysis we let xi = bi for all i. We make some simplification assumptions for the
discussion of the convergence behavior: f exists and for some k the following conditions hold: (a)
xk < ak ; (b) f (xk ) < 0 and f (ak ) > 0; (c) f (x) 0 for all x [xk , ak ].

x k +1

xk

ak

(a)

(b)

Under these assumptions, either f (xk+1 ) = 0 or f (xk+1 )f (xk ) > 0 and consequently
xi < xi+1 < ai+1 = ai .
To see this, use the remainder formula for polynomial interpolation at xk and ak :
f (x) p(x) = (x xk )(x ak )

f ()
2

for x [xk , ak ] and a suitable [xk , ak ]. Under condition (c), the above equation implies that
f (x) p(x) 0. In particular, f (xk+1 ) p(xk+1 ) 0, which in turn implies that f (xk+1 ) 0 since
p(xk+1 ) = 0.
Unless f (xk+1 ) = 0, in which case the iteration stops at xk+1 , we can see that conditions (a),
(b), and (c) hold for all i k. Therefore ai = ak = a and
xi+1 =

af (xi ) xi f (a)
f (xi ) f (a)

for all i k. Furthermore, {xi } for i k form a monotone increasing sequence bounded by a. So
limi xi = exists. Consequently,
f () 0,

f (a) > 0,

and

af () f (a)
,
f () f (a)

which gives
( a)f () = 0.
But < a since f () 0 < f (a). Hence f () = 0 and {xi } converges to a zero of f .
The above discussion enables us to look at the order of convergence through the iteration
function
af (x) xf (a)
xi+1 = (xi ),
where
(x) =
.
f (x) f (a)
3

## Since f () = 0, we obtain that



af () f (a) f (a) + f (a)f ()
a
() =
= 1 f ()
.
2
f (a)
f () f (a)
By the mean value theorem, there exist 1 , 2 such that
f () f (a)
a
f (xi ) f ()
xi

=
=

f (a)
= f (1 ),
a
f (xi )
= f (2 ),
xi

< 1 < a;

(1)

xi < 2 < .

(2)

## Since f (x) 0, f (x) increases monotonically in [xi , a], So f (2 ) f () f (1 ). Meanwhile,

condition (2), xi < , and f (xi ) < 0 together imply that 0 < f (2 ). Therefore 0 < f () f (1 ).
We have thus shown that
f ()
< 1.
0 () = 1
f (1 )
So the regula falsi method converges linearly.
From the previous discussion we see that the method of regula falsi will almost always end up
with the one-sided convergence demonstrated before.

## In secant method, the iteration is in the form

xi+1 = xi

f (xi )
,
f [xi1 , xi ]

i = 0, 1, . . .

(3)

To determine the local convergence rate, without loss of generality we assume that the sequence
{xi } is in a small enough neighborhood of the zero and that f is twice differentiable. Subtract
from both sides of (3):
f (xi )
xi+1 = (xi )
f [xi1 , xi ]


f (xi )
f [xi , ]
f (xi ) f ()
=
= (xi ) 1
,
since f [xi , ] =
f [xi1 , xi ]
xi
xi
f [xi1 , xi ] f [xi , ]
= (xi )(xi1 )
(xi1 )f [xi1 , xi ]
f [xi1 , xi , ]
= (xi )(xi1 )
.
f [xi1 , xi ]
From error estimation of polynomial interpolation, we learned that
f [xi1 , xi ] = f (1 ),
1
f (2 ),
f [xi1 , xi , ] =
2
4

1 I[xi1 , xi ];
2 I[xi1 , xi , ],

(4)

where I[xi1 , xi ] is the smallest interval containing xi1 and xi , and I[xi1 , xi , ] the smallest
interval containing xi1 , xi ,
n

If is a simple zero, that is, f () 6= 0, there exists a bound M and an interval J = x |x|
o
for some > 0 such that

1 f (2 )

(5)
2 f (1 ) M,
for any 1 , 2 J.
Let ei = M |xi | and e0 , e1 < min{1, M }. By induction and using (4) and (5) we can easily
show that
ei ei1
ei+1 = M |xi+1 | M

M = ei ei1 ,
M M
and
|ei | min{1, M },
for i = 1, 2, . . ..
Let q = (1 + 5)/2 be the root of the equation z 2 z 1 = 0. Then we have
i

ei K q ,

where K = max{e0 ,

i = 0, 1, 2, . . .

q
e1 } < 1. This is because (by induction)
i

ei+1 ei ei1 K q K q

i1

= Kq

i1 (q+1)

= Kq

i1 q 2

= Kq

i+1

Thus the secant method converges at least as well as a method of order p = 1+2 5 = 1.618 . . ..
One-step secant requires one additional function evaluation. But one-step Newton requires two
(f and f ). Therefore two secant steps are as expensive as single Newton step. But two secant
steps has a convergence order of (1.618)2 2.618. This explains why in practice the secant method
always dominates Newtons method with numerical derivatives.

References
 J. Stoer and R. Bulirsch. Introduction to Numerical Analysis. Springer-Verlag New York, Inc.,
2nd edition, 1993.
 M. Erdmann. Lecture notes for 16-811 Mathematical Fundamentals for Robotics. The Robotics
Institute, Carnegie Mellon University, 1998.
 W. H. Press, et al. Numerical Recipes in C++: The Art of Scientific Computing. Cambridge
University Press, 2nd edition, 2002.