Cme4041 04

SECANT METHOD
Newton’s method is an extremely powerful technique, but it has a

major weakness: the need to know the value of the derivative of f
at each approximation. Frequently, f 0 (x) is far more difficult and
needs more arithmetic operations to calculate than f (x).
√
√ 0 cos x
f (x) = sin x ⇒ f (x) = √
2 x
To circumvent the problem of the derivative evaluation in

Newton’s method, we introduce a slight variation.
Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 1/ 46

Definition (Derivative Function):
The derivative of the function f (x) with respect to the variable x

is the function f 0 whose value at x is
f (x + h) − f (x)
f 0 (x) = lim
h→0 h
Provided the limit exists.
Alternative formula for the derivative
f (z) − f (x)
f 0 (x) = lim
z→x z−x

h→0 tan(β) → tan(α)
f (x0 +h)−f (x0 )
where tan(β) = x0 +h−x0 and tan(α) = f 0 (x0 )

When h is sufficiently small (it means x0 and x0 + h are very close
to each other)
f (x0 + h) − f (x0 )
≈ f 0 (x0 )
x0 + h − x0
If we set x0 + h = xn and x0 = xn−1 in the above formula, we get

an approximation to f 0 (xn ) for the Newton iteration.
f (xn ) − f (xn−1 )
≈ f 0 (xn )
xn − xn−1

When this replacement is made, the resulting algorithm is called
the SECANT METHOD. Its formula is

xn − xn−1
xn+1 = xn − f (xn ) n≥1
f (xn ) − f (xn−1 )
Since the calculation of xn+1 requires xn and xn−1 , two initial

points must be prescribed at the beginning.
However, each new xn+1 requires only one new evaluation of f .

The graphical interpretation of the secant method is similar to that
of Newton’s method. The tangent line to the curve is replaced by
a secant line.

ERROR ANALYSIS
For secant method with en = xn − r,the order of convergence of
the secant method,
√
|en+1 | ≈ A|en |(1+ 5)/2
where,
0.62
f 00 (r)

A=
2f 0 (r)

√
|en+1 | ≈ A|en |(1+ 5)/2
√
Since (1 + 5)/2 ≈ 1.62 < 2, the rapidity of convergence of the
secant method is not good as Newton’s method but better than
the bisection method.
However, each step of the secant method requires only one new
function evaluation, while each step of the Newton algorithm
requires two function evaluations, namely f (x) and f 0 (x).

Since function evaluations constitute the principal computational
burden in these algorithms, a pair of steps in the secant method is
comparable to one step in the Newton method.
|en+2 | ∼ A|en |α
2
√
∼ A1+α |en |α = A1+α |en |(3+ 5)/2
This is considerably better than

√ the quadratic convergence of the
Newton method since (3 + 5)/2 ≈ 2.62. Of course, two steps of
the secant method would require more work per iteration.

Example:
Find an approximation to root of the following function using

Secant method.
f (x) = cos(x) − x
Since f (0).f (π/2) < 0 has a root in [0, π/2] Let’s write Secant
method iteration for this function

xn − xn−1
xn+1 = xn − f (xn ) n≥1
f (xn ) − f (xn−1

xn − xn−1
xn+1 = xn − (cos(xn ) − xn )
(cos(xn ) − xn ) − (cos(xn−1 ) − xn−1 )
we need x0 and x1

Choose x0 = 0.5 and x1 = π/4 for n = 1

x1 − x0
x2 = x1 − (cos(x1 ) − x1 )
(cos(x1 ) − x1 ) − (cos(x0 ) − x0 )
π
4 − 0.5

π π π
x2 = − (cos( ) − )
4 4 4 (cos( π4 ) − π4 ) − (cos(0.5) − 0.5)
x2 = 0.736384

for n = 2,

x2 − x1
x3 = x2 − (cos(x2 ) − x2 )
(cos(x2 ) − x2 ) − (cos(x1 ) − x1 )
x3 = 0.739058

step(n − 1) xn f (xn )
1 0.7363841388 0.00451772
2 0.7390581392 0.0000451772
3 0.7390851493 −2.69822 × 10−8
4 0.7390851332 1.60871 × 10−13
5 0.7390851332 0.
6 0.7390851332 0.

For the same problem if we use Newton-Raphson Method with
starting point x0 = π/4
step xi f (xi )
1 0.7395361335 -0.000754875
2 0.7390851781 −7.51299 × 10−8
3 0.7390851332 −7.77156 × 10−16
4 0.7390851332 0.
5 0.7390851332 0.

FIXED POINTS AND FUNCTIONAL
ITERATION
Newton’s method example of procedures whereby a sequence of
points is computed by a formula of the form
xn+1 = F (xn ) n≥0 (∗)

The algorithm defined by such an equation is called functional
iteration. In Newton’s method, the function F is given by
f (x)
F (x) = x − (**)
f 0 (x)

Formula ** can be used to generate sequences that do not
converge, as for example the sequence 1, 3, 9, 27, . . . , which arises
if x0 = 1 and F (x) = 3x. However, our interest lies mainly in
those cases where lim xn exists. Suppose, then, that

n→∞
lim xn = s
n→∞
What is the relation between s and F ?

If F is continuous, then
F (s) = F ( lim xn ) = lim F (xn ) = lim xn+1 = s

n→∞ n→∞ n→∞
Thus, F (s) = s, and we call s a fixed point of the function F .

Example:
The function g(x) = x2 − 2 for −2 ≤ x ≤ 3, has fixed point at

x = 1 and x = 2 since g(−1) = (−1)2 − 2 = −1 and
g(2) = 22 − 2 = 2.

Given a root finding problem f (p) = 0, we can define functions g
with a fixed point at p in a number of ways, for example, as
g(x) = x − f (x) or as g(x) = x + 3f (x).
Conversely, if the function g has a fixed point at p, then the

function defined by
f (x) = x − g(x)
has a zero at p.

Example:
Consider solving the equation
x2 − 5 = 0
−x + x2 − 5 = 0 − x
multiply by −1
x = 5 + x − x2
fixed point for −x2 + x + 5 functional iteration
xn+1 = 5 + xn − x2n (1)

x2 − 5 = 0
1 2 1
x =5
x x
5
x=
x
fixed point for x5 , functional iteration
5
xn+1 = (2)
xn

x2 − 5 = 0
x2
5( − 1) = 0
5
x2
−x + −1=0−x
5
2
multiplying by −1 gives x = 1 + x − x5
2
fixed point for 1 + x − x5 . Functional iteration:
x2n
xn+1 = 1 + xn − (3)
5

x2 − 5 = 0
x2 + x2 − 5 = 0 + x2
1 2 1
2x = (x2 + 5)
2x 2x
1 5
x = (x + )
2 x
fixed point for 21 (x + x5 ). Functional iteration:
1 5
xn+1 = (xn + ) (4)
2 xn

Which one is convergent?
xn+1 = 5 + xn − x2n
5
xn+1 =
xn
x2n
xn+1 = 1 + xn −
5
1 5
xn+1 = (xn + )
2 xn

We first need to become comfortable with this new type of problem
and to decide when a function has a fixed point and how the fixed
points can be approximated to within a specified accuracy.

Definition:
A mapping (or function) F is said to be contractive if there exists
a number λ less than 1 such that
|F (x) − F (y)| ≤ λ|x − y|

for all points x and y in the domain of F . The distance between x
and y is mapped into a smaller distance between F (x) and F (y)
by the contractive function F .

Graphical example of contractive mapping.

Theorem:
Let F be a contractive mapping of a close set C ⊂ R into C.
Then F has a unique fixed point. Moreover, this fixed point is the
limit of every sequence obtained from equation (xn+1 = F (xn ))
with any starting point x0 ∈ C.

Example:
Prove that the sequence [xn ] defined recursively as follows is

convergent.
x0 = −15
1
xn+1 = 3 − |xn | n≥0
2
The function F (x) = 3 − 21 |x| is a contraction because

1 1
|F (x) − F (y)| = 3 − |x| − 3 + |y|

2 2
1 1
= ||y| − |x|| ≤ |y − x|
2 2
So F (x) has a fixed point, it means [xn ] converges.

Theorem:
Assume g(x) and g 0 (x) are continuous for a ≤ x ≤ b and assume
a ≤ g(x) ≤ b for a ≤ x ≤ b. Further assume that
λ ≡ max |g 0 (x)| < 1

a≤x≤b
There is a unique solution α of x = g(x) in the interval [a, b].

For any initial estimate x0 in [a, b], the iterates xn will converge to
α.
λn
|α − xn | ≤ |x0 − x1 |, n≥0
1−λ
for xn close to α α − xn+1 ≈ g 0 (α)(α − xn )

We can examine the iterations (1), (2), (3), (4).
For (1);
g(x) = 5 + x − x2
we have g 0 (x) = 1 − 2x. Since g(2).g(3) < 0, g has a root in [2, 3],
√
Thus max |g 0 (x)| = 5 > 1. Hence iteration will not converge 5.
For (2);
5
g(x) =
x
−5
we have g 0 (x) = x2
. Since g(2).g(3) < 0, g has a root in [2, 3],
Thus max |g 0 (x)| = | −5
4 | > 1. Hence iteration will not converge
√
5.

For (3);
1
g(x) = 1 + x − x2
5
we have g 0 (x) = 1 − 25 x. Since g(2).g(3) < 0, g has a root in [2, 3],
√
Thus max |g 0 (x)| = 0.2 < 1. Hence iteration will converge 5.
For (4);
1 5
g(x) = (x + )
2 x
we have g 0 (x) = 12 (1 − 5
x2
). Since g(2).g(3) < 0, g has a root in
[2, 3],
√
Thus max |g 0 (x)| = 0.22 < 1. Hence iteration will converge 5.

Example:
Find the fixed point of cos x in first quadrant.

We must solve cos x = x. It is clear that solution will be in [0, π/2].
Functional iteration is
xn+1 = cos xn

Since (cos x)0 = − sin x we have maxπ | − sin x| < 1 Therefore
x∈(0, 2 )
xn+1 = cos xn
will converge, for any start point x0 ∈ (0, π2 ).
Choose x0 = 1
n=0 ⇒ x1 = cos x0 = cos 1 = 0.540302

n=1 ⇒ x2 = cos x1 = cos(0.540302) = 0.857553
..
.

System of Nonlinear Equations
Newton’s method for systems of nonlinear equations follows the

same strategy that was employed for a single equation. Thus we
linearize and solve. Let us illustrate with a pair of equations
involving two variables
f (x, y) = 0
g(x, y) = 0

System of nonlinear equations

x0
The Newton method iteration for systems with starting point y0

xn+1 xn −1 f (xn , yn )
= −J (xn , yn ) , n≥0
yn+1 yn g(xn , yn )
where
fx (xn , yn ) fy (xn , yn )
J(xn , yn ) =
gx (xn , yn ) gy (xn , yn )

We shall require J to be nonsingular, i.e. det(J) 6= 0.

Or we solve the following linear system for xn+1 and yn+1

xn+1 xn f (xn , yn )
J(xn , yn ) − =−
yn+1 yn g(xn , yn )

Example
Find the
approximate
roots of the following system with starting
2
point 2
x2 + y 2 = 18
x−y =0
Here f (x, y) = x2 + y 2 − 18 and g(x, y) = x − y. Thus

fx fy 2x 2y
J= =
gx gy 1 −1

2
det(J) = −2x − 2y 6= 0 for starting point
2

So
−1 1 −1 −2y
J =
−2x − 2y −1 2x
Lets start Newton iteration

x1 x0 −1 f (x0 , y0 )
n=0 = − J (x0 , y0 )
y1 y0 g(x0 , y0 )
2
x0 + y02 − 18

x1 x0 1 −1 −2y0
= + 0.5
y1 x0 + y0 −1 2x0
y0 x0 − y0
2
2 + 22 − 18

x1 2 1 −1 −2 × 2
= + 0.5
y1 2 2 + 2 −1 2 × 2 2−2

x1 3.25
=
y1 3.25


3.25
Again det(J) = −2x − 2y 6= 0 for starting point
3.25
Next iteration is ( for n = 1)
x21 + y12 − 18

x2 x1 1 −1 −2y1
= + 0.5
y2 y1 x 1 + y1 −1 2x1 x1 − y1
3.252 + 3.252 − 18

x2 3.25 1 −1 −2 × 3.25
= +0.5
y2 3.25 3.25 + 3.25 −1 2 × 3.25 3.25 − 3.25

x2 3.0096
=
y2 3.0096

If we drive one iteration, we can find

x3 3.00002
=
y3 3.00002
We can stop drive the iteration, when we reach a fixed iteration

number N or when successive approximations satisfying stopping
criteria
max{(xn+1 − xn ), (yn+1 − yn )} <

For large number of equations, calculation of J −1 is not convenient
because of chopping or round off errors. Therefore we solve the
following linear system of xn and yn to drive the newton iteration

xn+1 xn f (xn , yn )
J(xn , yn ) − =− , n≥0
yn+1 yn g(xn , yn )
Lets try this for previous example...

x0 2
We have = .
y0 2
For n = 1

x1 x0 f (x0 , y0 )
J(x0 , y0 ) − =−
y1 y0 g(x0 , y0 )
x20 + y02 − 18

2x0 2y0 x1 x0
− =−
1 −1 y1 y0 x 0 − y0
22 + 22 − 18

4 4 x1 2
− =−
1 −1 y1 2 2−2

4 4 x1 − 2 10
=
1 −1 y1 − 2 0
This equivalent to the following
4x1 + 4y1 = 26
x1 − y1 = 0
Solution of above system is

x1 3.25
=
y1 3.25
..
.

Cme4041 04

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Cme4041 04

Hochgeladen von

Copyright:

Verfügbare Formate

SECANT METHOD

Newton’s method is an extremely powerful technique, but it has a

To circumvent the problem of the derivative evaluation in

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 1/ 46

The derivative of the function f (x) with respect to the variable x

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 2/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 3/ 46

If we set x0 + h = xn and x0 = xn−1 in the above formula, we get

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 4/ 46

Since the calculation of xn+1 requires xn and xn−1 , two initial

However, each new xn+1 requires only one new evaluation of f .

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 5/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 6/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 7/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 8/ 46

This is considerably better than

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 9/ 46

Find an approximation to root of the following function using

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 10/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 11/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 12/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 13/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 14/ 46

xn+1 = F (xn ) n≥0 (∗)

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 15/ 46

those cases where lim xn exists. Suppose, then, that

What is the relation between s and F ?

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 16/ 46

F (s) = F ( lim xn ) = lim F (xn ) = lim xn+1 = s

Thus, F (s) = s, and we call s a fixed point of the function F .

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 17/ 46

The function g(x) = x2 − 2 for −2 ≤ x ≤ 3, has fixed point at

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 18/ 46

Conversely, if the function g has a fixed point at p, then the

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 19/ 46

Consider solving the equation

xn+1 = 5 + xn − x2n (1)

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 20/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 21/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 22/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 23/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 24/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 25/ 46

|F (x) − F (y)| ≤ λ|x − y|

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 26/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 27/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 28/ 46

Prove that the sequence [xn ] defined recursively as follows is

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 29/ 46

λ ≡ max |g 0 (x)| < 1

There is a unique solution α of x = g(x) in the interval [a, b].

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 30/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 31/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 32/ 46

Find the fixed point of cos x in first quadrant.

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 34/ 46

will converge, for any start point x0 ∈ (0, π2 ).

n=0 ⇒ x1 = cos x0 = cos 1 = 0.540302

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 35/ 46

Newton’s method for systems of nonlinear equations follows the

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 37/ 46

Lecturer Volkan ÖĞER CME 4401 Numerical Analysis - Lecture 4 38/ 46