Department of Mathematics

Amer Rasheed
Lecture 2
Numerical Analysis
Math344, Math541

Amer Rasheed (MATH344) Department of Mathematics 1 / 20

 Solution of Non-Linear Equations Fixed Point Iteration

Fixed Point Iteration

Iteration
The repetition of a process, particularly, repetition of a mathematical or
computational procedure applied to obtain successively closer approxi-
mations to the desired result.

Fixed Point
A fixed point of a (real) function g(x) is a real number P such that

P = g(P )

Fixed Point Iteration In order to obtain fixed point of a function g(x), if

it exists, we set an iteration

Pn+1 = g(Pn )

called fixed point iterations.

Amer Rasheed (MATH344) Department of Mathematics 2 / 20
 Solution of Non-Linear Equations Fixed Point Iteration

Fixed Point Iteration

Example
Find the fixed point of

g(x) = ex , x0 = 0.5.
We write the fixed point iteration as

xn+1 = g(xn ) = exn , n = 0, 1, 2, 3, . . .

Amer Rasheed (MATH344) Department of Mathematics 3 / 20

 Solution of Non-Linear Equations Fixed Point Iteration

Fixed Point Iteration

MatLab Code

Amer Rasheed (MATH344) Department of Mathematics 4 / 20

 Solution of Non-Linear Equations Fixed Point Iteration

Fixed Point Iteration

Iterations

Amer Rasheed (MATH344) Department of Mathematics 5 / 20

 Solution of Non-Linear Equations Fixed Point Iteration

Fixed Point Iteration

Theorem
Assume that g is a continuous function and that {Pn } n=0 is a sequence
generated by fixed point iteration. If lim Pn = P . Then P is a fixed point
n
of g(x).
Proof Given lim Pn = P . Which implies that lim Pn+1 = P
n n
Thus

P = lim Pn+1 = lim g(Pn )

n n
= g( lim Pn ), g is continuous.
n
= g(P ).

Thus P is a fixed point.

Amer Rasheed (MATH344) Department of Mathematics 6 / 20

 Solution of Non-Linear Equations Fixed Point Iteration

Fixed Point Iteration

Existence and Uniqueness

Theorem (Existence and Uniqueness)

Assume that g C[a, b]. If range of the mapping y = g(x) satisfies

a y b, a < x < b,

Then g has a fix point in [a, b].

Furthermore, suppose that g 0 (x) exists and defined over (a, b) and that
a positive constant K < 1 exists with

|g 0 (x)| K < 1, x (a, b)

Then g has a unique fix point in (a, b).

Amer Rasheed (MATH344) Department of Mathematics 7 / 20

 Solution of Non-Linear Equations Fixed Point Iteration

Fixed Point Iteration

Existence and Uniqueness

Proof (Existence)
If g(a) = a or g(b) = b, the statement is true.
Otherwise, the range values g(a) and g(b) must satisfy
a < g(a) b & a g(b) < b
We can write
a g(a) < 0 & b g(b) > 0
We define a function
f (x) = x g(x), x (a, b)
which has the property
f (a) = a g(a) < 0, & f (b) = b g(b) > 0
Thus
f (a) < L < f (b), L=0
Amer Rasheed (MATH344) Department of Mathematics 8 / 20
 Solution of Non-Linear Equations Fixed Point Iteration

Fixed Point Iteration

Existence and Uniqueness

Thus
f (a) < L < f (b), L=0
Then by intermediate value theorem for f (x) on [a, b], there exists a
number P such that a < P < b with

f (P ) = L = 0

Therefore x = P is the desired fixed point of g(x).

Amer Rasheed (MATH344) Department of Mathematics 9 / 20

 Solution of Non-Linear Equations Fixed Point Iteration

Fixed Point Iteration

Existence and Uniqueness

Proof (Uniqueness)
We show that the fixed point is unique. Let us assume that there exists two
fixed points P1 and P2 of g(x).
As g(x) C[a, b], then by using Mean Value Theorem, there exists a number c
with a < c < b, such that

g(P2 ) g(P1 )
= g 0 (c)
P2 P1
As P1 and P2 are fixed points of g(x), thus
P2 P1
= g 0 (c) g 0 (c) = 1
P2 P1
which is contradiction to the hypothesis (given) that

g 0 (c) < 1, x [a, b]

Therefore, g(x) has unique fixed point P in [a, b].

Amer Rasheed (MATH344) Department of Mathematics 10 / 20
 Solution of Non-Linear Equations Fixed Point Theorem

Fixed Point Iteration

Fixed Point Theorem

Assume that g(x) & g 0 (x) are continuous on (a, b) = (P , P + ) that
contains the unique fixed point P and the starting value P0 is chosen in
the interval.
1). If g 0 (c) 1, x (a, b), then the iteration Pn = g(Pn1 ) will
converge to P. (attractive fixed point)
2). If g 0 (c) > 1, x (a, b), then the iteration Pn = g(Pn1 ) will not
converge to P. (repulsive fixed point)

Amer Rasheed (MATH344) Department of Mathematics 11 / 20

 Solution of Non-Linear Equations Fixed Point Theorem

Fixed Point Iteration

Proof
First, we show that the points {Pn } all lie in (a, b). Starting with P0 and
using Mean Value theorem, there exists a value c (a, b) such that

|P P1 | = |g(P ) g(P0 )| = |g 0 (c0 )(P P0 )|

|P P1 | K|P P0 |
As P0 (a, b) = (P , P + ) |P P0 | < , thus

|P P1 | < , K<1

P1 (a, b) = (P , P + )
Similarly, we can show that

|P P2 | K|P P1 |

Amer Rasheed (MATH344) Department of Mathematics 12 / 20

 Solution of Non-Linear Equations Fixed Point Theorem

Fixed Point Iteration

In general

|P Pn | = |g(P ) g(Pn1 )| = |g 0 (cn1 )||P Pn1 |

|P Pn | K|P Pn1 | <
Therefore Pn (P , P + ) and hence by induction all the points {Pn }
lie in (P , P + ) Next, we show that lim Pn = P . Again by using
n
Mean Value theorem

|P P1 | = |g(P ) g(P0 )| = |g 0 (c0 )(P P0 )|

|P P1 | K|P P0 |
Similarly, we can show that

|P P2 | K 2 |P P0 |

Amer Rasheed (MATH344) Department of Mathematics 13 / 20

 Solution of Non-Linear Equations Fixed Point Theorem

Fixed Point Iteration

Proof
In general, we can prove that

|P Pn | K n |P P0 |

As K < 1, by applying limit n on both sides

0 < lim |P Pn | lim K n |P P0 |

n n

Therefore
0 lim |P Pn | 0
n

and thus
lim Pn = P
n

Amer Rasheed (MATH344) Department of Mathematics 14 / 20

 Solution of Non-Linear Equations Fixed Point Theorem

Fixed Point Iteration

Order/Speed of Convergence
Assume that {pn }n=0 converges to p and set en = p pn for n 0. If
two positive constant A 6= 0 and R > 0 exist, and

|p pn+1 | |en+1 |
lim R
= = A,
n0 |p pn | |en |R

Then the sequence {pn } n=0 is said to converge to p with order of

convergence R. The number A is called asymptotic error constant.
The cases R = 1, 2 are given special considerations.
If R = 1, the convergence of {pn }
n=0 is called LINEAR.
If R = 2, the convergence of {pn }
n=0 is called QUADRATIC.

e.g., if R = 2 and |en | = 102 . Then |en+1 | = A 104

Amer Rasheed (MATH344) Department of Mathematics 15 / 20

 Solution of Non-Linear Equations Fixed Point Theorem

Fixed Point Iteration

Theorem (Order of Convergence of Fixed Point Iteration)
Let p be a fixed point of the iteration
pn+1 = g(pn ), n0
and suppose that g 0 (p) 6= 0. Then the iteration will have a linear rate of
convergence.
Proof : Using Taylors expansion of g(x) about x = p
g 00 ()
g(x) = g(p) + g 0 (p)(x p) + (x p)2 , (x, p)
2
Evaluating at pn and noting that pn+1 = g(pn ) and g(p) = p, we obtain
g 00 ()
pn+1 = p + g 0 (p)(pn p) + (pn p)2 ,
2
pn+1 p g 00 ()
= g 0 (p) + (pn p),
pn p 2
Amer Rasheed (MATH344) Department of Mathematics 16 / 20
 Solution of Non-Linear Equations Fixed Point Theorem

Fixed Point Iteration

pn+1 p g 00 ()
= g 0 (p) + (pn p),
pn p 2

pn+1 p g 00 ()
lim = g 0 (p) + lim (pn p),
n pn p n 2
pn+1 p
lim = g 0 (p),
n pn p
|pn+1 p|
lim = |g 0 (p)|,
n |pn p|
|en+1 |
lim =A
n |en |1

Therefore, fixed point iteration converges LINEARLY.

Amer Rasheed (MATH344) Department of Mathematics 17 / 20
 Solution of Non-Linear Equations Fixed Point Iteration

Fixed Point Iteration

Updated Code

Amer Rasheed (MATH344) Department of Mathematics 18 / 20

 Solution of Non-Linear Equations Fixed Point Iteration

Fixed Point Iteration

Iterations

Amer Rasheed (MATH344) Department of Mathematics 19 / 20

 Solution of Non-Linear Equations Fixed Point Iteration

Fixed Point Iteration

Graphical Interpretation

Amer Rasheed (MATH344) Department of Mathematics 20 / 20