Computational Physics (PHY 362)

FA 16

Dr. Muhammad Kamran

Department of Physics
CIIT Islamabad

Lecture 11

17/10/2016

Lecture 11

Newton-Raphson method
We assume that the function `f ' is dierentiable.

The graph of the function has a denite slope at each point

and, hence, a unique tangent line.

Computational Physics

Dr. M. Kamran

3 / 18

17/10/2016

Lecture 11

Newton-Raphson method
We assume that the function `f ' is dierentiable.

The graph of the function has a denite slope at each point

and, hence, a unique tangent line.

At a certain point

(x0 , f (x0 ))

on the graph of `f ', there is a

tangent, which is a rather good approximation to the curve

in the vicinity of that point.

Computational Physics

Dr. M. Kamran

3 / 18

17/10/2016

Lecture 11

Newton-Raphson method
We assume that the function `f ' is dierentiable.

The graph of the function has a denite slope at each point

and, hence, a unique tangent line.

At a certain point

(x0 , f (x0 ))

on the graph of `f ', there is a

tangent, which is a rather good approximation to the curve

in the vicinity of that point.
Analytically, it means that the linear function
g (x)

= f (x0 ) (x x0 ) + f (x0 ) ,

(1)

is close to the given function `f ' near x0 .

Computational Physics

Dr. M. Kamran

3 / 18

17/10/2016

Lecture 11

Newton-Raphson method
We assume that the function `f ' is dierentiable.

The graph of the function has a denite slope at each point

and, hence, a unique tangent line.

At a certain point

(x0 , f (x0 ))

on the graph of `f ', there is a

tangent, which is a rather good approximation to the curve

in the vicinity of that point.
Analytically, it means that the linear function
g (x)

= f (x0 ) (x x0 ) + f (x0 ) ,

(1)

is close to the given function `f ' near x0 .

At x0 , the two functions, `f ' and `g' agree.

We take the root of `g' as an approximation to the root of

`f'.

Computational Physics

Dr. M. Kamran

3 / 18

17/10/2016

Lecture 11

Newton-Raphson method
The root of g (x) can be easily found from Eq. 1.
x1

Computational Physics

= x0

(x0 )
.
0)

f (x

Dr. M. Kamran

(2)

4 / 18

17/10/2016

Lecture 11

Newton-Raphson method
The root of g (x) can be easily found from Eq. 1.
x1

= x0

(x0 )
.
0)

f (x

(2)

The point x0 is used as an approximation to the root

sought.

Starting with point x0 , we pass to a new point x1 .

Computational Physics

Dr. M. Kamran

4 / 18

17/10/2016

Lecture 11

Newton-Raphson method
The root of g (x) can be easily found from Eq. 1.
x1

= x0

(x0 )
.
0)

(2)

f (x

The point x0 is used as an approximation to the root

sought.

Starting with point x0 , we pass to a new point x1 .

The process can be repeated to produce a sequence of
points.
x2

= x1

(x1 )
,
1)

f (x

Computational Physics

x3

= x2

(x2 )
.
2)

f (x

Dr. M. Kamran

(3)

4 / 18

17/10/2016

Lecture 11

Newton-Raphson method
The root of g (x) can be easily found from Eq. 1.
x1

= x0

(x0 )
.
0)

(2)

f (x

The point x0 is used as an approximation to the root

sought.

Starting with point x0 , we pass to a new point x1 .

The process can be repeated to produce a sequence of
points.
x2

= x1

(x1 )
,
1)

f (x

x3

= x2

(x2 )
.
2)

f (x

(3)

Under favorable conditions, the sequence of points will

approach a zero of `f '.
Computational Physics

Dr. M. Kamran

4 / 18

17/10/2016

Lecture 11

Newton-Raphson method

Figure: Newton-Raphson method

Computational Physics

Dr. M. Kamran

5 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Describing the Newton's method in terms of a sequence
x0 , x1 , . . ., we can write
xn+1

Computational Physics

= xn

(xn )
.
n)

f (x

Dr. M. Kamran

(4)

6 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Describing the Newton's method in terms of a sequence
x0 , x1 , . . ., we can write
xn+1

= xn

(xn )
.
n)

f (x

(4)

There are other ways of interpreting Newton's method.

Computational Physics

Dr. M. Kamran

6 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Describing the Newton's method in terms of a sequence
x0 , x1 , . . ., we can write
xn+1

= xn

(xn )
.
n)

f (x

(4)

There are other ways of interpreting Newton's method.

Suppose again that x0 is an initial approximation to a root
of f .

Computational Physics

Dr. M. Kamran

6 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Describing the Newton's method in terms of a sequence
x0 , x1 , . . ., we can write
xn+1

= xn

(xn )
.
n)

f (x

(4)

There are other ways of interpreting Newton's method.

Suppose again that x0 is an initial approximation to a root
of f .
We add a correction h to x0 to obtain the root precisely.
f

Computational Physics

(x0 + h) = 0.

Dr. M. Kamran

(5)

6 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Describing the Newton's method in terms of a sequence
x0 , x1 , . . ., we can write
xn+1

= xn

(xn )
.
n)

f (x

(4)

There are other ways of interpreting Newton's method.

Suppose again that x0 is an initial approximation to a root
of f .
We add a correction h to x0 to obtain the root precisely.
f

(x0 + h) = 0.

(5)

If f is a suciently well-behaved function, it will have a

Taylor series at x0 .
f

(x0 ) + hf (x0 ) +

Computational Physics

(x0 ) + . . . = 0.
Dr. M. Kamran

(6)
6 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Determining `h' from this equation is, of course, not easy.
Therefore, we give up the expectation of arriving at the
true root in one step and seek only an approximation to h.

Computational Physics

Dr. M. Kamran

7 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Determining `h' from this equation is, of course, not easy.
Therefore, we give up the expectation of arriving at the
true root in one step and seek only an approximation to h.
We retain the rst two terms in the series, and ignore
higher terms.
f

Computational Physics

(x0 ) + hf (x0 ) = 0

Dr. M. Kamran

(7)

7 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Determining `h' from this equation is, of course, not easy.
Therefore, we give up the expectation of arriving at the
true root in one step and seek only an approximation to h.
We retain the rst two terms in the series, and ignore
higher terms.
f

(x0 ) + hf (x0 ) = 0

(7)

Note: The h that solves Eq. 7 is not the one that solves Eq.
6.

Computational Physics

Dr. M. Kamran

7 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Determining `h' from this equation is, of course, not easy.
Therefore, we give up the expectation of arriving at the
true root in one step and seek only an approximation to h.
We retain the rst two terms in the series, and ignore
higher terms.
f

(x0 ) + hf (x0 ) = 0

(7)

Note: The h that solves Eq. 7 is not the one that solves Eq.
6.
From Eq. 7, we obtain
h

Computational Physics

(x0 )
.
0)

f (x

Dr. M. Kamran

(8)

7 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Determining `h' from this equation is, of course, not easy.
Therefore, we give up the expectation of arriving at the
true root in one step and seek only an approximation to h.
We retain the rst two terms in the series, and ignore
higher terms.
f

(x0 ) + hf (x0 ) = 0

(7)

Note: The h that solves Eq. 7 is not the one that solves Eq.
6.
From Eq. 7, we obtain
h

(x0 )
.
0)

f (x

(8)

Our new approximation is then

Computational Physics

Dr. M. Kamran

7 / 18

17/10/2016

Lecture 11

Newton-Raphson method

x1

= x0 + h = x0

Computational Physics

(x0 )
.
0)

f (x

Dr. M. Kamran

(9)

8 / 18

17/10/2016

Lecture 11

Newton-Raphson method

x1

= x0 + h = x0

(x0 )
.
0)

f (x

(9)

The process can be repeated to get a more accurate root.

Computational Physics

Dr. M. Kamran

8 / 18

17/10/2016

Lecture 11

Newton-Raphson method

x1

= x0 + h = x0

(x0 )
.
0)

(9)

f (x

The process can be repeated to get a more accurate root.

Problem: If f

(x) = x3 x + 1

and x0

= 1,

calculate x1 and x2

in the Newton-Raphson iteration.

Computational Physics

Dr. M. Kamran

8 / 18

17/10/2016

Lecture 11

Newton-Raphson method

x1

= x0 + h = x0

(x0 )
.
0)

(9)

f (x

The process can be repeated to get a more accurate root.

Problem: If f

(x) = x3 x + 1

and x0

= 1,

calculate x1 and x2

in the Newton-Raphson iteration.

f

(x0 ) = 1;

Computational Physics

(x0 ) = 2.

Dr. M. Kamran

8 / 18

17/10/2016

Lecture 11

Newton-Raphson method

x1

= x0 + h = x0

(x0 )
.
0)

(9)

f (x

The process can be repeated to get a more accurate root.

Problem: If f

(x) = x3 x + 1

and x0

= 1,

calculate x1 and x2

in the Newton-Raphson iteration.

f

(x0 ) = 1;
x1

Computational Physics

(x0 ) = 2.

= .
2

Dr. M. Kamran

8 / 18

17/10/2016

Lecture 11

Newton-Raphson method

x1

= x0 + h = x0

(x0 )
.
0)

(9)

f (x

The process can be repeated to get a more accurate root.

Problem: If f

(x) = x3 x + 1

and x0

= 1,

calculate x1 and x2

in the Newton-Raphson iteration.

f

(x0 ) = 1;
x1

(x1 ) = ;

Computational Physics

(x0 ) = 2.

= .
2
f

(x1 ) = .
4

Dr. M. Kamran

8 / 18

17/10/2016

Lecture 11

Newton-Raphson method

x1

= x0 + h = x0

(x0 )
.
0)

(9)

f (x

The process can be repeated to get a more accurate root.

Problem: If f

(x) = x3 x + 1

and x0

= 1,

calculate x1 and x2

in the Newton-Raphson iteration.

f

(x0 ) = 1;
x1

(x1 ) = ;
8

(x0 ) = 2.

= .
2
f

x2
Computational Physics

(x1 ) = .
4

= 3.
Dr. M. Kamran

8 / 18

17/10/2016

Lecture 11

Newton-Raphson method
In the use of Newton's method, consideration must be given
to the proper choice of a starting point.
Its convergence depends upon hypotheses that are dicult
to verify a priori.

Computational Physics

Dr. M. Kamran

9 / 18

17/10/2016

Lecture 11

Newton-Raphson method
In the use of Newton's method, consideration must be given
to the proper choice of a starting point.
Its convergence depends upon hypotheses that are dicult
to verify a priori.
Sometimes, due to a poor choice of the initial point, it may
happen that the tangent to the graph of the function  f  at
x0 intersects the x-axis at a point remote from the root  r.

Computational Physics

Dr. M. Kamran

9 / 18

17/10/2016

Lecture 11

Newton-Raphson method
In the use of Newton's method, consideration must be given
to the proper choice of a starting point.
Its convergence depends upon hypotheses that are dicult
to verify a priori.
Sometimes, due to a poor choice of the initial point, it may
happen that the tangent to the graph of the function  f  at
x0 intersects the x-axis at a point remote from the root  r.

Figure: Runaway
Computational Physics

Dr. M. Kamran

9 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Successive points in Newton's iteration will recede from the
root instead of converging to it.

Computational Physics

Dr. M. Kamran

10 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Successive points in Newton's iteration will recede from the
root instead of converging to it.
It may happen that the tangent to the curve is parallel to
the x-axis and x1

= ,

or it is assigned the value of

machine innity in a computer.

Computational Physics

Dr. M. Kamran

10 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Successive points in Newton's iteration will recede from the
root instead of converging to it.
It may happen that the tangent to the curve is parallel to
the x-axis and x1

= ,

or it is assigned the value of

machine innity in a computer.

It may also happen that the iteration values cycle because
x2

= x0 .

Computational Physics

Dr. M. Kamran

10 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Successive points in Newton's iteration will recede from the
root instead of converging to it.
It may happen that the tangent to the curve is parallel to
the x-axis and x1

= ,

or it is assigned the value of

machine innity in a computer.

It may also happen that the iteration values cycle because
x2

= x0 .

Figure: Flat spot

Computational Physics

Dr. M. Kamran

10 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Successive points in Newton's iteration will recede from the
root instead of converging to it.
It may happen that the tangent to the curve is parallel to
the x-axis and x1

= ,

or it is assigned the value of

machine innity in a computer.

It may also happen that the iteration values cycle because
x2

= x0 .

Figure: Flat spot

Computational Physics

Figure: Cycle
Dr. M. Kamran

10 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Roundo errors or limited precision may eventually cause
this situation to become unbalanced.

The iterates will either spiral inward and converge or spiral

outward and diverge.

Computational Physics

Dr. M. Kamran

11 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Roundo errors or limited precision may eventually cause
this situation to become unbalanced.

The iterates will either spiral inward and converge or spiral

If f

outward and diverge.

(r) = 0, then `r' is a zero

of the function and its

derivative. Such a zero is termed a multiple zero of f .

In this case, it is, at least, a double zero.

Computational Physics

Dr. M. Kamran

11 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Roundo errors or limited precision may eventually cause
this situation to become unbalanced.

The iterates will either spiral inward and converge or spiral

If f

outward and diverge.

(r) = 0, then `r' is a zero

of the function and its

derivative. Such a zero is termed a multiple zero of f .

In this case, it is, at least, a double zero.

Suppose that the multiplicity of the root is m, then,
Newton's method is modied to read
xn+1

Computational Physics

= xn m

f
f

(xn )
.
(xn )

Dr. M. Kamran

(10)

11 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Problem: Carry out two iterations of Newton's method using
x0

=1

and f

(x) = x2 + 10x 3.

Computational Physics

Dr. M. Kamran

12 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Problem: Carry out two iterations of Newton's method using
x0

=1

and f

(x) = x2 + 10x 3.
f

(x) = x2 + 10x 3;

Computational Physics

(x) = 2x + 10.

Dr. M. Kamran

12 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Problem: Carry out two iterations of Newton's method using
x0

=1

and f

(x) = x2 + 10x 3.
f

(x) = x2 + 10x 3;
f

(x0 ) = 8;

Computational Physics

(x) = 2x + 10.

(x0 ) = 12.

Dr. M. Kamran

12 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Problem: Carry out two iterations of Newton's method using
x0

=1

and f

(x) = x2 + 10x 3.
f

(x) = x2 + 10x 3;
f

(x0 ) = 8;
x1

Computational Physics

(x) = 2x + 10.

(x0 ) = 12.

= 1/3.

Dr. M. Kamran

12 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Problem: Carry out two iterations of Newton's method using
x0

=1

and f

(x) = x2 + 10x 3.
f

(x) = x2 + 10x 3;
f

(x0 ) = 8;
x1

(x1 ) = 4/9;

Computational Physics

(x) = 2x + 10.

(x0 ) = 12.

= 1/3.
f

(x1 ) = 32/3.

Dr. M. Kamran

12 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Problem: Carry out two iterations of Newton's method using
x0

=1

and f

(x) = x2 + 10x 3.
f

(x) = x2 + 10x 3;
f

(x0 ) = 8;
x1

(x1 ) = 4/9;
x2

Computational Physics

(x) = 2x + 10.

(x0 ) = 12.

= 1/3.
f

(x1 ) = 32/3.

= 7/24.
Dr. M. Kamran

12 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Dening the function (using symbolic toolbox)
syms x
f

= symfun (x3 x 1, x);

Computational Physics

Dr. M. Kamran

13 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Dening the function (using symbolic toolbox)
syms x
f

= symfun (x3 x 1, x);

Derivative of the function

f_der = di(f , x);

Computational Physics

Dr. M. Kamran

13 / 18

17/10/2016

Lecture 11

Newton-Raphson method
Dening the function (using symbolic toolbox)
syms x
f

= symfun (x3 x 1, x);

Derivative of the function

f_der = di(f , x);
Plotting the function
fplot(f,`b

')

grid on
xlabel(`$$x$$',`Interpreter',`Latex')
ylabel(`$$f

(x)$$',`Interpreter',`Latex')
3

title( [ `Plot of the function ',`$$x

x 1$$'],

`Interpreter',`Latex')

Computational Physics

Dr. M. Kamran

13 / 18

17/10/2016

Lecture 11

Newton-Raphson method
The initial guess
x0

= 1.0;

Computational Physics

Dr. M. Kamran

14 / 18

17/10/2016

Lecture 11

Newton-Raphson method
The initial guess
x0

= 1.0;

Tolerance allowed

min_err = 10

Computational Physics

Dr. M. Kamran

14 / 18

17/10/2016

Lecture 11

Newton-Raphson method
The initial guess
x0

= 1.0;

Tolerance allowed

min_err = 10

Calling function to calculate the root

[x_root,iterations_vec,rel_err_vec] =
new_raph_fun(f,f_der,x_0,min_err);

Computational Physics

Dr. M. Kamran

14 / 18

17/10/2016

Lecture 11

Newton-Raphson method
The initial guess
x0

= 1.0;

Tolerance allowed

min_err = 10

Calling function to calculate the root

[x_root,iterations_vec,rel_err_vec] =
new_raph_fun(f,f_der,x_0,min_err);
Printing the result
formatspec = `The calculated root is % 3.7f \n \n';
fprintf(formatspec,x_root)

Computational Physics

Dr. M. Kamran

14 / 18

17/10/2016

Lecture 11

Newton-Raphson method
The initial guess
x0

= 1.0;

Tolerance allowed

min_err = 10

Calling function to calculate the root

[x_root,iterations_vec,rel_err_vec] =
new_raph_fun(f,f_der,x_0,min_err);
Printing the result
formatspec = `The calculated root is % 3.7f \n \n';
fprintf(formatspec,x_root)
Plotting the relative error
gure
plot(iterations_vec,rel_err_vec,`r')
xlabel(`Number of iterations')
ylabel(`Relative error')
title(`Plot of the relative error')
Computational Physics

Dr. M. Kamran

14 / 18

17/10/2016

Lecture 11

Newton-Raphson method
The function le
function [x_root,iterations_vec,rel_err_vec] =
new_raph_fun(f,f_der,x_0,min_err)
x_old = x_0;
rel_err = 1;
iterations = 0;
counter = 1;
while rel_err > min_err
f_x_old = double(subs(f,x_old));
f_der_x_old = double(subs(f_der,x_old));
x_new = x_old

f_x_old / f_der_x_old;

rel_err = abs( (x_new

Computational Physics

x_old) / x_old );

Dr. M. Kamran

15 / 18

17/10/2016

Lecture 11

Newton-Raphson method
x_old = x_new;
iterations_vec (counter) = iterations;
rel_err_vec(counter) = rel_err;
counter = counter + 1;
iterations = iterations + 1;
end
x_root = x_new;

Computational Physics

Dr. M. Kamran

16 / 18

17/10/2016

Lecture 11

Newton-Raphson method

Figure: The function

Computational Physics

Dr. M. Kamran

17 / 18

17/10/2016

Lecture 11

Newton-Raphson method

Figure: Variation of relative error

Computational Physics

Dr. M. Kamran

18 / 18