10.3 Newtons Method | Department of Electrical and Computer Enginee...

https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/10RootFinding/...

Search:

This Text

ECE

UW

Numerical Analysis for Engineering

Skip to the content of the web site. ECE Home Undergraduate Home My Home Numerical Analysis Table of Contents 0 Introduction 1 Error Analysis 2 Numeric Representation 3 Iteration 4 Linear Algebra 5 Interpolation 6 Least Squares 7 Taylor Series 8 Bracketing 9 The Five Techniques 10 Root Finding 10.1 Bisection Method 10.2 False-Position Method 10.3 Newtons Method 10.4 Secant Method 10.5 Polynomials 10.6 Mllers Method 10.7 Newtons Method in N Dimensions 11 Optimization 12 Differentiation 13 Integration 14 Initial-value Problems 15 Boundary-value Problems Appendices

10.3 Newtons Method

Introduction Theory HOWTO Error Analysis Examples Questions Applications in Engineering Matlab Maple

1 of 9

30/01/2014 9:19

10.3 Newtons Method | Department of Electrical and Computer Enginee...

https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/10RootFinding/...

Introduction
Newtons method is a technique for finding the root of a scalar-valued function f(x) of a single variable x. It has rapid convergence properties but requires that model information providing the derivative exists.

Background
Useful background for this topic includes: 3. Iteration 7. Taylor series

References
Bradie, Section 2.4, Newtons Method, p.95. Mathews, Section 2.4, Newton-Raphson and Secant Methods, p.70. Weisstein, http://mathworld.wolfram.com/NewtonsMethod.html.

Theory Assumptions
Newtons method is based on the assumption that functions with continuous derivatives look like straight lines when you zoom in closely enough to the functions. This is demonstrated here. We will assume that f(x) is a scalar-valued function of a single variable x and that f(x) has a continuous derivative f(1)(x) which we can compute.

Derivation
Suppose we have an approximation xa to a root r of f(x), that is, f(r) = 0. Figure 1 shows a function with a root and an approximation to that root.

2 of 9

30/01/2014 9:19

10.3 Newtons Method | Department of Electrical and Computer Enginee...

https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/10RootFinding/...

Figure 1. A function f(x), a root r, and an approximation to that root xa. Because f(x) has a continuous derivative, it follows that at any point on the curve of f(x), if we examine it closely enough, that it will look like a straight line. If this is the case, why not approximate the function at (xa, f(xa)) by a straight line which is tangent Txa to the curve at that point? This is shown in Figure 2.

Figure 2. The line tangent to the point (xa, f(xa)).

3 of 9

30/01/2014 9:19

10.3 Newtons Method | Department of Electrical and Computer Enginee...

https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/10RootFinding/...

The formula for this line may be deduced quite easily: the linear polynomial f(1)(xa) (x - xa) is zero at xa and has a slope of f(1)(xa), and therefore, if we add f(xa), it will be tangent to the given point on the curve, that is, the linear polynomial is the tangent line. Because the tangent line is a good approximation to the function, it follows that the root of the tangent line should be a better approximation to the root than xa, and solving for the root of the tangent is straight-forward:

HOWTO Problem
Given a function of one variable, f(x), find a value r (called a root) such that f(r) = 0.

Assumptions
We will assume that the function f(x) is continuous and has a continuous derivative.

Tools
We will use sampling, the derivative, and iteration. Information about the derivative is derived from the model. We use Taylor series for error analysis.

Initial Requirements
We have an initial approximation x0 of the root.

Iteration Process
Given the approximation xn, the next approximation xn + 1 is defined to be

Halting Conditions
There are three conditions which may cause the iteration process to halt: 1. We halt if both of the following conditions are met: The step between successive iterates is sufficiently small, |xn + 1 - xn| < step, and The function evaluated at the point xn + 1 is sufficiently small, |f(xn + 1)| < abs.

4 of 9

30/01/2014 9:19

10.3 Newtons Method | Department of Electrical and Computer Enginee...

https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/10RootFinding/...

2. If the derivative f(1)(xn) = 0, the iteration process fails (division-by-zero) and we halt. 3. If we have iterated some maximum number of times, say N, and have not met Condition 1, we halt and indicate that a solution was not found. If we halt due to Condition 1, we state that xn + 1 is our approximation to the root. If we halt due to either Condition 2 or 3, we may either choose a different initial approximation x0, or state that a solution may not exist.

Error Analysis
Given that we are using Newtons method to approximate a root of the function f(x). Suppose we have an approximation of the root xn which has an error of (r - xn). What is the error of the next approximation xn + 1 found after one iteration of Newtons method? Suppose r is the actual root of f(x). Then from the Taylor series, we have that: where [r, xn]. Note, however, that f(r) = 0, so if we set the left-hand side to zero and divide both sides by f(1)(xn), we get: We can bring the first two terms to the left-hand side and multiple each side by -1. For the next step, I will group two of the terms on the terms on the left-hand side: Note that the object in the parentheses on the left-hand side is, by definition, xn + 1 (after all, xn + 1 = xn f(xn)/f(1)(xn) ), and thus we have: But the left hand side is the error of xn + 1, and therefore we see that error is reduced by a scalar multiple of the square of the previous error. To demonstrate this, let us find the root of f(x) = ex - 2 starting with x0 = 1. We note that the 1st and 2nd derivatives of f(x) are equal, so we will approximate f(2)()/f(1)(xn) by . Table 1 shows the Newton iterates, their absolute errors, and the approximation of the error based on the square previous error. Table 1. Newton iterates in finding a root of f(x) = ex - 2. n 0 1.0 xn errn = ln(2) - xn -3.069 10-1 errn - 12 N/A -4.708 10-2 -9.079 10-4

1 0.735758882342885 -4.261 10-2 2 0.694042299918915 -8.951 10-4

5 of 9

30/01/2014 9:19

10.3 Newtons Method | Department of Electrical and Computer Enginee...

https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/10RootFinding/...

3 0.693147581059771 -4.005 10-7 4 0.693147180560025 -8.016 10-14

-4.006 10-7 -8.020 10-14

Note that the error at the nth step is very closely approximated by the error of the (n - 1)th step. Now, in reality, we do not know what the actual error is (otherwise, we wouldnt be using Newtons method, would we?) but this reassures us that, under reasonable conditions, Newtons method will converge very quickly.

Failure of Newtons Method

The above formula suggests that there are three situations where Newtons method may not converge quickly: 1. Our approximation is far away from the actual root, 2. The 2nd derivative is very large, or 3. The derivative at xn is close to zero.

Examples Example 1
As an example of Newtons method, suppose we wish to find a root of the function f(x) = cos(x) + 2 sin(x) + x2. A closed form solution for x does not exist so we must use a numerical technique. We will use x0 = 0 as our initial approximation. We will let the two values step = 0.001 and abs = 0.001 and we will halt after a maximum of N = 100 iterations. From calculus, we know that the derivative of the given function is f(1)(x) = -sin(x) + 2 cos(x) + 2x. We will use four decimal digit arithmetic to find a solution and the resulting iteration is shown in Table 1. Table 1. Newtons method applied to f(x) = cos(x) + 2 sin(x) + x2. n xn xn + 1 |f(xn + 1)| |xn + 1 - xn| 0.5000 0.1368

0 0.0

-0.5000 0.1688

1 -0.5000 -0.6368 0.0205

2 -0.6368 -0.6589 0.0008000 0.02210 3 -0.6589 -0.6598 0.0006 0.0009

Thus, with the last step, both halting conditions are met, and therefore, after four iterations, our approximation to the root is -0.6598 .

6 of 9

30/01/2014 9:19

10.3 Newtons Method | Department of Electrical and Computer Enginee...

https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/10RootFinding/...

Questions Question 1
Find a root of the function f(x) = e-x cos(x) starting with x0 = 1.3 . The terminating conditions are given by abs = 1e-5 and step = 1e-5. Answer: 1.57079632679490 after five iterations.

Question 2
Perform three steps of Newtons method for the function f(x) = x2 - 2 starting with x0 = 1. Use a calculator for the third step. Answer: 3/2, 17/12, 577/408 1.414215686274510

Applications to Engineering
Consider the circuit, consisting of a voltage source, a resistor, and a diode, shown in Figure 1.

Figure 1. A resistor-diode circuit. Suppose we wish to find the current running through this circuit. To do so, we can use Kirchhoffs voltage law (KVL) which says that the sum of the voltages around a loop is zero. For this, we need the model of diode which states that the relationship between the current and the voltage across a diode is given by the equation Solving this equation for the voltage v and using values IS = 8.3e-10 A, VT = 0.7 V, and n = 2, we get from

7 of 9

30/01/2014 9:19

10.3 Newtons Method | Department of Electrical and Computer Enginee...

https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/10RootFinding/...

KVL that This equation cannot be solved exactly for the current with any of the tools available in an undergraduate program (it requires the use of the Lambert W function). Therefore we must resort to using numerical methods: Defining the left hand side of the equation to be v(i), we have to solve v(i) = 0 for i, and we will continue iterating until step < 1e-10 and abs < 1e-5.Table 1. Newtons method applied to v(i). n 0 0.0 in in + 1 |v(in + 1)| |in + 1 - in| 2.96e-10 5.83e-11 1.53e-12 9.94e-16

2.964283918e-10 0.0724

1 2.964283918e-10 3.547336588e-10 1.81e-3 2 3.547336588-10 3.562680160-10 1.17e-5

3 3.562680160e-10 3.562690102e-10 2.63e-9

Therefore, the current is approximately 3.562690102e-10 A.

Matlab
Finding a root of f(x) = cos(x):
eps_step = 1e-5; eps_abs = 1e-5; N = 100; x = 0.2; for i=1:N xn = x - cos(x)/( -sin(x) ); if abs( x - xn ) < eps_step && abs( cos( xn ) ) < eps_abs break; elseif i == N error( 'Newton\'s method did not converge' ); end x = xn; end xn

What happens if you start with smaller values of x? Why?

Maple
The following commands in Maple:
with( Student[Calculus1] ): NewtonsMethod( cos(x), x = 0.5, output = plot, iterations = 5 );

8 of 9

30/01/2014 9:19

10.3 Newtons Method | Department of Electrical and Computer Enginee...

https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/10RootFinding/...

produces the plot in Figure 1 (click to enlarge).

Figure 1. Newtons method applied to f(x) = cos(x). For more help on Newtons method or on the Student[Calculus1] package, enter:
?NewtonsMethod ?Student,Calculus1

Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo 200 University Avenue West Waterloo, Ontario, Canada N2L 3G1 Phone: +1 519 888 4567 extension 37023 Fax: +1 519 746 3077 Contact the Author Department of Electrical and Computer Engineering Faculty of Engineering

9 of 9

30/01/2014 9:19